research methods textual analysis

• Privacy Policy

Home » Textual Analysis – Types, Examples and Guide

Textual Analysis – Types, Examples and Guide

Table of Contents

Textual Analysis

Textual analysis is the process of examining a text in order to understand its meaning. It can be used to analyze any type of text, including literature , poetry, speeches, and scientific papers. Textual analysis involves analyzing the structure, content, and style of a text.

Textual analysis can be used to understand a text’s author, date, and audience. It can also reveal how a text was constructed and how it functions as a piece of communication.

Textual Analysis in Research

Textual analysis is a valuable tool in research because it allows researchers to examine and interpret text data in a systematic and rigorous way. Here are some ways that textual analysis can be used in research:

• To explore research questions: Textual analysis can be used to explore research questions in various fields, such as literature, media studies, and social sciences. It can provide insight into the meaning, interpretation, and communication patterns of text.
• To identify patterns and themes: Textual analysis can help identify patterns and themes within a set of text data, such as analyzing the representation of gender or race in media.
• To evaluate interventions: Textual analysis can be used to evaluate the effectiveness of interventions, such as analyzing the language and messaging of public health campaigns.
• To inform policy and practice: Textual analysis can provide insights that inform policy and practice, such as analyzing legal documents to inform policy decisions.
• To analyze historical data: Textual analysis can be used to analyze historical data, such as letters, diaries, and newspapers, to provide insights into historical events and social contexts.

Textual Analysis in Cultural and Media Studies

Textual analysis is a key tool in cultural and media studies as it enables researchers to analyze the meanings, representations, and discourses present in cultural and media texts. Here are some ways that textual analysis is used in cultural and media studies:

• To analyze representation: Textual analysis can be used to analyze the representation of different social groups, such as gender, race, and sexuality, in media and cultural texts. This analysis can provide insights into how these groups are constructed and represented in society.
• To analyze cultural meanings: Textual analysis can be used to analyze the cultural meanings and symbols present in media and cultural texts. This analysis can provide insights into how culture and society are constructed and understood.
• To analyze discourse: Textual analysis can be used to analyze the discourse present in cultural and media texts. This analysis can provide insights into how language is used to construct meaning and power relations.
• To analyze media content: Textual analysis can be used to analyze media content, such as news articles, TV shows, and films, to understand how they shape our understanding of the world around us.
• To analyze advertising : Textual analysis can be used to analyze advertising campaigns to understand how they construct meanings, identities, and desires.

Textual Analysis in the Social Sciences

Textual analysis is a valuable tool in the social sciences as it enables researchers to analyze and interpret text data in a systematic and rigorous way. Here are some ways that textual analysis is used in the social sciences:

• To analyze interview data: Textual analysis can be used to analyze interview data, such as transcribed interviews, to identify patterns and themes in the data.
• To analyze survey responses: Textual analysis can be used to analyze survey responses to identify patterns and themes in the data.
• To analyze social media data: Textual analysis can be used to analyze social media data, such as tweets and Facebook posts, to identify patterns and themes in the data.
• To analyze policy documents: Textual analysis can be used to analyze policy documents, such as government reports and legislation, to identify discourses and power relations present in the policy.
• To analyze historical data: Textual analysis can be used to analyze historical data, such as letters and diaries, to provide insights into historical events and social contexts.

Textual Analysis in Literary Studies

Textual analysis is a key tool in literary studies as it enables researchers to analyze and interpret literary texts in a systematic and rigorous way. Here are some ways that textual analysis is used in literary studies:

• To analyze narrative structure: Textual analysis can be used to analyze the narrative structure of a literary text, such as identifying the plot, character development, and point of view.
• To analyze language and style: Textual analysis can be used to analyze the language and style used in a literary text, such as identifying figurative language, symbolism, and rhetorical devices.
• To analyze themes and motifs: Textual analysis can be used to analyze the themes and motifs present in a literary text, such as identifying recurring symbols, themes, and motifs.
• To analyze historical and cultural context: Textual analysis can be used to analyze the historical and cultural context of a literary text, such as identifying how the text reflects the social and political context of its time.
• To analyze intertextuality: Textual analysis can be used to analyze the intertextuality of a literary text, such as identifying how the text references or is influenced by other literary works.

Textual Analysis Methods

Textual analysis methods are techniques used to analyze and interpret various types of text, including written documents, audio and video recordings, and online content. These methods are commonly used in fields such as linguistics, communication studies, sociology, psychology, and literature.

Some common textual analysis methods include:

Content Analysis

This involves identifying patterns and themes within a set of text data. This method is often used to analyze media content or other types of written materials, such as policy documents or legal briefs.

Discourse Analysis

This involves examining how language is used to construct meaning in social contexts. This method is often used to analyze political speeches or other types of public discourse.

Critical Discourse Analysis

This involves examining how power and social relations are constructed through language use, particularly in political and social contexts.

Narrative Analysis

This involves examining the structure and content of stories or narratives within a set of text data. This method is often used to analyze literary texts or oral histories.

This involves analyzing the meaning of signs and symbols within a set of text data. This method is often used to analyze advertising or other types of visual media.

Text mining

This involves using computational techniques to extract patterns and insights from large sets of text data. This method is often used in fields such as marketing and social media analysis.

Close Reading

This involves a detailed and in-depth analysis of a particular text, focusing on the language, style, and literary techniques used by the author.

How to Conduct Textual Analysis

Here are some general steps to conduct textual analysis:

• Choose your research question: Define your research question and identify the text or set of texts that you want to analyze.
• F amiliarize yourself with the text: Read and re-read the text, paying close attention to its language, structure, and content. Take notes on key themes, patterns, and ideas that emerge.
• Choose your analytical approach: Select the appropriate analytical approach for your research question, such as close reading, thematic analysis, content analysis, or discourse analysis.
• Create a coding scheme: If you are conducting content analysis, create a coding scheme to categorize and analyze the content of the text. This may involve identifying specific words, themes, or ideas to code.
• Code the text: Apply your coding scheme to the text and systematically categorize the content based on the identified themes or patterns.
• Analyze the data: Once you have coded the text, analyze the data to identify key patterns, themes, or trends. Use appropriate software or tools to help with this process if needed.
• Draw conclusions: Draw conclusions based on your analysis and answer your research question. Present your findings and provide evidence to support your conclusions.
• R eflect on limitations and implications: Reflect on the limitations of your analysis, such as any biases or limitations of the selected method. Also, discuss the implications of your findings and their relevance to the broader research field.

When to use Textual Analysis

Textual analysis can be used in various research fields and contexts. Here are some situations when textual analysis can be useful:

• Understanding meaning and interpretation: Textual analysis can help understand the meaning and interpretation of text, such as literature, media, and social media.
• Analyzing communication patterns: Textual analysis can be used to analyze communication patterns in different contexts, such as political speeches, social media conversations, and legal documents.
• Exploring cultural and social contexts: Textual analysis can be used to explore cultural and social contexts, such as the representation of gender, race, and identity in media.
• Examining historical documents: Textual analysis can be used to examine historical documents, such as letters, diaries, and newspapers.
• Evaluating marketing and advertising campaigns: Textual analysis can be used to evaluate marketing and advertising campaigns, such as analyzing the language, symbols, and imagery used.

Examples of Textual Analysis

Here are a few examples:

• Media Analysis: Textual analysis is frequently used in media studies to examine how news outlets and social media platforms frame and present news stories. Researchers can use textual analysis to examine the language and images used in news articles, tweets, and other forms of media to identify patterns and biases.
• Customer Feedback Analysis: Textual analysis is often used by businesses to analyze customer feedback, such as online reviews or social media posts, to identify common themes and areas for improvement. This allows companies to make data-driven decisions and improve their products or services.
• Political Discourse Analysis: Textual analysis is commonly used in political science to analyze political speeches, debates, and other forms of political communication. Researchers can use this method to identify the language and rhetoric used by politicians, as well as the strategies they employ to appeal to different audiences.
• Literary Analysis: Textual analysis is a fundamental tool in literary criticism, allowing scholars to examine the language, structure, and themes of literary works. This can involve close reading of individual texts or analysis of larger literary movements.
• Sentiment Analysis: Textual analysis is used to analyze social media posts, customer feedback, or other sources of text data to determine the sentiment of the text. This can be useful for businesses or organizations to understand how their brand or product is perceived in the market.

Purpose of Textual Analysis

There are several specific purposes for using textual analysis, including:

• To identify and interpret patterns in language use: Textual analysis can help researchers identify patterns in language use, such as common themes, recurring phrases, and rhetorical devices. This can provide insights into the values and beliefs that underpin the text.
• To explore the cultural context of the text: Textual analysis can help researchers understand the cultural context in which the text was produced, including the historical, social, and political factors that shaped the language and messages.
• To examine the intended and unintended meanings of the text: Textual analysis can help researchers uncover both the intended and unintended meanings of the text, and to explore how the language is used to convey certain messages or values.
• To understand how texts create and reinforce social and cultural identities: Textual analysis can help researchers understand how texts contribute to the creation and reinforcement of social and cultural identities, such as gender, race, ethnicity, and nationality.

Applications of Textual Analysis

Here are some common applications of textual analysis:

Media Studies

Textual analysis is frequently used in media studies to analyze news articles, advertisements, and social media posts to identify patterns and biases in media representation.

Literary Criticism

Textual analysis is a fundamental tool in literary criticism, allowing scholars to examine the language, structure, and themes of literary works.

Political Science

Textual analysis is commonly used in political science to analyze political speeches, debates, and other forms of political communication.

Marketing and Consumer Research

Textual analysis is used to analyze customer feedback, such as online reviews or social media posts, to identify common themes and areas for improvement.

Healthcare Research

Textual analysis is used to analyze patient feedback and medical records to identify patterns in patient experiences and improve healthcare services.

Social Sciences

Textual analysis is used in various fields within social sciences, such as sociology, anthropology, and psychology, to analyze various forms of data, including interviews, field notes, and documents.

Linguistics

Textual analysis is used in linguistics to study language use and its relationship to social and cultural contexts.

Advantages of Textual Analysis

There are several advantages of textual analysis in research. Here are some of the key advantages:

• Systematic and objective: Textual analysis is a systematic and objective method of analyzing text data. It enables researchers to analyze text data in a consistent and rigorous way, minimizing the risk of bias or subjectivity.
• Versatile : Textual analysis can be used to analyze a wide range of text data, including interview transcripts, survey responses, social media data, policy documents, and literary texts.
• Efficient : Textual analysis can be a more efficient method of data analysis compared to manual coding or other methods of qualitative analysis. With the help of software tools, researchers can process large volumes of text data more quickly and accurately.
• Allows for in-depth analysis: Textual analysis enables researchers to conduct in-depth analysis of text data, uncovering patterns and themes that may not be visible through other methods of data analysis.
• Can provide rich insights: Textual analysis can provide rich and detailed insights into complex social phenomena. It can uncover subtle nuances in language use, reveal underlying meanings and discourses, and shed light on the ways in which social structures and power relations are constructed and maintained.

Limitations of Textual Analysis

While textual analysis can provide valuable insights into the ways in which language is used to convey meaning and create social and cultural identities, it also has several limitations. Some of these limitations include:

• Limited Scope : Textual analysis is only able to analyze the content of written or spoken language, and does not provide insights into non-verbal communication such as facial expressions or body language.
• Subjectivity: Textual analysis is subject to the biases and interpretations of the researcher, as well as the context in which the language was produced. Different researchers may interpret the same text in different ways, leading to inconsistencies in the findings.
• Time-consuming: Textual analysis can be a time-consuming process, particularly if the researcher is analyzing a large amount of text. This can be a limitation in situations where quick analysis is necessary.
• Lack of Generalizability: Textual analysis is often used in qualitative research, which means that its findings cannot be generalized to larger populations. This limits the ability to draw conclusions that are applicable to a wider range of contexts.
• Limited Accessibility: Textual analysis requires specialized skills and training, which may limit its accessibility to researchers who are not trained in this method.

About the author

Muhammad Hassan

Researcher, Academic Writer, Web developer

You may also like

Cluster Analysis – Types, Methods and Examples

Questionnaire – Definition, Types, and Examples

Discriminant Analysis – Methods, Types and...

Case Study – Methods, Examples and Guide

MANOVA (Multivariate Analysis of Variance) –...

Observational Research – Methods and Guide

Textual Analysis: Definition, Types & 10 Examples

Textual analysis is a research methodology that involves exploring written text as empirical data. Scholars explore both the content and structure of texts, and attempt to discern key themes and statistics emergent from them.

This method of research is used in various academic disciplines, including cultural studies, literature, bilical studies, anthropology , sociology, and others (Dearing, 2022; McKee, 2003).

This method of analysis involves breaking down a text into its constituent parts for close reading and making inferences about its context, underlying themes, and the intentions of its author.

Textual Analysis Definition

Alan McKee is one of the preeminent scholars of textual analysis. He provides a clear and approachable definition in his book Textual Analysis: A Beginner’s Guide (2003) where he writes:

“When we perform textual analysis on a text we make an educated guess at some of the most likely interpretations that might be made of the text […] in order to try and obtain a sense of the ways in which, in particular cultures at particular times, people make sense of the world around them.”

A key insight worth extracting from this definition is that textual analysis can reveal what cultural groups value, how they create meaning, and how they interpret reality.

This is invaluable in situations where scholars are seeking to more deeply understand cultural groups and civilizations – both past and present (Metoyer et al., 2018).

As such, it may be beneficial for a range of different types of studies, such as:

• Studies of Historical Texts: A study of how certain concepts are framed, described, and approached in historical texts, such as the Bible.
• Studies of Industry Reports: A study of how industry reports frame and discuss concepts such as environmental and social responsibility.
• Studies of Literature: A study of how a particular text or group of texts within a genre define and frame concepts. For example, you could explore how great American literature mythologizes the concept of the ‘The American Dream’.
• Studies of Speeches: A study of how certain politicians position national identities in their appeals for votes.
• Studies of Newspapers: A study of the biases within newspapers toward or against certain groups of people.
• Etc. (For more, see: Dearing, 2022)

McKee uses the term ‘textual analysis’ to also refer to text types that are not just written, but multimodal. For a dive into the analysis of multimodal texts, I recommend my article on content analysis , where I explore the study of texts like television advertisements and movies in detail.

Features of a Textual Analysis

When conducting a textual analysis, you’ll need to consider a range of factors within the text that are worthy of close examination to infer meaning. Features worthy of considering include:

• Content: What is being said or conveyed in the text, including explicit and implicit meanings, themes, or ideas.
• Context: When and where the text was created, the culture and society it reflects, and the circumstances surrounding its creation and distribution.
• Audience: Who the text is intended for, how it’s received, and the effect it has on its audience.
• Authorship: Who created the text, their background and perspectives, and how these might influence the text.
• Form and structure: The layout, sequence, and organization of the text and how these elements contribute to its meanings (Metoyer et al., 2018).

Textual Analysis Coding Methods

The above features may be examined through quantitative or qualitative research designs , or a mixed-methods angle.

1. Quantitative Approaches

You could analyze several of the above features, namely, content, form, and structure, from a quantitative perspective using computational linguistics and natural language processing (NLP) analysis.

From this approach, you would use algorithms to extract useful information or insights about frequency of word and phrase usage, etc. This can include techniques like sentiment analysis, topic modeling, named entity recognition, and more.

2. Qualitative Approaches

In many ways, textual analysis lends itself best to qualitative analysis. When identifying words and phrases, you’re also going to want to look at the surrounding context and possibly cultural interpretations of what is going on (Mayring, 2015).

Generally, humans are far more perceptive at teasing out these contextual factors than machines (although, AI is giving us a run for our money).

One qualitative approach to textual analysis that I regularly use is inductive coding, a step-by-step methodology that can help you extract themes from texts. If you’re interested in using this step-by-step method, read my guide on inductive coding here .

See more Qualitative Research Approaches Here

Textual Analysis Examples

Title: “Discourses on Gender, Patriarchy and Resolution 1325: A Textual Analysis of UN Documents”  Author: Nadine Puechguirbal Year: 2010 APA Citation: Puechguirbal, N. (2010). Discourses on Gender, Patriarchy and Resolution 1325: A Textual Analysis of UN Documents, International Peacekeeping, 17 (2): 172-187. doi: 10.1080/13533311003625068

Summary: The article discusses the language used in UN documents related to peace operations and analyzes how it perpetuates stereotypical portrayals of women as vulnerable individuals. The author argues that this language removes women’s agency and keeps them in a subordinate position as victims, instead of recognizing them as active participants and agents of change in post-conflict environments. Despite the adoption of UN Security Council Resolution 1325, which aims to address the role of women in peace and security, the author suggests that the UN’s male-dominated power structure remains unchallenged, and gender mainstreaming is often presented as a non-political activity.

Title: “Racism and the Media: A Textual Analysis”  Author: Kassia E. Kulaszewicz Year: 2015 APA Citation: Kulaszewicz, K. E. (2015). Racism and the Media: A Textual Analysis . Dissertation. Retrieved from: https://sophia.stkate.edu/msw_papers/477

Summary: This study delves into the significant role media plays in fostering explicit racial bias. Using Bandura’s Learning Theory, it investigates how media content influences our beliefs through ‘observational learning’. Conducting a textual analysis, it finds differences in representation of black and white people, stereotyping of black people, and ostensibly micro-aggressions toward black people. The research highlights how media often criminalizes Black men, portraying them as violent, while justifying or supporting the actions of White officers, regardless of their potential criminality. The study concludes that news media likely continues to reinforce racism, whether consciously or unconsciously.

Title: “On the metaphorical nature of intellectual capital: a textual analysis” Author: Daniel Andriessen Year: 2006 APA Citation: Andriessen, D. (2006). On the metaphorical nature of intellectual capital: a textual analysis. Journal of Intellectual capital , 7 (1), 93-110.

Summary: This article delves into the metaphorical underpinnings of intellectual capital (IC) and knowledge management, examining how knowledge is conceptualized through metaphors. The researchers employed a textual analysis methodology, scrutinizing key texts in the field to identify prevalent metaphors. They found that over 95% of statements about knowledge are metaphor-based, with “knowledge as a resource” and “knowledge as capital” being the most dominant. This study demonstrates how textual analysis helps us to understand current understandings and ways of speaking about a topic.

Title: “Race in Rhetoric: A Textual Analysis of Barack Obama’s Campaign Discourse Regarding His Race” Author: Andrea Dawn Andrews Year: 2011 APA Citation: Andrew, A. D. (2011) Race in Rhetoric: A Textual Analysis of Barack Obama’s Campaign Discourse Regarding His Race. Undergraduate Honors Thesis Collection. 120 . https://digitalcommons.butler.edu/ugtheses/120

This undergraduate honors thesis is a textual analysis of Barack Obama’s speeches that explores how Obama frames the concept of race. The student’s capstone project found that Obama tended to frame racial inequality as something that could be overcome, and that this was a positive and uplifting project. Here, the student breaks-down times when Obama utilizes the concept of race in his speeches, and examines the surrounding content to see the connotations associated with race and race-relations embedded in the text. Here, we see a decidedly qualitative approach to textual analysis which can deliver contextualized and in-depth insights.

Sub-Types of Textual Analysis

While above I have focused on a generalized textual analysis approach, a range of sub-types and offshoots have emerged that focus on specific concepts, often within their own specific theoretical paradigms. Each are outlined below, and where I’ve got a guide, I’ve linked to it in blue:

• Content Analysis : Content analysis is similar to textual analysis, and I would consider it a type of textual analysis, where it’s got a broader understanding of the term ‘text’. In this type, a text is any type of ‘content’, and could be multimodal in nature, such as television advertisements, movies, posters, and so forth. Content analysis can be both qualitative and quantitative, depending on whether it focuses more on the meaning of the content or the frequency of certain words or concepts (Chung & Pennebaker, 2018).
• Discourse Analysis : Emergent specifically from critical and postmodern/ poststructural theories, discourse analysis focuses closely on the use of language within a social context, with the goal of revealing how repeated framing of terms and concepts has the effect of shaping how cultures understand social categories. It considers how texts interact with and shape social norms, power dynamics, ideologies, etc. For example, it might examine how gender is socially constructed as a distinct social category through Disney films. It may also be called ‘critical discourse analysis’.
• Narrative Analysis: This approach is used for analyzing stories and narratives within text. It looks at elements like plot, characters, themes, and the sequence of events to understand how narratives construct meaning.
• Frame Analysis: This approach looks at how events, ideas, and themes are presented or “framed” within a text. It explores how these frames can shape our understanding of the information being presented. While similar to discourse analysis, a frame analysis tends to be less associated with the loaded concept of ‘discourse’ that exists specifically within postmodern paradigms (Smith, 2017).
• Semiotic Analysis: This approach studies signs and symbols, both visual and textual, and could be a good compliment to a content analysis, as it provides the language and understandings necessary to describe how signs make meaning in cultural contexts that we might find with the fields of semantics and pragmatics . It’s based on the theory of semiotics, which is concerned with how meaning is created and communicated through signs and symbols.
• Computational Textual Analysis: In the context of data science or artificial intelligence, this type of analysis involves using algorithms to process large amounts of text. Techniques can include topic modeling, sentiment analysis, word frequency analysis, and others. While being extremely useful for a quantitative analysis of a large dataset of text, it falls short in its ability to provide deep contextualized understandings of words-in-context.

Each of these methods has its strengths and weaknesses, and the choice of method depends on the research question, the type of text being analyzed, and the broader context of the research.

See More Examples of Analysis Here

Strengths and Weaknesses of Textual Analysis

When writing your methodology for your textual analysis, make sure to define not only what textual analysis is, but (if applicable) the type of textual analysis, the features of the text you’re analyzing, and the ways you will code the data. It’s also worth actively reflecting on the potential weaknesses of a textual analysis approach, but also explaining why, despite those weaknesses, you believe this to be the most appropriate methodology for your study.

Chung, C. K., & Pennebaker, J. W. (2018). Textual analysis. In  Measurement in social psychology  (pp. 153-173). Routledge.

Dearing, V. A. (2022).  Manual of textual analysis . Univ of California Press.

McKee, A. (2003). Textual analysis: A beginner’s guide.  Textual analysis , 1-160.

Mayring, P. (2015). Qualitative content analysis: Theoretical background and procedures.  Approaches to qualitative research in mathematics education: Examples of methodology and methods , 365-380. doi: https://doi.org/10.1007/978-94-017-9181-6_13

Metoyer, R., Zhi, Q., Janczuk, B., & Scheirer, W. (2018, March). Coupling story to visualization: Using textual analysis as a bridge between data and interpretation. In  23rd International Conference on Intelligent User Interfaces  (pp. 503-507). doi: https://doi.org/10.1145/3172944.3173007

Smith, J. A. (2017). Textual analysis.  The international encyclopedia of communication research methods , 1-7.

Chris Drew (PhD)

Dr. Chris Drew is the founder of the Helpful Professor. He holds a PhD in education and has published over 20 articles in scholarly journals. He is the former editor of the Journal of Learning Development in Higher Education. [Image Descriptor: Photo of Chris]

• Chris Drew (PhD) https://helpfulprofessor.com/author/chris-drew-phd/ 5 Top Tips for Succeeding at University
• Chris Drew (PhD) https://helpfulprofessor.com/author/chris-drew-phd/ 50 Durable Goods Examples
• Chris Drew (PhD) https://helpfulprofessor.com/author/chris-drew-phd/ 100 Consumer Goods Examples
• Chris Drew (PhD) https://helpfulprofessor.com/author/chris-drew-phd/ 30 Globalization Pros and Cons

Leave a Comment Cancel Reply

Your email address will not be published. Required fields are marked *

The Practical Guide to Textual Analysis

• Getting Started
• How Does It Work?
• Use Cases & Applications

Textual analysis is the process of gathering and examining qualitative data to understand what it’s about.

But making sense of qualitative information is a major challenge. Whether analyzing data in business or performing academic research, manually reading, analyzing, and tagging text is no longer effective – it’s time-consuming, results are often inaccurate, and the process far from scalable.

Fortunately, developments in the sub-fields of Artificial Intelligence (AI) like machine learning and natural language processing (NLP) are creating unprecedented opportunities to process and analyze large collections of text data.

Thanks to algorithms trained with machine learning it is possible to perform a myriad of tasks that involve analyzing text, like topic classification (automatically tagging texts by topic), feature extraction (identifying specific characteristics in a text) and sentiment analysis (recognizing the emotions that underlie a given text).

Below, we’ll dive into textual analysis with machine learning, what it is and how it works, and reveal its most important applications in business and academic research:

Getting started with textual analysis

• What is textual analysis?
• Difference between textual analysis and content analysis?
• What is computer-assisted textual analysis?
• Methods and techniques
• Why is it important?

How does textual analysis work?

• Text classification
• Text extraction

Use cases and applications

• Customer service
• Customer feedback
• Academic research

Let’s start with the basics!

Getting Started With Textual Analysis

What is textual analysis.

While similar to text analysis , textual analysis is mainly used in academic research to analyze content related to media and communication studies, popular culture, sociology, and philosophy.

In this case, the purpose of textual analysis is to understand the cultural and ideological aspects that underlie a text and how they are connected with the particular context in which the text has been produced. In short, textual analysis consists of describing the characteristics of a text and making interpretations to answer specific questions.

One of the challenges of textual analysis resides in how to turn complex, large-scale data into manageable information. Computer-assisted textual analysis can be instrumental at this point, as it allows you to perform certain tasks automatically (without having to read all the data) and makes it simple to observe patterns and get unexpected insights. For example, you could perform automated textual analysis on a large set of data and easily tag all the information according to a series of previously defined categories. You could also use it to extract specific pieces of data, like names, countries, emails, or any other features.

Companies are using computer-assisted textual analysis to make sense of unstructured business data , and find relevant insights that lead to data-driven decisions. It’s being used to automate everyday tasks like ticket tagging and routing, improving productivity, and saving valuable time.

Difference Between Textual Analysis and Content Analysis?

When we talk about textual analysis we refer to a data-gathering process for analyzing text data. This qualitative methodology examines the structure, content, and meaning of a text, and how it relates to the historical and cultural context in which it was produced. To do so, textual analysis combines knowledge from different disciplines, like linguistics and semiotics.

Content analysis can be considered a subcategory of textual analysis, which intends to systematically analyze text, by coding the elements of the text to get quantitative insights. By coding text (that is, establishing different categories for the analysis), content analysis makes it possible to examine large sets of data and make replicable and valid inferences.

Sitting at the intersection between qualitative and quantitative approaches, content analysis has proved to be very useful to study a wide array of text data ― from newspaper articles to social media messages ― within many different fields, that range from academic research to organizational or business studies.

What is Computer-Assisted Textual Analysis?

Computer-assisted textual analysis involves using a software, digital platform, or computational tools to perform tasks related to text analysis automatically.

The developments in machine learning make it possible to create algorithms that can be trained with examples and learn a series of tasks, from identifying topics on a given text to extracting relevant information from an extensive collection of data. Natural Language Processing (NLP), another sub-field of AI, helps machines process unstructured data and transform it into manageable information that’s ready to analyze.

Automated textual analysis enables you to analyze large amounts of data that would require a significant amount of time and resources if done manually. Not only is automated textual analysis fast and straightforward, but it’s also scalable and provides consistent results.

Let’s look at an example. During the US elections 2016, we used MonkeyLearn to analyze millions of tweets referring to Donald Trump and Hillary Clinton . A text classification model allowed us to tag each Tweet into the two predefined categories: Trump and Hillary. The results showed that, on an average day, Donald Trump was getting around 450,000 Twitter mentions while Hillary Clinton was only getting about 250,000. And that was just the tip of the iceberg! What was really interesting was the nuances of those mentions: were they favorable or unfavorable? By performing sentiment analysis , we were able to discover the feelings behind those messages and gain some interesting insights about the polarity of those opinions.

For example, this is how Trump’s Tweets looked like when counted by sentiment:

And this graphic shows the same for Hillary Clinton:

There are many methods and techniques for automated textual analysis. In the following section, we’ll take a closer look at each of them so that you have a better idea of what you can do with computer-assisted textual analysis.

Textual Analysis Methods & Techniques

• Word frequency

Collocation

Concordance, basic methods, word frequency.

Word frequency helps you find the most recurrent terms or expressions within a set of data. Counting the times a word is mentioned in a group of texts can lead you to interesting insights, for example, when analyzing customer feedback responses. If the terms ‘hard to use’ or ‘complex’ often appear in comments about your product, it may indicate you need to make UI/UX adjustments.

By ‘collocation’ we mean a sequence of words that frequently occur together. Collocations are usually bigrams (a pair of words) and trigrams (a combination of three words). ‘Average salary’ , ‘global market’ , ‘close a deal’ , ‘make an appointment’ , ‘attend a meeting’ are examples of collocations related to business.

In textual analysis, identifying collocations is useful to understand the semantic structure of a text. Counting bigrams and trigrams as one word improves the accuracy of the analysis.

Human language is ambiguous: depending on the context, the same word can mean different things. Concordance is used to identify instances in which a word or a series of words appear, to understand its exact meaning. For example, here are a few sentences from product reviews containing the word ‘time’:

Advanced Methods

Text classification.

Text classification is the process of assigning tags or categories to unstructured data based on its content.

When we talk about unstructured data we refer to all sorts of text-based information that is unorganized, and therefore complex to sort and manage. For businesses, unstructured data may include emails, social media posts, chats, online reviews, support tickets, among many others. Text classification ― one of the essential tasks of Natural Language Processing (NLP) ― makes it possible to analyze text in a simple and cost-efficient way, organizing the data according to topic, urgency, sentiment or intent. We’ll take a closer look at each of these applications below:

Topic Analysis consists of assigning predefined tags to an extensive collection of text data, based on its topics or themes. Let’s say you want to analyze a series of product reviews to understand what aspects of your product are being discussed, and a review reads ‘the customer service is very responsive, they are always ready to help’ . This piece of feedback will be tagged under the topic ‘Customer Service’ .

Sentiment Analysis , also known as ‘opinion mining’, is the automated process of understanding the attributes of an opinion, that is, the emotions that underlie a text (e.g. positive, negative, and neutral). Sentiment analysis provides exciting opportunities in all kinds of fields. In business, you can use it to analyze customer feedback, social media posts, emails, support tickets, and chats. For instance, you could analyze support tickets to identify angry customers and solve their issues as a priority. You may also combine topic analysis with sentiment analysis (it is called aspect-based sentiment analysis ) to identify the topics being discussed about your product, and also, how people are reacting towards those topics. For example, take the product review we mentioned earlier for topic analysis: ‘the customer service is very responsive, they are always ready to help’ . This statement would be classified as both Positive and Customer Service .

Language detection : this allows you to classify a text based on its language. It’s particularly useful for routing purposes. For example, if you get a support ticket in Spanish, it could be automatically routed to a Spanish-speaking customer support team.

Intent detection : text classifiers can also be used to recognize the intent of a given text. What is the purpose behind a specific message? This can be helpful if you need to analyze customer support conversations or the results of a sales email campaign. For example, you could analyze email responses and classify your prospects based on their level of interest in your product.

Text Extraction

Text extraction is a textual analysis technique which consists of extracting specific terms or expressions from a collection of text data. Unlike text classification, the result is not a predefined tag but a piece of information that is already present in the text. For example, if you have a large collection of emails to analyze, you could easily pull out specific information such as email addresses, company names or any keyword that you need to retrieve. In some cases, you can combine text classification and text extraction in the same analysis.

The most useful text extraction tasks include:

Named-entity recognition : used to extract the names of companies , people , or organizations from a set of data.

Keyword extraction : allows you to extract the most relevant terms within a text. You can use keyword extraction to index data to be searched, create tags clouds, summarize the content of a text, among many other things.

Feature extraction : used to identify specific characteristics within a text. For example, if you are analyzing a series of product descriptions, you could create customized extractors to retrieve information like brand, model, color, etc .

Why is Textual Analysis Important?

Every day, we create a colossal amount of digital data. In fact, in the last two years alone we generated 90% percent of all the data in the world . That includes social media messages, emails, Google searches, and every other source of online data.

At the same time, books, media libraries, reports, and other types of databases are now available in digital format, providing researchers of all disciplines opportunities that didn’t exist before.

But the problem is that most of this data is unstructured. Since it doesn’t follow any organizational criteria, unstructured text is hard to search, manage, and examine. In this scenario, automated textual analysis tools are essential, as they help make sense of text data and find meaningful insights in a sea of information.

Text analysis enables businesses to go through massive collections of data with minimum human effort, saving precious time and resources, and allowing people to focus on areas where they can add more value. Here are some of the advantages of automated textual analysis:

Scalability

You can analyze as much data as you need in just seconds. Not only will you save valuable time, but you’ll also make your teams much more productive.

Real-time analysis

For businesses, it is key to detect angry customers on time or be warned of a potential PR crisis. By creating customized machine learning models for text analysis, you can easily monitor chats, reviews, social media channels, support tickets and all sorts of crucial data sources in real time, so you’re ready to take action when needed.

Academic researchers, especially in the political science field , may find real-time analysis with machine learning particularly useful to analyze polls, Twitter data, and election results.

Consistent criteria

Routine manual tasks (like tagging incoming tickets or processing customer feedback, for example) often end up being tedious and time-consuming. There are more chances of making mistakes and the criteria applied within team members often turns out to be inconsistent and subjective. Machine learning algorithms, on the other hand, learn from previous examples and always use the same criteria to analyze data .

How does Textual Analysis Work?

Computer-assisted textual analysis makes it easy to analyze large collections of text data and find meaningful information. Thanks to machine learning, it is possible to create models that learn from examples and can be trained to classify or extract relevant data.

But how easy is to get started with textual analysis?

As with most things related to artificial intelligence (AI), automated text analysis is perceived as a complex tool, only accessible to those with programming skills. Fortunately, that’s no longer the case. AI platforms like MonkeyLearn are actually very simple to use and don’t require any previous machine learning expertise. First-time users can try different pre-trained text analysis models right away, and use them for specific purposes even if they don’t have coding skills or have never studied machine learning.

However, if you want to take full advantage of textual analysis and create your own customized models, you should understand how it works.

There are two steps you need to follow before running an automated analysis: data gathering and data preparation. Here, we’ll explain them more in detail:

Data gathering : when we think of a topic we want to analyze, we should first make sure that we can obtain the data we need. Let’s say you want to analyze all the customer support tickets your company has received over a designated period of time. You should be able to export that information from your software and create a CSV or an Excel file. The data can be either internal (that is, data that’s only available to your business, like emails, support tickets, chats, spreadsheets, surveys, databases, etc) or external (like review sites, social media, news outlets or other websites).

Data preparation : before performing automated text analysis it’s necessary to prepare the data that you are going to use. This is done by applying a series of Natural Language Processing (NLP) techniques. Tokenization , parsing , lemmatization , stemming and stopword removal are just a few of them.

Once these steps are complete, you will be all set up for the data analysis itself. In this section, we’ll refer to how the most common textual analysis methods work: text classification and text extraction.

Text classification is the process of assigning tags to a collection of data based on its content.

When done manually, text categorization is a time-consuming task that often leads to mistakes and inaccuracies. By doing this automatically, it is possible to obtain very good results while spending less time and resources. Automatic text classification consists of three main approaches: rule-based, machine learning and hybrid.

Rule-based systems

Rule-based systems follow an ‘if-then’ (condition-action) structure based on linguistic rules. Basically, rules are human-made associations between a linguistic pattern on a text and a predefined tag. These linguistic patterns often refer to morphological, syntactic, lexical, semantic, or phonological aspects.

For instance, this could be a rule to classify a series of laptop descriptions:

( Lenovo | Sony | Hewlett Packard | Apple ) → Brand

In this case, when the text classification model detects any of those words within a text (the ‘if’ portion), it will assign the predefined tag ‘brand’ to them (the ‘then’ portion).

One of the main advantages of rule-based systems is that they are easy to understand by humans. On the downside, creating complex systems is quite tricky, because you need to have good knowledge of linguistics and of the topics present in the text that you want to analyze. Besides, adding new rules can be tough as it requires several tests, making rule-based systems hard to scale.

Machine learning-based systems

Machine learning-based systems are trained to make predictions based on examples. This means that a person needs to provide representative and consistent samples and assign the expected tags manually so that the system learns to make its own predictions from those past observations. The collection of manually tagged data is called training data .

But how does machine learning actually work?

Suppose you are training a machine learning-based classifier. The system needs to transform the training data into something it can understand: in this case, vectors (an array of numbers with encoded data). Vectors contain a set of relevant features from the given text, and use them to learn and make predictions on future data.

One of the most common methods for text vectorization is called bag of words and consists of counting how many times a particular word (from a predetermined list of words) appears in the text you want to analyze.

So, the text is transformed into vectors and fed into a machine learning algorithm along with its expected tags, creating a text classification model:

After being trained, the model can make predictions over unseen data:

Machine learning algorithms

The most common algorithms used in text classification are Naive Bayes family of algorithms (NB), Support Vector Machines (SVM), and deep learning algorithms .

Naive Bayes family of algorithms (NB) is a probabilistic algorithm that uses Bayes’ theorem to calculate the probability of each tag for a given text. It then provides the tag with the highest likelihood of occurrence. This algorithm provides good results as long as the training data is scarce.

Support Vector Machines (SVM) is a machine learning algorithm that divides vectors into two different groups within a three-dimensional space. In one group, you have vectors that belong to a given tag, and in the other group vectors that don’t belong to that tag. Using this algorithm requires more coding skills, but the results are better than the ones with Naive Bayes.

Deep learning algorithms try to emulate the way the human brain thinks. They use millions of training examples and generate very rich representations of texts, leading to much more accurate predictions than other machine learning algorithms. The downside is that they need vast amounts of training data to provide accurate results and require intensive coding.

Hybrid systems

These systems combine rule-based systems and machine learning-based systems to obtain more accurate predictions.

There are different parameters to evaluate the performance of a text classifier: accuracy , precision , recall , and F1 score .

You can measure how your text classifier works by comparing it to a fixed testing set (that is, a group of data that already includes its expected tags) or by using cross-validation, a process that divides your training data into two groups – one used to train the model, and the other used to test the results.

Let’s go into more detail about each of these parameters:

Accuracy : this is the number of correct predictions that the text classifier makes divided by the total number of predictions. However, accuracy alone is not the best parameter to analyze the performance of a text classifier. When the number of examples is imbalanced (for example, a lot of the data belongs to one of the categories) you may experience an accuracy paradox , that is, a model with high accuracy, but one that’s not necessarily able to make accurate predictions for all tags. In this case, it’s better to look at precision and recall, and F1 score.

Precision : this metric indicates the number of correct predictions for a given tag, divided by the total number of correct and incorrect predictions for that tag. In this case, a high precision level indicates there were less false positives. For some tasks ― like sending automated email responses ― you will need text classification models with a high level of precision, that will only deliver an answer when it’s highly likely that the recipient belongs to a given tag.

Recall : it shows the number of correct predictions for a given tag, over the number of predictions that should have been predicted as belonging to that tag. High recall metrics indicate there were less false negatives and, if routing support tickets for example, it means that tickets will be sent to the right teams.

F1 score : this metric considers both precision and recall results, and provides an idea of how well your text classifier is working. It allows you to see how accurate is your model for all the tags you’re using.

Cross-validation

Cross-validation is a method used to measure the accuracy of a text classifier model. It consists of splitting the training dataset into a number of equal-length subsets, in a random way. For instance, let’s imagine you have four subsets and each of them contains 25% of your training data.

All of those subsets except one are used to train the text classifier. Then, the classifier is used to make predictions over the remaining subset. After this, you need to compile all the metrics we mentioned before (accuracy, precision, recall, and F1 score), and start the process all over again, until all the subsets have been used for testing. Finally, all the results are compiled to obtain the average performance of each metric.

Text extraction is the process of identifying specific pieces of text from unstructured data. This is very useful for a variety of purposes, from extracting company names from a Linkedin dataset to pulling out prices on product descriptions.

Text extraction allows to automatically visualize where the relevant terms or expressions are, without needing to read or scan all the text by yourself. And that is particularly relevant when you have massive databases, which would otherwise take ages to analyze manually.

There are different approaches to text extraction. Here, we’ll refer to the most commonly used and reliable:

Regular expressions

Regular expressions are similar to rules for text classification models. They can be defined as a series of characters that define a pattern.

Every time the text extractor detects a coincidence with a pattern, it assigns the corresponding tag.

This approach allows you to create text extractors quickly and with good results, as long as you find the right patterns for the data you want to analyze. However, as it gets more complex, it can be hard to manage and scale.

Conditional Random Fields

Conditional Random Fields (CRF) is a statistical approach used for text extraction with machine learning. It identifies different patterns by assigning a weight to each of the word sequences within a text. CRF’s also allow you to create additional parameters related to the patterns, based on syntactic or semantic information.

This approach creates more complex and richer patterns than regular expressions and can encode a large volume of information. However, if you want to train the text extractor properly, you will need to have in-depth NLP and computing knowledge.

You can use the same performance metrics that we mentioned for text classification (accuracy, precision, recall, and F1 score), although these metrics only consider exact matches as positive results, leaving partial matches aside.

If you want partial matches to be included in the results, you should use a performance metric called ROUGE (Recall-Oriented Understudy for Gisting Evaluation). This group of metrics measures lengths and numbers of sequences to make a match between the source text and the extraction performed by the model.

The parameters used to compare these two texts need to be defined manually. You may define ROUGE-n metrics (n is the length of the units you want to measure) or ROUGE-L metrics (to compare the longest common sentence).

Use Cases and Applications

Automated textual analysis is the process of obtaining meaningful information out of raw data. Considering unstructured data is getting closer to 80% of the existing information in the digital world , it’s easy to understand why this brings outstanding opportunities for businesses, organizations, and academic researchers.

For companies, it is now possible to obtain real-time insights on how their users feel about their products and make better business decisions based on data. Shifting to a data-driven approach is one of the main challenges of businesses today.

Textual analysis has many exciting applications across different areas of a company, like customer service, marketing, product, or sales. By allowing the automation of specific tasks that used to be manual, textual analysis is helping teams become more productive and efficient, and allowing them to focus on areas where they can add real value.

In the academic research field, computer-assisted textual analysis (and mainly, machine learning-based models) are expanding the horizons of investigation, by providing new ways of processing, classifying, and obtaining relevant data.

In this section, we’ll describe the most significant applications related to customer service, customer feedback, and academic research.

Customer Service

It’s not all about having an amazing product or investing a lot of money on advertising. What really tips the balance when it comes to business success is to provide high-quality customer service. Stats claim that 70% of the customer journey is defined by how people feel they are being treated .

So, how can textual analysis help companies deliver a better customer service experience?

Automatically tag support tickets

Every time a customer sends a request, comment, or complain, there’s a new support ticket to be processed. Customer support teams need to categorize every incoming message based on its content, a routine task that can be boring, time-consuming, and inconsistent if done manually.

Textual analysis with machine learning allows you automatically identify the topic of each support ticket and tag it accordingly. How does it work?

• First, a person defines a set of categories and trains a classifier model by applying the appropriate tags to a number of representative samples.
• The model analyzes the words and expressions used in each ticket. For example: ‘I’m having problems when paying with my credit card’ , and it compares it with previous examples.
• Finally, it automatically tags the ticket according to its content. In this case, the ticket would be tagged as Payment Issues .

Automatically route and triage support tickets

Once support tickets are tagged, they need to be routed to the appropriate team in charge to deal with that issue. Machine learning enables teams to send a ticket to the right person in real-time , based on the ticket’s topic, language or complexity. For example, a ticket previously tagged as Payment Issues will be automatically routed to the Billing Area .

Detect the urgency of a ticket

A simple task, like being able to prioritize tickets based on their urgency, can have a substantial positive impact on your customer service. By analyzing the content of each ticket, a textual analysis model can let you assess which of them are more critical and prioritize accordingly . For instance, a ticket containing the words or expressions ‘as soon as possible’ or ‘immediately’ would be automatically classified as Urgent .

Get insights from ticket analytics

The performance of customer service teams is usually measured by KPI’s, like first response time, the average time of resolution, and customer satisfaction (CSAT).

Textual analysis algorithms can be used to analyze the different interactions between customers and the customer service area, like chats, support tickets, emails, and customer satisfaction surveys.

You can use aspect-based sentiment analysis to understand the main topics discussed by your customers and how they are feeling about those topics. For example, you may have a lot of mentions referring to the topic ‘UI/UX’ . But, are all those customers’ opinions positive, negative, or neutral? This type of analysis can provide a more accurate perspective of what they think about your product and get a deeper understanding the overall customer satisfaction.

Customer Feedback

Listening to the Voice of Customer (VoC) is critical to understand the customers’ expectations, experience and opinion about your brand. Two of the most common tools to monitor and examine customer feedback are customer surveys and product reviews.

By analyzing customer feedback data, companies can detect topics for improvement, spot product flaws, get a better understanding of your customer’s needs and measure their level of satisfaction, among many other things.

But how do you process and analyze tons of reviews or thousands of customer surveys? Here are some ideas of how you can use textual analysis algorithms to analyze different kinds of customer feedback:

Analyze NPS Responses

Net Promoter Score (NPS) is the most popular tool to measure customer satisfaction. The first part of the survey involves giving the brand a score from 0 to 10 based on the question: 'How likely is it that you would recommend [brand] to a friend or colleague?' . The results allow you to classify your customers as promoters , passives , and detractors .

Then, there’s a follow-up question, inquiring about the reasons for your previous score. These open-ended responses often provide the most insightful information about your company. At the same time, it’s the most complex data to process. Yes, you could read and tag each of the responses manually, but what if there are thousands of them?

Textual analysis with machine learning enables you to detect the main topics that your customers are referring to, and even extract the most relevant keywords related to those topics . To make the most of your data, you could also perform sentiment analysis and find out if your customers are talking about a given topic positively or negatively.

Analyze Customer Surveys

Besides NPS, textual analysis algorithms can help you analyze all sorts of customer surveys. Using a text classification model to tag your responses can make you save a lot of valuable time and resources while allowing you to obtain consistent results.

Analyze Product Reviews

Product reviews are a significant factor when buying a product. Prospective buyers read at least 10 reviews before feeling they can trust a local business and that’s just one of the (many) reasons why you should keep a close eye on what people are saying about your brand online.

Analyzing product reviews can give you an idea of what people love and hate the most about your product and service. It can provide useful insights and opportunities for improvement. And it can show you what to do to get one step ahead of your competition.

The truth is that going through pages and pages of product reviews is not a very exciting task. Categorizing all those opinions can take teams hours and in the end, it becomes an expensive and unproductive process. That’s why automated textual analysis is a game-changer.

Imagine you want to analyze a set of product reviews from your SaaS company in G2 Crowd. A textual analysis model will allow you to tag g each review based on topic, like Ease of Use , Price , UI/UX , Integrations . You could also run a sentiment analysis to discover how your customers feel about those topics: do they think the price is suitable or too expensive? Do they find it too complex or easy to use?

Thanks to textual analysis algorithms, you can get powerful information to help you make data-driven decisions, and empower your teams to be more productive by reducing manual tasks to a minimum.

Academic Research

What if you were able to sift through tons of papers and journals, and discover data that is relevant to your research in just seconds? Just imagine if you could easily classify years of news articles and extract meaningful keywords from them, or analyze thousands of tweets after a significant political change .

Even though machine learning applications in business and science seem to be more frequent, social science research is also benefiting from ML to perform tasks related to the academic world.

Social science researchers need to deal with vast volumes of unstructured data. Therefore, one of the major opportunities provided by computer-assisted textual analysis is being able to classify data, extract relevant information, or identify different groups in extensive collections of data.

Another application of textual analysis with machine learning is supporting the coding process . Coding is one of the early steps of any qualitative textual analysis. It involves a detailed examination of what you want to analyze to become familiar with the data. When done manually, this task can be very time consuming and often inaccurate or inconsistent. Fortunately, machine learning algorithms (like text classifier models) can help you do this in very little time and allow you to scale up the coding process easily.

Finally, using machine learning algorithms to scan large amounts of papers, databases, and journal articles can lead to new investigation hypotheses .

Final Words

In a world overloaded with data, textual analysis with machine learning is a powerful tool that enables you to make sense of unstructured information and find what’s relevant in just seconds.

With promising use cases across many fields from marketing to social science research, machine learning algorithms are far from being a niche technology only available for a few. Moreover, they are turning into user-friendly applications that are dominated by workers with little or no coding skills.

Thanks to text analysis models, teams are becoming more productive by being released from manual and routine tasks that used to take valuable time from them. At the same time, companies can make better decisions based on valuable, real-time insights obtained from data.

By now, you probably have an idea of what textual analysis is with machine learning and how you can use it to make your everyday tasks more efficient and straightforward. Ready to get started? MonkeyLearn makes it very simple to take your first steps. Just contact us and get a personalized demo from one of our experts!

MonkeyLearn Inc. All rights reserved 2024

Analyzing Text Data

An introduction to text analysis and text mining, an overview of text analysis methods, additional resources.

• Text Analysis Methods
• Library Databases
• Social Media
• Open Source
• Language Corpora
• Web Scraping
• Software for Text Analysis
• Text Data Citation

Library Data Services

What is text analysis.

Text analysis is a broad term that encompasses the examination and interpretation of textual data. It involves various techniques to understand, organize, and derive insights from text, including methods from linguistics, statistics, and machine learning. Text analysis often includes processes like text categorization, sentiment analysis, and entity recognition, to gain valuable insights from textual data.

What is text mining?

Text mining , also known as text data mining, is a process of using computer programs and algorithms to dig through large amounts of text, like books, articles, websites, or social media posts, to find valuable and hidden information. This information could be patterns, trends, insights, or specific pieces of knowledge that are not immediately obvious when you read the texts on your own. Text data mining helps people make sense of vast amounts of text data quickly and efficiently, making it easier to discover useful information and gain new perspectives from written content.

This video is an introduction to text mining and how it can be used in research.

There are many different methods for text analysis, such as:

• word frequency analysis
• natural language processing
• sentiment analysis

These text analysis techniques serve various purposes, from organizing and understanding text data to making predictions, extracting knowledge, and automating tasks.

Before beginning your text analysis project, it is important to specify your goals and then choose the method that will allow you to meet those goals. Then, consider how much data you need, and identify a sampling plan , before beginning data collection.

• Examples of Text and Data Mining Research Using Copyrighted Materials By Sean Flynn and Lokesh Vyas, an exploration of text and data mining across disciplines, from medicine to literature. Published December 5, 2022.
• Next: Text Analysis Methods >>
• Last Updated: Feb 7, 2024 10:26 AM
• URL: https://libguides.gwu.edu/textanalysis

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Published: 11 April 2024

Quantitative text analysis

• Kristoffer L. Nielbo   ORCID: orcid.org/0000-0002-5116-5070 1 ,
• Folgert Karsdorp 2 ,
• Melvin Wevers   ORCID: orcid.org/0000-0001-8177-4582 3 ,
• Alie Lassche   ORCID: orcid.org/0000-0002-7607-0174 4 ,
• Rebekah B. Baglini   ORCID: orcid.org/0000-0002-2836-5867 5 ,
• Mike Kestemont 6 &
• Nina Tahmasebi   ORCID: orcid.org/0000-0003-1688-1845 7  

Nature Reviews Methods Primers volume  4 , Article number:  25 ( 2024 ) Cite this article

3497 Accesses

53 Altmetric

Metrics details

• Computational science
• Interdisciplinary studies

Text analysis has undergone substantial evolution since its inception, moving from manual qualitative assessments to sophisticated quantitative and computational methods. Beginning in the late twentieth century, a surge in the utilization of computational techniques reshaped the landscape of text analysis, catalysed by advances in computational power and database technologies. Researchers in various fields, from history to medicine, are now using quantitative methodologies, particularly machine learning, to extract insights from massive textual data sets. This transformation can be described in three discernible methodological stages: feature-based models, representation learning models and generative models. Although sequential, these stages are complementary, each addressing analytical challenges in the text analysis. The progression from feature-based models that require manual feature engineering to contemporary generative models, such as GPT-4 and Llama2, signifies a change in the workflow, scale and computational infrastructure of the quantitative text analysis. This Primer presents a detailed introduction of some of these developments, offering insights into the methods, principles and applications pertinent to researchers embarking on the quantitative text analysis, especially within the field of machine learning.

You have full access to this article via your institution.

Similar content being viewed by others

Augmenting interpretable models with large language models during training

An open source machine learning framework for efficient and transparent systematic reviews

The shaky foundations of large language models and foundation models for electronic health records

Introduction.

Qualitative analysis of textual data has a long research history. However, a fundamental shift occurred in the late twentieth century when researchers began investigating the potential of computational methods for text analysis and interpretation 1 . Today, researchers in diverse fields, such as history, medicine and chemistry, commonly use the quantification of large textual data sets to uncover patterns and trends, producing insights and knowledge that can aid in decision-making and offer novel ways of viewing historical events and current realities. Quantitative text analysis (QTA) encompasses a range of computational methods that convert textual data or natural language into structured formats before subjecting them to statistical, mathematical and numerical analysis. With the increasing availability of digital text from numerous sources, such as books, scientific articles, social media posts and online forums, these methods are becoming increasingly valuable, facilitated by advances in computational technology.

Given the widespread application of QTA across disciplines, it is essential to understand the evolution of the field. As a relatively consolidated field, QTA embodies numerous methods for extracting and structuring information in textual data. It gained momentum in the late 1990s as a subset of the broader domain of data mining, catalysed by advances in database technologies, software accessibility and computational capabilities 2 , 3 . However, it is essential to recognize that the evolution of QTA extends beyond computer science and statistics. It has heavily incorporated techniques and algorithms derived from  corpus linguistics 4 , computer linguistics 5 and information retrieval 6 . Today, QTA is largely driven by  machine learning , a crucial component of  data science , artificial intelligence (AI) and natural language processing (NLP).

Methods of QTA are often referred to as techniques that are innately linked with specific tasks (Table  1 ). For example, the sentiment analysis aims to determine the emotional tone of a text 7 , whereas entity and concept extraction seek to identify and categorize elements in a text, such as names, locations or key themes 8 , 9 . Text classification refers to the task of sorting texts into groups with predefined labels 10 — for example, sorting news articles into semantic categories such as politics, sports or entertainment. In contrast to machine-learning tasks that use supervised learning , text clustering, which uses  unsupervised learning , involves finding naturally occurring groups in unlabelled texts 11 . A significant subset of tasks primarily aim to simplify and structure natural language. For example, representation learning includes tasks that automatically convert texts into numerical representations, which can then be used for other tasks 12 . The lines separating these techniques can be blurred and often vary depending on the research context. For example, topic modelling, a type of statistical modelling used for concept extraction, serves simultaneously as a clustering and representation learning technique 13 , 14 , 15 .

QTA, similar to machine learning, learns from observation of existing data rather than by manipulating variables as in scientific experiments 16 . In QTA, experiments encompass the design and implementation of empirical tests to explore and evaluate the performance of models, algorithms and techniques in relation to specific tasks and applications. In practice, this involves a series of steps. First, text data are collected from real-world sources such as newspaper articles, patient records or social media posts. Then, a specific type of machine-learning model is selected and designed. The model could be a tree-based decision model, a clustering technique or more complex encoder–decoder models for tasks such as translation. Subsequently, the selected model is trained on the collected data, learning to make categorizations or predictions based on the data. The performance of the model is evaluated using predominantly intrinsic performance metrics (such as accuracy for a classification task) and, to a lesser degree, extrinsic metrics that measure how the output of the model impacts a broader task or system.

Three distinct methodological stages can be observed in the evolution of QTA: feature-based models, representation learning models and generative models (Fig.  1 ). Feature-based models use efficient machine-learning techniques, collectively referred to as shallow learning, which are ideal for tabular data but require manual feature engineering. They include models based on  bag-of-words models , decision trees and support vector machines and were some of the first methods applied in QTA. Representation learning models use deep learning techniques that automatically learn useful features from text. These models include architectures such as the highly influential  transformer architecture 17 and techniques such as masked language modelling, as used in language representation models such as Bidirectional Encoder Representations from Transformers (BERT) 18 . BERT makes use of the transformer architecture, as do most other large language models after the introduction of the architecture 17 . This shift towards automatic learning representations marked an important advance in natural language understanding. Generative models, trained using autoregressive techniques, represent the latest frontier. These models, such as generative pre-trained transformer GPT-3 (ref. 19 ), GPT-4 and Llama2 (ref. 20 ), can generate coherent and contextually appropriate responses and are powerful tools for natural language generation. Feature-based models preceded representation learning, which in turn preceded generative models.

a , Feature-based models in which data undergo preprocessing to generate features for model training and prediction. b , Representation learning models that can be trained from scratch using raw data or leverage pre-trained models fine-tuned with specific data. c , Generative models in which a prompt guides the generative deep learning model, potentially augmented by external data, to produce a result.

Although these models are temporally ordered, they do not replace each other. Instead, each offers unique methodological features and is suitable for different tasks. The progress from small models with limited computing capacity to today’s large models with billions of parameters encapsulates the transformation in the scale and complexity of the QTA.

The evolution of these models reflects the advancement of machine-learning infrastructure, particularly in the emergence and development of tooling frameworks. These frameworks, exemplified by platforms such as scikit-learn 21 and Hugging Face 22 , have served as essential infrastructure for democratizing and simplifying the implementation of increasingly sophisticated models. They offer user-friendly interfaces that mask the complexities of the algorithms, thereby empowering researchers to harness advanced methodologies with minimal prerequisite knowledge and coding expertise. The advent of high-level generative models such as GPT-3 (ref. 19 ), GPT-4 and Llama2 (ref. 20 ) marks milestones in the progression. Renowned for their unprecedented language understanding and generation capabilities, these models have the potential to redefine access to the sophisticated text analysis by operating on natural language prompts, effectively bypassing the traditional need for coding. It is important to emphasize that these stages represent an abstraction that points to fundamental changes to the workflow and underlying infrastructure of QTA.

This Primer offers an accessible introduction to QTA methods, principles and applications within feature-based models, representation learning and generative models. The focus is on how to extract and structure textual data using machine learning to enable quantitative analysis. The Primer is particularly suitable for researchers new to the field with a pragmatic interest in these techniques. By focusing on machine-learning methodologies, a comprehensive overview of several key workflows currently in use is presented. The focus consciously excludes traditional count-based and rule-based methods, such as keyword and collocation analysis. This decision is guided by the current dominance of machine learning in QTA, in terms of both performance and scalability. However, it is worth noting that machine-learning methods can encompass traditional approaches where relevant, adding to their versatility and broad applicability. The experiments in QTA are presented, including problem formulation, data collection, model selection and evaluation techniques. The results and real-world applications of these methodologies are discussed, underscoring the importance of reproducibility and robust data management practices. The inherent limitations and potential optimizations within the field are addressed, charting the evolution from basic feature-based approaches to advanced generative models. The article concludes with a forward-looking discussion on the ethical implications, practical considerations and methodological advances shaping the future of QTA. Regarding tools and software, references to specific libraries and packages are omitted as they are relatively easy to identify given a specific task. Generally, the use of programming languages that are well suited for QTA is recommended, such as Python, R and Julia, but it is also acknowledged that graphical platforms for data analysis provide similar functionalities and may be better suited for certain disciplines.

Experimentation

In QTA, the term experiment assumes a distinct character. Rather than mirroring the controlled conditions commonly associated with randomized controlled trials, it denotes a structured procedure that aims to validate, refine and compare models and findings. QTA experiments provide a platform for testing ideas, establishing hypotheses and paving the way for advancement. At the heart of these experiments lies a model — a mathematical and computational embodiment of discernible patterns drawn from data. A model can be considered a learned function that captures the intricate relationship between textual features and their intended outcomes, allowing for informed decisions on unseen data. For example, in the sentiment analysis, a model learns the association between specific words or phrases and the emotions they convey, later using this knowledge to assess the sentiment of new texts.

The following section delineates the required steps for a QTA experiment. This step-by-step description encompasses everything from problem definition and data collection to the nuances of model selection, training and validation. It is important to distinguish between two approaches in QTA: training or fine-tuning a model, and applying a (pre-trained) model (Fig.  1 ). In the first approach, a model is trained or fine-tuned to solve a QTA task. In the second approach, a pre-trained model is used to solve a QTA task. Finally, it is important to recognize that experimentation, much like other scientific pursuits, is inherently iterative. This cyclic process ensures that the devised models are not just accurate but also versatile enough to be applicable in real-world scenarios.

Problem formulation

Problem formulation is a crucial first step in QTA, laying the foundation for subsequent analysis and experimentation. This process involves several key considerations, which, when clearly defined beforehand, contributes to the clarity and focus of the experiment. First, every QTA project begins with the identification of a research question. The subsequent step is to determine the scope of the analysis, which involves defining the boundaries of the study, such as the time period, the type of texts to be analysed or the geographical or demographic considerations.

An integral part of this process is to identify the nature of the analytical task. This involves deciding whether the study is a classification task, for example, in which data are categorized into predefined classes; a clustering task, in which data are grouped based on similarities without predefined categories; or another type of analysis. The choice of task has significant implications for both the design of the study and the selection of appropriate data and analytical techniques. For instance, a classification task such as sentiment analysis requires clearly defined categories and suitable labelled data, whereas a clustering task might be used in the exploratory data analysis to uncover underlying patterns in the data.

After selecting data to support the analysis, an important next step is deciding on the level of analysis. QTA can be conducted at various levels, such as the document-level, paragraph-level, sentence-level or even word-level. The choice largely depends on the research question, as well as the nature of the data set.

Classification

A common application of a classification task in QTA is the sentiment analysis. For instance, in analysing social media comments, a binary classification might be employed in which comments are labelled as positive or negative. This straightforward example showcases the formulation of a problem in which the objective is clear-cut classification based on predefined sentiment labels. In this case, the level of analysis might be at the sentence level, focusing on the sentiment expressed in each individual comment.

From this sentence-level information, it is possible to extrapolate to general degrees of sentiment. This is often done when companies want to survey their products or when political parties want to analyse their support, for example, to determine how many people are positive or negative towards the party 23 . Finally, from changing degrees of sentiment, one can extract the most salient aspects that form this sentiment: recurring positive or negative sentiments towards price or quality, or different political issues.

Modelling of themes

The modelling of themes involves the identification of prevalent topics, for example, in a collection of news articles. Unlike the emotion classification task, here the researcher is interested in uncovering underlying themes or topics, rather than classifying texts into predefined categories. This problem formulation requires an approach that can discern and categorize emergent topics from the textual data, possibly at the document level, to capture broader thematic elements. This can be done without using any predefined hypotheses 24 , or by steering topic models towards certain seed topics (such as a given scientific paper or book) 25 . Using such topic detection tools, it can be determined how prevalent topics are in different time periods or across genre to determine significance or impact of both topics and authors.

Modelling of temporal change

Consider a study aiming to track the evolution of literary themes over time. In this scenario, the problem formulation would involve not only the selection of texts and features but also a temporal dimension, in which changes in themes are analysed across different time periods. This type of analysis might involve examining patterns and trends in literary themes, requiring a longitudinal approach to text analysis, for example, in the case of scientific themes or reports about important events 26 or themes as proxy for meaning change 27 . Often, when longitudinal analysis is considered, additional challenges are involved, such as statistical properties relating to increasing or decreasing quantity or quality of data that can influence results, see, for example, refs. 28 , 29 , 30 , 31 .

In similar fashion, temporal analysis of changing data happens in a multitude of disciplines from linguistics, as in computational detection of words that experience change in meaning 32 , to conceptual change in history 33 , poetry 34 , medicine 35 , political science 36 , 37 and to the study of ethnical biases and racism 38 , 39 , 40 .

The GIGO principle, meaning ‘garbage in, garbage out’, is ever present in QTA because without high-quality data even the most sophisticated models can falter, rendering analyses inaccurate or misleading. To ensure robustness in, for example, social media data, its inherently informal and dynamic nature must be acknowledged, often characterized by non-standard grammar, slang and evolving language use. Robustness here refers to the ability of the data to provide reliable, consistent analysis, despite these irregularities. This requires implementing specialized preprocessing techniques that can handle such linguistic variability without losing contextual meaning. For example, rather than discarding non-standard expressions or internet-specific abbreviations, these elements should be carefully processed to preserve their significant role in conveying sentiment and meaning. Additionally, ensuring representativeness and diversity in the data set is crucial; collecting data across different demographics, topics and time frames can mitigate biases and provide a more comprehensive view of the discourse if this is needed. Finally, it is important to pay attention to errors, anomalies and irregularities in the data, such as optical character recognition errors and missing values, and in some cases take steps to remediate these in preprocessing. More generally, it is crucial to emphasize that the quality of a given data set depends on the research question. Grammatically well-formed sentences may be high-quality data for training a linguistic parser; social media could never be studied as people on social media rarely abide by the rules of morphology and syntax. This underscores the vital role of data not just as input but also as an essential component that dictates the success and validity of the analytical endeavour.

Data acquisition

Depending on the research objective, data sets can vary widely in their characteristics. For the emotion classifier, a data set could consist of many social media comments. If the task is to train or fine-tune a model, each comment should be annotated with its corresponding sentiment label (labels). If the researcher wants to apply a pre-trained model, then only a subset of the data must be annotated to test the generalizability of the model. Labels can be annotated manually or automatically, for instance, by user-generated ratings, such as product reviews or social media posts, for example. Training data should have sufficient coverage of the phenomenon under investigation to capture its linguistic characteristics. For the emotion classifier, a mix of comments are needed, ranging from brief quips to lengthy rants, offering diverse emotional perspectives. Adhering to the principle that there are no data like more data, the breadth and depth of such a data set significantly enhance the accuracy of the model. Traditionally, data collection was arduous, but today QTA researchers can collect data from the web and archives using dedicated software libraries or an  application programming interface . For analogue data, optical character recognition and handwritten text recognition offer efficient conversion to machine-readable formats 41 . Similarly, for auditory language data, automatic speech recognition has emerged as an invaluable tool 42 .

Data preprocessing

In feature-based QTA, manual data preprocessing is one of the most crucial and time-consuming stages. Studies suggest that researchers can spend up to 80% of their project time refining and managing their data 43 . A typical preprocessing workflow for feature-based techniques requires data cleaning and text normalization. Standard procedures include transforming all characters to lower case for uniformity, eliminating punctuation marks and removing high-frequency functional words such as ‘and’, ‘the’ or ‘is’. However, it is essential to recognize that these preprocessing strategies should be closely aligned with the specific research question at hand. For example, in the sentiment analysis, retaining emotive terms and expressions is crucial, whereas in syntactic parsing, the focus might be on the structural elements of language, requiring a different approach to what constitutes ‘noise’ in the data. More nuanced challenges arise in ensuring the integrity of a data set. For instance, issues with character encoding require attention to maintain language and platform interoperability, which means resorting to universally accepted encoding formats such as UTF-8. Other normalization steps, such as  stemming or lemmatization , involve reducing words to their root forms to reduce lexical variation. Although these are standard practices, their application might vary depending on the research objective. For example, in a study focusing on linguistic diversity, aggressive stemming may erase important stylistic or dialectal markers. Many open-source software libraries exist nowadays that can help automate such processes for various languages. The impact of these steps on research results underscores the necessity of a structured and well-documented approach to preprocessing, including detailed reporting of all preprocessing steps and software used, to ensure that analyses are both reliable and reproducible. The practice of documenting preprocessing is crucial, yet often overlooked, reinforcing its importance for the integrity of research.

With representation learning and generative techniques, QTA has moved towards end-to-end models that take raw text input such as social media comments and directly produces the final desired output such as emotion classification, handling all intermediate steps without manual intervention 44 . However, removal of non-textual artefacts such as HTML codes and unwanted textual elements such as pornographic material can still require substantial work to prepare data to train an end-to-end model.

Annotation and labelling

Training and validating a (pre-trained) model requires annotating the textual data set. These data sets come in two primary flavours: pre-existing collections with established labels and newly curated sets awaiting annotation. Although pre-existing data sets offer a head-start, owing to their readymade labels, they must be validated to ensure alignment with research objectives. By contrast, crafting a data set from scratch confers flexibility to tailor the data to precise research needs, but it also ushers in the intricate task of collecting and annotating data. Annotation is a meticulous endeavour that demands rigorous consistency and reliability. To ensure inter-annotator agreement (IAA) 45 , for example, annotations from multiple annotators are compared using metrics such as  Fleiss’ kappa ( κ ) to assess consistency. A high IAA score not only indicates annotation consistency but also lends confidence in the reliability of the data set. There is no universally accepted manner to interpret κ statistics, although κ  ≥ 0. 61 is generally considered to indicate ‘substantial agreement’ 46 .

Various tools and platforms support the annotation process. Specialized software for research teams provides controlled environments for annotation tasks. Crowdsourcing is another approach, in which tasks are distributed among a large group of people. This can be done through non-monetized campaigns, focusing on volunteer participation or gamification strategies to encourage user engagement in annotation tasks 47 . Monetized platforms, such as Amazon Mechanical Turk, represent a different facet of crowdsourcing in which microtasks are outsourced for financial compensation. It is important to emphasize that, although these platforms offer a convenient way to gather large-scale annotations, they raise ethical concerns regarding worker exploitation and fair compensation. Critical studies, such as those of Paolacci, Chandler and Ipeirotis 48 and Bergvall-Kåreborn and Howcroft 49 , highlight the need for awareness and responsible use of such platforms in research contexts.

Provenance and ethical considerations

Data provenance is of utmost importance in QTA. Whenever feasible, preference should be given to open and well-documented data sets that comply with the principles of FAIR (findable, accessible, interoperable and reusable) 50 . However, the endeavour to harness data, especially online, requires both legal and ethical considerations. For instance, the General Data Protection Regulation delineates the rights of European data subjects and sets stringent data collection and usage criteria. Unstructured data can complicate standard techniques for data depersonalization (for example, data masking, swapping and pseudonymization). Where these techniques fail, differential privacy may be a viable alternative to ensure that the probability of any specific output of the model does not depend on the information of any individual in the data set 51 .

Recognition of encoded biases is equally important. Data sets can inadvertently perpetuate cultural biases towards attributes such as gender and race, resulting in sampling bias. Such bias compromises research integrity and can lead to models that reinforce existing inequalities. Gender, for instance, can have subtle effects that are not easily detected in textual data 52 . A popular approach to rectifying biases is  data augmentation , which can be used to increase the diversity of a data set without collecting new data 53 . This is achieved by applying transformations to existing textual data, creating new and diverse examples. The main goal of data augmentation is to improve model generalization by exposing it to a broader range of data variations.

Model selection and design

Model selection and design set the boundaries for efficiency, accuracy and generalizability of any QTA experiment. Choosing the right model architecture depends on several considerations and will typically require experimentation to compare the performance of multiple models. Although the methodological trajectory of QTA provides a roadmap, specific requirements of the task, coupled with available data volume, often guide the final choice. Although some tasks require that the model be trained from scratch owing to, for instance, transparency and security requirements, it has become common to use pre-trained models that provide text representations originating from training on massive data sets. Pre-trained models can be fine-tuned for a specific task, for example, emotion classification. Training feature-based models may be optimal for smaller data sets, focusing on straightforward interpretability. By contrast, the complexities of expansive textual data often require representation learning or generative models. In QTA, achieving peak performance is a trade-off among model interpretability, computational efficiency and predictive power. As the sophistication of a model grows, hyperparameter tuning, regularization and loss function require meticulous consideration. These decisions ensure that a model is not only accurate but also customized for research-specific requirements.

Training and evaluation

During the training phase, models learn patterns from the data to predict or classify textual input. Evaluation is the assessment phase that determines how the trained model performs on unseen data. Evaluation serves multiple purposes, but first and foremost, it is used to assess how well the model performs on a specific task using metrics such as accuracy, precision and recall. For example, knowing how accurately the emotion classifier identifies emotions is crucial for any research application. Evaluation of this model also allows researchers to assess whether it is biased towards common emotions and whether it generalizes across different types of text sources. When an emotion classifier is trained on social media posts, a common practice, its effectiveness can be evaluated on different data types, such as patient journals or historical newspapers, to determine its performance across varied contexts. Evaluation enables us to compare multiple models to select the most relevant for the research problem. Additional evaluation involves hyperparameter tuning, resource allocation, benchmarking and model fairness audits.

Overfitting is often a challenge in model training, which can occur when a model is excessively tailored to the peculiarities of the training data and becomes so specialized that its generalizability is compromised. Such a model performs accurately on the specific data set but underperforms on unseen examples. Overfitting can be counteracted by dividing the data into three distinct subsets: the training set, the validation set and the test set. The training set is the primary data set from which the model learns patterns, adjusts its weights and fine-tunes itself based on the labelled examples provided. The validation set is used to monitor and assess the performance of the model during training. It acts as a checkpoint, guides hyperparameter tuning and ensures that the model is not veering off track. The test set is the final held-out set on which the performance of the model is evaluated. The test set is akin to a final examination, assessing how well the model generalizes to unseen data. If a pre-trained model is used, only the data sets used to fine-tune the model are necessary to evaluate the model.

The effectiveness of any trained model is gauged not just by how well it fits the training data but also by its performance on unseen samples. Evaluation metrics provide objective measures to assess performance on validation and test sets as well as unseen examples. The evaluation process is fundamental to QTA experiments, as demonstrated in the text classification research 10 . Several evaluation metrics are used to measure performance. The most prominent are accuracy (the proportion of all predictions that are correct), precision (the proportion of positive predictions that are actually correct) and recall (the proportion of actual positives that were correctly identified). The F1 score amalgamates precision and recall and emerges as a balanced metric, especially when class distributions are skewed. An effective evaluation typically uses various complementary metrics.

In QTA, a before-and-after dynamic often emerges, encapsulating the transformation from raw data to insightful conclusions 54 . This paradigm is especially important in QTA, in which the raw textual data can be used to distil concrete answers to research questions. In the preceding section, the preliminary before phase, the process of setting up an experiment in QTA, is explored with emphasis on the importance of model training and thorough evaluation to ensure robustness. For the after phase, the focus pivots to the critical step of applying the trained model to new, unseen data, aiming to answer the research questions that guide exploration.

Research questions in QTA are often sophisticated and complex, encompassing a range of inquiries either directly related to the text being analysed or to the external phenomena the text reflects. The link between the output of QTA models and the research question is often vague and under-specified. When dealing with a complex research question, for example, the processes that govern the changing attitudes towards different migrant groups, the outcome of any one QTA model is often insufficient. Even several models might not provide a complete answer to the research question. Consequently, challenges surface during the transition from before to after, from setting up and training to applying and validating. One primary obstacle is the validation difficulty posed by the uniqueness and unseen nature of the new data.

Validating QTA models on new, unseen data introduces a layer of complexity that highlights the need for robust validation strategies, to ensure stability, generalizability and replicability of results. Although the effectiveness of a model might have been calibrated in a controlled setup, its performance can oscillate when exposed to the multifaceted layers of new real-world data. Ensuring consistent model performance is crucial to deriving meaningful conclusions aligned with the research question. This dual approach of applying the model and subsequently evaluating its performance in fresh terrains is central to the after phase of QTA. In addition to validating the models, the results that stem from the models need to be validated with respect to the research question. The results need to be representative for the data as a whole; they need to be stable such that the answer does not change if different choices are made in the before phase; and they need to provide an answer to the research question at hand.

This section provides a road map for navigating the application of QTA models to new data and a chart of methodologies for evaluating the outcomes in line with the research question (questions). The goal is to help researchers cross the bridge between the theoretical foundations of QTA and its practical implementation, illuminating the steps that support the successful application and assessment of QTA models. The ensuing discussion covers validation strategies that cater to the challenges brought forth by new data, paving the way towards more insightful analysis.

Application to new data

After the training and evaluation phases have been completed, the next step is applying the trained model to new, unseen data (Fig.  2 ). The goal is to ensure that the application aligns with the research questions and aids in extracting meaningful insights. However, applying the model to new data is not without challenges.

Although the illustration demonstrates a feature-based modelling approach, the fundamental principle remains consistent across different methodologies, be it feature-based, representation learning or generative. A critical consideration is ensuring the consistency in content and preprocessing between the training data and any new data subjected to inference.

Before application of the model, it is crucial to preprocess the new data similar to the training data. This involves routine tasks such as tokenization and lemmatization, but also demands vigilance for anomalies such as divergent text encoding formats or missing values. In such cases, additional preprocessing steps might be required and should be documented carefully to ensure reproducibility.

Another potential hurdle is the discrepancy in data distributions between the training data and new data, often referred to as domain shift. If not addressed, domain shifts may hinder the efficacy of the model. Even thematically, new data may unearth categories or motifs that were absent during training, thus challenging the interpretative effectiveness of the model. In such scenarios, transfer learning or domain adaptation techniques are invaluable tools for adjusting the model so that it aligns better with the characteristics of the new data. In transfer learning, a pre-trained model provides general language understanding and is fine-tuned with a small data set for a specific task (for example, fine-tuning a large language model such as GPT or BERT for emotion classification) 55 , 56 . Domain adaptation techniques similarly adjust a model from a source domain to a target domain; for example, an emotion classifier trained on customer reviews can be adapted to rate social media comments.

Given the iterative nature of QTA, applying a model is not necessarily an end point; it may simply be a precursor to additional refinement and analysis. Therefore, the adaptability of the validation strategies is paramount. As nuances in the new data are uncovered, validation strategies may need refinement or re-adaptation to ensure the predictions of the model remain accurate and insightful, ensuring that the answers to the research questions are precise and meaningful. Through careful application and handling of the new data, coupled with adaptable validation strategies, researchers can significantly enhance the value of their analysis in answering the research question.

Evaluation metrics

QTA models are often initially developed and validated on well-defined data sets, ensuring their reliability in controlled settings. This controlled environment allows researchers to set aside a held-out test set to gauge the performance of a model, simulating how it will fare on new data. The real world, however, is considerably more complex than any single data set can capture. The challenge is how to transition from a controlled setting to novel data sets.

One primary challenge is the mismatch between the test set and real-world texts. Even with the most comprehensive test sets, capturing the linguistic variation, topic nuance and contextual subtlety present in new data sets is not a trivial task, and researchers should not be overconfident regarding the universal applicability of a model 57 . The situation does not become less complicated when relying on pre-trained or off-the-shelf models. The original training data and its characteristics might not be transparent or known with such models. Without appropriate documentation, predicting the behaviour of a model on new data may become a speculative endeavour 58 .

The following sections summarize strategies for evaluating models on new data.

Model confidence scores

In QTA, models often generate confidence or probability scores alongside predictions, indicating the confidence of the model in its accuracy. However, high scores do not guarantee correctness and can be misleading. Calibrating the model refines these scores to align better with true label likelihoods 59 . This is especially crucial in high-stakes QTA applications such as legal or financial text analysis 60 . Calibration techniques adjust the original probability estimates, enhancing model reliability and the trustworthiness of predictions, thereby addressing potential discrepancies between the expressed confidence of the model and its actual performance.

Precision at k

Precision at k (P@ k ) is useful for tasks with rankable predictions, such as determining document relevance. P@ k measures the proportion of relevant items among the top- k ranked items, providing a tractable way to gauge the performance of a model on unseen data by focusing on a manageable subset, especially when manual evaluation of the entire data set is infeasible. Although primarily used in information retrieval and recommender system , its principles apply to QTA, in which assessing the effectiveness of a model in retrieving or categorizing relevant texts is crucial.

External feedback mechanisms

Soliciting feedback from domain experts is invaluable in evaluating models on unseen data. Domain experts can provide qualitative insights into the output of the model, identifying strengths and potential missteps. For example, in topic modelling, domain experts can assess the coherence and relevance of the generated topics. This iterative feedback helps refine the model, ensuring its robustness and relevance when applied to new, unseen data, thereby bridging the gap between model development and practical application.

Software and tools

When analysing and evaluating QTA models on unseen data, researchers often turn to specialized tools designed to increase model transparency and explain model predictions. Among these tools, LIME (Local Interpretable Model-agnostic Explanations) 61 and SHAP (SHapley Additive exPlanations) 62 have gained traction for their ability to provide insights into model behaviour per instance, which is crucial when transitioning to new data domains.

LIME focuses on the predictions of machine-learning models by creating locally faithful explanations. It operates by perturbing the input data and observing how the predictions change, making it a useful tool to understand model behaviour on unseen data. Using LIME, researchers can approximate complex models with simpler, interpretable models locally around the prediction point. By doing so, they can gain insight into how different input features contribute to the prediction of the model, which can be instrumental in understanding how a model might generalize to new, unseen data.

SHAP, by contrast, provides a unified measure of feature importance across different data types, including text. It uses game theoretic principles to attribute the output of machine-learning models to their input features. This method allows for a more precise understanding of how different words or phrases in text data influence the output of the model, thereby offering a clearer picture of the behaviour of the model on new data domains. The SHAP library provides examples of how to explain predictions from text analysis models applied to various NLP tasks including sentiment analysis, text generation and translation.

Both LIME and SHAP offer visual tools to help researchers interpret the predictions of the model, making it easier to identify potential issues when transitioning to unseen data domains. For instance, visualizations allow researchers to identify words or phrases that heavily influence the decisions of the model, which can be invaluable in understanding and adjusting the model for new text data.

Interpretation

Interpretability is paramount in QTA as it facilitates the translation of complex model outcomes into actionable insights relevant to the research questions. The nature and complexity of the research question can significantly mould the interpretation process by requiring various information signals to be extracted from the text, see, for example, ref.  63 . For example, in predicting election outcomes based on sentiments expressed in social media 64 , it is essential to account for both endorsements of parties as expressed in the text and a count of individuals (that is, statistical signals) to avoid the results being skewed because some individuals make a high number of posts. It is also important to note whether voters of some political parties are under-represented in the data.

The complexity amplifies when delving into understanding why people vote (or do not vote) for particular parties and what arguments sway their decisions. Such research questions demand a more comprehensive analysis, often necessitating the amalgamation of insights from multiple models, for example, argument mining, aspect-based sentiment analysis and topic models. There is a discernible gap between the numerical or categorical outputs of QTA models — such as classification values, proportions of different stances or vectors representing individual words — and the nuanced understanding required to fully address the research question. This understanding is achieved either using qualitative human analysis or applying additional QTA methods and extracts a diverse set of important arguments in support of different stances, or provides qualitative summaries of a large set of different comments. Because it is not only a matter of ‘what’ results are found using QTA, but the value that can be attributed to those results.

When interpreting the results of a computational model applied to textual data for a specific research question, it is important to consider the completeness of the answer (assess whether the output of the model sufficiently addresses the research question or whether there are aspects left unexplored), the necessity of additional models (determine whether the insights from more models are needed to fully answer the research question), the independence or co-dependence of results (in cases in which multiple models are used, ascertain whether their results are independent or co-dependent and adjust for any overlap in insights accordingly), clarify how the results are used to support an answer (such as the required occurrence of a phenomenon in the text to accept a concept, or how well a derived topic is understood and represented) and the effect of methodology (evaluate the impact of the chosen method or preprocessing on the results, ensuring the reproducibility and robustness of the findings against changes in preprocessing or methods).

Using these considerations alongside techniques such as LIME and SHAP enhances the evaluation of the application of the model. For instance, in a scenario in which a QTA model is used to analyse customer reviews, LIME and SHAP could provide nuanced insights on a peer-review basis and across all reviews, respectively. Such insights are pivotal in assessing the alignment of the model with the domain-relevant information necessary to address the research questions and in making any adjustments needed to enhance its relevance and performance. Moreover, these techniques and considerations catalyse a dialogue between model and domain experts, enabling a more nuanced evaluation that extends beyond mere quantitative metrics towards a qualitative understanding of the application of the model.

Applications

The applicability of QTA can be found in its ability to address research questions across various disciplines. Although these questions are varied and tasks exist that do not fit naturally into categories, they can be grouped into four primary tasks: extracting, categorizing, predicting and generating. Each task is important in advancing understanding of large textual data sets, either by examining phenomena specific to a text or by using texts as a proxy for phenomena outside the text.

Extracting information

In the context of QTA, information extraction goes beyond mere data retrieval; it also involves identifying and assessing patterns, structures and entities within extensive textual data sets. At its core are techniques such as frequency analysis, in which words or sets of words are counted and their occurrences plotted over periods to reveal trends or shifts in usage and syntactical analysis, which targets specific structures such as nouns, verbs and intricate patterns such as passive voice constructions. Named entity recognition pinpoints entities such as persons, organizations and locations using syntactic information and lexicons of entities.

These methodologies have proven useful in various academic domains. For example, humanities scholars have applied QTA to track the evolution of literary themes 65 . Word embedding has been used to shed light on broader sociocultural shifts such as the conceptual change of ‘racism’, or detecting moments of linguistic change in American foreign relations 40 , 66 . In a historical context, researchers have used diachronic word embeddings to scrutinize the role of abolitionist newspapers in influencing public opinion about the abolition of slavery, revealing pathways of lexical semantic influence, distinguishing leaders from followers and identifying others who stood out based on the semantic changes that swept through this period 67 . Topic modelling and topic linkage (the extent to which two topics tend to co-appear) have been applied to user comments and submissions from the ‘subreddit’ group r/TheRedPill to study how people interact with ideology 68 . In the medical domain 69 , QTA tools have been used to study narrative structures in personal birth stories. The authors utilized a topic model based on latent Dirichlet allocation (LDA) to not only represent the sequence of events in every story but also detect outlier stories using the probability of transitioning between topics.

Historically, the focus was predominantly on feature-based models that relied on manual feature engineering. Such methods were transparent but rigid, constraining the richness of the textual data. Put differently, given the labour-intensive selection of features and the need to keep them interpretable, the complexity of a text was reduced to a limited set of features. However, the advent of representation learning has catalysed a significant paradigm shift. It enables more nuanced extraction, considers contextual variations and allows for sophisticated trend analysis. Studies using these advanced techniques have been successful in, for example, analysing how gender stereotypes and attitudes towards ethnic minorities in the USA evolved during the twentieth and twenty-first centuries 38 and tracking the emergence of ideas in the domains of politics, law and business through contextual embeddings combined with statistical modelling 70 (Box  1 ).

Box 1 Using text mining to model prescient ideas

Vicinanza et al. 70 focused on the predictive power of linguistic markers within the domains of politics, law and business, positing that certain shifts in language can serve as early indicators of deeper cognitive changes. They identified two primary attributes of prescient ideas: their capacity to challenge existing contextual assumptions, and their ability to foreshadow the future evolution of a domain. To quantify this, they utilized Bidirectional Encoder Representations from Transformers, a type 2 language model, to calculate a metric termed contextual novelty to gauge the predictability of an utterance within the prevailing discourse.

Their study presents compelling evidence that prescient ideas are more likely to emerge from the periphery of a domain than from its core. This suggests that prescience is not solely an individual trait but also significantly influenced by contextual factors. Thus, the researchers extended the notion of prescience to include the environments in which innovative ideas are nurtured, adding another layer to our understanding of how novel concepts evolve and gain acceptance.

Categorizing content

It remains an indispensable task in QTA to categorize content, especially when dealing with large data sets. The challenge is not only logistical but also methodological, demanding sophisticated techniques to ensure precision and utility. Text classification algorithms, supervised or unsupervised, continue to have a central role in labelling and organizing content. They serve crucial functions beyond academic settings; for instance, digital libraries use these algorithms to manage and make accessible their expansive article collections. These classification systems also contribute significantly to the systematic review of the literature, enabling more focused and effective investigations of, for example, medical systematic reviews 71 . In addition, unsupervised techniques such as topic modelling have proven invaluable in uncovering latent subject matter within data sets 72 (Box  2 ). This utility extends to multiple scenarios, from reducing redundancies in large document sets to facilitating the analysis of open-ended survey responses 73 , 74 .

Earlier approaches to categorization relied heavily on feature-based models that used manually crafted features for organization. This traditional paradigm has been disrupted by advances in representation learning, deep neural networks and word embeddings, which has introduced a new age of dynamic unsupervised and semi-supervised techniques for content categorization. GPT models represent another leap forward in text classification tasks, outpacing existing benchmarks across various applications. From the sentiment analysis to text labelling and psychological construct detection, generative models have demonstrated a superior capability for context understanding, including the ability to parse complex linguistic cues such as sarcasm and mixed emotions 75 , 76 , 77 . Although the validity of these models is a matter of debate, they offer explanations for their reasoning, which adds a layer of interpretability.

Box 2 Exploring molecular data with topic modelling

Schneider et al. 72 introduced a novel application of topic modelling to the field of medicinal chemistry. The authors adopt a probabilistic topic modelling approach to organize large molecular data sets into chemical topics, enabling the investigation of relationships between these topics. They demonstrate the effectiveness of the quantitative text analysis method in identifying and retrieving chemical series from molecular sets. The authors are able to reproduce concepts assigned by humans in the identification and retrieval of chemical series from sets of molecules. Using topic modelling, the authors are able to show chemical topics intuitively with data visualization and efficiently extend the method to a large data set (ChEMBL22) containing 1.6 million molecules.

Predicting outcomes

QTA is not limited to understanding or classifying text but extends its reach into predictive analytics, which is an invaluable tool across many disciplines and industries. In the financial realm, sentiment analysis tools are applied to news articles and social media data to anticipate stock market fluctuations 78 . Similarly, political analysts use sentiment analysis techniques to make election forecasts, using diverse data sources ranging from Twitter (now X) feeds to party manifestos 79 . Authorship attribution offers another intriguing facet, in which predictive abilities of the QTA are harnessed to identify potential authors of anonymous or pseudonymous works 80 . A notable instance was the unmasking of J.K. Rowling as the author behind the pseudonym Robert Galbraith 81 . Health care has also tapped into predictive strengths of the QTA: machine-learning models that integrate natural language and binary features from patient records have been shown to have potential as early warning systems to prevent unnecessary mechanical restraint of psychiatric inpatients 82 (Box  3 ).

In the era of feature-based models, predictions often hinged on linear or tree-based structures using manually engineered features. Representation learning introduced embeddings and sequential models that improved prediction capabilities. These learned representations enrich predictive tasks, enhancing accuracy and reliability while decreasing interpretability.

Box 3 Predicting mechanical restraint: assessing the contribution of textual data

Danielsen et al. 82 set out to assess the potential of electronic health text data to predict incidents of mechanical restraint of psychiatric patients. Mechanical restraint is used during inpatient treatments to avert potential self-harm or harm to others. The research team used feature-based supervised machine learning to train a predictive model on clinical notes and health records from the Central Denmark Region, specifically focusing on the first hour of admission data. Of 5,050 patients and 8,869 admissions, 100 patients were subjected to mechanical restraint between 1 h and 3 days after admission. Impressively, a random forest algorithm could predict mechanical restraint with considerable precision, showing an area under the curve of 0.87. Nine of the ten most influential predictors stemmed directly from clinical notes, that is, unstructured textual data. The results show the potential of textual data for the creation of an early detection system that could pave the way for interventions that minimize the use of mechanical restraint. It is important to emphasize that the model was limited by a narrow scope of data from the Central Denmark Region, and by the fact that only initial mechanical restraint episodes were considered (in other words, recurrent incidents were not included in the study).

Generating content

Although the initial QTA methodologies were not centred on content generation, the rise of generative models has been transformative. Models such as GPT-4 and Llama2 (ref. 20 ) have brought forth previously unimagined capabilities, expanding the potential of QTA to create content, including coherent and contextually accurate paragraphs to complete articles. Writers and content creators are now using tools based on models such as GPT-4 to augment their writing processes by offering suggestions or even drafting entire sections of texts. In education, such models aid in developing customized content for students, ensuring adaptive learning 83 . The capacity to create synthetic data also heralds new possibilities. Consider the domain of historical research, in which generative models can simulate textual content, offering speculative yet data-driven accounts of alternate histories or events that might have been; for example, relying on generative models to create computational software agents that simulate human behaviour 84 . However, the risks associated with text-generating models are exemplified by a study in which GPT-3 was used for storytelling. The generated stories were found to exhibit many known gender stereotypes, even when prompts did not contain explicit gender cues or stereotype-related content 85 .

Reproducibility and data deposition

Given the rapidly evolving nature of the models, methods and practices in QTA, reproducibility is essential for validating the results and creating a foundation upon which other researchers can build. Sharing code and trained models in well-documented repositories are important to enable reproducible experiments. However, sharing and depositing raw data can be challenging, owing to the inherent limitations of unstructured data and regulations related to proprietary and sensitive data.

Code and model sharing

In QTA research, using open source code has become the norm and the need to share models and code to foster innovation and collaboration has been widely accepted. QTA is interdisciplinary by nature, and by making code and models public, the field has avoided unnecessary silos and enabled collaboration between otherwise disparate disciplines. A further benefit of open source software is the flexibility and transparency that comes from freely accessing and modifying software to meet specific research needs. Accessibility enables an iterative feedback loop, as researchers can validate, critique and build on the existing work. Software libraries, such as scikit-learn, that have been drivers for adopting machine learning in QTA are testimony to the importance of open source software 21 .

Sharing models is not without challenges. QTA is evolving rapidly, and models may use specific versions of software and hardware configurations that no longer work or that yield different results with other versions or configurations. This variability can complicate the accessibility and reproducibility of research results. The breakthroughs of generative AI in particular have introduced new proprietary challenges to model sharing as data owners and sources raise objections to the use of models that have been trained on their data. This challenge is complicated, but fundamentally it mirrors the disputes about intellectual property rights and proprietary code in software engineering. Although QTA as a field benefits from open source software, individual research institutions may have commercial interests or intellectual property rights related to their software.

On the software side, there is currently a preference for scripting languages, especially Python, that enable rapid development, provide access to a wide selection of software libraries and have a large user community. QTA is converging towards code and model sharing through open source platforms such as GitHub and GitLab with an appropriate open source software license such as the MIT license . Models often come with additional disclaimers or use-based restrictions to promote responsible use of AI, such as in the RAIL licenses . Pre-trained models are also regularly shared on dedicated machine-learning platforms such as Hugging Face 22 to enable efficient fine-tuning and deployment. It is important to emphasize that although these platforms support open science, these services are provided by companies with commercial interests. Open science platforms such as Zenodo and OSF can also be used to share code and models for the purpose of reproducibility.

Popular containerization software has been widely adopted in the machine-learning community and has spread to QTA. Containerization, that is, packaging all parts of a QTA application — including code and other dependencies — into a single standalone unit ensures that model and code run consistently across various computing environments. It offers a powerful solution to challenges such as reproducibility, specifically variability in software and hardware configurations.

Data management and storage

Advances in QTA in recent years are mainly because of the availability of vast amounts of text data and the rise of deep learning techniques. However, the dependency on large unstructured data sets, many of which are proprietary or sensitive, poses unique data management challenges. Pre-trained models irrespective of their use (for example, representation learning or generative) require extensive training on large data sets. When these data sets are proprietary or sensitive, they cannot be readily available, which limits the ability of researchers to reproduce results and develop competitive models. Furthermore, models trained on proprietary data sets often lack transparency regarding their collection and curation processes, which can hide potential biases in the data. Finally, there can be data privacy issues related to training or using models that are trained on sensitive data. Individuals whose data are included may not have given their explicit consent for their information to be used in research, which can pose ethical and legal challenges.

It is a widely adopted practice in QTA to share data and metadata with an appropriate license whenever possible. Data can be deposited in open science platforms such as Zenodo, but specialized machine-learning platforms are also used for this purpose. However, it should be noted that QTA data are rarely unique, unlike experimental data collected through random controlled trials. In many cases, access to appropriate metadata and documentation would enable the data to be reconstructed. In almost all cases, it is therefore strongly recommended that researchers share metadata and documentation for data, as well as code and models, using a standardized document or framework, a so-called datasheet. Although QTA is not committed to one set of principles for (meta)data management, European research institutions are increasingly adopting the FAIR principles 50 .

Documentation

Although good documentation is vital in all fields of software development and research, the reliance of QTA on code, models and large data sets makes documentation particularly crucial for reproducibility. Popular resources for structuring projects include project templating tools and documentation generators such as Cookiecutter and Sphinx . Models are often documented with model cards that provide a detailed overview of the development, capabilities and biases of the model to promote transparency and accountability 86 . Similarly, datasheets or data cards can be used to promote transparency for data used in QTA 87 . Finally, it is considered good practice to provide logs for models that document parameters, metrics and events for QTA experiments, especially during training and fine-tuning. Although not strictly required, logs are also important for documenting the iterative process of model refinement. There are several platforms that support the creation and visualization of training logs ( Weights & Biases and MLflow ).

Limitations and optimizations

The application of QTA requires scrutiny of its inherent limitations and potentials. This section discusses these aspects and elucidates the challenges and opportunities for further refinement.

Limitations in QTA

Defining research questions.

In QTA, the framing of research questions is often determined by the capabilities and limitations of the available text analysis tools, rather than by intellectual inquiry or scientific curiosity. This leads to task-driven limitations, in which inquiry is confined to areas where the tools are most effective. For example, relying solely on bag-of-words models might skew research towards easily quantifiable aspects, distorting the intellectual landscape. Operationalizing broad and nuanced research questions into specific tasks may strip them of their depth, forcing them to conform to the constraints of existing analytical models 88 .

Challenges in interpretation

The representation of language of underlying phenomena is often ambiguous or indirect, requiring careful interpretation. Misinterpretations can arise, leading to challenges related to historical, social and cultural context of a text, in which nuanced meanings that change across time, class and cultures are misunderstood 89 . Overlooking other modalities such as visual or auditory information can lead to a partial understanding of the subject matter and limit the full scope of insights. This can to some extent be remedied by the use of grounded models (such as GPT-4), but it remains a challenge for the community to solve long term.

Determining reliability and validation

The reliability and stability of the conclusions drawn from the QTA require rigorous validation, which is often neglected in practice. Multiple models, possibly on different types of data, should be compared to ensure that conclusions are not artefacts of a particular method or of a different use of the method. Furthermore, cultural phenomena should be evolved to avoid misguided insights. Building a robust framework that allows testing and comparison enhances the integrity and applicability of QTA in various contexts 90 .

Connecting analysis to cultural insights

Connecting text analysis to larger cultural claims necessitates foundational theoretical frameworks, including recognizing linguistic patterns, sociolinguistic variables and theories of cultural evolution that may explain changes. Translating textual patterns into meaningful cultural observations requires understanding how much (or how little) culture is expressed in text so that findings can be generalized beyond isolated observations. A theoretical foundation is vital to translate textual patterns into culturally relevant insights, making QTA a more effective tool for broader cultural analysis.

Balancing factors in machine learning

Balancing factors is critical in aligning machine-learning techniques with research objectives. This includes the trade-off between quality and control. Quality refers to rigorous, robust and valid findings, and control refers to the ability to manage specific variables for clear insights. It is also vital to ensure a balance between quantity and quality in data source to lead to more reliable conclusions. Balance is also needed between correctness and accuracy, in which the former ensures consistent application of rules, and the latter captures the true nature of the text.

From features-based to generative models

QTA has undergone a profound evolution, transitioning from feature-based approaches to representation learning and finally to generative models. This progression demonstrates growing complexity in our understanding of language, reflecting the maturity in the field of QTA. Each stage has its characteristics, strengths and limitations.

In the early stages, feature-based models were both promising and limiting. The simplicity of their design, relying on explicit feature engineering, allowed for the targeted analysis. However, this simplicity limited their ability to grasp complex, high-level patterns in language. For example, the use of bag-of-words models in the sentiment analysis showcased direct applicability, but also revealed limitations in understanding contextual nuances. The task-driven limitations of these models sometimes overshadowed genuine intellectual inquiry. Using a fixed (often modern) list of words with corresponding emotional valences may limit our ability to fully comprehend the complexity of emotional stances in, for example, historical literature. Despite these drawbacks, the ability to customize features provided researchers with a direct and specific understanding of language phenomena that could be informed by specialized domain knowledge 91 .

With the emergence of representation learning, a shift occurred within the field of QTA. These models offered the ability to capture higher-level abstractions, forging a richer understanding of language. Their scalability to handle large data sets and uncover complex relationships became a significant strength. However, this complexity introduced new challenges, such as a loss of specificity in analysis and difficulties in translating broad research questions into specific tasks. Techniques such as Word2Vec enabled the capture of semantic relationships but made it difficult to pinpoint specific linguistic features. Contextualized models, in turn, allow for more specificity, but are typically pre-trained on huge data sets (not available for scrutiny) and then applied to a research question without any discussion of how well the model fits the data at hand. In addition, these contextualized models inundate with information. Instead of providing one representation for a word (similar to Word2Vec does), they provide one representation for each occurrence of the word. Each of these representations is one order of magnitude larger than vectors typical for Word2Vec (768–1,600 dimensions compared with 50–200) and comes in several varieties, one for each of the layers of the mode, typically 12.

The introduction of generative models represents the latest stage of this evolution, providing even greater complexity and potential. Innovative in their design, generative models provide opportunities to address more complex and open-ended research questions. They fuel the generation of new ideas and offer avenues for novel approaches. However, these models are not without their challenges. Their high complexity can make interpretation and validation demanding, and if not properly managed, biases and ethical dilemmas will emerge. The use of generative models in creating synthetic text must be handled with care to avoid reinforcing stereotypes or generating misleading information. In addition, if the enormous amounts of synthetically generated text are used to further train the models, this will lead to a spiral of decaying quality as eventually a majority of the training data will have been generated by machines (the models often fail to distinguish synthetic text from genuine human-created text) 92 . However, it will also allow researchers to draw insights from a machine that is learning on data it has generated itself.

The evolution from feature-based to representation learning to generative models reflects increasing maturity in the field of QTA. As models become more complex, the need for careful consideration, ethical oversight and methodological innovation intensifies. The challenge now lies in ensuring that these methodologies align with intellectual and scientific goals, rather than being constrained by their inherent limitations. This growing complexity mirrors the increasing demands of this information-driven society, requiring interdisciplinary collaboration and responsible innovation. Generative models require a nuanced understanding of the complex interplay between language, culture, time and society, and a clear recognition of constraints of the QTA. Researchers must align their tools with intellectual goals and embrace active efforts to address the challenges through optimization strategies. The evolution in QTA emphasizes not only technological advances but also the necessity of aligning the ever-changing landscape of computational methodologies with research questions. By focusing on these areas and embracing the accompanying challenges, the field can build robust, reliable conclusions and move towards more nuanced applications of the text analysis. This progress marks a significant step towards an enriched exploration of textual data, widening the scope for understanding multifaceted relationships. The road ahead calls for a further integration of theory and practice. It is essential that evolution of QTA ensures that technological advancement serves both intellectual curiosity and ethical responsibility, resonating with the multifaceted dynamics of language, culture, time and society 93 .

Balancing size and quality

In QTA, the relationship between data quantity and data quality is often misconceived. Although large data sets serve as the basis for training expansive language models, they are not always required when seeking answers to nuanced research questions. The wide-ranging scope of large data sets can offer comprehensive insights into broad trends and general phenomena. However, this often comes at the cost of a detailed understanding of context-specific occurrences. An issue such as  frequency bias exemplifies this drawback. Using diverse sampling strategies, such as stratified sampling to ensure representation across different social groups and bootstrapping methods to correct for selection bias, can offer a more balanced, contextualized viewpoint. Also, relying on methods such as burst or change-point detection can help to pinpoint moments of interest in data sets with a temporal dimension. Triangulating these methods across multiple smaller data sets can enhance reliability and depth of the analysis.

The design of machine-learning models should account for both the frequency and the significance of individual data points. In other words, the models should be capable of learning not just from repetitive occurrences but also from singular, yet critical, events. This enables the machine to understand rare but important phenomena such as revolutions, seminal publications or watershed individual actions, which would typically be overlooked in a conventional data-driven approach. The capacity to learn from such anomalies can enhance the interpretative depth of the model, enabling them to offer more nuanced insights.

Although textual data have been the mainstay for computational analyses, it is not the only type of data that matters, especially when the research questions involve cultural and societal nuances. Diverse data types including images, audio recordings and even physical artefacts should be integrated into the research to provide a more rounded analysis. Additionally, sourcing data from varied geographical and sociocultural contexts can bring multiple perspectives into the frame, thus offering a multifaceted understanding that textual data from English sources alone cannot capture.

Ethical, practical and efficient models

The evolving landscape of machine learning, specifically with respect to model design and utility, reflects a growing emphasis on efficiency and interpretive value. One notable shift is towards smaller, more energy-efficient models. This transition is motivated by both environmental sustainability and economic pragmatism. With computational costs soaring and the environmental toll becoming untenable, the demand for smaller models that maintain or even exceed the quality of larger models is escalating 94 .

Addressing the data sources used to train models is equally critical, particularly when considering models that will serve research or policy purposes. The provenance and context of data dictate its interpretive value, requiring models to be designed with a hierarchical evaluation of data sources. Such an approach could improve the understanding of a model of the importance of each data type given a specific context, thereby improving the quality and reliability of its analysis. Additionally, it is important to acknowledge the potential ethical and legal challenges within this process, including the exploitation of workers during the data collection and model development.

Transparency remains another pressing issue as these models become integral to research processes. Future iterations should feature a declaration of content that enumerates not only the origin of the data but also its sociocultural and temporal context, preprocessing steps, any known biases, along with the analytical limitations of the model. This becomes especially important for generative models, which may produce misleading or even harmful content if the original data sources are not properly disclosed and understood. Important steps have already been taken with the construction of model cards and data sheets 95 .

Finally, an emergent concern is the risk of feedback loops compromising the quality of machine-learning models. If a model is trained on its own output, errors and biases risk being amplified over time. This necessitates constant vigilance as it poses a threat to the long-term reliability and integrity of AI models. The creation of a gold-standard version of the Internet, not polluted by AI-generated data, is also important 96 .

Refining the methodology and ethos

The rapid advances in QTA, particularly the rise of generative models, have opened up a discourse that transcends mere technological prowess. Although earlier feature-based models require domain expertise and extensive human input before they could be used, generative models can already generate convincing output based on relatively short prompts. This shift raises crucial questions about the interplay between machine capability and human expertise. The notion that advanced algorithms might eventually replace researchers is a common misplaced apprehension. These algorithms and models should be conceived as tools to enhance human scholarship by automating mundane tasks, spawning new research questions and even offering novel pathways for data analysis that might be too complex or time-consuming for human cognition.

This paradigm shift towards augmentative technologies introduces a nuanced problem-solving framework that accommodates the complexities intrinsic to studying human culture and behaviour. The approach of problem decomposition, a cornerstone in computer science, also proves invaluable here, converting overarching research queries into discrete, operationalizable components. These elements can then be addressed individually through specialized algorithms or models, whose results can subsequently be synthesized into a comprehensive answer. As we integrate increasingly advanced tuning methods into generative models — such as prompt engineering, retrieval augmented generation and parameter-efficient fine-tuning — it is important to remember that these models are tools, not replacements. They are most effective when employed as part of a broader research toolkit, in which their strengths can complement traditional scholarly methods.

Consequently, model selection becomes pivotal and should be intricately aligned with the nature of the research inquiry. Unsupervised learning algorithms such as clustering are well suited to exploratory research aimed at pattern identification. Conversely, confirmatory questions, which seek to validate theories or test hypotheses, are better addressed through supervised learning models such as regression.

The importance of a well-crafted interpretation stage cannot be overstated. This is where the separate analytical threads are woven into a comprehensive narrative that explains how the individual findings conjoin to form a cohesive answer to the original research query. However, the lack of standardization across methodologies is a persistent challenge. This absence hinders the reliable comparison of research outcomes across various studies. To remedy this, a shift towards establishing guidelines or best practices is advocated. These need not be rigid frameworks but could be adapted to fit specific research contexts, thereby ensuring methodological rigor alongside innovative freedom.

Reflecting on the capabilities and limitations of current generative models in QTA research is crucial. Beyond recognizing their utility, the blind spots — questions they cannot answer and challenges they have yet to overcome — need to be addressed 97 , 98 . There is a growing need to tailor these models to account for nuances such as frequency bias and to include various perspectives, possibly through more diverse data sets or a polyvocal approach.

In summary, a multipronged approach that synergizes transparent and informed data selection, ethical and critical perspectives on model building and selection, and an explicit and reproducible result interpretation offers a robust framework for tackling intricate research questions. By adopting such a nuanced strategy, we make strides not just in technological capability but also in the rigor, validity and credibility of QTA as a research tool.

Miner, G. Practical Text Mining and Statistical Analysis for Non-structured Text Data Applications (Academic Press, 2012).

Fayyad, U., Piatetsky-Shapiro, G. & Smyth, P. From data mining to knowledge discovery in databases. AI Mag. 17 , 37 (1996).

Google Scholar  

Hand, D. J. Data mining: statistics and more? Am. Stat. 52 , 112–116 (1998).

Article   Google Scholar  

McEnery, T. & Wilson, A. Corpus Linguistics: An Introduction (Edinburgh University Press, 2001).

Manning, C. D. & Schütze, H. Foundations of Statistical Natural Language Processing 1st edn (The MIT Press, 1999).

Manning, C., Raghavan, P. & Schütze, H. Introduction to Information Retrieval 1st edn (Cambridge University Press, 2008).

Wankhade, M., Rao, A. C. S. & Kulkarni, C. A survey on sentiment analysis methods, applications, and challenges. Artif. Intell. Rev. 55 , 5731–5780 (2022).

Jehangir, B., Radhakrishnan, S. & Agarwal, R. A survey on named entity recognition — datasets, tools, and methodologies. Nat. Lang. Process. J. 3 , 100017 (2023).

Fu, S. et al. Clinical concept extraction: a methodology review. J. Biomed. Inform. 109 , 103526 (2020).

Sebastiani, F. Machine learning in automated text categorization. ACM Comput. Surv. 34 , 1–47 (2002).

Talley, E. M. et al. Database of NIH grants using machine-learned categories and graphical clustering. Nat. Meth. 8 , 443–444 (2011).

Bommasani, R. et al. On the opportunities and risks of foundation models. Preprint at https://arxiv.org/abs/2108.07258 (2022).

Blei, D. M., Ng, A. Y. & Jordan, M. I. Latent Dirichlet allocation. J. Mach. Learn. Res. 3 , 993–1022 (2003).

Angelov, D. Top2Vec: distributed representations of topics. Preprint at https://arxiv.org/abs/2008.09470 (2020).

Barron, A. T. J., Huang, J., Spang, R. L. & DeDeo, S. Individuals, institutions, and innovation in the debates of the French Revolution. Proc. Natl Acad. Sci. USA 115 , 4607–4612 (2018).

Article   ADS   Google Scholar  

Mitchell, T. M. Machine Learning 1st edn (McGraw-Hill, 1997).

Vaswani, A. et al. Attention is all you need. in Advances in Neural Information Processing Systems (eds Guyon, I. et al.) Vol. 30 (Curran Associates, Inc., 2017).

Devlin, J., Chang, M.-W., Lee, K. & Toutanova, K. BERT: pre-training of deep bidirectional transformers for language understanding. https://arxiv.org/abs/1810.04805 (2018).

Brown, T. et al. Language models are few-shot learners. in Advances in Neural Information Processing Systems Vol. 33 (eds Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M. & Lin, H.) 1877–1901 (Curran Associates, Inc., 2020).

Touvron, H. et al. Llama 2: open foundation and fine-tuned chat models. Preprint at https://arxiv.org/abs/2307.09288 (2023).

Pedregosa, F. et al. Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12 , 2825–2830 (2011).

MathSciNet   Google Scholar  

Wolf, T. et al. Transformers: state-of-the-art natural language processing. in Proc. 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations 38–45 (Association for Computational Linguistics, Online, 2020).

Demartini, G., Siersdorfer, S., Chelaru, S. & Nejdl, W. Analyzing political trends in the blogosphere. in Proceedings of the International AAAI Conference on Web and Social Media vol. 5 466–469 (AAAI, 2011).

Goldstone, A. & Underwood, T. The quiet transformations of literary studies: what thirteen thousand scholars could tell us. New Lit. Hist. 45 , 359–384 (2014).

Tangherlini, T. R. & Leonard, P. Trawling in the sea of the great unread: sub-corpus topic modeling and humanities research. Poetics 41 , 725–749 (2013).

Mei, Q. & Zhai, C. Discovering evolutionary theme patterns from text: an exploration of temporal text mining. in Proceedings of the Eleventh ACM SIGKDD International Conference on Knowledge Discovery and Data Mining 198–207 (Association for Computing Machinery, 2005).

Frermann, L. & Lapata, M. A Bayesian model of diachronic meaning change. Trans. Assoc. Comput. Linguist. 4 , 31–45 (2016).

Koplenig, A. Analyzing Lexical Change in Diachronic Corpora . PhD thesis, Mannheim https://nbn-resolving.org/urn:nbn:de:bsz:mh39-48905 (2016).

Dubossarsky, H., Weinshall, D. & Grossman, E. Outta control: laws of semantic change and inherent biases in word representation models. in Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing 1136–1145 (Association for Computational Linguistics, 2017).

Dubossarsky, H., Hengchen, S., Tahmasebi, N. & Schlechtweg, D. Time-out: temporal referencing for robust modeling of lexical semantic change. in Proc. 57th Annual Meeting of the Association for Computational Linguistics 457–470 (Association for Computational Linguistics, 2019).

Koplenig, A. Why the quantitative analysis of diachronic corpora that does not consider the temporal aspect of time-series can lead to wrong conclusions. Digit. Scholarsh. Humanit. 32 , 159–168 (2017).

Tahmasebi, N., Borin, L. & Jatowt, A. Survey of computational approaches to lexical semantic change detection. Zenodo https://doi.org/10.5281/zenodo.5040302 (2021).

Bizzoni, Y., Degaetano-Orttlieb, S., Fankhauser, P. & Teich, E. Linguistic variation and change in 250 years of English scientific writing: a data-driven approach. Front. Artif. Intell. 3 , 73 (2020).

Haider, T. & Eger, S. Semantic change and emerging tropes in a large corpus of New High German poetry. in Proc. 1st International Workshop on Computational Approaches to Historical Language Change 216–222 (Association for Computational Linguistics, 2019).

Vylomova, E., Murphy, S. & Haslam, N. Evaluation of semantic change of harm-related concepts in psychology. in Proc. 1st International Workshop on Computational Approaches to Historical Language Change 29–34 (Association for Computational Linguistics, 2019).

Marjanen, J., Pivovarova, L., Zosa, E. & Kurunmäki, J. Clustering ideological terms in historical newspaper data with diachronic word embeddings. in 5th International Workshop on Computational History, HistoInformatics 2019 (CEUR-WS, 2019).

Tripodi, R., Warglien, M., Levis Sullam, S. & Paci, D. Tracing antisemitic language through diachronic embedding projections: France 1789–1914. in Proc. 1st International Workshop on Computational Approaches to Historical Language Change 115–125 (Association for Computational Linguistics, 2019).

Garg, N., Schiebinger, L., Jurafsky, D. & Zou, J. Word embeddings quantify 100 years of gender and ethnic stereotypes. Proc. Natl. Acad. Sci. USA 115 , E3635–E3644 (2018).

Wevers, M. Using word embeddings to examine gender bias in Dutch newspapers, 1950–1990. in Proc. 1st International Workshop on Computational Approaches to Historical Language Change 92–97 (Association for Computational Linguistics, 2019).

Sommerauer, P. & Fokkens, A. Conceptual change and distributional semantic models: an exploratory study on pitfalls and possibilities. in Proc. 1st International Workshop on Computational Approaches to Historical Language Change 223–233 (Association for Computational Linguistics, 2019). This article examines the effects of known pitfalls on digital humanities studies, using embedding models, and proposes guidelines for conducting such studies while acknowledging the need for further research to differentiate between artefacts and actual conceptual changes .

Doermann, D. & Tombre, K. (eds) Handbook of Document Image Processing and Recognition 2014th edn (Springer, 2014).

Yu, D. & Deng, L. Automatic Speech Recognition: A Deep Learning Approach 2015th edn (Springer, 2014).

Dasu, T. & Johnson, T. Exploratory Data Mining and Data Cleaning (John Wiley & Sons, Inc., 2003).

Prabhavalkar, R., Hori, T., Sainath, T. N., Schlüter, R. & Watanabe, S. End-to-end speech recognition: a survey https://arxiv.org/abs/2303.03329 (2023).

Pustejovsky, J. & Stubbs, A. Natural Language Annotation for Machine Learning: A Guide to Corpus-Building for Applications 1st edn (O’Reilly Media, 2012). A hands-on guide to data-intensive humanities research, including the quantitative text analysis, using the Python programming language .

Landis, J. R. & Koch, G. G. The measurement of observer agreement for categorical data. Biometrics 33 , 159–174 (1977).

Gurav, V., Parkar, M. & Kharwar, P. Accessible and ethical data annotation with the application of gamification. in Data Science and Analytics (eds Batra, U., Roy, N. R. & Panda, B.) 68–78 (Springer Singapore, 2020).

Paolacci, G., Chandler, J. & Ipeirotis, P. G. Running experiments on Amazon Mechanical Turk. Judgm. Decis. Mak. 5 , 411–419 (2010).

Bergvall-Kåreborn, B. & Howcroft, D. Amazon mechanical turk and the commodification of labour. New Technol. Work Employ. 29 , 213–223 (2014).

Wilkinson, M. D. et al. The FAIR guiding principles for scientific data management and stewardship. Sci. Data 3 , 160018 (2016).

Klymenko, O., Meisenbacher, S. & Matthes, F. Differential privacy in natural language processing the story so far. in Proc. Fourth Workshop on Privacy in Natural Language Processing 1–11 (Association for Computational Linguistics, 2022).

Lassen, I. M. S., Almasi, M., Enevoldsen, K. & Kristensen-McLachlan, R. D. Detecting intersectionality in NER models: a data-driven approach. in Proc. 7th Joint SIGHUM Workshop on Computational Linguistics for Cultural Heritage, Social Sciences, Humanities and Literature 116–127 (Association for Computational Linguistics, 2023).

DaCy: A Unified Framework for Danish NLP Vol. 2989, 206–216 (CEUR Workshop Proceedings, 2021).

Karsdorp, F., Kestemont, M. & Riddell, A. Humanities Data Analysis: Case Studies with Python (Princeton Univ. Press, 2021).

Ruder, S., Peters, M. E., Swayamdipta, S. & Wolf, T. Transfer learning in natural language processing. in Proc. 2019 Conference of the North American Chapter of the Association for Computational Linguistics : Tutorials 15–18 (Association for Computational Linguistics, 2019). The paper presents an overview of modern transfer learning methods in natural language processing, highlighting their emergence, effectiveness in improving the state of the art across various tasks and potential to become a standard tool in natural language processing .

Malte, A. & Ratadiya, P. Evolution of transfer learning in natural language processing. Preprint at https://arxiv.org/abs/1910.07370 (2019).

Groh, M. Identifying the context shift between test benchmarks and production data. Preprint at https://arxiv.org/abs/2207.01059 (2022).

Wang, H., Li, J., Wu, H., Hovy, E. & Sun, Y. Pre-trained language models and their applications. Engineering 25 , 51–65 (2023). This article provides a comprehensive review of the recent progress and research on pre-trained language models in natural language processing , including their development, impact, challenges and future directions in the field.

Wilks, D. S. On the combination of forecast probabilities for consecutive precipitation periods. Weather Forecast. 5 , 640–650 (1990).

Loughran, T. & McDonald, B. Textual analysis in accounting and finance: a survey. J. Account. Res. 54 , 1187–1230 (2016).

Ribeiro, M. T., Singh, S. & Guestrin, C. ‘Why should I trust you?’: explaining the predictions of any classifier. Preprint at https://arxiv.org/abs/1602.04938 (2016).

Lundberg, S. M. & Lee, S.-I. A unified approach to interpreting model predictions. in Advances in Neural Information Processing Systems Vol. 30 (eds Guyon, I. et al.) 4765–4774 (Curran Associates, Inc., 2017).

Tahmasebi, N. & Hengchen, S. The strengths and pitfalls of large-scale text mining for literary studies. Samlaren 140 , 198–227 (2019).

Jaidka, K., Ahmed, S., Skoric, M. & Hilbert, M. Predicting elections from social media: a three-country, three-method comparative study. Asian J. Commun. 29 , 252–273 (2019).

Underwood, T. Distant Horizons : Digital Evidence and Literary Change (Univ. Chicago Press, 2019).

Jo, E. S. & Algee-Hewitt, M. The long arc of history: neural network approaches to diachronic linguistic change. J. Jpn Assoc. Digit. Humanit. 3 , 1–32 (2018).

Soni, S., Klein, L. F. & Eisenstein, J. Abolitionist networks: modeling language change in nineteenth-century activist newspapers. J. Cultural Anal. 6 , 1–43 (2021).

Perry, C. & Dedeo, S. The cognitive science of extremist ideologies online. Preprint at https://arxiv.org/abs/2110.00626 (2021).

Antoniak, M., Mimno, D. & Levy, K. Narrative paths and negotiation of power in birth stories. Proc. ACM Hum. Comput. Interact. 3 , 1–27 (2019).

Vicinanza, P., Goldberg, A. & Srivastava, S. B. A deep-learning model of prescient ideas demonstrates that they emerge from the periphery. PNAS Nexus 2 , pgac275 (2023). Using deep learning on text data, the study identifies markers of prescient ideas, revealing that groundbreaking thoughts often emerge from the periphery of domains rather than their core.

Adeva, J. G., Atxa, J. P., Carrillo, M. U. & Zengotitabengoa, E. A. Automatic text classification to support systematic reviews in medicine. Exp. Syst. Appl. 41 , 1498–1508 (2014).

Schneider, N., Fechner, N., Landrum, G. A. & Stiefl, N. Chemical topic modeling: exploring molecular data sets using a common text-mining approach. J. Chem. Inf. Model. 57 , 1816–1831 (2017).

Kayi, E. S., Yadav, K. & Choi, H.-A. Topic modeling based classification of clinical reports. in 51st Annual Meeting of the Association for Computational Linguistics Proceedings of the Student Research Workshop 67–73 (Association for Computational Linguistics, 2013).

Roberts, M. E. et al. Structural topic models for open-ended survey responses. Am. J. Political Sci. 58 , 1064–1082 (2014).

Kheiri, K. & Karimi, H. SentimentGPT: exploiting GPT for advanced sentiment analysis and its departure from current machine learning. Preprint at https://arxiv.org/abs/2307.10234 (2023).

Pelaez, S., Verma, G., Ribeiro, B. & Shapira, P. Large-scale text analysis using generative language models: a case study in discovering public value expressions in AI patents. Preprint at https://arxiv.org/abs/2305.10383 (2023).

Rathje, S. et al. GPT is an effective tool for multilingual psychological text analysis. Preprint at https://psyarxiv.com/sekf5/ (2023).

Bollen, J., Mao, H. & Zeng, X. Twitter mood predicts the stock market. J. Comput. Sci. 2 , 1–8 (2011). Analysing large-scale Twitter feeds, the study finds that certain collective mood states can predict daily changes in the Dow Jones Industrial Average with 86.7% accuracy .

Tumasjan, A., Sprenger, T. O., Sandner, P. G. & Welpe, I. M. Election forecasts with twitter: how 140 characters reflect the political landscape. Soc. Sci. Comput. Rev. 29 , 402–418 (2011).

Koppel, M., Schler, J. & Argamon, S. Computational methods in authorship attribution. J. Am. Soc. Inf. Sci. Tech. 60 , 9–26 (2009).

Juola, P. The Rowling case: a proposed standard analytic protocol for authorship questions. Digit. Scholarsh. Humanit. 30 , i100–i113 (2015).

Danielsen, A. A., Fenger, M. H. J., Østergaard, S. D., Nielbo, K. L. & Mors, O. Predicting mechanical restraint of psychiatric inpatients by applying machine learning on electronic health data. Acta Psychiatr. Scand. 140 , 147–157 (2019). The study used machine learning from electronic health data to predict mechanical restraint incidents within 3 days of psychiatric patient admission, achieving an accuracy of 0.87 area under the curve, with most predictive factors coming from clinical text notes .

Rudolph, J., Tan, S. & Tan, S. ChatGPT: bullshit spewer or the end of traditional assessments in higher education? J. Appl. Learn. Teach. 6 , 342–363 (2023).

Park, J. S. et al. Generative agents: interactive Simulacra of human behavior. in Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology (UIST ‘23) 1–22 (Association for Computing Machinery, 2023).

Lucy, L. & Bamman, D. Gender and representation bias in GPT-3 generated stories. in Proc. Third Workshop on Narrative Understanding 48–55 (Association for Computational Linguistics, Virtual, 2021). The paper shows how GPT-3-generated stories exhibit gender stereotypes, associating feminine characters with family and appearance, and showing them as less powerful than masculine characters, prompting concerns about social biases in language models for storytelling.

Mitchell, M. et al. Model cards for model reporting. in Proc. Conference on Fairness, Accountability, and Transparency (Association for Computing Machinery, 2019). The paper introduces model cards for documentation of machine-learning models, detailing their performance characteristics across diverse conditions and contexts to promote transparency and responsible use .

Gebru, T. et al. Datasheets for datasets. Commun. ACM 64 , 86–92 (2021).

Bailer-Jones, D. M. When scientific models represent. Int. Stud. Philos. Sci. 17 , 59–74 (2010).

Guldi, J. The Dangerous Art of Text Mining: A Methodology for Digital History 1st edn (Cambridge Univ. Press, (2023).

Da, N. Z. The computational case against computational literary studies. Crit. Inquiry 45 , 601–639 (2019).

Mäntylä, M. V., Graziotin, D. & Kuutila, M. The evolution of sentiment analysis — a review of research topics, venues, and top cited papers. Comp. Sci. Rev. 27 , 16–32 (2018).

Alemohammad, S. et al. Self-consuming generative models go mad. Preprint at https://arxiv.org/abs/2307.01850 (2023).

Bockting, C. L., van Dis, E. A., van Rooij, R., Zuidema, W. & Bollen, J. Living guidelines for generative AI — why scientists must oversee its use. Nature 622 , 693–696 (2023).

Wu, C.-J. et al. Sustainable AI: environmental implications, challenges and opportunities. in Proceedings of Machine Learning and Systems 4 (MLSys 2022) vol. 4, 795–813 (2022).

Pushkarna, M., Zaldivar, A. & Kjartansson, O. Data cards: purposeful and transparent dataset documentation for responsible AI. in 2022 ACM Conference on Fairness, Accountability, and Transparency 1776–1826 (Association for Computing Machinery, 2022).

Shumailov, I. et al. The curse of recursion: training on generated data makes models forget. Preprint at https://arxiv.org/abs/2305.17493 (2023).

Mitchell, M. How do we know how smart AI systems are? Science https://doi.org/10.1126/science.adj5957 (2023).

Wu, Z. et al. Reasoning or reciting? Exploring the capabilities and limitations of language models through counterfactual tasks. Preprint at https://arxiv.org/abs/2307.02477 (2023).

Birjali, M., Kasri, M. & Beni-Hssane, A. A comprehensive survey on sentiment analysis: approaches, challenges and trends. Knowl. Based Syst. 226 , 107134 (2021).

Acheampong, F. A., Wenyu, C. & Nunoo Mensah, H. Text based emotion detection: advances, challenges, and opportunities. Eng. Rep. 2 , e12189 (2020).

Pauca, V. P., Shahnaz, F., Berry, M. W. & Plemmons, R. J. Text mining using non-negative matrix factorizations. in Proc. 2004 SIAM International Conference on Data Mining 452–456 (Society for Industrial and Applied Mathematics, 2004).

Sharma, A., Amrita, Chakraborty, S. & Kumar, S. Named entity recognition in natural language processing: a systematic review. in Proc. Second Doctoral Symposium on Computational Intelligence (eds Gupta, D., Khanna, A., Kansal, V., Fortino, G. & Hassanien, A. E.) 817–828 (Springer Singapore, 2022).

Nasar, Z., Jaffry, S. W. & Malik, M. K. Named entity recognition and relation extraction: state-of-the-art. ACM Comput. Surv. 54 , 1–39 (2021).

Sedighi, M. Application of word co-occurrence analysis method in mapping of the scientific fields (case study: the field of informetrics). Library Rev. 65 , 52–64 (2016).

El-Kassas, W. S., Salama, C. R., Rafea, A. A. & Mohamed, H. K. Automatic text summarization: a comprehensive survey. Exp. Syst. Appl. 165 , 113679 (2021).

Download references

Acknowledgements

K.L.N. was supported by grants from the Velux Foundation (grant title: FabulaNET) and the Carlsberg Foundation (grant number: CF23-1583). N.T. was supported by the research programme Change is Key! supported by Riksbankens Jubileumsfond (grant number: M21-0021).

Author information

Authors and affiliations.

Center for Humanities Computing, Aarhus University, Aarhus, Denmark

Kristoffer L. Nielbo

Meertens Institute, Royal Netherlands Academy of Arts and Sciences, Amsterdam, The Netherlands

Folgert Karsdorp

Department of History, University of Amsterdam, Amsterdam, The Netherlands

Melvin Wevers

Institute of History, Leiden University, Leiden, The Netherlands

Alie Lassche

Department of Linguistics, Aarhus University, Aarhus, Denmark

Rebekah B. Baglini

Department of Literature, University of Antwerp, Antwerp, Belgium

Mike Kestemont

Department of Philosophy, Linguistics and Theory of Science, University of Gothenburg, Gothenburg, Sweden

Nina Tahmasebi

You can also search for this author in PubMed   Google Scholar

Contributions

Introduction (K.L.N. and F.K.); Experimentation (K.L.N., F.K., M.K. and R.B.B.); Results (F.K., M.K., R.B.B. and N.T.); Applications (K.L.N., M.W. and A.L.); Reproducibility and data deposition (K.L.N. and A.L.); Limitations and optimizations (M.W. and N.T.); Outlook (M.W. and N.T.); overview of the Primer (K.L.N.).

Corresponding author

Correspondence to Kristoffer L. Nielbo .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Peer review

Peer review information.

Nature Reviews Methods Primers thanks F. Jannidis, L. Nelson, T. Tangherlini and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

A set of rules, protocols and tools for building software and applications, which programs can query to obtain data.

A model that represents text as a numerical vector based on word frequency or presence. Each text corresponds to a predefined vocabulary dictionary, with the vector.

Intersection of linguistics, computer science and artificial intelligence that is concerned with computational aspects of human language. It involves the development of algorithms and models that enable computers to understand, interpret and generate human language.

The branch of linguistics that studies language as expressed in corpora (samples of real-world text) and uses computational methods to analyse large collections of textual data.

A technique used to increase the size and diversity of language data sets to train machine-learning models.

The application of statistical, analytical and computational techniques to extract insights and knowledge from data.

( κ ). A statistical measure used to assess the reliability of agreement between multiple raters when assigning categorical ratings to a number of items.

A phenomenon in which elements that are over-represented in a data set receive disproportionate attention or influence in the analysis.

A field of study focused on the science of searching for information within documents and retrieving relevant documents from large databases.

A text normalization technique used in natural language processing in which words are reduced to their base or dictionary form.

In quantitative text analysis, machine learning refers to the application of algorithms and statistical models to enable computers to identify patterns, trends and relationships in textual data without being explicitly programmed. It involves training these models on large data sets to learn and infer from the structure and nuances of language.

A field of artificial intelligence using computational methods for analysing and generating natural language and speech.

A type of information filtering system that seeks to predict user preferences and recommend items (such as books, movies and products) that are likely to be of interest to the user.

A set of techniques in machine learning in which the system learns to automatically identify and extract useful features or representations from raw data.

A text normalization technique used in natural language processing, in which words are reduced to their base or root form.

A machine-learning approach in which models are trained on labelled data, such that each training text is paired with an output label. The model learns to predict the output from the input data, with the aim of generalizing the training set to unseen data.

A deep learning model that handles sequential data, such as text, using mechanisms called attention and self-attention, allowing it to weigh the importance of different parts of the input data. In the quantitative text analysis, transformers are used for tasks such as sentiment analysis, text classification and language translation, offering superior performance in understanding context and nuances in large data sets.

A type of machine learning in which models are trained on data without output labels. The goal is to discover underlying patterns, groupings or structures within the data, often through clustering or dimensionality reduction techniques.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Cite this article.

Nielbo, K.L., Karsdorp, F., Wevers, M. et al. Quantitative text analysis. Nat Rev Methods Primers 4 , 25 (2024). https://doi.org/10.1038/s43586-024-00302-w

Download citation

Accepted : 21 February 2024

Published : 11 April 2024

DOI : https://doi.org/10.1038/s43586-024-00302-w

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing: AI and Robotics newsletter — what matters in AI and robotics research, free to your inbox weekly.

• Subject List
• Take a Tour
• For Authors
• Subscriber Services
• Publications
• African American Studies
• African Studies
• American Literature
• Anthropology
• Architecture Planning and Preservation
• Art History
• Atlantic History
• Biblical Studies
• British and Irish Literature
• Childhood Studies
• Chinese Studies
• Cinema and Media Studies

Communication

• Criminology
• Environmental Science
• Evolutionary Biology
• International Law
• International Relations
• Islamic Studies
• Jewish Studies
• Latin American Studies
• Latino Studies
• Linguistics
• Literary and Critical Theory
• Medieval Studies
• Military History
• Political Science
• Public Health
• Renaissance and Reformation
• Social Work
• Urban Studies
• Victorian Literature
• Browse All Subjects

How to Subscribe

• Free Trials

In This Article Expand or collapse the "in this article" section Textual Analysis and Communication

Introduction, theoretical background.

• General Introductions
• Analytical Strategies
• Methodological Antecedents
• Methodological Debate
• Types of Textual Analysis
• Qualitative Research in Media and Communication Studies

Related Articles Expand or collapse the "related articles" section about

About related articles close popup.

Lorem Ipsum Sit Dolor Amet

Vestibulum ante ipsum primis in faucibus orci luctus et ultrices posuere cubilia Curae; Aliquam ligula odio, euismod ut aliquam et, vestibulum nec risus. Nulla viverra, arcu et iaculis consequat, justo diam ornare tellus, semper ultrices tellus nunc eu tellus.

• Approaches to Multimodal Discourse Analysis
• Critical and Cultural Studies
• Feminist Theory
• Food Studies and Communication
• Gender and the Media
• Rhetoric and Communication
• Symbolic Interactionism in Communication

Other Subject Areas

Forthcoming articles expand or collapse the "forthcoming articles" section.

• Culture Shock and Communication
• LGBTQ+ Family Communication
• Queerbaiting
• Find more forthcoming titles...
• Export Citations
• Share This Facebook LinkedIn Twitter

Textual Analysis and Communication by Elfriede Fürsich LAST REVIEWED: 25 September 2018 LAST MODIFIED: 25 September 2018 DOI: 10.1093/obo/9780199756841-0216

Textual analysis is a qualitative method used to examine content in media and popular culture, such as newspaper articles, television shows, websites, games, videos, and advertising. The method is linked closely to cultural studies. Based on semiotic and interpretive approaches, textual analysis is a type of qualitative analysis that focuses on the underlying ideological and cultural assumptions of a text. In contrast to systematic quantitative content analysis, textual analysis reaches beyond manifest content to understand the prevailing ideologies of a particular historical and cultural moment that make a specific coverage possible. Critical-cultural scholars understand media content and other cultural artifacts as indicators of how realities are constructed and which ideas are accepted as normal. Following the French cultural philosopher Roland Barthes, content is understood as “text,” i.e., not as a fixed entity but as a complex set of discursive strategies that is generated in a special social, political, historic, and cultural context ( Barthes 2013 , cited under Theoretical Background ). Any text can be interpreted in multiple ways; the possibility of multiple meanings within a text is called “polysemy.” The goal of textual analysis is not to find one “true” interpretation—in contrast to traditional hermeneutic approaches to text exegesis—but to explain the variety of possible meanings inscribed in the text. Researchers who use textual analysis do not follow a single established approach but employ a variety of analysis types, such as ideological, genre, narrative, rhetorical, gender, or discourse analysis. Therefore, the term “textual analysis” could also be understood as a collective term for a variety of qualitative, interpretive, and critical content analysis techniques of popular culture artifacts. This method, just as cultural studies itself, draws on an eclectic mix of disciplines, such as anthropology, literary studies, rhetorical criticism, and cultural sociology, along with intellectual traditions, such as semiotics, (post)structuralism, and deconstruction. What distinguishes textual analysis from other forms of qualitative content analysis in the sociological tradition is its critical-cultural focus on power and ideology. Moreover, textual analysts normally do not use linguistic aspects as central evidence (such as in critical discourse analysis), nor do they use a pre-established code book, such as some traditional qualitative content methods. Textual analysis follows an inductive, interpretive approach by finding patterns in the material that lead to “readings” grounded in the back and forth between observation and contextual analysis. Of central interest is the deconstruction of representations (especially but not always of Others with regard to race, class, gender, sexuality, and ability) because these highlight the relationship of media and content to overall ideologies. The method is based in a constructionist framework. For textual analysts, media content does not simply reflect reality; instead, media, popular culture, and society are mutually constituted. Media and popular culture are arenas in which representations and ideas about reality are produced, maintained, and also challenged.

Central to textual analysis is the idea that content as “text” is a coming together of multiple meanings in a specific moment. Barthes 2013 and Barthes 1977 discuss this idea in detail and provide groundbreaking analysis of cultural phenomena in postwar France. Fiske 1987 , Fiske 2010 , and Fiske 2011 provide the standard on how popular culture can be “read,” i.e., interpreted for its ideological assumptions. Because the central aim of textual analysis is to understand how representations are produced in media content, Stuart Hall’s chapter 1 “The Work of Representation” in the renowned textbook Hall, et al. 2013 delivers a compact but comprehensive explanation of representation as a concept. To understand the shift to post-structuralism and concepts, such as discourse, hegemony, and the relationship between language and power, that are central to textual work, one can turn to the works of original theorists such as Foucault 1972 as well as Best and Kellner 1991 for contextualized clarification. Moreover, Deleuze and Guattari 2004 is a foundational post-structural text that radically rethinks the relationship between meaning and practice.

Barthes, Roland. 2013. Mythologies . New York: Hill and Wang.

English translation by Richard Howard and Annette Lavers of the original book published in 1957. Part 1: “Mythologies” consists of a series of short essayistic analyses of cultural phenomena, such as the Blue Guide travel books or advertising for detergents. Part 2: “Myth Today” lays out Barthes’s semiotic-structural approach. Although Barthes later acknowledged the historic contingencies of his interpretations, they remain important as they provided relevant perspective and methodological vocabulary for textual analysis for years to come.

Barthes, Roland. 1977. Image, music, text . Essays selected and translated by Stephen Heath. New York: Hill and Wang.

Classic collection of Barthes’s writing on semiotics and structuralism. For methodological considerations, the chapters “Introduction to the Structural Analysis of Narrative” and “The Death of the Author” are especially relevant.

Best, Steven, and Douglas Kellner. 1991. Postmodern theory: Critical interventions . New York: Guilford.

DOI: 10.1007/978-1-349-21718-2

Accessible introduction to leading postmodern and post-structuralist theorists. For textual analysis, especially the chapter 2 “Foucault and the Critique of Modernity,” chapter 4 “Baudrillard en route to Postmodernity,” and chapter 5 “Lyotard and Postmodern Gaming” provide relevant context for understanding post-structural and postmodern principles.

Deleuze, Gilles, and Félix Guattari. 2004. A thousand plateaus . Translated by Brian Massumi. London and New York: Continuum.

This book is the second part of Deleuze and Guattari’s groundbreaking philosophical project, “Capitalism and Schizophrenia.” Originally published in 1980, it explains central post-structural concepts such as rhizomes, multiplicity, and nomadic thought. Foundational for understanding the production of knowledge and meaning, these ideas have stood the test of time and resonate in networked and digitalized societies in the early 21st century.

Fiske, John. 1987. Television culture . New York: Routledge.

A classic book by Fiske. His “codes of television” (pp. 4–20) explain even for beginning researchers the important relationship among reality, representation, and ideology that is foundational for the textual analysis of any media content even beyond television.

Fiske, John. 2010. Understanding popular culture . 2d ed. London and New York: Routledge.

Important work by Fiske, originally published in 1989, that lays out the theoretical foundations for cultural analysis.

Fiske, John. 2011. Reading the popular . 2d ed. London and New York: Routledge.

Recently reissued companion book to Fiske 2010 . Provides a variety of examples for cultural analysis ranging from Madonna and shopping malls to news and quiz shows.

Foucault, Michel. 1972. The discourse on language. In The archeology of knowledge and the discourse of language . By Michel Foucault, 215–237. Translated by A. M. Sheridan Smith. New York: Pantheon.

Based on the author’s inaugural lecture at the Collège de France in 1970, this appendix provides a fairly succinct introduction to Foucault’s scholarly program and outlines his specific concepts of “discourse.” The author begins to connect discourses to structures of power and knowledge, an argument that becomes more central in his later writings.

Hall, Stuart, Jessica Evans, and Sean Nixon, eds. 2013. Representation: Cultural representations and signifying practices . 2d ed. London and Thousand Oaks, CA: SAGE.

One of the central goals of textual analysis is to understand and interpret media representations. Chapter 1 “The Work of Representation,” by Stuart Hall, is the most comprehensive introduction to this central post-structural cultural studies concept.

back to top

Users without a subscription are not able to see the full content on this page. Please subscribe or login .

Oxford Bibliographies Online is available by subscription and perpetual access to institutions. For more information or to contact an Oxford Sales Representative click here .

• About Communication »
• Meet the Editorial Board »
• Accounting Communication
• Acculturation Processes and Communication
• Action Assembly Theory
• Action-Implicative Discourse Analysis
• Activist Media
• Adherence and Communication
• Adolescence and the Media
• Advertisements, Televised Political
• Advertising
• Advertising, Children and
• Advertising, International
• Advocacy Journalism
• Agenda Setting
• Annenberg, Walter H.
• Apologies and Accounts
• Applied Communication Research Methods
• Argumentation
• Artificial Intelligence (AI) Advertising
• Attitude-Behavior Consistency
• Audience Fragmentation
• Audience Studies
• Authoritarian Societies, Journalism in
• Bakhtin, Mikhail
• Bandwagon Effect
• Baudrillard, Jean
• Blockchain and Communication
• Bourdieu, Pierre
• Brand Equity
• British and Irish Magazine, History of the
• Broadcasting, Public Service
• Capture, Media
• Castells, Manuel
• Celebrity and Public Persona
• Civil Rights Movement and the Media, The
• Co-Cultural Theory and Communication
• Codes and Cultural Discourse Analysis
• Cognitive Dissonance
• Collective Memory, Communication and
• Comedic News
• Communication Apprehension
• Communication Campaigns
• Communication, Definitions and Concepts of
• Communication History
• Communication Law
• Communication Management
• Communication Networks
• Communication, Philosophy of
• Community Attachment
• Community Journalism
• Community Structure Approach
• Computational Journalism
• Computer-Mediated Communication
• Content Analysis
• Corporate Social Responsibility and Communication
• Crisis Communication
• Critical Race Theory and Communication
• Cross-tools and Cross-media Effects
• Cultivation
• Cultural and Creative Industries
• Cultural Imperialism Theories
• Cultural Mapping
• Cultural Persuadables
• Cultural Pluralism and Communication
• Cyberpolitics
• Death, Dying, and Communication
• Debates, Televised
• Deliberation
• Developmental Communication
• Diffusion of Innovations
• Digital Divide
• Digital Gender Diversity
• Digital Intimacies
• Digital Literacy
• Diplomacy, Public
• Distributed Work, Comunication and
• Documentary and Communication
• E-democracy/E-participation
• E-Government
• Elaboration Likelihood Model
• Electronic Word-of-Mouth (eWOM)
• Embedded Coverage
• Entertainment
• Entertainment-Education
• Environmental Communication
• Ethnic Media
• Ethnography of Communication
• Experiments
• Families, Multicultural
• Family Communication
• Federal Communications Commission
• Feminist and Queer Game Studies
• Feminist Data Studies
• Feminist Journalism
• Focus Groups
• Freedom of the Press
• Friendships, Intercultural
• Gatekeeping
• Global Englishes
• Global Media, History of
• Global Media Organizations
• Glocalization
• Goffman, Erving
• Habermas, Jürgen
• Habituation and Communication
• Health Communication
• Hermeneutic Communication Studies
• Homelessness and Communication
• Hook-Up and Dating Apps
• Hostile Media Effect
• Identification with Media Characters
• Identity, Cultural
• Image Repair Theory
• Implicit Measurement
• Impression Management
• Infographics
• Information and Communication Technology for Development
• Information Management
• Information Overload
• Information Processing
• Infotainment
• Innis, Harold
• Instructional Communication
• Integrated Marketing Communications
• Interactivity
• Intercultural Capital
• Intercultural Communication
• Intercultural Communication, Tourism and
• Intercultural Communication, Worldview in
• Intercultural Competence
• Intercultural Conflict Mediation
• Intercultural Dialogue
• Intercultural New Media
• Intergenerational Communication
• Intergroup Communication
• International Communications
• Interpersonal Communication
• Interpersonal LGBTQ Communication
• Interpretation/Reception
• Interpretive Communities
• Journalism, Accuracy in
• Journalism, Alternative
• Journalism and Trauma
• Journalism, Citizen
• Journalism, Citizen, History of
• Journalism Ethics
• Journalism, Interpretive
• Journalism, Peace
• Journalism, Tabloid
• Journalists, Violence against
• Knowledge Gap
• Lazarsfeld, Paul
• Leadership and Communication
• Mass Communication
• McLuhan, Marshall
• Media Activism
• Media Aesthetics
• Media and Time
• Media Convergence
• Media Credibility
• Media Dependency
• Media Ecology
• Media Economics
• Media Economics, Theories of
• Media, Educational
• Media Effects
• Media Ethics
• Media Events
• Media Exposure Measurement
• Media, Gays and Lesbians in the
• Media Literacy
• Media Logic
• Media Management
• Media Policy and Governance
• Media Regulation
• Media, Social
• Media Sociology
• Media Systems Theory
• Merton, Robert K.
• Message Characteristics and Persuasion
• Mobile Communication Studies
• Multimodal Discourse Analysis, Approaches to
• Multinational Organizations, Communication and Culture in
• Murdoch, Rupert
• Narrative Engagement
• Narrative Persuasion
• Net Neutrality
• News Framing
• News Media Coverage of Women
• NGOs, Communication and
• Online Campaigning
• Open Access
• Organizational Change and Organizational Change Communicat...
• Organizational Communication
• Organizational Communication, Aging and
• Parasocial Theory in Communication
• Participation, Civic/Political
• Participatory Action Research
• Patient-Provider Communication
• Peacebuilding and Communication
• Perceived Realism
• Personalized Communication
• Persuasion and Social Influence
• Persuasion, Resisting
• Photojournalism
• Political Advertising
• Political Communication, Normative Analysis of
• Political Economy
• Political Knowledge
• Political Marketing
• Political Scandals
• Political Socialization
• Polls, Opinion
• Product Placement
• Public Interest Communication
• Public Opinion
• Public Relations
• Public Sphere
• Queer Intercultural Communication
• Queer Migration and Digital Media
• Race and Communication
• Racism and Communication
• Radio Studies
• Reality Television
• Reasoned Action Frameworks
• Religion and the Media
• Reporting, Investigative
• Rhetoric and Intercultural Communication
• Rhetoric and Social Movements
• Rhetoric, Religious
• Rhetoric, Visual
• Risk Communication
• Rumor and Communication
• Schramm, Wilbur
• Science Communication
• Scripps, E. W.
• Selective Exposure
• Sense-Making/Sensemaking
• Sesame Street
• Sex in the Media
• Small-Group Communication
• Social Capital
• Social Change
• Social Cognition
• Social Construction
• Social Identity Theory and Communication
• Social Interaction
• Social Movements
• Social Network Analysis
• Social Protest
• Sports Communication
• Stereotypes
• Strategic Communication
• Superdiversity
• Surveillance and Communication
• Synchrony in Intercultural Communication
• Tabloidization
• Telecommunications History/Policy
• Television, Cable
• Textual Analysis and Communication
• Third Culture Kids
• Third-Person Effect
• Time Warner
• Transgender Media Studies
• Transmedia Storytelling
• Two-Step Flow
• United Nations and Communication
• Urban Communication
• Uses and Gratifications
• Video Deficit
• Video Games and Communication
• Violence in the Media
• Virtual Reality and Communication
• Visual Communication
• Web Archiving
• Whistleblowing
• Whiteness Theory in Intercultural Communication
• Youth and Media
• Zines and Communication
• Privacy Policy
• Cookie Policy
• Legal Notice
• Accessibility

Powered by:

• [66.249.64.20|185.194.105.172]
• 185.194.105.172

Table of Contents

Ai, ethics & human agency, collaboration, information literacy, writing process, textual research methods.

Create more agency in your life. Sharpen your critical reading and thinking competencies by engaging in critical analysis of texts. Understand disparities in textual interpretations. Learn different ways to interpret texts.

Synonymous Terms

Textual research methods is also known as scholarship. Other synonyms include

• Textual Analysis
• Scholarly Research
• Academic Research
• Library Research
• Desk Research
• Secondary Research .

Related Concepts: Dialectic ; Hermeneutics ; Semiotics ; Text & Intertextuality

What are textual research methods?

Textual research methods refers to the methods scholars use to interpret texts , to assess knowledge claims , and to develop new knowledge . Any time you are interpreting, learning from, describing, and discussing texts, you are engaging in textual research .

Texts are so ubiquitous in our modern lives that we often engage in textual research informally without thinking consciously about our actions as a form of textual research. For instance, we may

• use a GPS system to navigate a trip
• search the web for pricing and reviews
• read nonverbal language and back channel discourse.

Even simple literacy tasks like looking up how to pay a parking ticket or investigate travel options requires some mastery of textual methods .

There are many different ways to conduct textual research. Academic disciplines and professions have unique approaches to engaging in textual research. For instance, lawyers may engage in textual research by surveying applicable laws, policies, and precedents. In contrast, a clinical psychologist might read peer-reviewed research on personality constructs. Thus, it’s helpful to think of textual research as a suite of practices , a range of options, that are deployed based on the rhetorical situation.

Formal textual research methods are taught in high school, college, and professional workplace settings. At a minimum, teachers in school settings—from middle schools, high schools, to colleges—train students to write with sources (e.g., how to summarize, paraphrase , and quote). Masters and doctoral programs provide discipline-specific training in research methods. As an example, consider legal training in the U.S.: following undergraduate studies, law students enroll in three additional years of training that focus on how to interpret texts from a legal perspective and engage in legal reasoning.

To help develop our competencies as textual researchers, teachers in school settings ask students to

• read/analyze a text to learn and think about existing knowledge , knowledge claims , research questions, hypotheses, theses , information
• speculate and engage in reasoned debate with others about texts and textual interpretations
• share our subjective readings of others’ texts (reader response)
• explore how interpretations of texts are historically and culturally situated and how those interpretations change over time, place, and cultures
• engage in intertextual analysis —e.g., analyze how thinking about a particular topic and knowledge claim evolves over time
• to inform an interpretation of a text
• to understand what a text and past interpretations of a text say about humanity and culture
• develop knowledge and knowledge claims by composing in response to other texts .

Hermeneutics

Textual Research Methods are informed by hermeneutics , a philosophy of interpretation and understanding, and the concept of a hermeneutic circle—i.e., the idea that interpretation is an integrative, reiterative process informed by

• readings change over time in response to the idiosyncrasies of readers, changes in culture and media
• the historical context of the document
• the author’s and reader’s relationship to the historical context.

Related Topics

Textual research methods & empirical research methods.

Textual Research Methods may play a major or minor role in how a text is developed:

• an author may use solely textual methods to develop a text. For example, the author may review peer-reviewed publications and compose an annotated bibliography or review of literature
• an author may rely primarily on empirical methods (e.g., qualitative and quantitative research ) and only conduct textual research during the early stages of research, when trying to define and contextualize the research question, hypothesis, and methods .

Textual Research Methods & Information Literacy

Textual Research Methods are informed by information literacy practices . People can make informed interpretations when they have access to information and can assess the accuracy, authority , currency, purpose , and relevance of a text .

Textual Research Methods & Mindset

Textual Research Methods are also informed by the writer’s mindset : being curious and open to information, especially when it threatens your deeply held values and beliefs, requires intellectual openness , self-regulation , professionalism , and persistence .

Textual Research Methods & Writing with Sources

Textual Research requires some mastery of conventions governing Writing with Sources , particularly the practices surrounding attribution : summarizing, paraphrasing , and citing secondary sources.

Textual Research Methods are informed by other competencies, especially

• hermeneutics
• Information Literacy, especially scholarship as a conversation ;
• Attribution, Citation, & References
• Rhetorical Reasoning , especially Logos, Reasoning ; Rhetorical Stance ; Rhetorical Knowledge
• Flow: Integrate Textual Evidence (Quotes, Paraphrases , Summaries).

Related Articles:

Literary criticism, rhetorical analysis, suggested edits.

• Please select the purpose of your message. * - Corrections, Typos, or Edits Technical Support/Problems using the site Advertising with Writing Commons Copyright Issues I am contacting you about something else
• Your full name
• Your email address *
• Page URL needing edits *
• Phone This field is for validation purposes and should be left unchanged.
• Angela Eward-Mangione

Literary Criticism refers to critical methods for interpreting texts and for substantiating arguments. This article focuses on literary interpretation, which may be called second-level literary criticism. The difference between first-...

• Joseph M. Moxley , Julie Staggers

How can I use rhetorical analysis of texts, people, events, and situations as an interpretive method? Learn how rhetorical analysis can help you understand why people say and do what...

Featured Articles

Academic Writing – How to Write for the Academic Community

Professional Writing – How to Write for the Professional World

Authority – How to Establish Credibility in Speech & Writing

• Architecture and Design
• Asian and Pacific Studies
• Business and Economics
• Classical and Ancient Near Eastern Studies
• Computer Sciences
• Cultural Studies
• Engineering
• General Interest
• Geosciences
• Industrial Chemistry
• Islamic and Middle Eastern Studies
• Jewish Studies
• Library and Information Science, Book Studies
• Life Sciences
• Linguistics and Semiotics
• Literary Studies
• Materials Sciences
• Mathematics
• Social Sciences
• Sports and Recreation
• Theology and Religion
• Publish your article
• The role of authors
• Promoting your article
• Abstracting & indexing
• Publishing Ethics
• Why publish with De Gruyter
• How to publish with De Gruyter
• Our book series
• Our subject areas
• Your digital product at De Gruyter
• Contribute to our reference works
• Product information
• Tools & resources
• Product Information
• Promotional Materials
• Orders and Inquiries
• FAQ for Library Suppliers and Book Sellers
• Repository Policy
• Free access policy
• Open Access agreements
• Database portals
• For Authors
• Customer service
• People + Culture
• Journal Management
• How to join us
• Working at De Gruyter
• Mission & Vision
• De Gruyter Foundation
• De Gruyter Ebound
• Our Responsibility
• Partner publishers

Your purchase has been completed. Your documents are now available to view.

9. Textual Analysis as a Research Method

From the book research methods for english studies.

• Catherine Belsey
• X / Twitter

Supplementary Materials

Please login or register with De Gruyter to order this product.

Chapters in this book (17)

Literature Review: What Is Textual Analysis, and Can We Use It to Measure Corporate Culture?

• First Online: 25 April 2023

Cite this chapter

• Terry Harris 2  

310 Accesses

The textual analysis approach also sometimes referred to as content analysis, computational linguistics, information retrieval, natural language processing, etc., refers to the systematic and objective quantification of the semantic content contained in a body of text. This notion of the parsing text to discover patterns allows for the unearthing of valuable information in text and has a long history and has been applied in many different contexts and disciplines.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

• Available as PDF
• Read on any device
• Instant download
• Own it forever
• Available as EPUB and PDF
• Compact, lightweight edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info
• Durable hardcover edition

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

In later work, Bodnaruk et al. ( 2015 ) adopt a similar methodology to that used in Loughran and McDonald ( 2011 ) and propose “constraining” words, which are those words that suggest some financial constraint, for inclusion in the Loughran and McDonald word list.

Ahern, K. R., & Sosyura, D. (2014). Who Writes the News? Corporate Press Releases During Merger Negotiations. The Journal of Finance, 69 , 241–291.

Article   Google Scholar  

Andreou, P. C., Fiordelisi, F., Harris, T., & Philip, D. (2021). Institutional Ownership and Firms’ Thrust to Compete. British Journal of Management, 33 , 1346–1370.

Andreou, P. C., Harris, T., & Philip, D. (2020). Measuring Firms’ Market Orientation Using Textual Analysis of 10-K Filings. British Journal of Management, 31 , 872–895.

Antweiler, W., & Frank, M. Z. (2004). Is All That Talk Just Noise? The Information Content of Internet Stock Message Boards. The Journal of Finance, 59 , 1259–1294.

Audi, R., Loughran, T., & McDonald, B. (2016). Trust, but Verify: MD&A Language and the Role of Trust in Corporate Culture. Journal of Business Ethics, 139 , 551–561.

Barth, A. (2016). The Role of Corporate Culture in the Financial Industry . Available at SSRN 2707078.

Google Scholar  

Bellstam, G., Bhagat, S., & Cookson, J. A. (2019). A Text-Based Analysis of Corporate Innovation . Available at SSRN 2803232.

Bhandari, A., Mammadov, B., Thevenot, M., & Vakilzadeh, S. H. (2017). The Invisible Hand: Corporate Culture and Its Implications for Earnings Management (Working Paper). Warrington College of Business, Gainesville, FL.

Biddle, G. C., Hilary, G., & Verdi, R. S. (2009). How does Financial Reporting Quality Relate to Investment Efficiency? Journal of Accounting and Economics, 48 , 112–131.

Blei, D. M., Ng, A. Y., & Jordan, M. I. (2003). Latent Dirichlet Allocation. Journal of Machine Learning Research, 3 , 993–1022.

Bodnaruk, A., Loughran, T., & McDonald, B. (2015). Using 10-K Text to Gauge Financial Constraints. Journal of Financial and Quantitative Analysis, 50 , 623–646.

Bonsall, S. B., IV., Leone, A. J., Miller, B. P., & Rennekamp, K. (2017). A Plain English Measure of Financial Reporting Readability. Journal of Accounting and Economics, 63 , 329–357.

Boukus, E., & Rosenberg, J. V. (2006). The Information Content of FOMC Minutes . Available at SSRN 922312.

Bushman, R. M., Hendricks, B. E., & Williams, C. D. (2016). Bank Competition: Measurement, Decision-Making, and Risk-Taking. Journal of Accounting Research, 54 , 777–826.

Chen, H., De, P., Hu, Y. J., & Hwang, B. H. (2014). Wisdom of Crowds: The Value of Stock Opinions Transmitted Through Social Media. The Review of Financial Studies , 27 , 1367–1403.

Cohen, L., Malloy, C., & Nguyen, Q. (2020). Lazy prices. The Journal of Finance, 75 , 1371–1415.

Da, Z., Engelberg, J., & Gao, P. (2015). The Sum of All FEARS Investor Sentiment and Asset Prices. The Review of Financial Studies, 28 , 1–32.

Davis, A. K., Piger, J. M., & Sedor, L. M. (2012). Beyond the Numbers: Measuring the Information Content of Earnings Press Release Language. Contemporary Accounting Research, 29 , 845–868.

Davis, A. K., & Tama-Sweet, I. (2012). Managers’ Use of Language Across Alternative Disclosure Outlets: Earnings Press Releases Versus MD&A. Contemporary Accounting Research, 29 , 804–837.

Das, S. R., & Chen, M. Y. (2007). Yahoo! for Amazon: Sentiment Extraction from Small Talk on the Web. Management Science, 53 , 1375–1388.

Doran, J. S., Peterson, D. R., & Price, S. M. (2012). Earnings Conference Call Content and Stock Price: The Case of REITs. The Journal of Real Estate Finance and Economics, 45 , 402–434.

Dougal, C., Engelberg, J., Garcia, D., & Parsons, C. A. (2012). Journalists and the Stock Market. The Review of Financial Studies, 25 , 639–679.

Dyer, T., Lang, M., & Stice-Lawrence, L. (2017). The Evolution of 10-K Textual Disclosure: Evidence from Latent Dirichlet Allocation. Journal of Accounting and Economics, 64 , 221–245.

Feldman, R., Govindaraj, S., Livnat, J., & Segal, B. (2010). Management’s Tone Change, Post Earnings Announcement Drift and Accruals. Review of Accounting Studies, 15 , 915–953.

Ferris, S. P., Hao, Q., & Liao, M. Y. (2013). The Effect of Issuer Conservatism on IPO Pricing and Performance. Review of Finance, 17 , 993–1027.

Fiordelisi, F., & Ricci, O. (2014). Corporate Culture and CEO Turnover. Journal of Corporate Finance, 28 , 66–82.

Fiordelisi, F., & Ricci, O. (2021). Corrigendum to “Corporate Culture and CEO Turnover”. Journal of Corporate Finance, 70 , 102064.

Fiordelisi, F., Renneboog, L., Ricci, O., & Lopes, S. S. (2019). Creative Corporate Culture and Innovation. Journal of International Financial Markets, Institutions and Money, 63 , 101137.

Garcia, D. (2013). Sentiment During Recessions. The Journal of Finance, 68 , 1267–1300.

Graham, J. R., Harvey, C. R., Grennen, J., & Rajgopal, S. (2022). Corporate Culture: Evidence from the Field. Journal of Financial Economics, 146 , 552–593.

Grennan, J. (2019). A Corporate Culture Channel: How Increased Shareholder Governance Reduces Firm Value . Available at SSRN 2345384.

Guiso, L., Sapienza, P., & Zingales, L. (2015a). Corporate Culture, Societal Culture, and Institutions. American Economic Review, 105 , 336–339.

Guiso, L., Sapienza, P., & Zingales, L. (2015b). The Value of Corporate Culture. Journal of Financial Economics, 117 , 60–76.

Guo, H., Yin, C., & Zeng, Y. (2022). How Does Firm‐Specific Investor Sentiment Affect the Value of Corporate Cash Holdings? British Journal of Management .

Hanley, K. W., & Hoberg, G. (2010). The Information Content of IPO Prospectuses. The Review of Financial Studies, 23 , 2821–2864.

Henry, E. (2008). Are Investors Influenced by How Earnings Press Releases Are Written? The Journal of Business Communication, 45 , 363–407.

Heston, S. L., & Sinha, N. R. (2015). News Versus Sentiment: Predicting Stock Returns from News Stories . Available at SSRN 2311310.

Hillert, A., Niessen-Ruenzi, A., & Ruenzi, S. (2014). Mutual Fund Shareholder Letter Tone-Do Investors Listen? (Working Paper). University of Mannheim, Mannheim, Germany.

Hoberg, G., & Phillips, G. (2010). Product Market Synergies and Competition in Mergers and Acquisitions: A Text-Based Analysis. Review of Financial Studies, 23 , 3773–3811.

Hoberg, G., & Phillips, G. (2016). Text-Based Network Industries and Endogenous Product Differentiation. Journal of Political Economy, 124 , 1423–1465.

Hoberg, G., Phillips, G., & Prabhala, N. (2014). Product Market Threats, Payouts, and Financial Flexibility. The Journal of Finance, 69 , 293–324.

Huang, A. H., Zang, A. Y., & Zheng, R. (2014a). Evidence on the Information Content of Text in Analyst Reports. The Accounting Review, 89 , 2151–2180.

Huang, X., Teoh, S. H., & Zhang, Y. (2014b). Tone Management. The Accounting Review, 89 , 1083–1113.

Kearney, C., & Liu, S. (2014). Textual Sentiment in Finance: A survey of Methods and Models. International Review of Financial Analysis, 33 , 171–185.

Kothari, S. P., Li, X., & Short, J. E. (2009). The Effect of Disclosures by Management, Analysts, and Business Press on Cost of Capital, Return Volatility, and Analyst Forecasts: A Study Using Content Analysis. The Accounting Review, 84 , 1639–1670.

Lawrence, A. (2013). Individual Investors and Financial Disclosure. Journal of Accounting and Economics, 56 , 130–147.

Lehavy, R., Li, F., & Merkley, K. (2011). The Effect of Annual Report Readability on Analyst Following and the Properties of Their Earnings Forecasts. The Accounting Review, 86 , 1087–1115.

Li, E. X., & Ramesh, K. (2009). Market Reaction Surrounding the Filing of Periodic SEC Reports. The Accounting Review, 84 , 1171–1208.

Li, F. (2008). Annual Report Readability, Current Earnings, and Earnings Persistence. Journal of Accounting and Economics, 45 , 221–247.

Li, F. (2010a). Textual Analysis of Corporate Disclosures: A Survey of the Literature. Journal of Accounting Literature, 29 , 143.

Li, F. (2010b). The Information Content of Forward-Looking Statements in Corporate Filings: A Naive Bayesian Machine Learning Approach. Journal of Accounting Research, 48 , 1049–1102.

Li, F. (2011). Managers’ Self-serving Attribution Bias, Overconfidence, and Corporate Financial Policies (Working Paper). University of Michigan, Ann Arbor, MI.

Li, F., Lundholm, R., & Minnis, M. (2013a). A Measure of Competition Based on 10-K Filings. Journal of Accounting Research, 51 , 399–436.

Li, K., Griffin, D., Yue, H., & Zhao, L. (2013b). How Does Culture Influence Corporate Risk-Taking? Journal of Corporate Finance, 23 , 1–22.

Li, K., Liu, X., Mai, F., & Zhang, T. (2020). The Role of Corporate Culture in Bad Times, Evidence from the COVID-19 Pandemic. Journal of Financial and Quantitative Analysis , 1–68.

Li, X. (2010c). The Impacts of Product Market Competition on the Quantity and Quality of Voluntary Disclosures. Review of Accounting Studies, 15 , 663–711.

Loughran, T., McDonald, B., & Yun, H. (2009). A Wolf in Sheep’s Clothing: The Use of Ethics-Related Terms in 10-K Reports. Journal of Business Ethics, 89 , 39–49.

Loughran, T., & McDonald, B. (2011). When Is a Liability Not a Liability? Textual Analysis, Dictionaries, and 10-Ks. The Journal of Finance, 66 , 35–65.

Loughran, T., & McDonald, B. (2013). IPO First-Day Returns, Offer Price Revisions, Volatility, and Form S-1 Language. Journal of Financial Economics, 109 , 307–326.

Loughran, T., & McDonald, B. (2014). Measuring Readability in Financial Disclosures. The Journal of Finance, 69 , 1643–1671.

Loughran, T., & McDonald, B. (2016). Textual Analysis in Accounting and Finance: A Survey. Journal of Accounting Research, 54 , 1187–1230.

Luu, H., Nguyen, L., & Wilson, J. O. S. (2022). Organizational Culture, Competition and Bank Loan Loss Provisions . Available at SSRN 3921043.

Mayew, W. J., & Venkatachalam, M. (2012). The Power of Voice: Managerial Affective States and Future Firm Performance. The Journal of Finance, 67 , 1–43.

Miller, B. P. (2010). The Effects of Reporting Complexity on Small and Large Investor Trading. The Accounting Review, 85 , 2107–2143.

Mosteller, F., & Wallace, D. (1964). Inference and Disputed Authorship: The Federalist . Addison-Wesley.

Nelson, K. K., & Pritchard, A. C. (2007). Litigation Risk and Voluntary Disclosure: The Use of Meaningful Cautionary Language. Available at SSRN 998590.

Nguyen, D. D., Hagendorff, J., & Eshraghi, A. (2018). Does a CEO’s Cultural Heritage Affect Performance Under Competitive Pressure? The Review of Financial Studies, 31 , 97–141.

Nguyen, D. D., Nguyen, L., & Sila, V. (2019). Does Corporate Culture Affect Bank Risk-Taking? Evidence from Loan-Level Data. British Journal of Management, 30 , 106–133.

Price, S. M., Doran, J. S., Peterson, D. R., & Bliss, B. A. (2012). Earnings Conference Calls and Stock Returns: The Incremental Informativeness of Textual Tone. Journal of Banking & Finance, 36 , 992–1011.

Rogers, J. L., Van Buskirk, A., & Zechman, S. L. (2011). Disclosure Tone and Shareholder Litigation. The Accounting Review, 86 , 2155–2183.

Simon, A. (1997). Administrative Behaviour (4th ed.). The Free Press.

Solomon, D. H., Soltes, E., & Sosyura, D. (2014). Winners in the Spotlight: Media Coverage of Fund Holdings as a Driver of Flows. Journal of Financial Economics, 113 , 53–72.

Tetlock, P. C. (2007). Giving Content to Investor Sentiment: The Role of Media in the Stock Market. The Journal of Finance, 62 , 1139–1168.

Tetlock, P. C., Saar-Tsechansky, M., & Macskassy, S. (2008). More Than Words: Quantifying Language to Measure Firms’ Fundamentals. The Journal of Finance, 63 , 1437–1467.

Download references

Author information

Authors and affiliations.

Accounting Department, Durham University, Durham, UK

Terry Harris

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Terry Harris .

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Harris, T. (2023). Literature Review: What Is Textual Analysis, and Can We Use It to Measure Corporate Culture?. In: Competition Culture and Corporate Finance. Palgrave Macmillan, Cham. https://doi.org/10.1007/978-3-031-30156-8_3

Download citation

DOI : https://doi.org/10.1007/978-3-031-30156-8_3

Published : 25 April 2023

Publisher Name : Palgrave Macmillan, Cham

Print ISBN : 978-3-031-30155-1

Online ISBN : 978-3-031-30156-8

eBook Packages : Economics and Finance Economics and Finance (R0)

Share this chapter

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

University Library, University of Illinois at Urbana-Champaign

Text Mining Tools and Methods

• Introduction
• Acquiring Text This link opens in a new window

Choosing a method

Word frequencies, machine learning, network and citation analysis, visualizations.

• Additional Downloadable Tools
• Web-Based Tools
• More resources

The text analysis method you choose will depend on your research question. When choosing a method to use, first consider what you expect to learn from your research and what form you would like your results to take. The methods described below can be combined in different ways during the course of a research project. For example, natural language processing algorithms might reveal the names of people in your text, to which you could apply network analysis to study how the actors are connected. 

Computing word frequencies is a basic building block of higher level textual analysis algorithms, although they can sometimes be revealing in themselves. This can include raw word counts, or calculating the percentage of words in a text or set of texts and comparing that across texts or time. Frequencies can also be counted for "n-grams," or phrases with a certain number (n) of words.

Related Tools in the Scholarly Commons:      

• NVivo  
• SAS Text Miner  

Related Tools Available Online:

• HathiTrust+Bookworm
• Google Ngram Viewer
• Voyant  
• HathiTrust Research Center Analytics

Related Library Guides:

• Plot Lexical Trends using HathiTrust+Bookworm  

Example Project Using Word Frequencies

• Clement, T.E. (2008). ‘ A Thing Not Beginning and Not Ending’: Using Digital Tools to Distant-Read Gertrude Stein’s The Making of Americans .  Literary and Linguistic Computing, vol. 23(3), 361-81. http://doi.org/10.1093/llc/fqn020. 

Text analysis often relies on machine learning, a branch of computer science that trains computers to recognize patterns. There are two kinds of machine learning used in text analysis: supervised learning, where a human helps to train the pattern-detecting model, and unsupervised learning, where the computer finds patterns in text with little human intervention. An example of supervised learning is Naive Bayes Classification. See Natural Language Processing and Topic Modeling for examples of unsupervised machine learning.

Example Project Using Classification (Supervised Machine Learning):

• Horton, R., Morrissey, R., Olsen, M., Roe, G., Voyer, R. (2009). Mining Eighteenth Century Ontologies: Machine Learning and Knowledge Classification in the Encyclopédie . Digital Humanities Quarterly , vol. (3)2. Retrieved from http://www.digitalhumanities.org/dhq/vol/3/2/000044/000044.html.
• Topic Modeling

Topic modeling, a form of machine learning, is a way of identifying patterns and themes in a body of text.  Topic modeling is done by statistical algorithms, such as Latent Dirichlet Allocation, which groups words into "topics" based on which words frequently co-occur in a text.

Related Tools in the Scholarly Commons: 

• R  (also available for free  online )
• Python  (also available for free  online )
• HathiTrust Research Center Analytics
• " Text Mine HathiTrust" on HathiTrust

Example Project using Topic Modeling:

• Mendenhall, R., Brown, N., Black, M., Van Moer, M., Lourentzou, I., Flynn, K., McKee, M., Zerai, A. (2016). Rescuing lost history: Using big data to recover black women's lived experiences.  In Proceedings of XSEDE 2016: Diversity, Big Data, and Science at Scale (Vol. 17-21-July-2016). https://doi.org/10.1145/2949550.2949642.  - Illinois Authors

Natural Language Processing

Natural language processing, a kind of machine learning, is the attempt to use computational methods to extract meaning from free text. Among other things, natural language processing algorithms can derive names of people and places, dates, sentiment, and parts of speech. 

Related Tools Available in the Scholarly Commons:

Related tools available online:     .

• Stanford Natural Language Processing Group  

Related Library Guides: ​

• " Text Mine HathiTrust " on HathiTrust

Example Project using Natural Language Processing:

• Underwood, T., Bamman, D., & Lee, S. (2018). The Transformation of Gender in English-Language Fiction .  Journal of Cultural Analytics. http://doi.org/10.22148/16.019. - Illinois Authors

Network analysis is a method for finding connections between nodes representing people, concepts, sources, and more. These networks are usually visualized into graphs that show the interconnectedness of the nodes.

Citation analysis can be used to discover connections and relationships between various citations of documents and then visualized.

• Gephi  (network analysis)
• VOSViewer  (citation and network analysis)

Example Project:

• Kaufman, M. (2014-2015). Quantifying Kissinger . Retrieved from http://blog.quantifyingkissinger.com/.

Generating visualizations is a way to "see" your data.  Text mining visualization can help researchers see relationships between certain concepts.  An example of a visualization of data can be word clouds, graphs, maps, and other graphics that produce a visual depiction the data.

Related Tools in the Scholarly Commons:

• ATLAS.ti  
• R  (also available for free  online ) 

Related Tools Available Online:        

• HathiTrust Research Center Analytics
• Visualizing your Data
• << Previous: Acquiring Text
• Next: ATLAS.ti >>
• Last Updated: Nov 8, 2023 3:52 PM
• URL: https://guides.library.illinois.edu/textmining

• Open access
• Published: 19 April 2024

A scoping review of continuous quality improvement in healthcare system: conceptualization, models and tools, barriers and facilitators, and impact

• Aklilu Endalamaw 1 , 2 ,
• Resham B Khatri 1 , 3 ,
• Tesfaye Setegn Mengistu 1 , 2 ,
• Daniel Erku 1 , 4 , 5 ,
• Eskinder Wolka 6 ,
• Anteneh Zewdie 6 &
• Yibeltal Assefa 1  

BMC Health Services Research volume  24 , Article number:  487 ( 2024 ) Cite this article

732 Accesses

Metrics details

The growing adoption of continuous quality improvement (CQI) initiatives in healthcare has generated a surge in research interest to gain a deeper understanding of CQI. However, comprehensive evidence regarding the diverse facets of CQI in healthcare has been limited. Our review sought to comprehensively grasp the conceptualization and principles of CQI, explore existing models and tools, analyze barriers and facilitators, and investigate its overall impacts.

This qualitative scoping review was conducted using Arksey and O’Malley’s methodological framework. We searched articles in PubMed, Web of Science, Scopus, and EMBASE databases. In addition, we accessed articles from Google Scholar. We used mixed-method analysis, including qualitative content analysis and quantitative descriptive for quantitative findings to summarize findings and PRISMA extension for scoping reviews (PRISMA-ScR) framework to report the overall works.

A total of 87 articles, which covered 14 CQI models, were included in the review. While 19 tools were used for CQI models and initiatives, Plan-Do-Study/Check-Act cycle was the commonly employed model to understand the CQI implementation process. The main reported purposes of using CQI, as its positive impact, are to improve the structure of the health system (e.g., leadership, health workforce, health technology use, supplies, and costs), enhance healthcare delivery processes and outputs (e.g., care coordination and linkages, satisfaction, accessibility, continuity of care, safety, and efficiency), and improve treatment outcome (reduce morbidity and mortality). The implementation of CQI is not without challenges. There are cultural (i.e., resistance/reluctance to quality-focused culture and fear of blame or punishment), technical, structural (related to organizational structure, processes, and systems), and strategic (inadequate planning and inappropriate goals) related barriers that were commonly reported during the implementation of CQI.

Conclusions

Implementing CQI initiatives necessitates thoroughly comprehending key principles such as teamwork and timeline. To effectively address challenges, it’s crucial to identify obstacles and implement optimal interventions proactively. Healthcare professionals and leaders need to be mentally equipped and cognizant of the significant role CQI initiatives play in achieving purposes for quality of care.

Peer Review reports

Continuous quality improvement (CQI) initiative is a crucial initiative aimed at enhancing quality in the health system that has gradually been adopted in the healthcare industry. In the early 20th century, Shewhart laid the foundation for quality improvement by describing three essential steps for process improvement: specification, production, and inspection [ 1 , 2 ]. Then, Deming expanded Shewhart’s three-step model into ‘plan, do, study/check, and act’ (PDSA or PDCA) cycle, which was applied to management practices in Japan in the 1950s [ 3 ] and was gradually translated into the health system. In 1991, Kuperman applied a CQI approach to healthcare, comprising selecting a process to be improved, assembling a team of expert clinicians that understands the process and the outcomes, determining key steps in the process and expected outcomes, collecting data that measure the key process steps and outcomes, and providing data feedback to the practitioners [ 4 ]. These philosophies have served as the baseline for the foundation of principles for continuous improvement [ 5 ].

Continuous quality improvement fosters a culture of continuous learning, innovation, and improvement. It encourages proactive identification and resolution of problems, promotes employee engagement and empowerment, encourages trust and respect, and aims for better quality of care [ 6 , 7 ]. These characteristics drive the interaction of CQI with other quality improvement projects, such as quality assurance and total quality management [ 8 ]. Quality assurance primarily focuses on identifying deviations or errors through inspections, audits, and formal reviews, often settling for what is considered ‘good enough’, rather than pursuing the highest possible standards [ 9 , 10 ], while total quality management is implemented as the management philosophy and system to improve all aspects of an organization continuously [ 11 ].

Continuous quality improvement has been implemented to provide quality care. However, providing effective healthcare is a complicated and complex task in achieving the desired health outcomes and the overall well-being of individuals and populations. It necessitates tackling issues, including access, patient safety, medical advances, care coordination, patient-centered care, and quality monitoring [ 12 , 13 ], rooted long ago. It is assumed that the history of quality improvement in healthcare started in 1854 when Florence Nightingale introduced quality improvement documentation [ 14 ]. Over the passing decades, Donabedian introduced structure, processes, and outcomes as quality of care components in 1966 [ 15 ]. More comprehensively, the Institute of Medicine in the United States of America (USA) has identified effectiveness, efficiency, equity, patient-centredness, safety, and timeliness as the components of quality of care [ 16 ]. Moreover, quality of care has recently been considered an integral part of universal health coverage (UHC) [ 17 ], which requires initiatives to mobilise essential inputs [ 18 ].

While the overall objective of CQI in health system is to enhance the quality of care, it is important to note that the purposes and principles of CQI can vary across different contexts [ 19 , 20 ]. This variation has sparked growing research interest. For instance, a review of CQI approaches for capacity building addressed its role in health workforce development [ 21 ]. Another systematic review, based on random-controlled design studies, assessed the effectiveness of CQI using training as an intervention and the PDSA model [ 22 ]. As a research gap, the former review was not directly related to the comprehensive elements of quality of care, while the latter focused solely on the impact of training using the PDSA model, among other potential models. Additionally, a review conducted in 2015 aimed to identify barriers and facilitators of CQI in Canadian contexts [ 23 ]. However, all these reviews presented different perspectives and investigated distinct outcomes. This suggests that there is still much to explore in terms of comprehensively understanding the various aspects of CQI initiatives in healthcare.

As a result, we conducted a scoping review to address several aspects of CQI. Scoping reviews serve as a valuable tool for systematically mapping the existing literature on a specific topic. They are instrumental when dealing with heterogeneous or complex bodies of research. Scoping reviews provide a comprehensive overview by summarizing and disseminating findings across multiple studies, even when evidence varies significantly [ 24 ]. In our specific scoping review, we included various types of literature, including systematic reviews, to enhance our understanding of CQI.

This scoping review examined how CQI is conceptualized and measured and investigated models and tools for its application while identifying implementation challenges and facilitators. It also analyzed the purposes and impact of CQI on the health systems, providing valuable insights for enhancing healthcare quality.

Protocol registration and results reporting

Protocol registration for this scoping review was not conducted. Arksey and O’Malley’s methodological framework was utilized to conduct this scoping review [ 25 ]. The scoping review procedures start by defining the research questions, identifying relevant literature, selecting articles, extracting data, and summarizing the results. The review findings are reported using the PRISMA extension for a scoping review (PRISMA-ScR) [ 26 ]. McGowan and colleagues also advised researchers to report findings from scoping reviews using PRISMA-ScR [ 27 ].

Defining the research problems

This review aims to comprehensively explore the conceptualization, models, tools, barriers, facilitators, and impacts of CQI within the healthcare system worldwide. Specifically, we address the following research questions: (1) How has CQI been defined across various contexts? (2) What are the diverse approaches to implementing CQI in healthcare settings? (3) Which tools are commonly employed for CQI implementation ? (4) What barriers hinder and facilitators support successful CQI initiatives? and (5) What effects CQI initiatives have on the overall care quality?

Information source and search strategy

We conducted the search in PubMed, Web of Science, Scopus, and EMBASE databases, and the Google Scholar search engine. The search terms were selected based on three main distinct concepts. One group was CQI-related terms. The second group included terms related to the purpose for which CQI has been implemented, and the third group included processes and impact. These terms were selected based on the Donabedian framework of structure, process, and outcome [ 28 ]. Additionally, the detailed keywords were recruited from the primary health framework, which has described lists of dimensions under process, output, outcome, and health system goals of any intervention for health [ 29 ]. The detailed search strategy is presented in the Supplementary file 1 (Search strategy). The search for articles was initiated on August 12, 2023, and the last search was conducted on September 01, 2023.

Eligibility criteria and article selection

Based on the scoping review’s population, concept, and context frameworks [ 30 ], the population included any patients or clients. Additionally, the concepts explored in the review encompassed definitions, implementation, models, tools, barriers, facilitators, and impacts of CQI. Furthermore, the review considered contexts at any level of health systems. We included articles if they reported results of qualitative or quantitative empirical study, case studies, analytic or descriptive synthesis, any review, and other written documents, were published in peer-reviewed journals, and were designed to address at least one of the identified research questions or one of the identified implementation outcomes or their synonymous taxonomy as described in the search strategy. Based on additional contexts, we included articles published in English without geographic and time limitations. We excluded articles with abstracts only, conference abstracts, letters to editors, commentators, and corrections.

We exported all citations to EndNote x20 to remove duplicates and screen relevant articles. The article selection process includes automatic duplicate removal by using EndNote x20, unmatched title and abstract removal, citation and abstract-only materials removal, and full-text assessment. The article selection process was mainly conducted by the first author (AE) and reported to the team during the weekly meetings. The first author encountered papers that caused confusion regarding whether to include or exclude them and discussed them with the last author (YA). Then, decisions were ultimately made. Whenever disagreements happened, they were resolved by discussion and reconsideration of the review questions in relation to the written documents of the article. Further statistical analysis, such as calculating Kappa, was not performed to determine article inclusion or exclusion.

Data extraction and data items

We extracted first author, publication year, country, settings, health problem, the purpose of the study, study design, types of intervention if applicable, CQI approaches/steps if applicable, CQI tools and procedures if applicable, and main findings using a customized Microsoft Excel form.

Summarizing and reporting the results

The main findings were summarized and described based on the main themes, including concepts under conceptualizing, principles, teams, timelines, models, tools, barriers, facilitators, and impacts of CQI. Results-based convergent synthesis, achieved through mixed-method analysis, involved content analysis to identify the thematic presentation of findings. Additionally, a narrative description was used for quantitative findings, aligning them with the appropriate theme. The authors meticulously reviewed the primary findings from each included material and contextualized these findings concerning the main themes1. This approach provides a comprehensive understanding of complex interventions and health systems, acknowledging quantitative and qualitative evidence.

Search results

A total of 11,251 documents were identified from various databases: SCOPUS ( n  = 4,339), PubMed ( n  = 2,893), Web of Science ( n  = 225), EMBASE ( n  = 3,651), and Google Scholar ( n  = 143). After removing duplicates ( n  = 5,061), 6,190 articles were evaluated by title and abstract. Subsequently, 208 articles were assessed for full-text eligibility. Following the eligibility criteria, 121 articles were excluded, leaving 87 included in the current review (Fig.  1 ).

Article selection process

Operationalizing continuous quality improvement

Continuous Quality Improvement (CQI) is operationalized as a cyclic process that requires commitment to implementation, teamwork, time allocation, and celebrating successes and failures.

CQI is a cyclic ongoing process that is followed reflexive, analytical and iterative steps, including identifying gaps, generating data, developing and implementing action plans, evaluating performance, providing feedback to implementers and leaders, and proposing necessary adjustments [ 31 , 32 , 33 , 34 , 35 , 36 , 37 , 38 ].

CQI requires committing to the philosophy, involving continuous improvement [ 19 , 38 ], establishing a mission statement [ 37 ], and understanding quality definition [ 19 ].

CQI involves a wide range of patient-oriented measures and performance indicators, specifically satisfying internal and external customers, developing quality assurance, adopting common quality measures, and selecting process measures [ 8 , 19 , 35 , 36 , 37 , 39 , 40 ].

CQI requires celebrating success and failure without personalization, leading each team member to develop error-free attitudes [ 19 ]. Success and failure are related to underlying organizational processes and systems as causes of failure rather than blaming individuals [ 8 ] because CQI is process-focused based on collaborative, data-driven, responsive, rigorous and problem-solving statistical analysis [ 8 , 19 , 38 ]. Furthermore, a gap or failure opens another opportunity for establishing a data-driven learning organization [ 41 ].

CQI cannot be implemented without a CQI team [ 8 , 19 , 37 , 39 , 42 , 43 , 44 , 45 , 46 ]. A CQI team comprises individuals from various disciplines, often comprising a team leader, a subject matter expert (physician or other healthcare provider), a data analyst, a facilitator, frontline staff, and stakeholders [ 39 , 43 , 47 , 48 , 49 ]. It is also important to note that inviting stakeholders or partners as part of the CQI support intervention is crucial [ 19 , 38 , 48 ].

The timeline is another distinct feature of CQI because the results of CQI vary based on the implementation duration of each cycle [ 35 ]. There is no specific time limit for CQI implementation, although there is a general consensus that a cycle of CQI should be relatively short [ 35 ]. For instance, a CQI implementation took 2 months [ 42 ], 4 months [ 50 ], 9 months [ 51 , 52 ], 12 months [ 53 , 54 , 55 ], and one year and 5 months [ 49 ] duration to achieve the desired positive outcome, while bi-weekly [ 47 ] and monthly data reviews and analyses [ 44 , 48 , 56 ], and activities over 3 months [ 57 ] have also resulted in a positive outcome.

Continuous quality improvement models and tools

There have been several models are utilized. The Plan-Do-Study/Check-Act cycle is a stepwise process involving project initiation, situation analysis, root cause identification, solution generation and selection, implementation, result evaluation, standardization, and future planning [ 7 , 36 , 37 , 45 , 47 , 48 , 49 , 50 , 51 , 53 , 56 , 57 , 58 , 59 , 60 , 61 , 62 , 63 , 64 , 65 , 66 , 67 , 68 , 69 , 70 ]. The FOCUS-PDCA cycle enhances the PDCA process by adding steps to find and improve a process (F), organize a knowledgeable team (O), clarify the process (C), understand variations (U), and select improvements (S) [ 55 , 71 , 72 , 73 ]. The FADE cycle involves identifying a problem (Focus), understanding it through data analysis (Analyze), devising solutions (Develop), and implementing the plan (Execute) [ 74 ]. The Logic Framework involves brainstorming to identify improvement areas, conducting root cause analysis to develop a problem tree, logically reasoning to create an objective tree, formulating the framework, and executing improvement projects [ 75 ]. Breakthrough series approach requires CQI teams to meet in quarterly collaborative learning sessions, share learning experiences, and continue discussion by telephone and cross-site visits to strengthen learning and idea exchange [ 47 ]. Another CQI model is the Lean approach, which has been conducted with Kaizen principles [ 52 ], 5 S principles, and the Six Sigma model. The 5 S (Sort, Set/Straighten, Shine, Standardize, Sustain) systematically organises and improves the workplace, focusing on sorting, setting order, shining, standardizing, and sustaining the improvement [ 54 , 76 ]. Kaizen principles guide CQI by advocating for continuous improvement, valuing all ideas, solving problems, focusing on practical, low-cost improvements, using data to drive change, acknowledging process defects, reducing variability and waste, recognizing every interaction as a customer-supplier relationship, empowering workers, responding to all ideas, and maintaining a disciplined workplace [ 77 ]. Lean Six Sigma, a CQI model, applies the DMAIC methodology, which involves defining (D) and measuring the problem (M), analyzing root causes (A), improving by finding solutions (I), and controlling by assessing process stability (C) [ 78 , 79 ]. The 5 C-cyclic model (consultation, collection, consideration, collaboration, and celebration), the first CQI framework for volunteer dental services in Aboriginal communities, ensures quality care based on community needs [ 80 ]. One study used meetings involving activities such as reviewing objectives, assigning roles, discussing the agenda, completing tasks, retaining key outputs, planning future steps, and evaluating the meeting’s effectiveness [ 81 ].

Various tools are involved in the implementation or evaluation of CQI initiatives: checklists [ 53 , 82 ], flowcharts [ 81 , 82 , 83 ], cause-and-effect diagrams (fishbone or Ishikawa diagrams) [ 60 , 62 , 79 , 81 , 82 ], fuzzy Pareto diagram [ 82 ], process maps [ 60 ], time series charts [ 48 ], why-why analysis [ 79 ], affinity diagrams and multivoting [ 81 ], and run chart [ 47 , 48 , 51 , 60 , 84 ], and others mentioned in the table (Table  1 ).

Barriers and facilitators of continuous quality improvement implementation

Implementing CQI initiatives is determined by various barriers and facilitators, which can be thematized into four dimensions. These dimensions are cultural, technical, structural, and strategic dimensions.

Continuous quality improvement initiatives face various cultural, strategic, technical, and structural barriers. Cultural dimension barriers involve resistance to change (e.g., not accepting online technology), lack of quality-focused culture, staff reporting apprehensiveness, and fear of blame or punishment [ 36 , 41 , 85 , 86 ]. The technical dimension barriers of CQI can include various factors that hinder the effective implementation and execution of CQI processes [ 36 , 86 , 87 , 88 , 89 ]. Structural dimension barriers of CQI arise from the organization structure, process, and systems that can impede the effective implementation and sustainability of CQI [ 36 , 85 , 86 , 87 , 88 ]. Strategic dimension barriers are, for example, the inability to select proper CQI goals and failure to integrate CQI into organizational planning and goals [ 36 , 85 , 86 , 87 , 88 , 90 ].

Facilitators are also grouped to cultural, structural, technical, and strategic dimensions to provide solutions to CQI barriers. Cultural challenges were addressed by developing a group culture to CQI and other rewards [ 39 , 41 , 80 , 85 , 86 , 87 , 90 , 91 , 92 ]. Technical facilitators are pivotal to improving technical barriers [ 39 , 42 , 53 , 69 , 86 , 90 , 91 ]. Structural-related facilitators are related to improving communication, infrastructure, and systems [ 86 , 92 , 93 ]. Strategic dimension facilitators include strengthening leadership and improving decision-making skills [ 43 , 53 , 67 , 86 , 87 , 92 , 94 , 95 ] (Table  2 ).

Impact of continuous quality improvement

Continuous quality improvement initiatives can significantly impact the quality of healthcare in a wide range of health areas, focusing on improving structure, the health service delivery process and improving client wellbeing and reducing mortality.

Structure components

These are health leadership, financing, workforce, technology, and equipment and supplies. CQI has improved planning, monitoring and evaluation [ 48 , 53 ], and leadership and planning [ 48 ], indicating improvement in leadership perspectives. Implementing CQI in primary health care (PHC) settings has shown potential for maintaining or reducing operation costs [ 67 ]. Findings from another study indicate that the costs associated with implementing CQI interventions per facility ranged from approximately $2,000 to $10,500 per year, with an average cost of approximately $10 to $60 per admitted client [ 57 ]. However, based on model predictions, the average cost savings after implementing CQI were estimated to be $5430 [ 31 ]. CQI can also be applied to health workforce development [ 32 ]. CQI in the institutional system improved medical education [ 66 , 96 , 97 ], human resources management [ 53 ], motivated staffs [ 76 ], and increased staff health awareness [ 69 ], while concerns raised about CQI impartiality, independence, and public accountability [ 96 ]. Regarding health technology, CQI also improved registration and documentation [ 48 , 53 , 98 ]. Furthermore, the CQI initiatives increased cleanliness [ 54 ] and improved logistics, supplies, and equipment [ 48 , 53 , 68 ].

Process and output components

The process component focuses on the activities and actions involved in delivering healthcare services.

Service delivery

CQI interventions improved service delivery [ 53 , 56 , 99 ], particularly a significant 18% increase in the overall quality of service performance [ 48 ], improved patient counselling, adherence to appropriate procedures, and infection prevention [ 48 , 68 ], and optimised workflow [ 52 ].

Coordination and collaboration

CQI initiatives improved coordination and collaboration through collecting and analysing data, onsite technical support, training, supportive supervision [ 53 ] and facilitating linkages between work processes and a quality control group [ 65 ].

Patient satisfaction

The CQI initiatives increased patient satisfaction and improved quality of life by optimizing care quality management, improving the quality of clinical nursing, reducing nursing defects and enhancing the wellbeing of clients [ 54 , 76 , 100 ], although CQI was not associated with changes in adolescent and young adults’ satisfaction [ 51 ].

CQI initiatives reduced medication error reports from 16 to 6 [ 101 ], and it significantly reduced the administration of inappropriate prophylactic antibiotics [ 44 ], decreased errors in inpatient care [ 52 ], decreased the overall episiotomy rate from 44.5 to 33.3% [ 83 ], reduced the overall incidence of unplanned endotracheal extubation [ 102 ], improving appropriate use of computed tomography angiography [ 103 ], and appropriate diagnosis and treatment selection [ 47 ].

Continuity of care

CQI initiatives effectively improve continuity of care by improving client and physician interaction. For instance, provider continuity levels showed a 64% increase [ 55 ]. Modifying electronic medical record templates, scheduling, staff and parental education, standardization of work processes, and birth to 1-year age-specific incentives in post-natal follow-up care increased continuity of care to 74% in 2018 compared to baseline 13% in 2012 [ 84 ].

The CQI initiative yielded enhanced efficiency in the cardiac catheterization laboratory, as evidenced by improved punctuality in procedure starts and increased efficiency in manual sheath-pulls inside [ 78 ].

Accessibility

CQI initiatives were effective in improving accessibility in terms of increasing service coverage and utilization rate. For instance, screening for cigarettes, nutrition counselling, folate prescription, maternal care, immunization coverage [ 53 , 81 , 104 , 105 ], reducing the percentage of non-attending patients to surgery to 0.9% from the baseline 3.9% [ 43 ], increasing Chlamydia screening rates from 29 to 60% [ 45 ], increasing HIV care continuum coverage [ 51 , 59 , 60 ], increasing in the uptake of postpartum long-acting reversible contraceptive use from 6.9% at the baseline to 25.4% [ 42 ], increasing post-caesarean section prophylaxis from 36 to 89% [ 62 ], a 31% increase of kangaroo care practice [ 50 ], and increased follow-up [ 65 ]. Similarly, the QI intervention increased the quality of antenatal care by 29.3%, correct partograph use by 51.7%, and correct active third-stage labour management, a 19.6% improvement from the baseline, but not significantly associated with improvement in contraceptive service uptake [ 61 ].

Timely access

CQI interventions improved the time care provision [ 52 ], and reduced waiting time [ 62 , 74 , 76 , 106 ]. For instance, the discharge process waiting time in the emergency department decreased from 76 min to 22 min [ 79 ]. It also reduced mean postprocedural length of stay from 2.8 days to 2.0 days [ 31 ].

Acceptability

Acceptability of CQI by healthcare providers was satisfactory. For instance, 88% of the faculty, 64% of the residents, and 82% of the staff believed CQI to be useful in the healthcare clinic [ 107 ].

Outcome components

Morbidity and mortality.

CQI efforts have demonstrated better management outcomes among diabetic patients [ 40 ], patients with oral mucositis [ 71 ], and anaemic patients [ 72 ]. It has also reduced infection rate in post-caesarean Sect. [ 62 ], reduced post-peritoneal dialysis peritonitis [ 49 , 108 ], and prevented pressure ulcers [ 70 ]. It is explained by peritonitis incidence from once every 40.1 patient months at baseline to once every 70.8 patient months after CQI [ 49 ] and a 63% reduction in pressure ulcer prevalence within 2 years from 2008 to 2010 [ 70 ]. Furthermore, CQI initiatives significantly reduced in-hospital deaths [ 31 ] and increased patient survival rates [ 108 ]. Figure  2 displays the overall process of the CQI implementations.

The overall mechanisms of continuous quality improvement implementation

In this review, we examined the fundamental concepts and principles underlying CQI, the factors that either hinder or assist in its successful application and implementation, and the purpose of CQI in enhancing quality of care across various health issues.

Our findings have brought attention to the application and implementation of CQI, emphasizing its underlying concepts and principles, as evident in the existing literature [ 31 , 32 , 33 , 34 , 35 , 36 , 39 , 40 , 43 , 45 , 46 ]. Continuous quality improvement has shared with the principles of continuous improvement, such as a customer-driven focus, effective leadership, active participation of individuals, a process-oriented approach, systematic implementation, emphasis on design improvement and prevention, evidence-based decision-making, and fostering partnership [ 5 ]. Moreover, Deming’s 14 principles laid the foundation for CQI principles [ 109 ]. These principles have been adapted and put into practice in various ways: ten [ 19 ] and five [ 38 ] principles in hospitals, five principles for capacity building [ 38 ], and two principles for medication error prevention [ 41 ]. As a principle, the application of CQI can be process-focused [ 8 , 19 ] or impact-focused [ 38 ]. Impact-focused CQI focuses on achieving specific outcomes or impacts, whereas process-focused CQI prioritizes and improves the underlying processes and systems. These principles complement each other and can be utilized based on the objectives of quality improvement initiatives in healthcare settings. Overall, CQI is an ongoing educational process that requires top management’s involvement, demands coordination across departments, encourages the incorporation of views beyond clinical area, and provides non-judgemental evidence based on objective data [ 110 ].

The current review recognized that it was not easy to implement CQI. It requires reasonable utilization of various models and tools. The application of each tool can be varied based on the studied health problem and the purpose of CQI initiative [ 111 ], varied in context, content, structure, and usability [ 112 ]. Additionally, overcoming the cultural, technical, structural, and strategic-related barriers. These barriers have emerged from clinical staff, managers, and health systems perspectives. Of the cultural obstacles, staff non-involvement, resistance to change, and reluctance to report error were staff-related. In contrast, others, such as the absence of celebration for success and hierarchical and rational culture, may require staff and manager involvement. Staff members may exhibit reluctance in reporting errors due to various cultural factors, including lack of trust, hierarchical structures, fear of retribution, and a blame-oriented culture. These challenges pose obstacles to implementing standardized CQI practices, as observed, for instance, in community pharmacy settings [ 85 ]. The hierarchical culture, characterized by clearly defined levels of power, authority, and decision-making, posed challenges to implementing CQI initiatives in public health [ 41 , 86 ]. Although rational culture, a type of organizational culture, emphasizes logical thinking and rational decision-making, it can also create challenges for CQI implementation [ 41 , 86 ] because hierarchical and rational cultures, which emphasize bureaucratic norms and narrow definitions of achievement, were found to act as barriers to the implementation of CQI [ 86 ]. These could be solved by developing a shared mindset and collective commitment, establishing a shared purpose, developing group norms, and cultivating psychological preparedness among staff, managers, and clients to implement and sustain CQI initiatives. Furthermore, reversing cultural-related barriers necessitates cultural-related solutions: development of a culture and group culture to CQI [ 41 , 86 ], positive comprehensive perception [ 91 ], commitment [ 85 ], involving patients, families, leaders, and staff [ 39 , 92 ], collaborating for a common goal [ 80 , 86 ], effective teamwork [ 86 , 87 ], and rewarding and celebrating successes [ 80 , 90 ].

The technical dimension barriers of CQI can include inadequate capitalization of a project and insufficient support for CQI facilitators and data entry managers [ 36 ], immature electronic medical records or poor information systems [ 36 , 86 ], and the lack of training and skills [ 86 , 87 , 88 ]. These challenges may cause the CQI team to rely on outdated information and technologies. The presence of barriers on the technical dimension may challenge the solid foundation of CQI expertise among staff, the ability to recognize opportunities for improvement, a comprehensive understanding of how services are produced and delivered, and routine use of expertise in daily work. Addressing these technical barriers requires knowledge creation activities (training, seminar, and education) [ 39 , 42 , 53 , 69 , 86 , 90 , 91 ], availability of quality data [ 86 ], reliable information [ 92 ], and a manual-online hybrid reporting system [ 85 ].

Structural dimension barriers of CQI include inadequate communication channels and lack of standardized process, specifically weak physician-to-physician synergies [ 36 ], lack of mechanisms for disseminating knowledge and limited use of communication mechanisms [ 86 ]. Lack of communication mechanism endangers sharing ideas and feedback among CQI teams, leading to misunderstandings, limited participation and misinterpretations, and a lack of learning [ 113 ]. Knowledge translation facilitates the co-production of research, subsequent diffusion of knowledge, and the developing stakeholder’s capacity and skills [ 114 ]. Thus, the absence of a knowledge translation mechanism may cause missed opportunities for learning, inefficient problem-solving, and limited creativity. To overcome these challenges, organizations should establish effective communication and information systems [ 86 , 93 ] and learning systems [ 92 ]. Though CQI and knowledge translation have interacted with each other, it is essential to recognize that they are distinct. CQI focuses on process improvement within health care systems, aiming to optimize existing processes, reduce errors, and enhance efficiency.

In contrast, knowledge translation bridges the gap between research evidence and clinical practice, translating research findings into actionable knowledge for practitioners. While both CQI and knowledge translation aim to enhance health care quality and patient outcomes, they employ different strategies: CQI utilizes tools like Plan-Do-Study-Act cycles and statistical process control, while knowledge translation involves knowledge synthesis and dissemination. Additionally, knowledge translation can also serve as a strategy to enhance CQI. Both concepts share the same principle: continuous improvement is essential for both. Therefore, effective strategies on the structural dimension may build efficient and effective steering councils, information systems, and structures to diffuse learning throughout the organization.

Strategic factors, such as goals, planning, funds, and resources, determine the overall purpose of CQI initiatives. Specific barriers were improper goals and poor planning [ 36 , 86 , 88 ], fragmentation of quality assurance policies [ 87 ], inadequate reinforcement to staff [ 36 , 90 ], time constraints [ 85 , 86 ], resource inadequacy [ 86 ], and work overload [ 86 ]. These barriers can be addressed through strengthening leadership [ 86 , 87 ], CQI-based mentoring [ 94 ], periodic monitoring, supportive supervision and coaching [ 43 , 53 , 87 , 92 , 95 ], participation, empowerment, and accountability [ 67 ], involving all stakeholders in decision-making [ 86 , 87 ], a provider-payer partnership [ 64 ], and compensating staff for after-hours meetings on CQI [ 85 ]. The strategic dimension, characterized by a strategic plan and integrated CQI efforts, is devoted to processes that are central to achieving strategic priorities. Roles and responsibilities are defined in terms of integrated strategic and quality-related goals [ 115 ].

The utmost goal of CQI has been to improve the quality of care, which is usually revealed by structure, process, and outcome. After resolving challenges and effectively using tools and running models, the goal of CQI reflects the ultimate reason and purpose of its implementation. First, effectively implemented CQI initiatives can improve leadership, health financing, health workforce development, health information technology, and availability of supplies as the building blocks of a health system [ 31 , 48 , 53 , 68 , 98 ]. Second, effectively implemented CQI initiatives improved care delivery process (counselling, adherence with standards, coordination, collaboration, and linkages) [ 48 , 53 , 65 , 68 ]. Third, the CQI can improve outputs of healthcare delivery, such as satisfaction, accessibility (timely access, utilization), continuity of care, safety, efficiency, and acceptability [ 52 , 54 , 55 , 76 , 78 ]. Finally, the effectiveness of the CQI initiatives has been tested in enhancing responses related to key aspects of the HIV response, maternal and child health, non-communicable disease control, and others (e.g., surgery and peritonitis). However, it is worth noting that CQI initiative has not always been effective. For instance, CQI using a two- to nine-times audit cycle model through systems assessment tools did not bring significant change to increase syphilis testing performance [ 116 ]. This study was conducted within the context of Aboriginal and Torres Strait Islander people’s primary health care settings. Notably, ‘the clinics may not have consistently prioritized syphilis testing performance in their improvement strategies, as facilitated by the CQI program’ [ 116 ]. Additionally, by applying CQI-based mentoring, uptake of facility-based interventions was not significantly improved, though it was effective in increasing community health worker visits during pregnancy and the postnatal period, knowledge about maternal and child health and exclusive breastfeeding practice, and HIV disclosure status [ 117 ]. The study conducted in South Africa revealed no significant association between the coverage of facility-based interventions and Continuous Quality Improvement (CQI) implementation. This lack of association was attributed to the already high antenatal and postnatal attendance rates in both control and intervention groups at baseline, leaving little room for improvement. Additionally, the coverage of HIV interventions remained consistently high throughout the study period [ 117 ].

Regarding health care and policy implications, CQI has played a vital role in advancing PHC and fostering the realization of UHC goals worldwide. The indicators found in Donabedian’s framework that are positively influenced by CQI efforts are comparable to those included in the PHC performance initiative’s conceptual framework [ 29 , 118 , 119 ]. It is clearly explained that PHC serves as the roadmap to realizing the vision of UHC [ 120 , 121 ]. Given these circumstances, implementing CQI can contribute to the achievement of PHC principles and the objectives of UHC. For instance, by implementing CQI methods, countries have enhanced the accessibility, affordability, and quality of PHC services, leading to better health outcomes for their populations. CQI has facilitated identifying and resolving healthcare gaps and inefficiencies, enabling countries to optimize resource allocation and deliver more effective and patient-centered care. However, it is crucial to recognize that the successful implementation of Continuous Quality Improvement (CQI) necessitates optimizing the duration of each cycle, understanding challenges and barriers that extend beyond the health system and settings, and acknowledging that its effectiveness may be compromised if these challenges are not adequately addressed.

Despite abundant literature, there are still gaps regarding the relationship between CQI and other dimensions within the healthcare system. No studies have examined the impact of CQI initiatives on catastrophic health expenditure, effective service coverage, patient-centredness, comprehensiveness, equity, health security, and responsiveness.

Limitations

In conducting this review, it has some limitations to consider. Firstly, only articles published in English were included, which may introduce the exclusion of relevant non-English articles. Additionally, as this review follows a scoping methodology, the focus is on synthesising available evidence rather than critically evaluating or scoring the quality of the included articles.

Continuous quality improvement is investigated as a continuous and ongoing intervention, where the implementation time can vary across different cycles. The CQI team and implementation timelines were critical elements of CQI in different models. Among the commonly used approaches, the PDSA or PDCA is frequently employed. In most CQI models, a wide range of tools, nineteen tools, are commonly utilized to support the improvement process. Cultural, technical, structural, and strategic barriers and facilitators are significant in implementing CQI initiatives. Implementing the CQI initiative aims to improve health system blocks, enhance health service delivery process and output, and ultimately prevent morbidity and reduce mortality. For future researchers, considering that CQI is context-dependent approach, conducting scale-up implementation research about catastrophic health expenditure, effective service coverage, patient-centredness, comprehensiveness, equity, health security, and responsiveness across various settings and health issues would be valuable.

Availability of data and materials

The data used and/or analyzed during the current study are available in this manuscript and/or the supplementary file.

Shewhart WA, Deming WE. Memoriam: Walter A. Shewhart, 1891–1967. Am Stat. 1967;21(2):39–40.

Article   Google Scholar  

Shewhart WA. Statistical method from the viewpoint of quality control. New York: Dover; 1986. ISBN 978-0486652320. OCLC 13822053. Reprint. Originally published: Washington, DC: Graduate School of the Department of Agriculture, 1939.

Moen R, editor Foundation and History of the PDSA Cycle. Asian network for quality conference Tokyo. https://www.deming.org/sites/default/files/pdf/2015/PDSA_History_Ron_MoenPdf . 2009.

Kuperman G, James B, Jacobsen J, Gardner RM. Continuous quality improvement applied to medical care: experiences at LDS hospital. Med Decis Making. 1991;11(4suppl):S60–65.

Article   CAS   PubMed   Google Scholar  

Singh J, Singh H. Continuous improvement philosophy–literature review and directions. Benchmarking: An International Journal. 2015;22(1):75–119.

Goldstone J. Presidential address: Sony, Porsche, and vascular surgery in the 21st century. J Vasc Surg. 1997;25(2):201–10.

Radawski D. Continuous quality improvement: origins, concepts, problems, and applications. J Physician Assistant Educ. 1999;10(1):12–6.

Shortell SM, O’Brien JL, Carman JM, Foster RW, Hughes E, Boerstler H, et al. Assessing the impact of continuous quality improvement/total quality management: concept versus implementation. Health Serv Res. 1995;30(2):377.

CAS   PubMed   PubMed Central   Google Scholar  

Lohr K. Quality of health care: an introduction to critical definitions, concepts, principles, and practicalities. Striving for quality in health care. 1991.

Berwick DM. The clinical process and the quality process. Qual Manage Healthc. 1992;1(1):1–8.

Article   CAS   Google Scholar  

Gift B. On the road to TQM. Food Manage. 1992;27(4):88–9.

CAS   PubMed   Google Scholar  

Greiner A, Knebel E. The core competencies needed for health care professionals. health professions education: A bridge to quality. 2003:45–73.

McCalman J, Bailie R, Bainbridge R, McPhail-Bell K, Percival N, Askew D et al. Continuous quality improvement and comprehensive primary health care: a systems framework to improve service quality and health outcomes. Front Public Health. 2018:6 (76):1–6.

Sheingold BH, Hahn JA. The history of healthcare quality: the first 100 years 1860–1960. Int J Afr Nurs Sci. 2014;1:18–22.

Google Scholar  

Donabedian A. Evaluating the quality of medical care. Milbank Q. 1966;44(3):166–206.

Institute of Medicine (US) Committee on Quality of Health Care in America. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington (DC): National Academies Press (US). 2001. 2, Improving the 21st-century Health Care System. Available from: https://www.ncbi.nlm.nih.gov/books/NBK222265/ .

Rubinstein A, Barani M, Lopez AS. Quality first for effective universal health coverage in low-income and middle-income countries. Lancet Global Health. 2018;6(11):e1142–1143.

Article   PubMed   Google Scholar  

Agency for Healthcare Reserach and Quality. Quality Improvement and monitoring at your fingertips USA,: Agency for Healthcare Reserach and Quality. 2022. Available from: https://qualityindicators.ahrq.gov/ .

Anderson CA, Cassidy B, Rivenburgh P. Implementing continuous quality improvement (CQI) in hospitals: lessons learned from the International Quality Study. Qual Assur Health Care. 1991;3(3):141–6.

Gardner K, Mazza D. Quality in general practice - definitions and frameworks. Aust Fam Physician. 2012;41(3):151–4.

PubMed   Google Scholar  

Loper AC, Jensen TM, Farley AB, Morgan JD, Metz AJ. A systematic review of approaches for continuous quality improvement capacity-building. J Public Health Manage Pract. 2022;28(2):E354.

Hill JE, Stephani A-M, Sapple P, Clegg AJ. The effectiveness of continuous quality improvement for developing professional practice and improving health care outcomes: a systematic review. Implement Sci. 2020;15(1):1–14.

Candas B, Jobin G, Dubé C, Tousignant M, Abdeljelil AB, Grenier S, et al. Barriers and facilitators to implementing continuous quality improvement programs in colonoscopy services: a mixed methods systematic review. Endoscopy Int Open. 2016;4(02):E118–133.

Peters MD, Marnie C, Colquhoun H, Garritty CM, Hempel S, Horsley T, et al. Scoping reviews: reinforcing and advancing the methodology and application. Syst Reviews. 2021;10(1):1–6.

Arksey H, O’Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8(1):19–32.

Tricco AC, Lillie E, Zarin W, O’Brien KK, Colquhoun H, Levac D, et al. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. 2018;169(7):467–73.

McGowan J, Straus S, Moher D, Langlois EV, O’Brien KK, Horsley T, et al. Reporting scoping reviews—PRISMA ScR extension. J Clin Epidemiol. 2020;123:177–9.

Donabedian A. Explorations in quality assessment and monitoring: the definition of quality and approaches to its assessment. Health Administration Press, Ann Arbor. 1980;1.

World Health Organization. Operational framework for primary health care: transforming vision into action. Geneva: World Health Organization and the United Nations Children’s Fund (UNICEF); 2020 [updated 14 December 2020; cited 2023 Nov Oct 17]. Available from: https://www.who.int/publications/i/item/9789240017832 .

The Joanna Briggs Institute. The Joanna Briggs Institute Reviewers’ Manual :2014 edition. Australia: The Joanna Briggs Institute. 2014:88–91.

Rihal CS, Kamath CC, Holmes DR Jr, Reller MK, Anderson SS, McMurtry EK, et al. Economic and clinical outcomes of a physician-led continuous quality improvement intervention in the delivery of percutaneous coronary intervention. Am J Manag Care. 2006;12(8):445–52.

Ade-Oshifogun JB, Dufelmeier T. Prevention and Management of Do not return notices: a quality improvement process for Supplemental staffing nursing agencies. Nurs Forum. 2012;47(2):106–12.

Rubenstein L, Khodyakov D, Hempel S, Danz M, Salem-Schatz S, Foy R, et al. How can we recognize continuous quality improvement? Int J Qual Health Care. 2014;26(1):6–15.

O’Neill SM, Hempel S, Lim YW, Danz MS, Foy R, Suttorp MJ, et al. Identifying continuous quality improvement publications: what makes an improvement intervention ‘CQI’? BMJ Qual Saf. 2011;20(12):1011–9.

Article   PubMed   PubMed Central   Google Scholar  

Sibthorpe B, Gardner K, McAullay D. Furthering the quality agenda in Aboriginal community controlled health services: understanding the relationship between accreditation, continuous quality improvement and national key performance indicator reporting. Aust J Prim Health. 2016;22(4):270–5.

Bennett CL, Crane JM. Quality improvement efforts in oncology: are we ready to begin? Cancer Invest. 2001;19(1):86–95.

VanValkenburgh DA. Implementing continuous quality improvement at the facility level. Adv Ren Replace Ther. 2001;8(2):104–13.

Loper AC, Jensen TM, Farley AB, Morgan JD, Metz AJ. A systematic review of approaches for continuous quality improvement capacity-building. J Public Health Manage Practice. 2022;28(2):E354–361.

Ryan M. Achieving and sustaining quality in healthcare. Front Health Serv Manag. 2004;20(3):3–11.

Nicolucci A, Allotta G, Allegra G, Cordaro G, D’Agati F, Di Benedetto A, et al. Five-year impact of a continuous quality improvement effort implemented by a network of diabetes outpatient clinics. Diabetes Care. 2008;31(1):57–62.

Wakefield BJ, Blegen MA, Uden-Holman T, Vaughn T, Chrischilles E, Wakefield DS. Organizational culture, continuous quality improvement, and medication administration error reporting. Am J Med Qual. 2001;16(4):128–34.

Sori DA, Debelew GT, Degefa LS, Asefa Z. Continuous quality improvement strategy for increasing immediate postpartum long-acting reversible contraceptive use at Jimma University Medical Center, Jimma, Ethiopia. BMJ Open Qual. 2023;12(1):e002051.

Roche B, Robin C, Deleaval PJ, Marti MC. Continuous quality improvement in ambulatory surgery: the non-attending patient. Ambul Surg. 1998;6(2):97–100.

O’Connor JB, Sondhi SS, Mullen KD, McCullough AJ. A continuous quality improvement initiative reduces inappropriate prescribing of prophylactic antibiotics for endoscopic procedures. Am J Gastroenterol. 1999;94(8):2115–21.

Ursu A, Greenberg G, McKee M. Continuous quality improvement methodology: a case study on multidisciplinary collaboration to improve chlamydia screening. Fam Med Community Health. 2019;7(2):e000085.

Quick B, Nordstrom S, Johnson K. Using continuous quality improvement to implement evidence-based medicine. Lippincotts Case Manag. 2006;11(6):305–15 ( quiz 16 – 7 ).

Oyeledun B, Phillips A, Oronsaye F, Alo OD, Shaffer N, Osibo B, et al. The effect of a continuous quality improvement intervention on retention-in-care at 6 months postpartum in a PMTCT Program in Northern Nigeria: results of a cluster randomized controlled study. J Acquir Immune Defic Syndr. 2017;75(Suppl 2):S156–164.

Nyengerai T, Phohole M, Iqaba N, Kinge CW, Gori E, Moyo K, et al. Quality of service and continuous quality improvement in voluntary medical male circumcision programme across four provinces in South Africa: longitudinal and cross-sectional programme data. PLoS ONE. 2021;16(8):e0254850.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Wang J, Zhang H, Liu J, Zhang K, Yi B, Liu Y, et al. Implementation of a continuous quality improvement program reduces the occurrence of peritonitis in PD. Ren Fail. 2014;36(7):1029–32.

Stikes R, Barbier D. Applying the plan-do-study-act model to increase the use of kangaroo care. J Nurs Manag. 2013;21(1):70–8.

Wagner AD, Mugo C, Bluemer-Miroite S, Mutiti PM, Wamalwa DC, Bukusi D, et al. Continuous quality improvement intervention for adolescent and young adult HIV testing services in Kenya improves HIV knowledge. AIDS. 2017;31(Suppl 3):S243–252.

Le RD, Melanson SE, Santos KS, Paredes JD, Baum JM, Goonan EM, et al. Using lean principles to optimise inpatient phlebotomy services. J Clin Pathol. 2014;67(8):724–30.

Manyazewal T, Mekonnen A, Demelew T, Mengestu S, Abdu Y, Mammo D, et al. Improving immunization capacity in Ethiopia through continuous quality improvement interventions: a prospective quasi-experimental study. Infect Dis Poverty. 2018;7:7.

Kamiya Y, Ishijma H, Hagiwara A, Takahashi S, Ngonyani HAM, Samky E. Evaluating the impact of continuous quality improvement methods at hospitals in Tanzania: a cluster-randomized trial. Int J Qual Health Care. 2017;29(1):32–9.

Kibbe DC, Bentz E, McLaughlin CP. Continuous quality improvement for continuity of care. J Fam Pract. 1993;36(3):304–8.

Adrawa N, Ongiro S, Lotee K, Seret J, Adeke M, Izudi J. Use of a context-specific package to increase sputum smear monitoring among people with pulmonary tuberculosis in Uganda: a quality improvement study. BMJ Open Qual. 2023;12(3):1–6.

Hunt P, Hunter SB, Levan D. Continuous quality improvement in substance abuse treatment facilities: how much does it cost? J Subst Abuse Treat. 2017;77:133–40.

Azadeh A, Ameli M, Alisoltani N, Motevali Haghighi S. A unique fuzzy multi-control approach for continuous quality improvement in a radio therapy department. Qual Quantity. 2016;50(6):2469–93.

Memiah P, Tlale J, Shimabale M, Nzyoka S, Komba P, Sebeza J, et al. Continuous quality improvement (CQI) institutionalization to reach 95:95:95 HIV targets: a multicountry experience from the Global South. BMC Health Serv Res. 2021;21(1):711.

Yapa HM, De Neve JW, Chetty T, Herbst C, Post FA, Jiamsakul A, et al. The impact of continuous quality improvement on coverage of antenatal HIV care tests in rural South Africa: results of a stepped-wedge cluster-randomised controlled implementation trial. PLoS Med. 2020;17(10):e1003150.

Dadi TL, Abebo TA, Yeshitla A, Abera Y, Tadesse D, Tsegaye S, et al. Impact of quality improvement interventions on facility readiness, quality and uptake of maternal and child health services in developing regions of Ethiopia: a secondary analysis of programme data. BMJ Open Qual. 2023;12(4):e002140.

Weinberg M, Fuentes JM, Ruiz AI, Lozano FW, Angel E, Gaitan H, et al. Reducing infections among women undergoing cesarean section in Colombia by means of continuous quality improvement methods. Arch Intern Med. 2001;161(19):2357–65.

Andreoni V, Bilak Y, Bukumira M, Halfer D, Lynch-Stapleton P, Perez C. Project management: putting continuous quality improvement theory into practice. J Nurs Care Qual. 1995;9(3):29–37.

Balfour ME, Zinn TE, Cason K, Fox J, Morales M, Berdeja C, et al. Provider-payer partnerships as an engine for continuous quality improvement. Psychiatric Serv. 2018;69(6):623–5.

Agurto I, Sandoval J, De La Rosa M, Guardado ME. Improving cervical cancer prevention in a developing country. Int J Qual Health Care. 2006;18(2):81–6.

Anderson CI, Basson MD, Ali M, Davis AT, Osmer RL, McLeod MK, et al. Comprehensive multicenter graduate surgical education initiative incorporating entrustable professional activities, continuous quality improvement cycles, and a web-based platform to enhance teaching and learning. J Am Coll Surg. 2018;227(1):64–76.

Benjamin S, Seaman M. Applying continuous quality improvement and human performance technology to primary health care in Bahrain. Health Care Superv. 1998;17(1):62–71.

Byabagambi J, Marks P, Megere H, Karamagi E, Byakika S, Opio A, et al. Improving the quality of voluntary medical male circumcision through use of the continuous quality improvement approach: a pilot in 30 PEPFAR-Supported sites in Uganda. PLoS ONE. 2015;10(7):e0133369.

Hogg S, Roe Y, Mills R. Implementing evidence-based continuous quality improvement strategies in an urban Aboriginal Community Controlled Health Service in South East Queensland: a best practice implementation pilot. JBI Database Syst Rev Implement Rep. 2017;15(1):178–87.

Hopper MB, Morgan S. Continuous quality improvement initiative for pressure ulcer prevention. J Wound Ostomy Cont Nurs. 2014;41(2):178–80.

Ji J, Jiang DD, Xu Z, Yang YQ, Qian KY, Zhang MX. Continuous quality improvement of nutrition management during radiotherapy in patients with nasopharyngeal carcinoma. Nurs Open. 2021;8(6):3261–70.

Chen M, Deng JH, Zhou FD, Wang M, Wang HY. Improving the management of anemia in hemodialysis patients by implementing the continuous quality improvement program. Blood Purif. 2006;24(3):282–6.

Reeves S, Matney K, Crane V. Continuous quality improvement as an ideal in hospital practice. Health Care Superv. 1995;13(4):1–12.

Barton AJ, Danek G, Johns P, Coons M. Improving patient outcomes through CQI: vascular access planning. J Nurs Care Qual. 1998;13(2):77–85.

Buttigieg SC, Gauci D, Dey P. Continuous quality improvement in a Maltese hospital using logical framework analysis. J Health Organ Manag. 2016;30(7):1026–46.

Take N, Byakika S, Tasei H, Yoshikawa T. The effect of 5S-continuous quality improvement-total quality management approach on staff motivation, patients’ waiting time and patient satisfaction with services at hospitals in Uganda. J Public Health Afr. 2015;6(1):486.

PubMed   PubMed Central   Google Scholar  

Jacobson GH, McCoin NS, Lescallette R, Russ S, Slovis CM. Kaizen: a method of process improvement in the emergency department. Acad Emerg Med. 2009;16(12):1341–9.

Agarwal S, Gallo J, Parashar A, Agarwal K, Ellis S, Khot U, et al. Impact of lean six sigma process improvement methodology on cardiac catheterization laboratory efficiency. Catheter Cardiovasc Interv. 2015;85:S119.

Rahul G, Samanta AK, Varaprasad G A Lean Six Sigma approach to reduce overcrowding of patients and improving the discharge process in a super-specialty hospital. In 2020 International Conference on System, Computation, Automation and Networking (ICSCAN) 2020 July 3 (pp. 1-6). IEEE

Patel J, Nattabi B, Long R, Durey A, Naoum S, Kruger E, et al. The 5 C model: A proposed continuous quality improvement framework for volunteer dental services in remote Australian Aboriginal communities. Community Dent Oral Epidemiol. 2023;51(6):1150–8.

Van Acker B, McIntosh G, Gudes M. Continuous quality improvement techniques enhance HMO members’ immunization rates. J Healthc Qual. 1998;20(2):36–41.

Horine PD, Pohjala ED, Luecke RW. Healthcare financial managers and CQI. Healthc Financ Manage. 1993;47(9):34.

Reynolds JL. Reducing the frequency of episiotomies through a continuous quality improvement program. CMAJ. 1995;153(3):275–82.

Bunik M, Galloway K, Maughlin M, Hyman D. First five quality improvement program increases adherence and continuity with well-child care. Pediatr Qual Saf. 2021;6(6):e484.

Boyle TA, MacKinnon NJ, Mahaffey T, Duggan K, Dow N. Challenges of standardized continuous quality improvement programs in community pharmacies: the case of SafetyNET-Rx. Res Social Adm Pharm. 2012;8(6):499–508.

Price A, Schwartz R, Cohen J, Manson H, Scott F. Assessing continuous quality improvement in public health: adapting lessons from healthcare. Healthc Policy. 2017;12(3):34–49.

Gage AD, Gotsadze T, Seid E, Mutasa R, Friedman J. The influence of continuous quality improvement on healthcare quality: a mixed-methods study from Zimbabwe. Soc Sci Med. 2022;298:114831.

Chan YC, Ho SJ. Continuous quality improvement: a survey of American and Canadian healthcare executives. Hosp Health Serv Adm. 1997;42(4):525–44.

Balas EA, Puryear J, Mitchell JA, Barter B. How to structure clinical practice guidelines for continuous quality improvement? J Med Syst. 1994;18(5):289–97.

ElChamaa R, Seely AJE, Jeong D, Kitto S. Barriers and facilitators to the implementation and adoption of a continuous quality improvement program in surgery: a case study. J Contin Educ Health Prof. 2022;42(4):227–35.

Candas B, Jobin G, Dubé C, Tousignant M, Abdeljelil A, Grenier S, et al. Barriers and facilitators to implementing continuous quality improvement programs in colonoscopy services: a mixed methods systematic review. Endoscopy Int Open. 2016;4(2):E118–133.

Brandrud AS, Schreiner A, Hjortdahl P, Helljesen GS, Nyen B, Nelson EC. Three success factors for continual improvement in healthcare: an analysis of the reports of improvement team members. BMJ Qual Saf. 2011;20(3):251–9.

Lee S, Choi KS, Kang HY, Cho W, Chae YM. Assessing the factors influencing continuous quality improvement implementation: experience in Korean hospitals. Int J Qual Health Care. 2002;14(5):383–91.

Horwood C, Butler L, Barker P, Phakathi S, Haskins L, Grant M, et al. A continuous quality improvement intervention to improve the effectiveness of community health workers providing care to mothers and children: a cluster randomised controlled trial in South Africa. Hum Resour Health. 2017;15(1):39.

Hyrkäs K, Lehti K. Continuous quality improvement through team supervision supported by continuous self-monitoring of work and systematic patient feedback. J Nurs Manag. 2003;11(3):177–88.

Akdemir N, Peterson LN, Campbell CM, Scheele F. Evaluation of continuous quality improvement in accreditation for medical education. BMC Med Educ. 2020;20(Suppl 1):308.

Barzansky B, Hunt D, Moineau G, Ahn D, Lai CW, Humphrey H, et al. Continuous quality improvement in an accreditation system for undergraduate medical education: benefits and challenges. Med Teach. 2015;37(11):1032–8.

Gaylis F, Nasseri R, Salmasi A, Anderson C, Mohedin S, Prime R, et al. Implementing continuous quality improvement in an integrated community urology practice: lessons learned. Urology. 2021;153:139–46.

Gaga S, Mqoqi N, Chimatira R, Moko S, Igumbor JO. Continuous quality improvement in HIV and TB services at selected healthcare facilities in South Africa. South Afr J HIV Med. 2021;22(1):1202.

Wang F, Yao D. Application effect of continuous quality improvement measures on patient satisfaction and quality of life in gynecological nursing. Am J Transl Res. 2021;13(6):6391–8.

Lee SB, Lee LL, Yeung RS, Chan J. A continuous quality improvement project to reduce medication error in the emergency department. World J Emerg Med. 2013;4(3):179–82.

Chiang AA, Lee KC, Lee JC, Wei CH. Effectiveness of a continuous quality improvement program aiming to reduce unplanned extubation: a prospective study. Intensive Care Med. 1996;22(11):1269–71.

Chinnaiyan K, Al-Mallah M, Goraya T, Patel S, Kazerooni E, Poopat C, et al. Impact of a continuous quality improvement initiative on appropriate use of coronary CT angiography: results from a multicenter, statewide registry, the advanced cardiovascular imaging consortium (ACIC). J Cardiovasc Comput Tomogr. 2011;5(4):S29–30.

Gibson-Helm M, Rumbold A, Teede H, Ranasinha S, Bailie R, Boyle J. A continuous quality improvement initiative: improving the provision of pregnancy care for Aboriginal and Torres Strait Islander women. BJOG: Int J Obstet Gynecol. 2015;122:400–1.

Bennett IM, Coco A, Anderson J, Horst M, Gambler AS, Barr WB, et al. Improving maternal care with a continuous quality improvement strategy: a report from the interventions to minimize preterm and low birth weight infants through continuous improvement techniques (IMPLICIT) network. J Am Board Fam Med. 2009;22(4):380–6.

Krall SP, Iv CLR, Donahue L. Effect of continuous quality improvement methods on reducing triage to thrombolytic interval for Acute myocardial infarction. Acad Emerg Med. 1995;2(7):603–9.

Swanson TK, Eilers GM. Physician and staff acceptance of continuous quality improvement. Fam Med. 1994;26(9):583–6.

Yu Y, Zhou Y, Wang H, Zhou T, Li Q, Li T, et al. Impact of continuous quality improvement initiatives on clinical outcomes in peritoneal dialysis. Perit Dial Int. 2014;34(Suppl 2):S43–48.

Schiff GD, Goldfield NI. Deming meets Braverman: toward a progressive analysis of the continuous quality improvement paradigm. Int J Health Serv. 1994;24(4):655–73.

American Hospital Association Division of Quality Resources Chicago, IL: The role of hospital leadership in the continuous improvement of patient care quality. American Hospital Association. J Healthc Qual. 1992;14(5):8–14,22.

Scriven M. The Logic and Methodology of checklists [dissertation]. Western Michigan University; 2000.

Hales B, Terblanche M, Fowler R, Sibbald W. Development of medical checklists for improved quality of patient care. Int J Qual Health Care. 2008;20(1):22–30.

Vermeir P, Vandijck D, Degroote S, Peleman R, Verhaeghe R, Mortier E, et al. Communication in healthcare: a narrative review of the literature and practical recommendations. Int J Clin Pract. 2015;69(11):1257–67.

Eljiz K, Greenfield D, Hogden A, Taylor R, Siddiqui N, Agaliotis M, et al. Improving knowledge translation for increased engagement and impact in healthcare. BMJ open Qual. 2020;9(3):e000983.

O’Brien JL, Shortell SM, Hughes EF, Foster RW, Carman JM, Boerstler H, et al. An integrative model for organization-wide quality improvement: lessons from the field. Qual Manage Healthc. 1995;3(4):19–30.

Adily A, Girgis S, D’Este C, Matthews V, Ward JE. Syphilis testing performance in Aboriginal primary health care: exploring impact of continuous quality improvement over time. Aust J Prim Health. 2020;26(2):178–83.

Horwood C, Butler L, Barker P, Phakathi S, Haskins L, Grant M, et al. A continuous quality improvement intervention to improve the effectiveness of community health workers providing care to mothers and children: a cluster randomised controlled trial in South Africa. Hum Resour Health. 2017;15:1–11.

Veillard J, Cowling K, Bitton A, Ratcliffe H, Kimball M, Barkley S, et al. Better measurement for performance improvement in low- and middle-income countries: the primary Health Care Performance Initiative (PHCPI) experience of conceptual framework development and indicator selection. Milbank Q. 2017;95(4):836–83.

Barbazza E, Kringos D, Kruse I, Klazinga NS, Tello JE. Creating performance intelligence for primary health care strengthening in Europe. BMC Health Serv Res. 2019;19(1):1006.

Assefa Y, Hill PS, Gilks CF, Admassu M, Tesfaye D, Van Damme W. Primary health care contributions to universal health coverage. Ethiopia Bull World Health Organ. 2020;98(12):894.

Van Weel C, Kidd MR. Why strengthening primary health care is essential to achieving universal health coverage. CMAJ. 2018;190(15):E463–466.

Download references

Acknowledgements

Not applicable.

The authors received no fund.

Author information

Authors and affiliations.

School of Public Health, The University of Queensland, Brisbane, Australia

Aklilu Endalamaw, Resham B Khatri, Tesfaye Setegn Mengistu, Daniel Erku & Yibeltal Assefa

College of Medicine and Health Sciences, Bahir Dar University, Bahir Dar, Ethiopia

Aklilu Endalamaw & Tesfaye Setegn Mengistu

Health Social Science and Development Research Institute, Kathmandu, Nepal

Resham B Khatri

Centre for Applied Health Economics, School of Medicine, Grifth University, Brisbane, Australia

Daniel Erku

Menzies Health Institute Queensland, Grifth University, Brisbane, Australia

International Institute for Primary Health Care in Ethiopia, Addis Ababa, Ethiopia

Eskinder Wolka & Anteneh Zewdie

You can also search for this author in PubMed   Google Scholar

Contributions

AE conceptualized the study, developed the first draft of the manuscript, and managing feedbacks from co-authors. YA conceptualized the study, provided feedback, and supervised the whole processes. RBK provided feedback throughout. TSM provided feedback throughout. DE provided feedback throughout. EW provided feedback throughout. AZ provided feedback throughout. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Aklilu Endalamaw .

Ethics declarations

Ethics approval and consent to participate.

Not applicable because this research is based on publicly available articles.

Consent for publication

Competing interests.

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary material 1., supplementary material 2., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Endalamaw, A., Khatri, R.B., Mengistu, T.S. et al. A scoping review of continuous quality improvement in healthcare system: conceptualization, models and tools, barriers and facilitators, and impact. BMC Health Serv Res 24 , 487 (2024). https://doi.org/10.1186/s12913-024-10828-0

Download citation

Received : 27 December 2023

Accepted : 05 March 2024

Published : 19 April 2024

DOI : https://doi.org/10.1186/s12913-024-10828-0

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Continuous quality improvement
• Quality of Care

BMC Health Services Research

ISSN: 1472-6963

• General enquiries: [email protected]

• Open access
• Published: 26 April 2024

3D visualization technology for Learning human anatomy among medical students and residents: a meta- and regression analysis

• Junming Wang 1 , 2   na1 ,
• Wenjun Li 1 , 2   na1 ,
• Aishe Dun 3   na1 ,
• Ning Zhong 1 &
• Zhen Ye 1  

BMC Medical Education volume  24 , Article number:  461 ( 2024 ) Cite this article

Metrics details

3D visualization technology applies computers and other devices to create a realistic virtual world for individuals with various sensory experiences such as 3D vision, touch, and smell to gain a more effective understanding of the relationships between real spatial structures and organizations. The purpose of this study was to comprehensively evaluate the effectiveness of 3D visualization technology in human anatomy teaching/training and explore the potential factors that affect the training effects to better guide the teaching of classroom/laboratory anatomy.

We conducted a meta-analysis of randomized controlled studies on teaching human anatomy using 3D visualization technology. We extensively searched three authoritative databases, PubMed, Web of Science, and Embase; the main outcomes were the participants’ test scores and satisfaction, while the secondary outcomes were time consumption and enjoyment. Heterogeneity by I² was statistically determined because I²> 50%; therefore, a random-effects model was employed, using data processing software such as RevMan, Stata, and VOSviewer to process data, apply standardized mean difference and 95% confidence interval, and subgroup analysis to evaluate test results, and then conduct research through sensitivity analysis and meta-regression analysis.

Thirty-nine randomized controlled trials (2,959 participants) were screened and included in this study. The system analysis of the main results showed that compared with other methods, including data from all regions 3D visualization technology moderately improved test scores as well as satisfaction and enjoyment; however, the time that students took to complete the test was not significantly reduced. Meta-regression analysis also showed that regional factorsaffected test scores, whereas other factors had no significant impact. When the literature from China was excluded, the satisfaction and happiness of the 3D virtual-reality group were statistically significant compared to those of the traditional group; however, the test results and time consumption were not statistically significant.

3D visualization technology is an effective way to improve learners’ satisfaction with and enjoyment of human anatomical learning, but it cannot reduce the time required for testers to complete the test. 3D visualization technology may struggle to improve the testers’ scores. The literature test results from China are more prone to positive results and affected by regional bias.

Peer Review reports

Introduction

Human anatomy is a compulsory course for both clinical and medical students; it is important for clinicians—especially surgeons—to master the anatomy of the human body. However, inadequate anatomical knowledge among medical students and young residents has been reported [ 1 ]. This occurs for several reasons: limited teaching time in anatomy in undergraduate education, which is associated with increased costs; limited availability of cadavers; and reduced exposure to traditional autopsies [ 2 ]. Traditional learning of anatomy is based on elements such as cadaver specimen, regional/ topography anatomical models, and two-dimensional atlases. Two-dimensional atlases lack a sense of three-dimensional space, and it is difficult to reflect the real spatial structure and relationship among organizations. Autopsies offer a complete visual and tactile experience of anatomical learning that is essentially three-dimensional. The traditional cadaverspecimen has the following shortcomings: shortage of cadaver sources; irritant of antiseptic reagent; and nerves and blood vessels lacking a clear holistic view. Features such as stereo vision, dynamic exploration, and tactile feedback are essential for three-dimensional anatomy [ 3 ]. As patients are 3D objects, medical treatment and education involve learning and applying 3D information.

Therefore, digital 3D visualization technology using computer imaging has shown potential educational value, owing to its high fidelity to organizations. 3D visualization technologies include virtual reality (VR), augmented reality (AR), and mixed reality (MR), among others. VR is a process of visualizing a computer-generated environment in an interactive manner using software and hardware [ 4 ].AR is an experience that involves superimposition of digital elements such as graphics, audio, and other sensory enhancements onto video streams of the real world with real-time interaction between the user and the digital elements. Although VR replaces the real-world environment with a virtual world, AR supplements a user’s perception of the real world in an immersive manner without obscuring it completely [ 5 ].MR is a hybrid of the real and virtual worlds. MR is created when computer processing combines the user’s inputs and environment to create an immersive environment in which physical and virtual objects coexist and interact in real time [ 6 ].An example of this technology is the superposition of information or 3D models onto a head-mounted display (HMD); however, MR HMDs do not obfuscate the real world [ 5 ].

The AR, VR, and MR concepts can be distinguished based on three criteria: immersion, interaction, and information [ 7 ]. Immersion refers to the nature of the user experience brought by the technology. Although VR provides an entirely immersive virtual experience, AR augments a real-world view using virtual information. MR performs spatial mapping between the real and virtual worlds in real time. Interaction refers to the types of interactions that are feasible through the use of technology. VR allows interactions with virtual objects, whereas AR enables interactions with physical objects. MR allows interactions with both physical and virtual objects. Information refers to the type of data handled during visualization. In VR, a displayed virtual object is registered in a virtual 3D space. AR provides virtual annotations in real time within a user’s environment. In MR, the displayed virtual object is registered in 3D space and time, with a correlation to the user’s environment in the real world [ 5 ].

In contrast to 2D imaging methods, such as textbook diagrams, photographs, digital CT, and MRI scans, the most obvious advantage of 3D visualization is its ability to view the spatial relationships between different anatomical structures from numerous viewpoints and angles. While diagrams and radiological imaging provide static snapshots, videos using 3D visualization offer the possibility of a narrative timeline where students can pause, rewind, or fast-forward at their convenience. In addition, through 3D image reconstruction, the two-dimensional information contained in images generated by CT, MRI, X-ray, etc. can be converted into three-dimensional information, thereby helping doctors restore the three-dimensional shape of various tissues [ 8 ].3D images alsooffer the ability to rotate, flip, and invert a viewed structure [ 9 ].

To better grasp anatomical knowledge, physical3D printing technology was introduced in our previous two studies [ 10 , 11 ], and it can be used as a good auxiliary tool to learn anatomy structure. In our previous meta-analysis, for achievement tests, we found no statistical difference between the 3D printing model group and the 3D visualization group [ 10 ].

Despite the surge in the use of digital 3D visualization technology with computer imaging in medical anatomy education, a comprehensive evaluation of its effectiveness through randomized trials is lacking [ 12 ]. To better understand the effectiveness of digital 3D visualization technology using computer imaging in anatomy teaching, we systematically evaluated published literature to better guide anatomy teaching.

Given the above considerations, this study aims to

Provide a comprehensive summary of research evaluating the educational effectiveness (test scores and time consumption) of 3D visualization technology in medical anatomy education compared to conventional teaching methods.

Provide a comprehensive summary of research evaluating the popularity (satisfaction and enjoyment) of 3D visualization technology in medical anatomy education compared to conventional teaching methods.

Explore the potential factors affecting the effect of 3D visualization application.

There have been many studies on 3D visualization technology, but their conclusions and results differ. Our study differs from previous studies, and we have proposed new goals and perspectives and hope to provide a reference for future research on this technology.

Materials and methods

This meta-analysis complies with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines [ 13 ].

Literature search

PubMed, Web of Science, and EBSCO (with subset databases including MEDLINE Ultimate, MEDLINE, Academic Search Premier, APA Psyclnfo, and ERIC) were searched for relevant literature. The search keywords were as follows: (“3D” or “Three-dimensional”) and (“visualization” or “visuospatial” or “stereoscop” or “stereocept” or “stereopsis” or “stereoscopic vision” or “virtual reality”) and (“medical” or “medicine”) and(“education” or “teaching”) and (“students” or “residents”) and (“group” or “study”) and (“anatomy” or “dissect”). The language setting for the literature searched was English. The deadline for publication of the included studies was July 2023.

Literature screening and data extraction

We set the inclusion criteria for the literature to be included as follows: (1) types of research: randomized controlled trials; (2) research objects: medical students or residents; (3) intervention measures: in the experimental group, 3D imaging was used to display the anatomical structure; (4) outcome index: test scores, satisfaction, time consumption, enjoyment; (5) research articles on the teaching or training of human anatomy; (6) the population receiving training included medical students or resident physicians; and (7) a control group was required. Exclusion criteria were as follows: (1) experiments lacking comparative studies; (2) research including animal anatomy; and (3) reviews, case reports, and studies for which valid data could not be extracted.

We conducted preliminary and fine screening of the retrieved literature. Preliminary screening involved reading titles and abstracts and removing the literature that clearly did not meet the requirements. In the next step, we read the full text, and if we encountered difficulties, we negotiated with another participant to solve them together.

Quality assessment of the included literature

For the quality assessment of the included literature, we used Review Manager 5.3 ( https://www.duote.com/soft/911598.html ) to evaluate the risk level after reading the full articles.

The assessment methods included the following indicators: (1) random sequence generation, (2) allocation concealment, (3) blinding of participants and personnel, (4) blinding of outcome assessment, (5) incomplete outcome data, (6) selective reporting, and (7) other bias. Each indicator was evaluated using the following three options: high, low, or unclear risk. If disagreement arose, it was resolved through negotiation; therefore, the lower the risk, the higher the quality of the literature.

Text mining of the included literature

We used VOSviewer 1.6.19 ( https://www.vosviewer.com/download ) to mine the time, country, and anatomical parts of the included literature.VOSviewer can be used to construct maps of authors or journals based on co-citation data or maps of keywords based on co-occurrence data [ 14 ].

In this study, VOSviewer displays maps in two ways: scatter and density views. In the scatter view, items are indicated by small circles; the more important the project, the larger the circle. If colors have been assigned to the items, each item’s circle is displayed in the item’s color [ 14 ]. For example, in Supplementary Figure S1 , each circle represents the country to which the randomized controlled trial belongs, and the larger the circle, the more the literature from that country included in the study. In the density view, the approach is similar to that in the scatter view. For example, in Fig.  1 , each fluorescent circle represents an anatomical part corresponding to the randomized controlled trial, and the brighter the fluorescent color, the more documents on this anatomical part are included in this study.

Regional distribution of literature sources

Combined analysis of data

In forest plots, the size of heterogeneity is described by the square of I. According to experience, heterogeneity is sometimes described as low when the square of I is less than 50%, medium when the square of I is 50–75%, and high when the square of I is more than 75% [ 15 ].

Egger’s regression is a tool used to detect research bias in meta-analysis. It can be used to test the bias of pleiotropic effect, and its slope coefficient provides an estimate of the causal effect [ 16 ].

The choice of meta-analysis model depends on the existence or non-existence of heterogeneity. With no heterogeneity (heterogeneity p  < 0.10), a fixed-effects model is used; however, with heterogeneity (heterogeneity p  < 0.10) in the study, the random-effects model should be used for meta-analysis [ 17 ].

A funnel diagram, the most common method for identifying publication bias, is a scatter diagram made of sample content (or reciprocal of the standard error of effect quantity) and effect quantity (or logarithm of effect quantity). The funnel graph asymmetry test evaluates specific types of heterogeneity and is more powerful in this case [ 18 ].

Meta-regression is a regression analysis of the effect value at the research level. It is used to identify and screen for heterogeneity, analyze its source, and provide a basis for subsequent subgroup analysis. The application condition of meta-regression analysis is not less than 10 studies, and in this study, we included 25. The region was divided into China and other countries, and time was divided into before 2018 and after 2018 (including 2018).

Statistical analysis

In the forest map, we used the square of I to describe the heterogeneity of the data, and Egger’s test. For continuous data, due to different scoring standards, we used the standardized mean difference (SMD) to compare the results. For the joint analysis of continuous variables, we analyzed the averages (X) and standard deviations of the experimental and control groups. In addition, the statistical method used in meta-regression analysis is the t-test, which can test whether the predicted variables are significant. According to the size of heterogeneity, a random-effects model is used to merge the data. Finally, the stability of the data is evaluated by sensitivity analysis, in which the random-effects model is used; the statistical effect quantity is the OR value, and the 95% confidence interval (CI) statistical method is used. In case of no special explanation, a p-value of less than 0.05 is considered statistically significant.

Literature screening

We downloaded the retrieved literature catalog as a whole, incorporated it into Endnote, summarized and merged it, and then divided it into two stages of literature screening, namely coarse screening and fine screening. By reading the articles’ abstracts, we could roughly screen them, and by downloading the full text for reading, we could refine the screening process. Based on the set search conditions, we retrieved 126 studies from the PubMed database, 140 studies from the Web of Science database, and 295 studies from other databases. After reading the titles and eliminating repetitive references, 39 articles remained (Fig.  2 ). Overall, 39 studies (Supplementary Table S1 ) met the inclusion requirements [ 1 , 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 , 34 , 35 , 36 , 37 , 38 , 39 , 40 , 41 , 42 , 43 , 44 , 45 , 46 , 47 , 48 , 49 , 50 , 51 , 52 , 53 , 54 , 55 , 56 ]. These were 39 randomized controlled studies with 2,959 participants: eight were from the United States; seven from China; six from Canada; five from Germany; three from the Netherlands; two from France; and one each from Russia, Belgium, India, Switzerland, New Zealand, Tunisia, Turkey, Thailand, and Japan.

Flowchart of the search strategy

Literature quality analysis

As shown in the quality analysis, the risk of bias is relatively low in most studies (Fig.  3 & Supplementary Figure S2 ). A few studies lack information on performance and detection bias [ 21 , 23 , 26 , 38 ] as, due to the nature of the intervention, it was impractical to conduct blind checks on students and residents during the research process (selection bias). Most studies were determined to have a low risk of selection and low risk of attrition bias due to the complete data of the research results and the use of random selection for grouping [ 1 , 19 , 20 , 21 , 22 , 23 , 25 , 26 , 27 , 28 , 29 , 30 , 34 , 35 , 36 , 37 , 39 , 40 , 41 , 42 , 43 , 44 , 45 , 46 , 47 , 48 , 49 , 50 , 51 ]. The judgment of whether the research has selective reporting is based on whether the results are fully mentioned in the manuscript or discussion section. Three studies were judged to be at high risk of other bias as the experimental or control group had fewer than 10 participants [ 20 , 24 , 52 ]. Finally, one study was judged to have a high risk of performance bias due to the participants’ biased understanding of the assigned interventions during the study period [ 24 ].

Risk of bias summary of included studies

Literature information analysis

As shown in the Fig.  1 , the United States, China, Canada, and Germany conducted more randomized controlled trials on this topic, followed by the Netherlands.

As shown in Supplementary Figure S1 , a greater number of randomized controlled trial on this topic were conducted in the area of neuroanatomy, followed by head and neck, liver, and cardiac anatomy.

Data merging of test scores

Based on StataMP 17 (64-bit), we made a score forest map, and all the studies reported the influence of intervention on test scores (of the 39 articles we cited, 35 included the influence on test scores, and the data of 25 articles were included).

With regard to overall data consolidation, in the random-effects model, compared with traditional learning, 3D technology significantly improved learners’ test scores (SMD = 0.69, 95% CI = 0.24–1.14, p  < 0.05, I 2  = 93.8%, Fig.  4 ).

Comparison of the experimental and control groups for test scores

As subgroup analysis displayed, merging the literature data of China has statistical significance (SMD = 1.72, 95% CI = 1.04–2.40, p  < 0.05, I 2  = 88.8%, Fig.  4 ); however, merging data from regions outside of China shows no statistical significance (SMD = 0.29, 95% CI = -0.19–0.77, p  < 0.05, I 2  = 92.9%, Fig.  4 ).

The medical students subgroup displayed statistical significance (SMD = 0.68, 95% CI = -0.17–1.19, p  < 0.05, I 2  = 94%), while the residents subgroup showed no statistical significance (SMD = 0.74, 95% CI = -0.31–1.80, p  < 0.05, I 2  = 94.5%, Supplementary Figure S3 ).

Data merging of satisfaction degree

Ten studies [ 24 , 32 , 36 , 41 , 43 , 45 , 48 , 50 , 52 , 53 ] evaluated satisfaction as a secondary outcome (Fig.  5 A). The summary results based on the random-effects model show that most students are more interested in learning through 3D methods than traditional or 2D teaching methods (SMD = 0.70, 95% CI = 0.32–1.07, p  < 0.05, I 2  = 69.0%), which may be related to the more intuitive experience given by 3D technology. If the literature from China is excluded, the 3D group has statistical significance as well (SMD = 0.79, 95% CI = 0.30–1.29, p  < 0.05, I 2  = 69.0%,Fig.  5 B).

Comparison of the experimental and control groups for satisfaction outcome

Data merging of time and enjoyment degree

We included six documents on time consumption [ 20 , 24 , 25 , 29 , 46 , 51 ] and four documents in the forest map of enjoyment value [ 32 , 40 , 45 , 48 ]. The results showed no statistical difference between the 3D group and the traditional group (SMD = -0.55, 95% CI = -1.23–0.14, p  > 0.05, I 2  = 86.5%, Supplementary Figure S4 A). If the study from China is removed, the statistical significance of the results remains unchanged (Supplementary Figure S4 B). However, the results of the happiness value forest map show that 3D technology makes participants feel happier (SMD = 3.04, 95% CI = 1.05–5.04, p  < 0.05, I 2  = 95.8%, Fig.  6 A). If the study from China is deleted, the statistical significance of the results remains unchanged as well (SMD = 2.73, 95% CI = 0.32–5.15, p  < 0.05, I 2  = 96.5%, Fig.  6 B).

Comparison of the experimental and control groups for enjoyment outcome

Publication bias

According to the results, the funnel chart is basically symmetrical, with a vertical line in the middle representing the combined OR value, and all studies are generally evenly distributed on both sides of the vertical line, showing an inverted funnel shape ( Figures S5 A-C ). This shows no obvious bias in grades, test time, or satisfaction. At the same time, the results of the Egger’s test for test time and test performance showed non-significant asymmetry ( p  > 0.05); therefore, no apparent application bias was observed in the present study. However, the Egger’s test of satisfaction showed application bias ( p  < 0.05), and when the Chinese studies were removed, no application bias was found (p  > 0.05).

Sensitivity analysis

Due to the significant heterogeneity (I 2  > 75%), we created a sensitivity analysis chart to verify the reliability of the results. We found that when any research was removed from the model, the significant influence of 3D visualization on test scores, satisfaction, and test time remained unchanged (Fig.  7 A and B, Figure S6 ). Therefore, this result shows that the survey’s inspection results were reasonable.

(A) Sensitivity analysis of the test results of the experimental and control groups was performed by meta-analysis. (B) The sensitivity analysis of the satisfaction of the experimental and control groups was conducted by meta-analysis

Regressive analysis

To confirm the influence of various factors, we made a meta-regression analysis on the influence of four potential factors: learners, countries, courses, and time (Table  1 ). We grouped medical students versus other learners, China versus other countries, and neuroanatomy versus other anatomy. The results show that the P-value of the country is less than 0.05, which indicates that the national factors have a significant influence on the results, while the other factors have none.

In the recent 10 years, 3D visualization technologies such as VR, augmented reality, and mixed reality have become increasingly popular [ 5 ].This meta-analysis included 39 studies; interestingly, for countries with the highest number of studies, these studies are directly related to their economy and technology. For example, the United States, the largest economy in the Americas, is also the region with the highest number of studies. China, the country with the largest economy in Asia, has the highest number of Asian studies. Germany, the leading country in the European economy, has the highest number of studies in Europe. Generally, the risk of bias in most studies is due to a lack of data or unclear descriptions as well as to other descriptions [ 57 ]. In all 39 studies, the subjects were divided into random control groups, and the heterogeneity may have been caused by differences in teaching quality and test difficulty in different schools.

Most medical students learn about anatomy using traditional textbooks. Autopsy is a special teaching method that has many advantages but also limitations [ 58 ]. Therefore, if 3D technology is widely used in this subject, it can improve students’ understanding of three-dimensional graphics [ 59 ]. In addition, 3D technology has potential practicability not only in education/training but also in operation planning and intraoperative guidance [ 60 ].From a learning perspective, 3D visualization technology can stimulate students to explore their own understanding [ 61 ]. This is beneficial to their clinical diagnosis and treatment after study. However, whether 3D technology can improve participants’ test scores varies depending on the research findings. There are many conclusions from previous meta-analyses. Some have shown that 3D visualization technology can improve learner performance [ 57 ], while others have shown that 3D visualization is a more effective method for acquiring anatomical knowledge than traditional methods [ 62 ].Others also show that 3D visualization as a learning tool has potential beneficial effects on learning [ 63 ].However, we disagree with this viewpoint. Thus, we included a greater number of studies in our research compared to them. Similarly, a literature summary including all regions found that the 3D group performed better in the test than the control group. However, after excluding studies from China, the results changed. They may have a regional bias, with higher positivity rates in the study from China, and we cannot rule out the possibility that the authors might have had a preference for positive results to publish the study. Therefore, whether 3D visualization can improve test scores is a question that carries substantial weight, and we are more inclined not to answer it. Excitingly, regarding the cost of answering questions, satisfaction, enjoyment, excluding and not excluding literature from China, after merging the data, we found that the results were stable. Therefore, the credibility of our findings is high.

The advantage of this research lies mainly in the retrieval and extraction of the literature, and we developed our own literature screening process. Compared with most document screening processes, our two-step document screening method is more accurate and comprehensive; this significantly improves the efficiency of document screening. In addition, compared with most related studies, we included more documents. Although these technologies provide interesting, new pedagogical possibilities, they have some limitations [ 9 ], including restrictions on the conditions of use, such as the high cost of 3D visualization technology, which makes it difficult to popularize.Undeniably, the limitations of our meta-analysis also include that some related papers may be omitted. Furthermore, learners’ different understandings of space affected the experimental results. However, it takes considerable skill and practice to develop the ability to visualize in three dimensions and insufficient ability to visualize frequently expressed by the students [ 64 ]. Due to the small number of participants in some studies, the experimental data may have subjective influence.

Medical students and residents who use 3D visualization technology to learn about human anatomy do not improve their test scores. Regional factors (the countries to which the 39 included studies belong) have a significant impact on the test results; the literature from China is more likely to have positive results, whereas other factors, such as learners, courses, and time, have no significant impact. 3D technology cannot shorten the time required for participants to answer questions; however, it can improve the participants’ satisfaction and enjoyment. Overall, 3D visualization technology is a promising teaching-aid technology.

Data availability

All data and materials are available as supplementary materials.

Bogomolova K, van der Ham IJ, fDankbaar MEW, van den Broek WW, Hovius SER, van der Hage JA, et al. The Effect of Stereoscopic Augmented reality visualization on learning anatomy and the modifying effect of visual-spatial abilities: a double-Center Randomized Controlled Trial. Anat Sci Educ. 2020;13(5):558–67.

Article   Google Scholar  

Bergman EM, de Bruin ABH, Herrler A, Verheijen IWH, Scherpbier A, van der Vleuten CPM. Students’ perceptions of anatomy across the undergraduate problem-based learning medical curriculum: a phenomenographical study. BMC Med Educ. 2013;13:152.

Reid S, Shapiro L, Louw G. How Haptics and drawing enhance the learning of anatomy. Anat Sci Educ. 2019;12(2):164–72.

Peugnet F, Dubois P, Rouland JF. Virtual reality versus conventional training in retinal photocoagulation: a first clinical assessment. Comput Aided Surgery: Official J Int Soc Comput Aided Surg. 1998;3(1):20–6.

Venkatesan M, Mohan H, Ryan JR, Schürch CM, Nolan GP, Frakes DH, et al. Virtual and augmented reality for biomedical applications. Cell Rep Med. 2021;2(7):100348.

Hughes CE, Stapleton CB, Hughes DE, Smith EM. Mixed reality in education, entertainment, and training. IEEE Comput Graph Appl. 2005;25(6):24–30.

Rokhsaritalemi S, Sadeghi-Niaraki A, Choi S-M. A review on mixed reality: current trends, challenges and prospects. J Appl Sci. 2020;10(2):636.

Zhao F, Fang J, Li D, Hong Q, You R, Chen K. Clinical Research of Orthopedic Trauma Based on Computer 3D Image Analysis. Computational and mathematical methods in medicine. 2021; 2021:2728388.

Yammine K, Violato C. A meta-analysis of the educational effectiveness of three-dimensional visualization technologies in teaching anatomy. Anat Sci Educ. 2015;8(6):525–38.

Ye Z, Jiang HM, Bai SY, Wang T, Yang DX, Hou HF, et al. Meta-analyzing the efficacy of 3D printed models in anatomy education. Front Bioeng Biotechnol. 2023;11:1117555.

Ye Z, Dun AS, Jiang HM, Nie CF, Zhao SL, Wang T, et al. The role of 3D printed models in the teaching of human anatomy: a systematic review and meta-analysis. BMC Med Educ. 2020;20(1):335.

Fleming C, Sadaghiani MS, Stellon MA, Javan R. Effectiveness of three-dimensionally printed models in Anatomy Education for Medical Students and Resident Physicians: systematic review and Meta-analysis. J Am Coll Radiol. 2020;17(10):1220–9.

Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JP, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS Med. 2009;151(4):65–94.

Google Scholar  

van Eck Nj Fau -, Waltman L, Waltman L. Software survey: VOSviewer, a computer program for bibliometric mapping. Scientometrics. 2010;84(2):523–38.

Andrade C. Understanding the basics of Meta-Analysis and how to read a forest plot: as simple as it gets. J Clin Psychiatry. 2020;81(5):20f13698.

Bowden J, Davey Smith G, Burgess S. Mendelian randomization with invalid instruments: effect estimation and bias detection through Egger regression. Int J Epidemiol. 2015;44(2):512–25.

Ried K. Interpreting and understanding meta-analysis graphs–a practical guide. Aus Fam Physician. 2006;35(8):635–8.

Egger M, Davey Smith G, Fau - Schneider M, Schneider M, Fau - Minder C, Minder C. Bias in meta-analysis detected by a simple, graphical test. BMJ. 1997;315(7109):629–34.

Jurgaitis J, Paskonis M, Pivoriūnas J, Martinaityte I, Juska A, Jurgaitiene R, et al. The comparison of 2-dimensional with 3-dimensional hepatic visualization in the clinical hepatic anatomy education. Med (Kaunas Lithuania). 2008;44(6):428–38.

Solyar A, Cuellar H, Fau - Sadoughi B, Sadoughi B, Fau - Olson TR, Olson Tr Fau - Fried MP, Fried MP. Endoscopic Sinus Surgery Simulator as a teaching tool for anatomy education. The American Journal of Surgery. 2007; 196(1): 120–124.

Hu A, Wilson T, Fau - Ladak H, Ladak H, Fau - Haase P, Haase P, Fau - Fung K, Fung K. Three-dimensional educational computer model of the larynx: voicing a new direction. Arch Otolaryngol Head Neck Surg. 2009;135(7):677–81.

Abid B, Hentati N, Chevallier JM, Ghorbel A, Delmas V, Douard R. Traditional versus three-dimensional teaching of peritoneal embryogenesis: a comparative prospective study. Surg Radiol Anat. 2010;32(7):647–52.

Hampton BS, Sung VW. Improving medical student knowledge of female pelvic floor dysfunction and anatomy: a randomized trial. Am J Obstet Gynecol. 2009;202(6):601.

SITTAPONG SETTAPAT, TIRANEE ACHALAKUL. Web-based 3D medical image visualization Framework for Biomedical Engineering Education. Comput Appl Eng Educ. 2014;22(2):216–26.

Keedy AW, Durack JC, Sandhu P, Chen EM, O’Sullivan PS, Breiman RS. Comparison of traditional methods with 3D computer models in the instruction of Hepatobiliary anatomy. Anat Sci Educ. 2011;4(2):84–91.

Müller-Stich BP, Löb N, Wald D, Bruckner T, Meinzer HP, Kadmon M, et al. Regular three-dimensional presentations improve in the identification of surgical liver anatomy - a randomized study. BMC Med Educ. 2013;13(1):131.

Drapkin ZA, Lindgren KA, Lopez MJ, Stabio ME. Development and assessment of a new 3D neuroanatomy teaching tool for MRI training. Anat Sci Educ. 2015;8(6):502–9.

Kockro RA, Amaxopoulou C, Killeen T, Wagner W, Reisch R, Schwandt E, et al. Stereoscopic neuroanatomy lectures using a three-dimensional virtual reality environment. Annals Anat. 2015;201:91–8.

Li Z, Li Z, Xu R, Li M, Li J, Liu Y, et al. Three-dimensional printing models improve understanding of spinal fracture–A randomized controlled study in China. Sci Rep. 2015;5(1):11570.

de Faria JW, Teixeira MJ, de Moura Sousa Júnior L, Otoch JP, Figueiredo EG. Virtual and stereoscopic anatomy: when virtual reality meets medical education. J Neurosurg. 2016;125(5):1105–11.

Goodarzi A, Monti S, Lee D, Girgis F. Effect of Stereoscopic Anaglyphic 3-Dimensional Video Didactics on Learning Neuroanatomy. World Neurosurg. 2017;107:35–9.

Stepan K, Zeiger J, Hanchuk S, Del Signore A, Shrivastava R, Govindaraj S et al. Immersive virtual reality as a teaching tool for neuroanatomy. INTERNATIONAL FORUM OF ALLERGY & RHINOLOGY. 2017; 7(10):1006–1013.

Ellington DR, Shum PC, Dennis EA, Willis HL, Szychowski JM, Richter HE. Female pelvic floor Immersive Simulation: a Randomized Trial to test the effectiveness of a virtual reality anatomic model on Resident Knowledge of female pelvic anatomy. J Minim Invasive Gynecol. 2018;26(5):897–901.

Hackett MA-O, Proctor M. The effect of autostereoscopic holograms on anatomical knowledge: a randomised trial. Med Educ. 2018;52(11):1147–55.

Wainman B, Wolak L, Pukas G, Zheng EA-O, Norman GA-O. The superiority of three-dimensional physical models to two-dimensional computer presentations in anatomy learning. Med Educ. 2018;52(11):1138–46.

Lin C, Gao JY, Zheng H, Zhao J, Yang H, Lin GL, et al. Three-dimensional visualization technology used in pancreatic surgery: a valuable tool for surgical trainees. Pancreas. 2019;48(10):1477.

Maresky HS, Oikonomou A, Ali I, Ditkofsky N, Pakkal M, Ballyk B. Virtual reality and cardiac anatomy: exploring immersive three-dimensional cardiac imaging, a pilot study in undergraduate medical anatomy education. Volume 32. New York, N.Y.): Clinical anatomy; 2019. pp. 238–43. 2.

Iqbal MS, Ms AKK, Axelrod DMJAP, USING VIRTUAL REALITY, HEART MODELS TO TEACH CONGENITAL HEART DISEASE TO TRAINEES. Acad Pediatr. 2020;20(7):15.

Cai S, He Y, Cui H, Zhou X, Zhou D, Wang F, et al. Effectiveness of three-dimensional printed and virtual reality models in learning the morphology of craniovertebral junction deformities: a multicentre, randomised controlled study. BMJ open. 2020;10(9):e036853.

Chen S, Zhu JW, Cheng C, Pan ZX, Liu LS, Du JH, et al. Can virtual reality improve traditional anatomy education programmes? A mixed-methods study on the use of a 3D skull model. BMC Med Educ. 2020;20(1):395.

Gnanasegaram JJ, Leung R, Beyea JA. Evaluating the effectiveness of learning ear anatomy using holographic models. J Otolaryngol - Head Neck Surg. 2020;49(1):63.

Kurul R, Ögün MN, Neriman Narin A, Avci Ş, Yazgan B. An alternative method for anatomy training: immersive virtual reality. Anat Sci Educ. 2020;13(5):648–56.

Wang C, Daniel B, Asil M, Khwaounjoo P, Cakmak YO. A Randomised Control Trial and Comparative Analysis of Multi-dimensional Learning Tools in anatomy. Sci Rep. 2020;10(1):6120.

Zinchenko YP, Khoroshikh PP, Sergievich AA, Smirnov AS, Tumyalis AV, Kovalev AI, et al. Virtual reality is more efficient in learning human heart anatomy especially for subjects with low baseline knowledge. J New Ideas Psychol. 2020;59:100786.

Bogomolova K, Vorstenbosch M, El Messaoudi I, Holla M, Hovius SER, van der Hage JA, Hierck BP. Effect of binocular disparity on learning anatomy with stereoscopic augmented reality visualization: a double center randomized controlled trial. Anat Sci Educ. 2021;16(1):87–98.

Greuter L, De Rosa A, Cattin P, Croci DM, Soleman J, Guzman R. Randomized study comparing 3D virtual reality and conventional 2D on-screen teaching of cerebrovascular anatomy. 2021; 51(2):e18.

van Deursen M, Reuvers L, Duits JD, de Jong G, van den Hurk M, Henssen D. Virtual reality and annotated radiological data as effective and motivating tools to help Social Sciences students learn neuroanatomy. Sci Rep. 2021;11(1):12843.

Imai TA-O, Tanaka Y, Hatanaka Y, Suetsugu T, Sato Y, Matsuhashi N, et al. Incorporation of virtual reality in the clinical training of medical students studying esophageal and mediastinal anatomy and surgery. Surg Today. 2022;52(8):1212–7.

Vandenbossche V, Valcke M, Willaert W, Audenaert E. From bones to bytes: do manipulable 3D models have added value in osteology education compared to static images? Med Educ. 2022;57(4):359–68.

Shao X, Qiang D, Yuan Q. A new neuroanatomical two-dimensional fitting three-dimensional imaging techniques in neuroanatomy education. BMC Med Educ. 2023;23(1):333.

Wise PA, Nickel F, Preukschas AA, Bettscheider L, Speidel S, Wagner M, Pfeiffer M, et al. Virtual reality head mounted Display versus three-dimensional models and two-dimensional Tomography Display for Training of Liver Surgery-A Randomized Controlled Study. Surg Endosc. 2023;37:S7.

Bork FA-O, Lehner A, Eck U, Navab N, Waschke J, Kugelmann DA-O. The effectiveness of collaborative augmented reality in gross anatomy teaching: a quantitative and qualitative pilot study. Anat Sci Educ. 2021;14(5):590–604.

Weeks JK, Pakpoor J, Park BJ, Robinson NJ, Rubinstein NA, Prouty SM, et al. Harnessing augmented reality and CT to teach first-Year Medical Students Head and Neck anatomy. Acad Radiol. 2021;28(6):871–6.

Nicholson DT, Chalk C, Fau - Funnell WRJ, Funnell Wr Fau -, Daniel SJ, Daniel SJ. Can virtual reality improve anatomy education? A randomised controlled study of a computer-generated three-dimensional anatomical ear model. Med Educ. 2006;40(11):1081–7.

Hu KC, Salcedo D, Kang YN, Lin CW, Hsu CW, Cheng CY, et al. Impact of virtual reality anatomy training on ultrasound competency development: a randomized controlled trial. PLoS ONE. 2020;15(11):e0242731–0242731.

Yohannan DG, Oommen AM, Amogh BJ, Raju NK, Nair SJJASE. Air anatomy’– teaching complex spatial anatomy using simple hand gestures. Anat Sci Educ. 2021;15(3):552–65.

Zhao JJ, Xu XL, Jiang HL, Ding Y. The effectiveness of virtual reality-based technology on anatomy teaching: a meta-analysis of randomized controlled studies. BMC Med Educ. 2020;20(1):127.

Xu YJ, Hu YM, Xu T, Yan KW, Zhang T, Li Q, et al. RNF8-mediated regulation of akt promotes lung cancer cell survival and resistance to DNA damage. CELL Rep. 2019;37(3):109854.

Murgitroyd E, Madurska M, Gonzalez J, Watson A. 3D digital anatomy modelling - practical or pretty? Surgeon: J Royal Colleges Surg Edinb Irel. 2015;13(3):177–80.

Wang S, Frisbie J, Keepers Z, Bolten Z, Hevaganinge A, Boctor E, et al. The Use of three-dimensional visualization techniques for prostate procedures: a systematic review. Eur Urol Focus. 2021;7(6):1274–86.

Silén C, Wirell S, Kvist J, Nylander E, Smedby O. Advanced 3D visualization in student-centred medical education. Med Teach. 2008;30(5):115–24.

Triepels CPR, Smeets CFA, Notten KJB, Kruitwagen R, Futterer JJ, Vergeldt TFM et al. Does three-dimensional anatomy improve student understanding? Clinical anatomy (New York, NY).2020; 33(1):25–33.

Petersson H, Sinkvist D, Wang C, Smedby O. Web-based interactive 3D visualization as a tool for improved anatomy learning. Anat Sci Educ. 2009;2(2):61–8.

Heylings DJ. Anatomy 1999–2000: the curriculum, who teaches it and how? Med Educ. 2002;36(8):702–10.

Download references

Acknowledgements

We wish to express our gratitude to the PubMed, Embase, and Web of Science databases for providing the full texts of the articles considered in our literature search.

Shandong First Medical University of Medical Sciences 2022 Clinical Medicine Excellence Class Undergraduate Science and Technology Innovation Program Project (for J.W. and W.L.). Research Project of Ideological and Political Education Reform in Colleges and Universities of Shandong Province in 2023 (SZ2023071 for A.D.). General Project of Education and Teaching Reform Research of Shandong First Medical University in 2023 (XM2023027 for A.D.).

Author information

Junming Wang, Wenjun Li and Aishe Dun contributed equally to this work.

Authors and Affiliations

Department of Health Management, The First Affiliated Hospital, Shandong Provincial Qianfoshan Hospital, Shandong First Medical University, 250013, Jinan, Shandong, China

Junming Wang, Wenjun Li, Ning Zhong & Zhen Ye

School of clinical and basic medicine, Shandong First Medical University, Jinan, China

Junming Wang & Wenjun Li

School of Stomatology, Shandong First Medical University, Jinan, China

You can also search for this author in PubMed   Google Scholar

Contributions

Z.Y. and N.Z. conceived and designed the study J.W. and W.L. carried out literature retrieval, data extraction and performed the statistical analysis. J.W., W.L.and A.D. completed the article writing.

Corresponding authors

Correspondence to Ning Zhong or Zhen Ye .

Ethics declarations

Ethics approval and consent to participate.

Ethical approvals have been obtained for every articles included in the study.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Supplementary material 2, supplementary material 3, supplementary material 4, supplementary material 5, supplementary material 6, supplementary material 7, supplementary material 8, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Wang, J., Li, W., Dun, A. et al. 3D visualization technology for Learning human anatomy among medical students and residents: a meta- and regression analysis. BMC Med Educ 24 , 461 (2024). https://doi.org/10.1186/s12909-024-05403-4

Download citation

Received : 04 December 2023

Accepted : 08 April 2024

Published : 26 April 2024

DOI : https://doi.org/10.1186/s12909-024-05403-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• 3D visualization technology
• Human anatomy
• Meta-analysis
• Regression analysis
• Medical education

BMC Medical Education

ISSN: 1472-6920

• Submission enquiries: [email protected]
• General enquiries: [email protected]

• Open access
• Published: 20 April 2024

“I am in favour of organ donation, but I feel you should opt-in”—qualitative analysis of the #options 2020 survey free-text responses from NHS staff toward opt-out organ donation legislation in England

• Natalie L. Clark 1 ,
• Dorothy Coe 2 ,
• Natasha Newell 3 ,
• Mark N. A. Jones 4 ,
• Matthew Robb 4 ,
• David Reaich 1 &
• Caroline Wroe 2  

BMC Medical Ethics volume  25 , Article number:  47 ( 2024 ) Cite this article

227 Accesses

4 Altmetric

Metrics details

In May 2020, England moved to an opt-out organ donation system, meaning adults are presumed to be an organ donor unless within an excluded group or have opted-out. This change aims to improve organ donation rates following brain or circulatory death. Healthcare staff in the UK are supportive of organ donation, however, both healthcare staff and the public have raised concerns and ethical issues regarding the change. The #options survey was completed by NHS organisations with the aim of understanding awareness and support of the change. This paper analyses the free-text responses from the survey.

The #options survey was registered as a National Institute of Health Research (NIHR) portfolio trial [IRAS 275992] 14 February 2020, and was completed between July and December 2020 across NHS organisations in the North-East and North Cumbria, and North Thames. The survey contained 16 questions of which three were free-text, covering reasons against, additional information required and family discussions. The responses to these questions were thematically analysed.

The #options survey received 5789 responses from NHS staff with 1404 individuals leaving 1657 free-text responses for analysis. The family discussion question elicited the largest number of responses (66%), followed by those against the legislation (19%), and those requiring more information (15%). Analysis revealed six main themes with 22 sub-themes.

Conclusions

The overall #options survey indicated NHS staff are supportive of the legislative change. Analysis of the free-text responses indicates that the views of the NHS staff who are against the change reflect the reasons, misconceptions, and misunderstandings of the public. Additional concerns included the rationale for the change, informed decision making, easy access to information and information regarding organ donation processes. Educational materials and interventions need to be developed for NHS staff to address the concepts of autonomy and consent, organ donation processes, and promote family conversations. Wider public awareness campaigns should continue to promote the positives and refute the negatives thus reducing misconceptions and misunderstandings.

Trial registration

National Institute of Health Research (NIHR) [IRAS 275992].

Peer Review reports

In England May 2020, Max and Kiera’s Law, also known as the Organ Donation (Deemed Consent) Bill, came into effect [ 1 , 2 ]. This means adults in England are now presumed to have agreed to deceased organ donation unless they are within an excluded group, have actively recorded their decision to opt-out of organ donation on the organ donor register (ODR), or nominated an individual to make the decision on their behalf [ 1 , 2 ]. The rationale for the legislative change is to improve the organ donation rates and reduce the shortage of organs available to donate following brain or circulatory death within the UK [ 2 , 3 , 4 ]. This is particularly important considering the growing number of patients awaiting a transplant. Almost 7000 patients were waiting in the UK at the end of March 2023 [ 5 ]. Wales was the first to make the legislative change in December 2015, followed by Scotland in March 2021 and lastly Northern Ireland in June 2023 [ 2 ]. Following the change in Wales, consent rates had increased from 58% in 2015/16 to 77% in 2018/19 [ 6 ], suggesting the opt-out system can significantly increase consent, though it further suggests that it might take a few years to fully appreciate the impact [ 7 , 8 ]. Spain, for example, has had an opt-out legislation since 1979 with increases in organ donation seen 10 years later [ 9 ].

Research, however, has raised concerns from both the public and healthcare staff regarding the move to an opt-out system. These concerns predominantly relate to a loss of freedom and individual choice [ 9 , 10 ], as well as an increased perception of state ownership of organs [ 10 , 11 , 12 ] after death. Healthcare staff additionally fear of a loss of trust and a damaged relationship with their patients [ 9 , 11 ]. These concerns are frequently linked to emotional and attitudinal barriers towards organ donation, understanding and acceptance [ 9 ]. Four often referenced barriers include (1) jinx factor: superstitious beliefs [ 13 , 14 , 15 ]; (2) ick factor: feelings of disgust related to donating [ 13 , 14 , 15 ]; (3) bodily integrity: body must remain intact [ 13 , 14 , 15 ]; (4) medical mistrust: believing doctors will not save the life of someone on the ODR [ 13 , 14 , 15 ]. The latter barrier is mostly reported by the general public in countries with opt-out systems [ 13 , 14 , 16 ] although medical mistrust does feature as a barrier across all organ donation systems. In addition, it is a reported barrier healthcare staff believe will occur in the UK under an opt-out system [ 9 , 16 ].

Deceased donation from ethnic minority groups is low in the UK, with family consent being a predominant barrier in these groups. Consent rates are 35% for ethnic minority eligible donors compared to 65% for white eligible donors [ 5 ]. The reasons for declining commonly relate to being uncertain of the person’s wishes and believing it was against their religious/cultural beliefs. Healthcare staff, particularly in the intensive care setting, have expressed a lack of confidence in communication and supporting ethnic minority groups because of language barriers and differing religious/cultural beliefs to their own [ 17 ]. However, one study has highlighted that generally all religious groups are in favour of organ donation with respect to certain rules and processes. Therefore, increasing knowledge amongst healthcare staff of differing religious beliefs would improve communication and help to sensitively support families during this difficult time [ 18 , 19 ]. However, individually and combined, the attitudinal barriers, concerns towards an opt-out system, and lack of understanding about ethnic minority groups, can have a significant impact within a soft opt-out system whereby the family are still approached about donation and can veto if they wish [ 11 , 12 , 20 ].

The #options survey [ 21 ] was completed online by healthcare staff from National Health Service (NHS) organisations in North-East and North Cumbria (NENC) and North Thames. The aim was to gain an understanding of the awareness and support to the change in legislation. The findings of the survey suggested that NHS staff are more aware, supportive, and proactive about organ donation than the general public, including NHS staff from religious and ethnic minority groups. However, there were still a number who express direct opposition to the change in legislation due to personal choice, views surrounding autonomy, misconceptions or lack of information. This paper will focus on the qualitative analysis of free-text responses to three questions included in the #options survey. It aims to explore the reasons for being against the legislation, what additional information they require to make a decision, and why had they not discussed their organ donation decision with their family. It will further explore a subset analysis of place of work, ethnicity, and misconceptions. The findings will aid suggestions for future educational and engagement work.

Design, sample and setting

The #options survey was approved as a clinical research study through the integrated research application system (IRAS) and registered as a National Institute of Health Research (NIHR) portfolio trial [IRAS 275992]. The survey was based on a previously used public survey [ 22 ] and peer reviewed by NHS Blood and Transplant (NHSBT). The free-text responses used in #options were an addition to the closed questions used in both the #options and the public survey. Due to the COVID-19 pandemic, the start of the survey was delayed by 4 months, opening for responses between July to December 2020. All NHS organisations in the NENC and North Thames were invited to take part. Those that accepted invitations were supplied with a communication package to distribute to their staff. All respondents voluntarily confirmed their agreement to participate in the survey at the beginning. The COnsolidated criteria for REporting Qualitative research (COREQ) checklist was used to guide analysis and reporting of findings [ 23 ], see Supplementary material 1 .

Data collection and analysis

The survey contained 16 questions, including a brief description of the change in legislation. The questions consisted of demographic details (age, sex, ethnicity, religion), place of work, and if the respondent had contact with or worked in an area offering support to donors and recipients. Three of the questions filtered to a free-text response, see Supplementary material 2 . These responses were transferred to Microsoft Excel to be cleaned and thematically analysed by DC. Thematic analysis was chosen to facilitate identification of groups and patterns within large datasets [ 24 ]. Each response was read multiple times to promote familiarity and initially coded. Following coding, they were reviewed to allow areas of interest to form and derive themes and sub-themes. Additional subsets were identified and analysed to better reflect and contrast views. This included, at the request of NHSBT, the theme of ‘misconceptions’. The themes were reviewed within the team (DC, CW, NK, NC, MJ) and shared with NHSBT. Any disagreements were discussed and agreed within the team.

Overall, the #options survey received 5789 responses from NHS staff. The COVID-19 pandemic further impacted on NHS organisations from North Thames to participate, resulting in respondents predominantly being from NENC (86%). Of the respondents, 1404 individuals (24%) left 1657 free-text responses for analysis. The family discussion question elicited the largest number of responses, accounting for 66% of the responses ( n  = 1088), followed by against the legislation at 19% ( n  = 316) and more information needed at 15% ( n  = 253). The responses to the against legislation question provided the richest data as they contained the most information. Across the three questions, there were six main themes and 22 sub-themes, see Table  1 . The large number of free-text responses illustrate the multifaceted nature of individuals views with many quotes containing overlap between themes and sub-themes.

Respondent characteristics

In comparison to the whole #options survey respondents, the free-text response group contained proportionally more males (21% vs 27%), less females (78% vs 72%), and marginally more 18–24year-olds (7% vs 8%), respectively. There were 5% more 55 + year olds in the free-text group, however all other age groups were between 2–3% lower when compared to the whole group. Additionally, the free-text group were more ethnically diverse than the whole group (6.9% vs 15.4%), with all named religions also having a higher representation (3.9% vs 7.3%), respectively.

Question one: I am against the legislation – Can you help us understand why you are against this legislation?

Of the three questions, this elicited the largest number of responses from males ( n  = 94, 30%), those aged over 55 years ( n  = 103, 33%), and ethnic minority responders ( n  = 79, 25%). Subset analysis of place of employment indicates 27% were from the transplant centre ( n  = 84), 8% were from the mental health trust ( n  = 26), and 4% from the ambulance trust ( n  = 14). Thematic analysis uncovered four main themes and 12 sub-themes from the responses, with the predominant theme being a perceived loss of autonomy.

Theme one: loss of autonomy

Respondents’ reasons for a loss of autonomy were categorised into four sub-themes. Firstly, calling into question the nature of informed consent and secondly, peoples’ awareness of the legislative change. One respondent stated individuals need to be “fully aware and informed” [R2943] in order to have consented to organ donation. However, one respondent stated that they believe individuals have “not [been] informed well” [R930] and thus “if people are not aware of it, how are they making a choice on what happens to their organs” [R1166] . It was suggested that awareness of the change may have “been overshadowed by COVID” [R4119] .

Furthermore, there was concerns regarding the means to record an opt-out decision, specifically to those that are “not tech savvy” [R167] , “homeless” [R5721] , “vulnerable” [R4553] , and “elderly” [R2155] . Therefore, removing that individual’s right to record their decision due to being at a disadvantage.

Finally, respondents expressed concerns of a move to an authoritarian model of State ownership of organs. This elicited strong, negative reactions from individuals under the belief the State would own and “harvest” a person’s organs under a deemed consent approach, with some removing themselves as a donor consequently, “I am furious that the Government has decided that my organs are theirs to assign. It is MY gift to give, not theirs. I have now removed myself as a long-standing organ donor.” [R593] .

Theme two: consequences

Following respondents stating their reason for being against the legislative change, they discussed further what they believed to be the consequences of an opt-out legislation, with a focus on trust. Respondents cited a lack of trust towards the system, “I have no Trust in the UK government” [R5374] , with some surprisingly citing a lack of trust towards healthcare professionals, “Don’t trust doctors in regard to organ donation” [R3010], as well as a fear of eroding trust with the general public, “This brings the NHS Organ Donation directly into dispute with the public.” [R1237]. Respondents additionally believed the legislative change would lead to an increase in mistakes i.e., organ’s being removed against a person’s wishes by presuming, “not convinced that errors won't be made in my notifying my objection and that this won't be dealt with or handed over correctly” [R3018]. Finally, it is believed this change would also lead to, “additional upset” [R587], for already grieving families.

Theme three: legislation

Respondents were additionally against the legislation itself as they believed it lacked an evidence-base to prove it is successful at increasing the numbers of organs donated. As well as this, respondents perceived the legislation as one that removed the donor’s choice as to which organs they want to donate, some with a religious attribute “I don't mind donating but would like choice of what I like to i.e., not my cornea as for after life I want to see where I am going.” [R5274].

Theme four: religion and culture

Religion and culture was another common theme with sub-themes relating to maintaining bodily integrity following death and the lack of clarity around the definition of brain death. Many others stated that organ donation is against their religion or, were “unsure whether organ donation is permissible” [R1067].

Question two: I need more information to decide—What information would you like to help you decide?

This question elicited the most responses from females ( n  = 188, 74%), those aged over 55 years ( n  = 80, 32%), with 19% being from ethnic minority groups ( n  = 49). Subset analysis of place of employment indicates 18% were from the transplant centre ( n  = 46), 8% were from the mental health trust ( n  = 18), and 9% from the ambulance trust ( n  = 23). Thematic analysis uncovered a main theme of “everything” . There were many responses that did not specify what information was required, but indicated that more general information on organ donation was required, within this there were five sub-themes.

Sub-themes:

The first sub-theme identified a request for information around the influence a family has on the decision to donate and the information that will be provided to families. This included providing “emotional wellbeing” [R162] support, and information on whether families can “appeal against the decision” [R539] or “be consulted” [R923] following their loved one’s death. This was mainly requested by those employed by transplant centres.

The second request was for information on the “process involved after death for organ retrieval” [R171] , predominantly by ethnic minority groups and those employed by the mental health trusts, with specific requests on confirming eligibility. Other examples of requested information on the process and pathway included “how the organs will be used” [R1086] , “what will be donated” [R1629] , and “who benefits from them” [R3730] .

The third request was information regarding the publicity strategy to raise awareness of the legislative change. Many of the respondents stated they did not think there was enough “coverage in the media” [R3668]. Additional considerations of public dissemination were to ensure it was an “ easy read update” [R137 3 ] , specifically for “the elderly or those with poor understanding of English who may struggled with the process” [R1676] .

The fourth request was information around the systems in place to record a decision. There were additional requests for the opt-out processes if someone was within the excluded group and “what safeguards are in place” [R3777], as well as what if individuals change their mind and the ease of recording this new decision.

Finally, and similarly to the first question, the fifth request was information for an evidence-base. Respondents stated that they, “would like to know the reasons behind this change” [R3965] , believing that if they had a greater understanding then this might increase their support towards the legislative change.

Question three: Have you discussed your decision with a family member? If no, can you help us understand what has stopped you discussing this with your family?

The free-text responses to analyse were from those who responded “No” to, “Have you discussed your decision with a family member?”. This received 1430 responses with females ( n  = 1025, 27%) predominantly answering “No”. However, not everyone left a free-text response, leaving 1088 comments for analysis. These were predominantly made by those aged over 55 years ( n  = 268, 24%), with 5% being from ethnic minority groups ( n  = 49). Subset analysis of the 1088 responses regarding place of employment indicated 14% were from the transplant centre ( n  = 147), 7% were from the mental health trust ( n  = 78), and 9% from the ambulance trust ( n  = 96). The analysis uncovered a main theme of priority and relevance made up of five sub-themes.

The first sub-theme identified one reason to be that it was their “individual decision” [R3] and there would be “nothing to be gained” [R248] from a discussion. Some respondents stated that despite a lack of discussion, their family members would assume their wishes in relation to organ donation and support these, “I imagine they are all of the same mindset” [R4470]. However, some stated their reasons to be because they “don’t have a family” [R1127] to discuss this with or have “young ones whose understanding is limited about organ donation” [R356] . Positively, there were several respondents who suggested the question had acted as a prompt to speak to their family.

Another reason stated by respondents was that they found the topic to be too difficult to discuss due to “recent bereavements” [R444], “current environment” [R441] , and “a reluctance to address death” [R4486] . As evident in the latter quote, many respondents viewed discussions around death and dying as a “taboo subject” [R3285] , increasing the avoidance of having such conversations.

Finally, the fifth reason was that several respondents “had not made any decision yet” [R2478] . One respondent wanted to ensure they had reviewed all available information before deciding and having a well-informed discussion with them.

Misconceptions

A further subset analysis of responses coded as misconceptions was reviewed at the request of NHSBT, with interest in whether these occurred from healthcare staff working with donors and recipients. Misconceptions were identified across the three questions, with misconceptions accounting for 24% of the responses to the against the legislation question. Responses used emotive, powerful words with suggestions of State ownership of organs, abuse of the system to procure organs, change in treatment of donors to hasten death, religious and cultural objections, and recipient worthiness.

I worked in organ retrieval theatre during my career and I was uncomfortable with the way the operations were performed during this period. Although the 'brain dead' tests had been completed prior to the operation the vital signs of the patient often reflected that the patient was responding to painful stimuli. Sometimes the patient was not given the usual analgesia that is often given during routine operations. This made me rethink organ donation therefore I am uncomfortable with this. I always carried a donation card prior to my experience but subsequently would not wish to donate. This may be a personal feeling but that is my experience. [R660].

I think that this is a choice that should be left to individuals and families to make. After many years in nursing lots of it spent with transplant patients not all recipients embrace a 'healthy lifestyle' post-transplant with many going back to old lifestyle choices which made a transplant necessary in the first place. [R867].

Additional comments suggested certain medical conditions and advancing age precludes donation and that the ability to choose which organs to donate had been removed.

Most of them will be of no use as I have had a heart attack, I smoke and have Type 2 diabetes. [R595]

Further analysis indicated that 27% ( n  = 24) of these comments were made by individuals who worked with or in an area that supported donors and recipients.

In summary, this qualitative paper has evidenced that the ability to make an autonomous informed decision is foremost in the respondent’s thoughts regarding an opt-out system. This has been commonly cited as a reason throughout the literature by those against an opt-out system [ 9 , 10 , 25 , 26 ]. The loss of that ability was the primary reason for respondents being against the change in legislation with the notion that the decision is a personal choice cited as a reason for lack of discussion with family members. Respondents stated that the ability to make autonomous decisions needs to be adequately supported by evidence-based information that is accessible to all. If the latter is unavailable, they expressed concern for negative consequences. This includes an increase in the perceived belief of the potential for mistakes and abuse of the system, as well as family distress and loss of trust in the donation system and the staff who work in it, as supported by previous literature [ 9 , 11 ].

Our findings further coincide with that of previous literature, highlighting views suggesting that the opt-out system is a move towards an authoritarian system, illustrating the commercialisation of organs, and a system that is open to abuse and mistakes [ 10 , 11 , 12 , 27 , 28 , 29 ]. Healthcare staff require reassurance that the population, specifically the hard-to-reach groups like the elderly and homeless, have access to information and systems in order to be able to make an informed decision [ 30 , 31 ]. Whilst the findings from the overall #options survey demonstrated awareness is higher in NHS staff, there was a significant narrative in the free-text response regarding a lack of awareness and a concern the general public must also lack the same awareness of the system change. Some responses also reflected medical mistrust concerns of the general public [ 13 , 14 , 16 ] as well as expressing a fear of losing trust with the public [ 9 , 11 , 16 ], as found within previous work. Additional research articles raising awareness of the opt-out system in England suggest that despite publicising the change with carefully crafted positive messaging, negative views and attitudes are likely to influence interpretation leading to an increase in misinformation [ 28 ]. Targeted, evidence-based interventions and campaigns that address misinformation, particularly in sub-groups like ethnic minorities, is likely to provide reassurance to NHS staff and the general public, as well as providing reliable resources of information [ 28 ].

Respondents also requested more detailed information about the process of organ donation. The disparity of information and the knowledge of the processes of donation includes eligibility criteria, perceived religious and cultural exclusions, practical processes of brain and circulatory death, and subsequent organ retrieval. As well as, most importantly, more information on the care provided to the donor before and after the donation procedure. The gap of available factual knowledge is instead filled by misconceptions and misunderstandings which is perpetuated until new information and knowledge is acquired. It may also be attributed to the increased awareness of ethical and regulatory processes. These attitudes and views illustrate the complexity of opinions associated with religion, culture, medical mistrust, and ignorance of the donation processes [ 11 , 15 , 32 ]. There is evidently a need for healthcare staff to display openness and transparency about the processes of organ donation and how this is completed, particularly with the donor’s family. It further reinforces the need to increase the knowledge of differing religious and cultural beliefs to support conversations with families [ 18 , 19 ].

Both healthcare staff and the public would benefit from educational materials and interventions to address attitudes towards organ donation [ 19 , 28 , 33 ]. This would assist in correcting misconceptions and misunderstandings held by NHS staff, specifically those who support and work with organ donors and recipients. Previous work illustrates support for donation being higher in intensivists, recommending educational programmes to increase awareness across all healthcare staff [ 34 ]. The quantitative and qualitative findings of the #options survey would support this recommendation, adding that interventions need to be delivered by those working within organ donation and transplantation. This would build on the community work being conducted by NHSBT, hopefully leading NHS staff to become transplant ambassadors within their local communities.

A further finding was that of confusion and misunderstanding surrounding the role of the family, a finding also supported by the literature [ 11 ]. It was suggested that family distress would be heightened, and families would override the premise of opt-out. Literature also supports this could be further impacted if the family holds negative attitudes towards organ donation [ 20 ]. The uncertainty of the donors’ wishes was the most common reason for refusing from ethnic minority groups [ 35 ], further highlighting the need for family discussions. Without this, families feel they are left with no prior indication so they opt-out as a precaution. Making an opt-in decision known can aid the grieving process as the family takes comfort in knowing they are fulfilling the donors wishes [ 26 ] and reduces the likelihood of refusal due to uncertainty about their wishes [ 36 ]. The ambiguity around the role of the family, coupled with not explicitly stating a choice via the organ donor register or discussions with family can make it problematic for next of kin and NHS staff.

Limitations

It is acknowledged that the findings of this study could have been influenced by the COVID-19 pandemic beyond the changes to the research delivery plan including a shift in critical care priorities, initial increase of false information circulating social media, delayed specialist nurse training, and removal of planned public campaigns [ 37 , 38 ]. The degree of the impact is unknown and supports the view that ongoing research into healthcare staff attitudes is required. Additionally, the survey did not collect job titles and is therefore limited to combining all healthcare staff responses. It is understood not all staff, such as those working in mental health, would know in depth details of organ donation and legislation, but it is expected that their level of knowledge would be greater than that of the general public.

The quantitative analysis [ 21 ] of the #options survey showed that overall NHS staff are well informed and more supportive of the change in legislation when compared to the general public. This qualitative analysis of the free-text responses provides a greater insight into the views of the healthcare staff who against the change. The reasons given reflect the known misconceptions and misunderstandings held by the general public and evidenced within the literature [ 9 , 10 , 11 , 12 , 13 , 14 , 15 , 16 ]. There are further concerns about the rationale for the change, the nature of the informed decision making, ease of access to information including information regarding organ donation processes. We therefore propose that educational materials and interventions for NHS staff are developed to address the concepts of autonomy and consent, are transparent about organ donation processes, and address the need for conversations with family. Regarding the wider public awareness campaigns, there is a continued need to promote the positives and refute the negatives to fill the knowledge gap with evidence-based information [ 39 ] and reduce misconceptions and misunderstandings.

Availability of data and materials

The datasets analysed during the current study are available from the corresponding author on reasonable request.

Abbreviations

Coronavirus Disease 2019

Integrated research application system

North-East and North Cumbria

• National Health Service

National Health Service Blood and Transplant

National Institute of Health Research

Organ donor register

United Kingdom

NHS Blood and Transplant. Max and Keira’s Law come into effect in England, https://www.organdonation.nhs.uk/get-involved/news/max-and-keira-s-law-comes-into-effect-in-england (Accessed 24 Feb 2021).

NHS Blood and Transplant. Organ donation laws. https://www.organdonation.nhs.uk/uk-laws/ (Accessed 26 July 2023).

Human Tissue Authority. Human Tissue Act 2004: human tissue authority. 2017. https://www.hta.gov.uk/policies/human-tissue-act-2004 (Accessed 07 Apr 2023).

NHS Blood and Transplant. ODT clinical, donation after brainstem death. 2019. https://www.odt.nhs.uk/deceased-donation/best-practice-guidance/donation-after-brainstem-death/ (Accessed 07 Apr 2023).

NHS Blood and Transplant. Organ and Tissue Donation and Transplantation: Activity Report 2022/23. https://nhsbtdbe.blob.core.windows.net/umbraco-assets-corp/30198/activity-report-2022-2023-final.pdf (Accessed 26 July 2023).

NHS Blood and Transplant. Welsh Health Minister celebrates that ‘Opt-out organ donation scheme has transformed lives’. 2020. https://www.organdonation.nhs.uk/get-involved/news/welsh-health-minister-celebrates-that-opt-out-organ-donation-scheme-has-transformed-lives/#:~:text=Consent%20rates%20for%20donation%20have,from%20180%20in%202017%2F18 (Accessed 26 July 2023).

Noyes J, McLaughlin L, Morgan K, et al. Short-term impact of introducing a soft opt-out organ donation system in Wales: before and after study. BMJ Open. 2019;9: e025159. https://doi.org/10.1136/bmjopen-2018-025159 .

Article   Google Scholar  

Madden S, Collett D, Walton P, Empson K, Forsythe J, Ingham A, Morgan K, Murphy P, Neuberger J, Gardiner D. The effect on consent rates for deceased organ donation in Wales after the introduction of an opt-out system. Anaesthesia. 2020;75(9):1146–52. https://doi.org/10.1111/anae.15055 .

Rieu R. The potential impact of an opt-out system for organ donation in the UK. Law, ethics and medicine. 2010;36:534–8. https://doi.org/10.1136/jme.2009.031757 .

Miller J, Currie S, McGregor LM, O’Carroll RE. ‘It’s like being conscripted, one volunteer is better than 10 pressed men’: A qualitative study into the views of people who plan to opt-out of organ donation. 2020; 25: 257–274. https://doi.org/10.1111/bjhp.12406

Miller J, Currie S, O’Carroll RE. ‘If I donate my organs it’s a gift, if you take them it’s theft’: a qualitative study of planned donor decisions under opt-out legislation. BMC Public Health. 2019;19:1463. https://doi.org/10.1186/s12889-019-7774-1 .

Rudge CJ. Organ donation: opting in or opting out? British Journal of General Practice. 2018: 62–63. https://doi.org/10.3399/bjgp18X694445

Morgan SE, Stephenson MT, Harrison TR, Afifi WA, Long SD. Facts versus ‘feelings’: how rational is the decision to become an organ donor? J Health Psychol. 2008;13(5):644–58. https://doi.org/10.1177/1359105308090936 .

Clark NL, Copping L, Swainston K, McGeechan GJ. Attitudes to Organ Donor Registration in England Under Opt-Out Legislation. Progress In Transplantation. 2023; 0(0). doi: https://doi.org/10.1177/15269248231189869

Miller J, Currie S, O’Carroll RE. ‘What if I’m not dead?’ – Myth-busting and organ donation. Br J Health Psychol. 2019;24:141–58. https://doi.org/10.1111/bjhp.12344 .

Organ Donation Taskforce. The potential impact of an opt out system for organ donation in the UK. The National Archives. 2008:1–36.

Morgan M, Kenten C, Deedat S, Farsides B, Newton T, Randhawa G, et al. Increasing the acceptability and rates of organ donation among ethnic groups: a programme of observational and evaluative research on Donation, Transplantation and Ethnicity (DonaTE). Program Grants Appl Res. 2016;4(4):1–196. https://doi.org/10.3310/pgfar04040 .

Doerry K, Oh J, Vincent D, Fishcer L, Schulz-Jurgensen S. Religious and cultural aspects of organ donation: Narrowing the gap through understanding different religious beliefs. Pediatr Transplant. 2022;26: e14339. https://doi.org/10.1111/petr.14339 .

Witjes M, Jansen NE, van der Hoeven JG, Abdo WF. Interventions aimed at healthcare professionals to increase the number of organ donors: a systematic review. Crit Care. 2019;23:227. https://doi.org/10.1186/s13054-019-2509-3 .

Shepherd L, O’Carroll RE, Ferguson E. Assessing the factors that influence the donation of a deceased family member’s organs in an opt-out system for organ donation. Soc Sci Med. 2023;317: 115545. https://doi.org/10.1016/j.socscimed.2022.115545 .

Coe D, Newell N, Jones M, Robb M, Clark N, Reaich D, Wroe C. NHS staff awareness, attitudes and actions towards the change in organ donation law in England – results of the #options survey 2020. Archives of Public Health. 2023;81:88. https://doi.org/10.1186/s13690-023-01099-y .

Welsh Government Social Research. Survey of Public Attitudes to Organ Donation: Wave 2. 66/2013, Welsh Government, 2013. https://gov.wales/sites/default/files/statistics-and-research/2019-04/public-attitudes-organ-donation-wave-2.pdf (Accessed 29 June 2021).

Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care. 2007;19(6):349–57. https://doi.org/10.1093/intqhc/mzm042 .

Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2006;3:77–101.

Cantrell TK. The ‘opt-out’ approach to deceased organ donation in England: A misconceived policy which may precipitate moral harm. Clin Ethics. 2019;14:63–9. https://doi.org/10.1177/1477750919851052 .

Hoeyer K, Jensen AMB, Olejaz M. Transplantation as an abstract good: practising deliberate ignorance in deceased organ donation in Denmark. Sociol Health Illn. 2015;37:578–93. https://doi.org/10.1111/1467-9566.12211 .

Dalal AR. Philosophy of organ donation: Review of ethical facets. World J Transplant. 2015;5:44–51. https://doi.org/10.5500/wjt.v5.i2.44 .

Faherty G, Williams L, Noyes J, McLaughlin L, Bostock J, Mays N. Analysis of content and online public responses to media articles that raise awareness of the opt-out system of consent to organ donation in England. Front Public Health. 2022. https://doi.org/10.3389/fpubh.2022.1067635 .

Koplin JJ. From blood donation to kidney sales: the gift relationship and transplant commercialism. Monash Bioeth Rev. 2015;33:102–22. https://doi.org/10.3389/fpubh.2022.1067635 .

Morgan M, Kenten C, Deedat S, et al. Attitudes to deceased organ donation and registration as a donor among minority ethnic groups in North America and the UK: a synthesis of quantitative and qualitative research. Ethn Health. 2013;18:367–90. https://doi.org/10.1080/13557858.2012.752073 .

Truijens D, van Exel J. Views on deceased organ donation in the Netherlands: A q-methodology study. PLoS ONE. 2019;14: e0216479. https://doi.org/10.1371/journal.pone.0216479 .

Irving MJ, Tong A, Jan S, Cass A, Rose J, Chadban S, Allen RD, Craig JC, Wong G, Howard K. Factors that influence the decision to be an organ donor: a systematic review of the qualitative literature. Nephrol Dial Transplant. 2012;27:2526–33. https://doi.org/10.1093/ndt/gfr683 .

Radunz S, Hertel S, Schmid KW, Heuer M, Stommel P, Fruhauf NR, Saner FH, Paul A, Kaiser GM. Attitude of Health Care Professionals to Organ Donation: Two Surveys Among the Staff of a German University Hospital. Transplant Proc. 2010;42:126–9. https://doi.org/10.1016/j.transproceed.2009.12.034 .

Umana E, Grant O, Curran E, May P, Mohamed A, O’Donnell J. Attitudes and knowledge of healthcare professionals regarding organ donation. A survey of the Saolta University health care group. Ir Med J. 2018;111:838.

Google Scholar  

Daga S, Patel R, Howard D, et al. ‘Pass it on’ - New Organ Donation Law in England May 2020: What Black, Asian or Minority Ethnic (BAME) Communities should do and Why? The Physician; 6. Epub ahead of print 5 May 2020. DOI: https://doi.org/10.38192/1.6.1.7 .

NHS Organ Donation. The UK Opt-Out Experience. https://www.youtube.com/watch?v=22oCq5NKoiE&t=3211s . YouTube 25 February 2021. Accessed 05 Dec 2023.

UK Parliament, House of Lords, Lords Chamber, Volume 803: debated 18 May 2020. Draft Human Tissue (Permitted Material: Exceptions) (England) Regulations 2020. https://hansard.parliament.uk/lords/2020-05-18/debates/1a7747af-1951-4289-b8d5-639493c85bb1/LordsChamber . 18 May 2020. Accessed 05 Dec 2023.

Miller J, McGregor L, Currie S, O’Carroll RE. Investigating the effects of threatening language, message framing, and reactance in opt-out organ donation campaigns. Ann Behav Med. 2022. https://doi.org/10.1093/abm/kaab017 .

Download references

Acknowledgements

With thanks to the NHSBT legislation implementation team for peer review of the questionnaire and the Kantar population survey data.

Funding for the project was gained from the Northern Counties Kidney Research Fund. Grant number 16.01.

Author information

Authors and affiliations.

South Tees Hospitals NHS Foundation Trust, Middlesbrough, North Yorkshire, England, UK

Natalie L. Clark & David Reaich

Newcastle-Upon-Tyne Hospitals NHS Foundation Trust, Newcastle Upon Tyne, Tyne and Wear, England, UK

Dorothy Coe & Caroline Wroe

Centre for Process Innovation, Sedgefield, County Durham, England, UK

Natasha Newell

NHS Blood and Transplant, Bristol, England, UK

Mark N. A. Jones & Matthew Robb

You can also search for this author in PubMed   Google Scholar

Contributions

NC, DC, and CW were responsible for the drafting and revising of the manuscript. NN, MJ, MR, DR, and CW were responsible for the design of the study. DC completed the qualitative analysis. NC, DC, NN, MJ, MR, DR, and CW read and approved the final manuscript.

Corresponding author

Correspondence to Caroline Wroe .

Ethics declarations

Ethics approval and consent to participate.

The research was carried out in accordance with the Declaration of Helsinki. The study was reviewed and approved by a Health Research Authority (HRA) and Health and Care Research Wales (HCRW) [REC reference: 20/HRA/0150] via the integrated research application system (IRAS) and registered as a National Institute of Health Research (NIHR) portfolio trial [IRAS 275992]. Informed Consent was obtained from all the participants and/or their legal guardians.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary material 1., supplementary material 2., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Clark, N.L., Coe, D., Newell, N. et al. “I am in favour of organ donation, but I feel you should opt-in”—qualitative analysis of the #options 2020 survey free-text responses from NHS staff toward opt-out organ donation legislation in England. BMC Med Ethics 25 , 47 (2024). https://doi.org/10.1186/s12910-024-01048-6

Download citation

Received : 19 September 2023

Accepted : 17 April 2024

Published : 20 April 2024

DOI : https://doi.org/10.1186/s12910-024-01048-6

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Organ donation
• Legislation
• Qualitative

BMC Medical Ethics

ISSN: 1472-6939

• General enquiries: [email protected]