qualitative and quantitative case study

Qualitative vs Quantitative Research Methods & Data Analysis

Saul Mcleod, PhD

Editor-in-Chief for Simply Psychology

BSc (Hons) Psychology, MRes, PhD, University of Manchester

Saul Mcleod, PhD., is a qualified psychology teacher with over 18 years of experience in further and higher education. He has been published in peer-reviewed journals, including the Journal of Clinical Psychology.

Learn about our Editorial Process

Olivia Guy-Evans, MSc

Associate Editor for Simply Psychology

BSc (Hons) Psychology, MSc Psychology of Education

Olivia Guy-Evans is a writer and associate editor for Simply Psychology. She has previously worked in healthcare and educational sectors.

On This Page:

What is the difference between quantitative and qualitative?

The main difference between quantitative and qualitative research is the type of data they collect and analyze.

Quantitative research collects numerical data and analyzes it using statistical methods. The aim is to produce objective, empirical data that can be measured and expressed in numerical terms. Quantitative research is often used to test hypotheses, identify patterns, and make predictions.

Qualitative research , on the other hand, collects non-numerical data such as words, images, and sounds. The focus is on exploring subjective experiences, opinions, and attitudes, often through observation and interviews.

Qualitative research aims to produce rich and detailed descriptions of the phenomenon being studied, and to uncover new insights and meanings.

Quantitative data is information about quantities, and therefore numbers, and qualitative data is descriptive, and regards phenomenon which can be observed but not measured, such as language.

What Is Qualitative Research?

Qualitative research is the process of collecting, analyzing, and interpreting non-numerical data, such as language. Qualitative research can be used to understand how an individual subjectively perceives and gives meaning to their social reality.

Qualitative data is non-numerical data, such as text, video, photographs, or audio recordings. This type of data can be collected using diary accounts or in-depth interviews and analyzed using grounded theory or thematic analysis.

Qualitative research is multimethod in focus, involving an interpretive, naturalistic approach to its subject matter. This means that qualitative researchers study things in their natural settings, attempting to make sense of, or interpret, phenomena in terms of the meanings people bring to them. Denzin and Lincoln (1994, p. 2)

Interest in qualitative data came about as the result of the dissatisfaction of some psychologists (e.g., Carl Rogers) with the scientific study of psychologists such as behaviorists (e.g., Skinner ).

Since psychologists study people, the traditional approach to science is not seen as an appropriate way of carrying out research since it fails to capture the totality of human experience and the essence of being human.  Exploring participants’ experiences is known as a phenomenological approach (re: Humanism ).

Qualitative research is primarily concerned with meaning, subjectivity, and lived experience. The goal is to understand the quality and texture of people’s experiences, how they make sense of them, and the implications for their lives.

Qualitative research aims to understand the social reality of individuals, groups, and cultures as nearly as possible as participants feel or live it. Thus, people and groups are studied in their natural setting.

Some examples of qualitative research questions are provided, such as what an experience feels like, how people talk about something, how they make sense of an experience, and how events unfold for people.

Research following a qualitative approach is exploratory and seeks to explain ‘how’ and ‘why’ a particular phenomenon, or behavior, operates as it does in a particular context. It can be used to generate hypotheses and theories from the data.

Qualitative Methods

There are different types of qualitative research methods, including diary accounts, in-depth interviews , documents, focus groups , case study research , and ethnography.

The results of qualitative methods provide a deep understanding of how people perceive their social realities and in consequence, how they act within the social world.

The researcher has several methods for collecting empirical materials, ranging from the interview to direct observation, to the analysis of artifacts, documents, and cultural records, to the use of visual materials or personal experience. Denzin and Lincoln (1994, p. 14)

Here are some examples of qualitative data:

Interview transcripts : Verbatim records of what participants said during an interview or focus group. They allow researchers to identify common themes and patterns, and draw conclusions based on the data. Interview transcripts can also be useful in providing direct quotes and examples to support research findings.

Observations : The researcher typically takes detailed notes on what they observe, including any contextual information, nonverbal cues, or other relevant details. The resulting observational data can be analyzed to gain insights into social phenomena, such as human behavior, social interactions, and cultural practices.

Unstructured interviews : generate qualitative data through the use of open questions.  This allows the respondent to talk in some depth, choosing their own words.  This helps the researcher develop a real sense of a person’s understanding of a situation.

Diaries or journals : Written accounts of personal experiences or reflections.

Notice that qualitative data could be much more than just words or text. Photographs, videos, sound recordings, and so on, can be considered qualitative data. Visual data can be used to understand behaviors, environments, and social interactions.

Qualitative Data Analysis

Qualitative research is endlessly creative and interpretive. The researcher does not just leave the field with mountains of empirical data and then easily write up his or her findings.

Qualitative interpretations are constructed, and various techniques can be used to make sense of the data, such as content analysis, grounded theory (Glaser & Strauss, 1967), thematic analysis (Braun & Clarke, 2006), or discourse analysis.

For example, thematic analysis is a qualitative approach that involves identifying implicit or explicit ideas within the data. Themes will often emerge once the data has been coded .

Key Features

• Events can be understood adequately only if they are seen in context. Therefore, a qualitative researcher immerses her/himself in the field, in natural surroundings. The contexts of inquiry are not contrived; they are natural. Nothing is predefined or taken for granted.
• Qualitative researchers want those who are studied to speak for themselves, to provide their perspectives in words and other actions. Therefore, qualitative research is an interactive process in which the persons studied teach the researcher about their lives.
• The qualitative researcher is an integral part of the data; without the active participation of the researcher, no data exists.
• The study’s design evolves during the research and can be adjusted or changed as it progresses. For the qualitative researcher, there is no single reality. It is subjective and exists only in reference to the observer.
• The theory is data-driven and emerges as part of the research process, evolving from the data as they are collected.

Limitations of Qualitative Research

• Because of the time and costs involved, qualitative designs do not generally draw samples from large-scale data sets.
• The problem of adequate validity or reliability is a major criticism. Because of the subjective nature of qualitative data and its origin in single contexts, it is difficult to apply conventional standards of reliability and validity. For example, because of the central role played by the researcher in the generation of data, it is not possible to replicate qualitative studies.
• Also, contexts, situations, events, conditions, and interactions cannot be replicated to any extent, nor can generalizations be made to a wider context than the one studied with confidence.
• The time required for data collection, analysis, and interpretation is lengthy. Analysis of qualitative data is difficult, and expert knowledge of an area is necessary to interpret qualitative data. Great care must be taken when doing so, for example, looking for mental illness symptoms.

Advantages of Qualitative Research

• Because of close researcher involvement, the researcher gains an insider’s view of the field. This allows the researcher to find issues that are often missed (such as subtleties and complexities) by the scientific, more positivistic inquiries.
• Qualitative descriptions can be important in suggesting possible relationships, causes, effects, and dynamic processes.
• Qualitative analysis allows for ambiguities/contradictions in the data, which reflect social reality (Denscombe, 2010).
• Qualitative research uses a descriptive, narrative style; this research might be of particular benefit to the practitioner as she or he could turn to qualitative reports to examine forms of knowledge that might otherwise be unavailable, thereby gaining new insight.

What Is Quantitative Research?

Quantitative research involves the process of objectively collecting and analyzing numerical data to describe, predict, or control variables of interest.

The goals of quantitative research are to test causal relationships between variables , make predictions, and generalize results to wider populations.

Quantitative researchers aim to establish general laws of behavior and phenomenon across different settings/contexts. Research is used to test a theory and ultimately support or reject it.

Quantitative Methods

Experiments typically yield quantitative data, as they are concerned with measuring things.  However, other research methods, such as controlled observations and questionnaires , can produce both quantitative information.

For example, a rating scale or closed questions on a questionnaire would generate quantitative data as these produce either numerical data or data that can be put into categories (e.g., “yes,” “no” answers).

Experimental methods limit how research participants react to and express appropriate social behavior.

Findings are, therefore, likely to be context-bound and simply a reflection of the assumptions that the researcher brings to the investigation.

There are numerous examples of quantitative data in psychological research, including mental health. Here are a few examples:

Another example is the Experience in Close Relationships Scale (ECR), a self-report questionnaire widely used to assess adult attachment styles .

The ECR provides quantitative data that can be used to assess attachment styles and predict relationship outcomes.

Neuroimaging data : Neuroimaging techniques, such as MRI and fMRI, provide quantitative data on brain structure and function.

This data can be analyzed to identify brain regions involved in specific mental processes or disorders.

For example, the Beck Depression Inventory (BDI) is a clinician-administered questionnaire widely used to assess the severity of depressive symptoms in individuals.

The BDI consists of 21 questions, each scored on a scale of 0 to 3, with higher scores indicating more severe depressive symptoms. 

Quantitative Data Analysis

Statistics help us turn quantitative data into useful information to help with decision-making. We can use statistics to summarize our data, describing patterns, relationships, and connections. Statistics can be descriptive or inferential.

Descriptive statistics help us to summarize our data. In contrast, inferential statistics are used to identify statistically significant differences between groups of data (such as intervention and control groups in a randomized control study).

• Quantitative researchers try to control extraneous variables by conducting their studies in the lab.
• The research aims for objectivity (i.e., without bias) and is separated from the data.
• The design of the study is determined before it begins.
• For the quantitative researcher, the reality is objective, exists separately from the researcher, and can be seen by anyone.
• Research is used to test a theory and ultimately support or reject it.

Limitations of Quantitative Research

• Context: Quantitative experiments do not take place in natural settings. In addition, they do not allow participants to explain their choices or the meaning of the questions they may have for those participants (Carr, 1994).
• Researcher expertise: Poor knowledge of the application of statistical analysis may negatively affect analysis and subsequent interpretation (Black, 1999).
• Variability of data quantity: Large sample sizes are needed for more accurate analysis. Small-scale quantitative studies may be less reliable because of the low quantity of data (Denscombe, 2010). This also affects the ability to generalize study findings to wider populations.
• Confirmation bias: The researcher might miss observing phenomena because of focus on theory or hypothesis testing rather than on the theory of hypothesis generation.

Advantages of Quantitative Research

• Scientific objectivity: Quantitative data can be interpreted with statistical analysis, and since statistics are based on the principles of mathematics, the quantitative approach is viewed as scientifically objective and rational (Carr, 1994; Denscombe, 2010).
• Useful for testing and validating already constructed theories.
• Rapid analysis: Sophisticated software removes much of the need for prolonged data analysis, especially with large volumes of data involved (Antonius, 2003).
• Replication: Quantitative data is based on measured values and can be checked by others because numerical data is less open to ambiguities of interpretation.
• Hypotheses can also be tested because of statistical analysis (Antonius, 2003).

Antonius, R. (2003). Interpreting quantitative data with SPSS . Sage.

Black, T. R. (1999). Doing quantitative research in the social sciences: An integrated approach to research design, measurement and statistics . Sage.

Braun, V. & Clarke, V. (2006). Using thematic analysis in psychology . Qualitative Research in Psychology , 3, 77–101.

Carr, L. T. (1994). The strengths and weaknesses of quantitative and qualitative research : what method for nursing? Journal of advanced nursing, 20(4) , 716-721.

Denscombe, M. (2010). The Good Research Guide: for small-scale social research. McGraw Hill.

Denzin, N., & Lincoln. Y. (1994). Handbook of Qualitative Research. Thousand Oaks, CA, US: Sage Publications Inc.

Glaser, B. G., Strauss, A. L., & Strutzel, E. (1968). The discovery of grounded theory; strategies for qualitative research. Nursing research, 17(4) , 364.

Minichiello, V. (1990). In-Depth Interviewing: Researching People. Longman Cheshire.

Punch, K. (1998). Introduction to Social Research: Quantitative and Qualitative Approaches. London: Sage

Further Information

• Designing qualitative research
• Methods of data collection and analysis
• Introduction to quantitative and qualitative research
• Checklists for improving rigour in qualitative research: a case of the tail wagging the dog?
• Qualitative research in health care: Analysing qualitative data
• Qualitative data analysis: the framework approach
• Using the framework method for the analysis of
• Qualitative data in multi-disciplinary health research
• Content Analysis
• Grounded Theory
• Thematic Analysis

Related Articles

Research Methodology

Qualitative Data Coding

What Is a Focus Group?

Cross-Cultural Research Methodology In Psychology

What Is Internal Validity In Research?

Research Methodology , Statistics

What Is Face Validity In Research? Importance & How To Measure

Criterion Validity: Definition & Examples

The Ultimate Guide to Qualitative Research - Part 1: The Basics

• Introduction and overview
• What is qualitative research?
• What is qualitative data?
• Examples of qualitative data
• Qualitative vs. quantitative research
• Mixed methods
• Qualitative research preparation
• Theoretical perspective
• Theoretical framework
• Literature reviews

Research question

• Conceptual framework
• Conceptual vs. theoretical framework

Data collection

• Qualitative research methods
• Focus groups
• Observational research

What is a case study?

Applications for case study research, what is a good case study, process of case study design, benefits and limitations of case studies.

• Ethnographical research
• Ethical considerations
• Confidentiality and privacy
• Power dynamics
• Reflexivity

Case studies

Case studies are essential to qualitative research , offering a lens through which researchers can investigate complex phenomena within their real-life contexts. This chapter explores the concept, purpose, applications, examples, and types of case studies and provides guidance on how to conduct case study research effectively.

Whereas quantitative methods look at phenomena at scale, case study research looks at a concept or phenomenon in considerable detail. While analyzing a single case can help understand one perspective regarding the object of research inquiry, analyzing multiple cases can help obtain a more holistic sense of the topic or issue. Let's provide a basic definition of a case study, then explore its characteristics and role in the qualitative research process.

Definition of a case study

A case study in qualitative research is a strategy of inquiry that involves an in-depth investigation of a phenomenon within its real-world context. It provides researchers with the opportunity to acquire an in-depth understanding of intricate details that might not be as apparent or accessible through other methods of research. The specific case or cases being studied can be a single person, group, or organization – demarcating what constitutes a relevant case worth studying depends on the researcher and their research question .

Among qualitative research methods , a case study relies on multiple sources of evidence, such as documents, artifacts, interviews , or observations , to present a complete and nuanced understanding of the phenomenon under investigation. The objective is to illuminate the readers' understanding of the phenomenon beyond its abstract statistical or theoretical explanations.

Characteristics of case studies

Case studies typically possess a number of distinct characteristics that set them apart from other research methods. These characteristics include a focus on holistic description and explanation, flexibility in the design and data collection methods, reliance on multiple sources of evidence, and emphasis on the context in which the phenomenon occurs.

Furthermore, case studies can often involve a longitudinal examination of the case, meaning they study the case over a period of time. These characteristics allow case studies to yield comprehensive, in-depth, and richly contextualized insights about the phenomenon of interest.

The role of case studies in research

Case studies hold a unique position in the broader landscape of research methods aimed at theory development. They are instrumental when the primary research interest is to gain an intensive, detailed understanding of a phenomenon in its real-life context.

In addition, case studies can serve different purposes within research - they can be used for exploratory, descriptive, or explanatory purposes, depending on the research question and objectives. This flexibility and depth make case studies a valuable tool in the toolkit of qualitative researchers.

Remember, a well-conducted case study can offer a rich, insightful contribution to both academic and practical knowledge through theory development or theory verification, thus enhancing our understanding of complex phenomena in their real-world contexts.

What is the purpose of a case study?

Case study research aims for a more comprehensive understanding of phenomena, requiring various research methods to gather information for qualitative analysis . Ultimately, a case study can allow the researcher to gain insight into a particular object of inquiry and develop a theoretical framework relevant to the research inquiry.

Why use case studies in qualitative research?

Using case studies as a research strategy depends mainly on the nature of the research question and the researcher's access to the data.

Conducting case study research provides a level of detail and contextual richness that other research methods might not offer. They are beneficial when there's a need to understand complex social phenomena within their natural contexts.

The explanatory, exploratory, and descriptive roles of case studies

Case studies can take on various roles depending on the research objectives. They can be exploratory when the research aims to discover new phenomena or define new research questions; they are descriptive when the objective is to depict a phenomenon within its context in a detailed manner; and they can be explanatory if the goal is to understand specific relationships within the studied context. Thus, the versatility of case studies allows researchers to approach their topic from different angles, offering multiple ways to uncover and interpret the data .

The impact of case studies on knowledge development

Case studies play a significant role in knowledge development across various disciplines. Analysis of cases provides an avenue for researchers to explore phenomena within their context based on the collected data.

This can result in the production of rich, practical insights that can be instrumental in both theory-building and practice. Case studies allow researchers to delve into the intricacies and complexities of real-life situations, uncovering insights that might otherwise remain hidden.

Types of case studies

In qualitative research , a case study is not a one-size-fits-all approach. Depending on the nature of the research question and the specific objectives of the study, researchers might choose to use different types of case studies. These types differ in their focus, methodology, and the level of detail they provide about the phenomenon under investigation.

Understanding these types is crucial for selecting the most appropriate approach for your research project and effectively achieving your research goals. Let's briefly look at the main types of case studies.

Exploratory case studies

Exploratory case studies are typically conducted to develop a theory or framework around an understudied phenomenon. They can also serve as a precursor to a larger-scale research project. Exploratory case studies are useful when a researcher wants to identify the key issues or questions which can spur more extensive study or be used to develop propositions for further research. These case studies are characterized by flexibility, allowing researchers to explore various aspects of a phenomenon as they emerge, which can also form the foundation for subsequent studies.

Descriptive case studies

Descriptive case studies aim to provide a complete and accurate representation of a phenomenon or event within its context. These case studies are often based on an established theoretical framework, which guides how data is collected and analyzed. The researcher is concerned with describing the phenomenon in detail, as it occurs naturally, without trying to influence or manipulate it.

Explanatory case studies

Explanatory case studies are focused on explanation - they seek to clarify how or why certain phenomena occur. Often used in complex, real-life situations, they can be particularly valuable in clarifying causal relationships among concepts and understanding the interplay between different factors within a specific context.

Intrinsic, instrumental, and collective case studies

These three categories of case studies focus on the nature and purpose of the study. An intrinsic case study is conducted when a researcher has an inherent interest in the case itself. Instrumental case studies are employed when the case is used to provide insight into a particular issue or phenomenon. A collective case study, on the other hand, involves studying multiple cases simultaneously to investigate some general phenomena.

Each type of case study serves a different purpose and has its own strengths and challenges. The selection of the type should be guided by the research question and objectives, as well as the context and constraints of the research.

The flexibility, depth, and contextual richness offered by case studies make this approach an excellent research method for various fields of study. They enable researchers to investigate real-world phenomena within their specific contexts, capturing nuances that other research methods might miss. Across numerous fields, case studies provide valuable insights into complex issues.

Critical information systems research

Case studies provide a detailed understanding of the role and impact of information systems in different contexts. They offer a platform to explore how information systems are designed, implemented, and used and how they interact with various social, economic, and political factors. Case studies in this field often focus on examining the intricate relationship between technology, organizational processes, and user behavior, helping to uncover insights that can inform better system design and implementation.

Health research

Health research is another field where case studies are highly valuable. They offer a way to explore patient experiences, healthcare delivery processes, and the impact of various interventions in a real-world context.

Case studies can provide a deep understanding of a patient's journey, giving insights into the intricacies of disease progression, treatment effects, and the psychosocial aspects of health and illness.

Asthma research studies

Specifically within medical research, studies on asthma often employ case studies to explore the individual and environmental factors that influence asthma development, management, and outcomes. A case study can provide rich, detailed data about individual patients' experiences, from the triggers and symptoms they experience to the effectiveness of various management strategies. This can be crucial for developing patient-centered asthma care approaches.

Other fields

Apart from the fields mentioned, case studies are also extensively used in business and management research, education research, and political sciences, among many others. They provide an opportunity to delve into the intricacies of real-world situations, allowing for a comprehensive understanding of various phenomena.

Case studies, with their depth and contextual focus, offer unique insights across these varied fields. They allow researchers to illuminate the complexities of real-life situations, contributing to both theory and practice.

Whatever field you're in, ATLAS.ti puts your data to work for you

Download a free trial of ATLAS.ti to turn your data into insights.

Understanding the key elements of case study design is crucial for conducting rigorous and impactful case study research. A well-structured design guides the researcher through the process, ensuring that the study is methodologically sound and its findings are reliable and valid. The main elements of case study design include the research question , propositions, units of analysis, and the logic linking the data to the propositions.

The research question is the foundation of any research study. A good research question guides the direction of the study and informs the selection of the case, the methods of collecting data, and the analysis techniques. A well-formulated research question in case study research is typically clear, focused, and complex enough to merit further detailed examination of the relevant case(s).

Propositions

Propositions, though not necessary in every case study, provide a direction by stating what we might expect to find in the data collected. They guide how data is collected and analyzed by helping researchers focus on specific aspects of the case. They are particularly important in explanatory case studies, which seek to understand the relationships among concepts within the studied phenomenon.

Units of analysis

The unit of analysis refers to the case, or the main entity or entities that are being analyzed in the study. In case study research, the unit of analysis can be an individual, a group, an organization, a decision, an event, or even a time period. It's crucial to clearly define the unit of analysis, as it shapes the qualitative data analysis process by allowing the researcher to analyze a particular case and synthesize analysis across multiple case studies to draw conclusions.

Argumentation

This refers to the inferential model that allows researchers to draw conclusions from the data. The researcher needs to ensure that there is a clear link between the data, the propositions (if any), and the conclusions drawn. This argumentation is what enables the researcher to make valid and credible inferences about the phenomenon under study.

Understanding and carefully considering these elements in the design phase of a case study can significantly enhance the quality of the research. It can help ensure that the study is methodologically sound and its findings contribute meaningful insights about the case.

Ready to jumpstart your research with ATLAS.ti?

Conceptualize your research project with our intuitive data analysis interface. Download a free trial today.

Conducting a case study involves several steps, from defining the research question and selecting the case to collecting and analyzing data . This section outlines these key stages, providing a practical guide on how to conduct case study research.

Defining the research question

The first step in case study research is defining a clear, focused research question. This question should guide the entire research process, from case selection to analysis. It's crucial to ensure that the research question is suitable for a case study approach. Typically, such questions are exploratory or descriptive in nature and focus on understanding a phenomenon within its real-life context.

Selecting and defining the case

The selection of the case should be based on the research question and the objectives of the study. It involves choosing a unique example or a set of examples that provide rich, in-depth data about the phenomenon under investigation. After selecting the case, it's crucial to define it clearly, setting the boundaries of the case, including the time period and the specific context.

Previous research can help guide the case study design. When considering a case study, an example of a case could be taken from previous case study research and used to define cases in a new research inquiry. Considering recently published examples can help understand how to select and define cases effectively.

Developing a detailed case study protocol

A case study protocol outlines the procedures and general rules to be followed during the case study. This includes the data collection methods to be used, the sources of data, and the procedures for analysis. Having a detailed case study protocol ensures consistency and reliability in the study.

The protocol should also consider how to work with the people involved in the research context to grant the research team access to collecting data. As mentioned in previous sections of this guide, establishing rapport is an essential component of qualitative research as it shapes the overall potential for collecting and analyzing data.

Collecting data

Gathering data in case study research often involves multiple sources of evidence, including documents, archival records, interviews, observations, and physical artifacts. This allows for a comprehensive understanding of the case. The process for gathering data should be systematic and carefully documented to ensure the reliability and validity of the study.

Analyzing and interpreting data

The next step is analyzing the data. This involves organizing the data , categorizing it into themes or patterns , and interpreting these patterns to answer the research question. The analysis might also involve comparing the findings with prior research or theoretical propositions.

Writing the case study report

The final step is writing the case study report . This should provide a detailed description of the case, the data, the analysis process, and the findings. The report should be clear, organized, and carefully written to ensure that the reader can understand the case and the conclusions drawn from it.

Each of these steps is crucial in ensuring that the case study research is rigorous, reliable, and provides valuable insights about the case.

The type, depth, and quality of data in your study can significantly influence the validity and utility of the study. In case study research, data is usually collected from multiple sources to provide a comprehensive and nuanced understanding of the case. This section will outline the various methods of collecting data used in case study research and discuss considerations for ensuring the quality of the data.

Interviews are a common method of gathering data in case study research. They can provide rich, in-depth data about the perspectives, experiences, and interpretations of the individuals involved in the case. Interviews can be structured , semi-structured , or unstructured , depending on the research question and the degree of flexibility needed.

Observations

Observations involve the researcher observing the case in its natural setting, providing first-hand information about the case and its context. Observations can provide data that might not be revealed in interviews or documents, such as non-verbal cues or contextual information.

Documents and artifacts

Documents and archival records provide a valuable source of data in case study research. They can include reports, letters, memos, meeting minutes, email correspondence, and various public and private documents related to the case.

These records can provide historical context, corroborate evidence from other sources, and offer insights into the case that might not be apparent from interviews or observations.

Physical artifacts refer to any physical evidence related to the case, such as tools, products, or physical environments. These artifacts can provide tangible insights into the case, complementing the data gathered from other sources.

Ensuring the quality of data collection

Determining the quality of data in case study research requires careful planning and execution. It's crucial to ensure that the data is reliable, accurate, and relevant to the research question. This involves selecting appropriate methods of collecting data, properly training interviewers or observers, and systematically recording and storing the data. It also includes considering ethical issues related to collecting and handling data, such as obtaining informed consent and ensuring the privacy and confidentiality of the participants.

Data analysis

Analyzing case study research involves making sense of the rich, detailed data to answer the research question. This process can be challenging due to the volume and complexity of case study data. However, a systematic and rigorous approach to analysis can ensure that the findings are credible and meaningful. This section outlines the main steps and considerations in analyzing data in case study research.

Organizing the data

The first step in the analysis is organizing the data. This involves sorting the data into manageable sections, often according to the data source or the theme. This step can also involve transcribing interviews, digitizing physical artifacts, or organizing observational data.

Categorizing and coding the data

Once the data is organized, the next step is to categorize or code the data. This involves identifying common themes, patterns, or concepts in the data and assigning codes to relevant data segments. Coding can be done manually or with the help of software tools, and in either case, qualitative analysis software can greatly facilitate the entire coding process. Coding helps to reduce the data to a set of themes or categories that can be more easily analyzed.

Identifying patterns and themes

After coding the data, the researcher looks for patterns or themes in the coded data. This involves comparing and contrasting the codes and looking for relationships or patterns among them. The identified patterns and themes should help answer the research question.

Interpreting the data

Once patterns and themes have been identified, the next step is to interpret these findings. This involves explaining what the patterns or themes mean in the context of the research question and the case. This interpretation should be grounded in the data, but it can also involve drawing on theoretical concepts or prior research.

Verification of the data

The last step in the analysis is verification. This involves checking the accuracy and consistency of the analysis process and confirming that the findings are supported by the data. This can involve re-checking the original data, checking the consistency of codes, or seeking feedback from research participants or peers.

Like any research method , case study research has its strengths and limitations. Researchers must be aware of these, as they can influence the design, conduct, and interpretation of the study.

Understanding the strengths and limitations of case study research can also guide researchers in deciding whether this approach is suitable for their research question . This section outlines some of the key strengths and limitations of case study research.

Benefits include the following:

• Rich, detailed data: One of the main strengths of case study research is that it can generate rich, detailed data about the case. This can provide a deep understanding of the case and its context, which can be valuable in exploring complex phenomena.
• Flexibility: Case study research is flexible in terms of design , data collection , and analysis . A sufficient degree of flexibility allows the researcher to adapt the study according to the case and the emerging findings.
• Real-world context: Case study research involves studying the case in its real-world context, which can provide valuable insights into the interplay between the case and its context.
• Multiple sources of evidence: Case study research often involves collecting data from multiple sources , which can enhance the robustness and validity of the findings.

On the other hand, researchers should consider the following limitations:

• Generalizability: A common criticism of case study research is that its findings might not be generalizable to other cases due to the specificity and uniqueness of each case.
• Time and resource intensive: Case study research can be time and resource intensive due to the depth of the investigation and the amount of collected data.
• Complexity of analysis: The rich, detailed data generated in case study research can make analyzing the data challenging.
• Subjectivity: Given the nature of case study research, there may be a higher degree of subjectivity in interpreting the data , so researchers need to reflect on this and transparently convey to audiences how the research was conducted.

Being aware of these strengths and limitations can help researchers design and conduct case study research effectively and interpret and report the findings appropriately.

Ready to analyze your data with ATLAS.ti?

See how our intuitive software can draw key insights from your data with a free trial today.

Research Writing and Analysis

• NVivo Group and Study Sessions
• SPSS This link opens in a new window
• Statistical Analysis Group sessions
• Using Qualtrics
• Dissertation and Data Analysis Group Sessions
• Defense Schedule - Commons Calendar This link opens in a new window
• Research Process Flow Chart
• Research Alignment Chapter 1 This link opens in a new window
• Step 1: Seek Out Evidence
• Step 2: Explain
• Step 3: The Big Picture
• Step 4: Own It
• Step 5: Illustrate
• Annotated Bibliography
• Literature Review This link opens in a new window
• Systematic Reviews & Meta-Analyses
• How to Synthesize and Analyze
• Synthesis and Analysis Practice
• Synthesis and Analysis Group Sessions
• Problem Statement
• Purpose Statement
• Conceptual Framework
• Theoretical Framework
• Locating Theoretical and Conceptual Frameworks This link opens in a new window
• Quantitative Research Questions
• Qualitative Research Questions
• Trustworthiness of Qualitative Data
• Analysis and Coding Example- Qualitative Data
• Thematic Data Analysis in Qualitative Design
• Dissertation to Journal Article This link opens in a new window
• International Journal of Online Graduate Education (IJOGE) This link opens in a new window
• Journal of Research in Innovative Teaching & Learning (JRIT&L) This link opens in a new window

Writing a Case Study

What is a case study?

A Case study is: 

• An in-depth research design that primarily uses a qualitative methodology but sometimes​​ includes quantitative methodology.
• Used to examine an identifiable problem confirmed through research.
• Used to investigate an individual, group of people, organization, or event.
• Used to mostly answer "how" and "why" questions.

What are the different types of case studies?

Note: These are the primary case studies. As you continue to research and learn

about case studies you will begin to find a robust list of different types. 

Who are your case study participants?

What is triangulation ? 

Validity and credibility are an essential part of the case study. Therefore, the researcher should include triangulation to ensure trustworthiness while accurately reflecting what the researcher seeks to investigate.

How to write a Case Study?

When developing a case study, there are different ways you could present the information, but remember to include the five parts for your case study.

Was this resource helpful?

• << Previous: Thematic Data Analysis in Qualitative Design
• Next: Journal Article Reporting Standards (JARS) >>
• Last Updated: May 29, 2024 8:05 AM
• URL: https://resources.nu.edu/researchtools

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, automatically generate references for free.

• Knowledge Base
• Methodology
• Qualitative vs Quantitative Research | Examples & Methods

Qualitative vs Quantitative Research | Examples & Methods

Published on 4 April 2022 by Raimo Streefkerk . Revised on 8 May 2023.

When collecting and analysing data, quantitative research deals with numbers and statistics, while qualitative research  deals with words and meanings. Both are important for gaining different kinds of knowledge.

Common quantitative methods include experiments, observations recorded as numbers, and surveys with closed-ended questions. Qualitative research Qualitative research is expressed in words . It is used to understand concepts, thoughts or experiences. This type of research enables you to gather in-depth insights on topics that are not well understood.

Table of contents

The differences between quantitative and qualitative research, data collection methods, when to use qualitative vs quantitative research, how to analyse qualitative and quantitative data, frequently asked questions about qualitative and quantitative research.

Quantitative and qualitative research use different research methods to collect and analyse data, and they allow you to answer different kinds of research questions.

Prevent plagiarism, run a free check.

Quantitative and qualitative data can be collected using various methods. It is important to use a data collection method that will help answer your research question(s).

Many data collection methods can be either qualitative or quantitative. For example, in surveys, observations or case studies , your data can be represented as numbers (e.g. using rating scales or counting frequencies) or as words (e.g. with open-ended questions or descriptions of what you observe).

However, some methods are more commonly used in one type or the other.

Quantitative data collection methods

• Surveys :  List of closed or multiple choice questions that is distributed to a sample (online, in person, or over the phone).
• Experiments : Situation in which variables are controlled and manipulated to establish cause-and-effect relationships.
• Observations: Observing subjects in a natural environment where variables can’t be controlled.

Qualitative data collection methods

• Interviews : Asking open-ended questions verbally to respondents.
• Focus groups: Discussion among a group of people about a topic to gather opinions that can be used for further research.
• Ethnography : Participating in a community or organisation for an extended period of time to closely observe culture and behavior.
• Literature review : Survey of published works by other authors.

A rule of thumb for deciding whether to use qualitative or quantitative data is:

• Use quantitative research if you want to confirm or test something (a theory or hypothesis)
• Use qualitative research if you want to understand something (concepts, thoughts, experiences)

For most research topics you can choose a qualitative, quantitative or mixed methods approach . Which type you choose depends on, among other things, whether you’re taking an inductive vs deductive research approach ; your research question(s) ; whether you’re doing experimental , correlational , or descriptive research ; and practical considerations such as time, money, availability of data, and access to respondents.

Quantitative research approach

You survey 300 students at your university and ask them questions such as: ‘on a scale from 1-5, how satisfied are your with your professors?’

You can perform statistical analysis on the data and draw conclusions such as: ‘on average students rated their professors 4.4’.

Qualitative research approach

You conduct in-depth interviews with 15 students and ask them open-ended questions such as: ‘How satisfied are you with your studies?’, ‘What is the most positive aspect of your study program?’ and ‘What can be done to improve the study program?’

Based on the answers you get you can ask follow-up questions to clarify things. You transcribe all interviews using transcription software and try to find commonalities and patterns.

Mixed methods approach

You conduct interviews to find out how satisfied students are with their studies. Through open-ended questions you learn things you never thought about before and gain new insights. Later, you use a survey to test these insights on a larger scale.

It’s also possible to start with a survey to find out the overall trends, followed by interviews to better understand the reasons behind the trends.

Qualitative or quantitative data by itself can’t prove or demonstrate anything, but has to be analysed to show its meaning in relation to the research questions. The method of analysis differs for each type of data.

Analysing quantitative data

Quantitative data is based on numbers. Simple maths or more advanced statistical analysis is used to discover commonalities or patterns in the data. The results are often reported in graphs and tables.

Applications such as Excel, SPSS, or R can be used to calculate things like:

• Average scores
• The number of times a particular answer was given
• The correlation or causation between two or more variables
• The reliability and validity of the results

Analysing qualitative data

Qualitative data is more difficult to analyse than quantitative data. It consists of text, images or videos instead of numbers.

Some common approaches to analysing qualitative data include:

• Qualitative content analysis : Tracking the occurrence, position and meaning of words or phrases
• Thematic analysis : Closely examining the data to identify the main themes and patterns
• Discourse analysis : Studying how communication works in social contexts

Quantitative research deals with numbers and statistics, while qualitative research deals with words and meanings.

Quantitative methods allow you to test a hypothesis by systematically collecting and analysing data, while qualitative methods allow you to explore ideas and experiences in depth.

In mixed methods research , you use both qualitative and quantitative data collection and analysis methods to answer your research question .

The research methods you use depend on the type of data you need to answer your research question .

• If you want to measure something or test a hypothesis , use quantitative methods . If you want to explore ideas, thoughts, and meanings, use qualitative methods .
• If you want to analyse a large amount of readily available data, use secondary data. If you want data specific to your purposes with control over how they are generated, collect primary data.
• If you want to establish cause-and-effect relationships between variables , use experimental methods. If you want to understand the characteristics of a research subject, use descriptive methods.

Data collection is the systematic process by which observations or measurements are gathered in research. It is used in many different contexts by academics, governments, businesses, and other organisations.

There are various approaches to qualitative data analysis , but they all share five steps in common:

• Prepare and organise your data.
• Review and explore your data.
• Develop a data coding system.
• Assign codes to the data.
• Identify recurring themes.

The specifics of each step depend on the focus of the analysis. Some common approaches include textual analysis , thematic analysis , and discourse analysis .

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the ‘Cite this Scribbr article’ button to automatically add the citation to our free Reference Generator.

Streefkerk, R. (2023, May 08). Qualitative vs Quantitative Research | Examples & Methods. Scribbr. Retrieved 7 June 2024, from https://www.scribbr.co.uk/research-methods/quantitative-qualitative-research/

Is this article helpful?

Raimo Streefkerk

• Search Menu
• Sign in through your institution
• Browse content in Arts and Humanities
• Browse content in Archaeology
• Anglo-Saxon and Medieval Archaeology
• Archaeological Methodology and Techniques
• Archaeology by Region
• Archaeology of Religion
• Archaeology of Trade and Exchange
• Biblical Archaeology
• Contemporary and Public Archaeology
• Environmental Archaeology
• Historical Archaeology
• History and Theory of Archaeology
• Industrial Archaeology
• Landscape Archaeology
• Mortuary Archaeology
• Prehistoric Archaeology
• Underwater Archaeology
• Zooarchaeology
• Browse content in Architecture
• Architectural Structure and Design
• History of Architecture
• Residential and Domestic Buildings
• Theory of Architecture
• Browse content in Art
• Art Subjects and Themes
• History of Art
• Industrial and Commercial Art
• Theory of Art
• Biographical Studies
• Byzantine Studies
• Browse content in Classical Studies
• Classical History
• Classical Philosophy
• Classical Mythology
• Classical Literature
• Classical Reception
• Classical Art and Architecture
• Classical Oratory and Rhetoric
• Greek and Roman Papyrology
• Greek and Roman Epigraphy
• Greek and Roman Law
• Greek and Roman Archaeology
• Late Antiquity
• Religion in the Ancient World
• Digital Humanities
• Browse content in History
• Colonialism and Imperialism
• Diplomatic History
• Environmental History
• Genealogy, Heraldry, Names, and Honours
• Genocide and Ethnic Cleansing
• Historical Geography
• History by Period
• History of Emotions
• History of Agriculture
• History of Education
• History of Gender and Sexuality
• Industrial History
• Intellectual History
• International History
• Labour History
• Legal and Constitutional History
• Local and Family History
• Maritime History
• Military History
• National Liberation and Post-Colonialism
• Oral History
• Political History
• Public History
• Regional and National History
• Revolutions and Rebellions
• Slavery and Abolition of Slavery
• Social and Cultural History
• Theory, Methods, and Historiography
• Urban History
• World History
• Browse content in Language Teaching and Learning
• Language Learning (Specific Skills)
• Language Teaching Theory and Methods
• Browse content in Linguistics
• Applied Linguistics
• Cognitive Linguistics
• Computational Linguistics
• Forensic Linguistics
• Grammar, Syntax and Morphology
• Historical and Diachronic Linguistics
• History of English
• Language Evolution
• Language Reference
• Language Acquisition
• Language Variation
• Language Families
• Lexicography
• Linguistic Anthropology
• Linguistic Theories
• Linguistic Typology
• Phonetics and Phonology
• Psycholinguistics
• Sociolinguistics
• Translation and Interpretation
• Writing Systems
• Browse content in Literature
• Bibliography
• Children's Literature Studies
• Literary Studies (Romanticism)
• Literary Studies (American)
• Literary Studies (Asian)
• Literary Studies (European)
• Literary Studies (Eco-criticism)
• Literary Studies (Modernism)
• Literary Studies - World
• Literary Studies (1500 to 1800)
• Literary Studies (19th Century)
• Literary Studies (20th Century onwards)
• Literary Studies (African American Literature)
• Literary Studies (British and Irish)
• Literary Studies (Early and Medieval)
• Literary Studies (Fiction, Novelists, and Prose Writers)
• Literary Studies (Gender Studies)
• Literary Studies (Graphic Novels)
• Literary Studies (History of the Book)
• Literary Studies (Plays and Playwrights)
• Literary Studies (Poetry and Poets)
• Literary Studies (Postcolonial Literature)
• Literary Studies (Queer Studies)
• Literary Studies (Science Fiction)
• Literary Studies (Travel Literature)
• Literary Studies (War Literature)
• Literary Studies (Women's Writing)
• Literary Theory and Cultural Studies
• Mythology and Folklore
• Shakespeare Studies and Criticism
• Browse content in Media Studies
• Browse content in Music
• Applied Music
• Dance and Music
• Ethics in Music
• Ethnomusicology
• Gender and Sexuality in Music
• Medicine and Music
• Music Cultures
• Music and Media
• Music and Religion
• Music and Culture
• Music Education and Pedagogy
• Music Theory and Analysis
• Musical Scores, Lyrics, and Libretti
• Musical Structures, Styles, and Techniques
• Musicology and Music History
• Performance Practice and Studies
• Race and Ethnicity in Music
• Sound Studies
• Browse content in Performing Arts
• Browse content in Philosophy
• Aesthetics and Philosophy of Art
• Epistemology
• Feminist Philosophy
• History of Western Philosophy
• Metaphysics
• Moral Philosophy
• Non-Western Philosophy
• Philosophy of Language
• Philosophy of Mind
• Philosophy of Perception
• Philosophy of Science
• Philosophy of Action
• Philosophy of Law
• Philosophy of Religion
• Philosophy of Mathematics and Logic
• Practical Ethics
• Social and Political Philosophy
• Browse content in Religion
• Biblical Studies
• Christianity
• East Asian Religions
• History of Religion
• Judaism and Jewish Studies
• Qumran Studies
• Religion and Education
• Religion and Health
• Religion and Politics
• Religion and Science
• Religion and Law
• Religion and Art, Literature, and Music
• Religious Studies
• Browse content in Society and Culture
• Cookery, Food, and Drink
• Cultural Studies
• Customs and Traditions
• Ethical Issues and Debates
• Hobbies, Games, Arts and Crafts
• Natural world, Country Life, and Pets
• Popular Beliefs and Controversial Knowledge
• Sports and Outdoor Recreation
• Technology and Society
• Travel and Holiday
• Visual Culture
• Browse content in Law
• Arbitration
• Browse content in Company and Commercial Law
• Commercial Law
• Company Law
• Browse content in Comparative Law
• Systems of Law
• Competition Law
• Browse content in Constitutional and Administrative Law
• Government Powers
• Judicial Review
• Local Government Law
• Military and Defence Law
• Parliamentary and Legislative Practice
• Construction Law
• Contract Law
• Browse content in Criminal Law
• Criminal Procedure
• Criminal Evidence Law
• Sentencing and Punishment
• Employment and Labour Law
• Environment and Energy Law
• Browse content in Financial Law
• Banking Law
• Insolvency Law
• History of Law
• Human Rights and Immigration
• Intellectual Property Law
• Browse content in International Law
• Private International Law and Conflict of Laws
• Public International Law
• IT and Communications Law
• Jurisprudence and Philosophy of Law
• Law and Politics
• Law and Society
• Browse content in Legal System and Practice
• Courts and Procedure
• Legal Skills and Practice
• Primary Sources of Law
• Regulation of Legal Profession
• Medical and Healthcare Law
• Browse content in Policing
• Criminal Investigation and Detection
• Police and Security Services
• Police Procedure and Law
• Police Regional Planning
• Browse content in Property Law
• Personal Property Law
• Study and Revision
• Terrorism and National Security Law
• Browse content in Trusts Law
• Wills and Probate or Succession
• Browse content in Medicine and Health
• Browse content in Allied Health Professions
• Arts Therapies
• Clinical Science
• Dietetics and Nutrition
• Occupational Therapy
• Operating Department Practice
• Physiotherapy
• Radiography
• Speech and Language Therapy
• Browse content in Anaesthetics
• General Anaesthesia
• Neuroanaesthesia
• Clinical Neuroscience
• Browse content in Clinical Medicine
• Acute Medicine
• Cardiovascular Medicine
• Clinical Genetics
• Clinical Pharmacology and Therapeutics
• Dermatology
• Endocrinology and Diabetes
• Gastroenterology
• Genito-urinary Medicine
• Geriatric Medicine
• Infectious Diseases
• Medical Toxicology
• Medical Oncology
• Pain Medicine
• Palliative Medicine
• Rehabilitation Medicine
• Respiratory Medicine and Pulmonology
• Rheumatology
• Sleep Medicine
• Sports and Exercise Medicine
• Community Medical Services
• Critical Care
• Emergency Medicine
• Forensic Medicine
• Haematology
• History of Medicine
• Browse content in Medical Skills
• Clinical Skills
• Communication Skills
• Nursing Skills
• Surgical Skills
• Browse content in Medical Dentistry
• Oral and Maxillofacial Surgery
• Paediatric Dentistry
• Restorative Dentistry and Orthodontics
• Surgical Dentistry
• Medical Ethics
• Medical Statistics and Methodology
• Browse content in Neurology
• Clinical Neurophysiology
• Neuropathology
• Nursing Studies
• Browse content in Obstetrics and Gynaecology
• Gynaecology
• Occupational Medicine
• Ophthalmology
• Otolaryngology (ENT)
• Browse content in Paediatrics
• Neonatology
• Browse content in Pathology
• Chemical Pathology
• Clinical Cytogenetics and Molecular Genetics
• Histopathology
• Medical Microbiology and Virology
• Patient Education and Information
• Browse content in Pharmacology
• Psychopharmacology
• Browse content in Popular Health
• Caring for Others
• Complementary and Alternative Medicine
• Self-help and Personal Development
• Browse content in Preclinical Medicine
• Cell Biology
• Molecular Biology and Genetics
• Reproduction, Growth and Development
• Primary Care
• Professional Development in Medicine
• Browse content in Psychiatry
• Addiction Medicine
• Child and Adolescent Psychiatry
• Forensic Psychiatry
• Learning Disabilities
• Old Age Psychiatry
• Psychotherapy
• Browse content in Public Health and Epidemiology
• Epidemiology
• Public Health
• Browse content in Radiology
• Clinical Radiology
• Interventional Radiology
• Nuclear Medicine
• Radiation Oncology
• Reproductive Medicine
• Browse content in Surgery
• Cardiothoracic Surgery
• Gastro-intestinal and Colorectal Surgery
• General Surgery
• Neurosurgery
• Paediatric Surgery
• Peri-operative Care
• Plastic and Reconstructive Surgery
• Surgical Oncology
• Transplant Surgery
• Trauma and Orthopaedic Surgery
• Vascular Surgery
• Browse content in Science and Mathematics
• Browse content in Biological Sciences
• Aquatic Biology
• Biochemistry
• Bioinformatics and Computational Biology
• Developmental Biology
• Ecology and Conservation
• Evolutionary Biology
• Genetics and Genomics
• Microbiology
• Molecular and Cell Biology
• Natural History
• Plant Sciences and Forestry
• Research Methods in Life Sciences
• Structural Biology
• Systems Biology
• Zoology and Animal Sciences
• Browse content in Chemistry
• Analytical Chemistry
• Computational Chemistry
• Crystallography
• Environmental Chemistry
• Industrial Chemistry
• Inorganic Chemistry
• Materials Chemistry
• Medicinal Chemistry
• Mineralogy and Gems
• Organic Chemistry
• Physical Chemistry
• Polymer Chemistry
• Study and Communication Skills in Chemistry
• Theoretical Chemistry
• Browse content in Computer Science
• Artificial Intelligence
• Computer Architecture and Logic Design
• Game Studies
• Human-Computer Interaction
• Mathematical Theory of Computation
• Programming Languages
• Software Engineering
• Systems Analysis and Design
• Virtual Reality
• Browse content in Computing
• Business Applications
• Computer Security
• Computer Games
• Computer Networking and Communications
• Digital Lifestyle
• Graphical and Digital Media Applications
• Operating Systems
• Browse content in Earth Sciences and Geography
• Atmospheric Sciences
• Environmental Geography
• Geology and the Lithosphere
• Maps and Map-making
• Meteorology and Climatology
• Oceanography and Hydrology
• Palaeontology
• Physical Geography and Topography
• Regional Geography
• Soil Science
• Urban Geography
• Browse content in Engineering and Technology
• Agriculture and Farming
• Biological Engineering
• Civil Engineering, Surveying, and Building
• Electronics and Communications Engineering
• Energy Technology
• Engineering (General)
• Environmental Science, Engineering, and Technology
• History of Engineering and Technology
• Mechanical Engineering and Materials
• Technology of Industrial Chemistry
• Transport Technology and Trades
• Browse content in Environmental Science
• Applied Ecology (Environmental Science)
• Conservation of the Environment (Environmental Science)
• Environmental Sustainability
• Environmentalist Thought and Ideology (Environmental Science)
• Management of Land and Natural Resources (Environmental Science)
• Natural Disasters (Environmental Science)
• Nuclear Issues (Environmental Science)
• Pollution and Threats to the Environment (Environmental Science)
• Social Impact of Environmental Issues (Environmental Science)
• History of Science and Technology
• Browse content in Materials Science
• Ceramics and Glasses
• Composite Materials
• Metals, Alloying, and Corrosion
• Nanotechnology
• Browse content in Mathematics
• Applied Mathematics
• Biomathematics and Statistics
• History of Mathematics
• Mathematical Education
• Mathematical Finance
• Mathematical Analysis
• Numerical and Computational Mathematics
• Probability and Statistics
• Pure Mathematics
• Browse content in Neuroscience
• Cognition and Behavioural Neuroscience
• Development of the Nervous System
• Disorders of the Nervous System
• History of Neuroscience
• Invertebrate Neurobiology
• Molecular and Cellular Systems
• Neuroendocrinology and Autonomic Nervous System
• Neuroscientific Techniques
• Sensory and Motor Systems
• Browse content in Physics
• Astronomy and Astrophysics
• Atomic, Molecular, and Optical Physics
• Biological and Medical Physics
• Classical Mechanics
• Computational Physics
• Condensed Matter Physics
• Electromagnetism, Optics, and Acoustics
• History of Physics
• Mathematical and Statistical Physics
• Measurement Science
• Nuclear Physics
• Particles and Fields
• Plasma Physics
• Quantum Physics
• Relativity and Gravitation
• Semiconductor and Mesoscopic Physics
• Browse content in Psychology
• Affective Sciences
• Clinical Psychology
• Cognitive Psychology
• Cognitive Neuroscience
• Criminal and Forensic Psychology
• Developmental Psychology
• Educational Psychology
• Evolutionary Psychology
• Health Psychology
• History and Systems in Psychology
• Music Psychology
• Neuropsychology
• Organizational Psychology
• Psychological Assessment and Testing
• Psychology of Human-Technology Interaction
• Psychology Professional Development and Training
• Research Methods in Psychology
• Social Psychology
• Browse content in Social Sciences
• Browse content in Anthropology
• Anthropology of Religion
• Human Evolution
• Medical Anthropology
• Physical Anthropology
• Regional Anthropology
• Social and Cultural Anthropology
• Theory and Practice of Anthropology
• Browse content in Business and Management
• Business Ethics
• Business Strategy
• Business History
• Business and Technology
• Business and Government
• Business and the Environment
• Comparative Management
• Corporate Governance
• Corporate Social Responsibility
• Entrepreneurship
• Health Management
• Human Resource Management
• Industrial and Employment Relations
• Industry Studies
• Information and Communication Technologies
• International Business
• Knowledge Management
• Management and Management Techniques
• Operations Management
• Organizational Theory and Behaviour
• Pensions and Pension Management
• Public and Nonprofit Management
• Strategic Management
• Supply Chain Management
• Browse content in Criminology and Criminal Justice
• Criminal Justice
• Criminology
• Forms of Crime
• International and Comparative Criminology
• Youth Violence and Juvenile Justice
• Development Studies
• Browse content in Economics
• Agricultural, Environmental, and Natural Resource Economics
• Asian Economics
• Behavioural Finance
• Behavioural Economics and Neuroeconomics
• Econometrics and Mathematical Economics
• Economic History
• Economic Systems
• Economic Methodology
• Economic Development and Growth
• Financial Markets
• Financial Institutions and Services
• General Economics and Teaching
• Health, Education, and Welfare
• History of Economic Thought
• International Economics
• Labour and Demographic Economics
• Law and Economics
• Macroeconomics and Monetary Economics
• Microeconomics
• Public Economics
• Urban, Rural, and Regional Economics
• Welfare Economics
• Browse content in Education
• Adult Education and Continuous Learning
• Care and Counselling of Students
• Early Childhood and Elementary Education
• Educational Equipment and Technology
• Educational Strategies and Policy
• Higher and Further Education
• Organization and Management of Education
• Philosophy and Theory of Education
• Schools Studies
• Secondary Education
• Teaching of a Specific Subject
• Teaching of Specific Groups and Special Educational Needs
• Teaching Skills and Techniques
• Browse content in Environment
• Applied Ecology (Social Science)
• Climate Change
• Conservation of the Environment (Social Science)
• Environmentalist Thought and Ideology (Social Science)
• Natural Disasters (Environment)
• Social Impact of Environmental Issues (Social Science)
• Browse content in Human Geography
• Cultural Geography
• Economic Geography
• Political Geography
• Browse content in Interdisciplinary Studies
• Communication Studies
• Museums, Libraries, and Information Sciences
• Browse content in Politics
• African Politics
• Asian Politics
• Chinese Politics
• Comparative Politics
• Conflict Politics
• Elections and Electoral Studies
• Environmental Politics
• Ethnic Politics
• European Union
• Foreign Policy
• Gender and Politics
• Human Rights and Politics
• Indian Politics
• International Relations
• International Organization (Politics)
• International Political Economy
• Irish Politics
• Latin American Politics
• Middle Eastern Politics
• Political Behaviour
• Political Economy
• Political Institutions
• Political Methodology
• Political Communication
• Political Philosophy
• Political Sociology
• Political Theory
• Politics and Law
• Politics of Development
• Public Policy
• Public Administration
• Quantitative Political Methodology
• Regional Political Studies
• Russian Politics
• Security Studies
• State and Local Government
• UK Politics
• US Politics
• Browse content in Regional and Area Studies
• African Studies
• Asian Studies
• East Asian Studies
• Japanese Studies
• Latin American Studies
• Middle Eastern Studies
• Native American Studies
• Scottish Studies
• Browse content in Research and Information
• Research Methods
• Browse content in Social Work
• Addictions and Substance Misuse
• Adoption and Fostering
• Care of the Elderly
• Child and Adolescent Social Work
• Couple and Family Social Work
• Direct Practice and Clinical Social Work
• Emergency Services
• Human Behaviour and the Social Environment
• International and Global Issues in Social Work
• Mental and Behavioural Health
• Social Justice and Human Rights
• Social Policy and Advocacy
• Social Work and Crime and Justice
• Social Work Macro Practice
• Social Work Practice Settings
• Social Work Research and Evidence-based Practice
• Welfare and Benefit Systems
• Browse content in Sociology
• Childhood Studies
• Community Development
• Comparative and Historical Sociology
• Economic Sociology
• Gender and Sexuality
• Gerontology and Ageing
• Health, Illness, and Medicine
• Marriage and the Family
• Migration Studies
• Occupations, Professions, and Work
• Organizations
• Population and Demography
• Race and Ethnicity
• Social Theory
• Social Movements and Social Change
• Social Research and Statistics
• Social Stratification, Inequality, and Mobility
• Sociology of Religion
• Sociology of Education
• Sport and Leisure
• Urban and Rural Studies
• Browse content in Warfare and Defence
• Defence Strategy, Planning, and Research
• Land Forces and Warfare
• Military Administration
• Military Life and Institutions
• Naval Forces and Warfare
• Other Warfare and Defence Issues
• Peace Studies and Conflict Resolution
• Weapons and Equipment

• < Previous chapter
• Next chapter >

28 Case Selection for Case‐Study Analysis: Qualitative and Quantitative Techniques

John Gerring is Professor of Political Science, Boston University.

• Published: 02 September 2009
• Cite Icon Cite
• Permissions Icon Permissions

This article presents some guidance by cataloging nine different techniques for case selection: typical, diverse, extreme, deviant, influential, crucial, pathway, most similar, and most different. It also indicates that if the researcher is starting from a quantitative database, then methods for finding influential outliers can be used. In particular, the article clarifies the general principles that might guide the process of case selection in case-study research. Cases are more or less representative of some broader phenomenon and, on that score, may be considered better or worse subjects for intensive analysis. The article then draws attention to two ambiguities in case-selection strategies in case-study research. The first concerns the admixture of several case-selection strategies. The second concerns the changing status of a case as a study proceeds. Some case studies follow only one strategy of case selection.

Case ‐study analysis focuses on one or several cases that are expected to provide insight into a larger population. This presents the researcher with a formidable problem of case selection: Which cases should she or he choose?

In large‐sample research, the task of case selection is usually handled by some version of randomization. However, in case‐study research the sample is small (by definition) and this makes random sampling problematic, for any given sample may be wildly unrepresentative. Moreover, there is no guarantee that a few cases, chosen randomly, will provide leverage into the research question of interest.

In order to isolate a sample of cases that both reproduces the relevant causal features of a larger universe (representativeness) and provides variation along the dimensions of theoretical interest (causal leverage), case selection for very small samples must employ purposive (nonrandom) selection procedures. Nine such methods are discussed in this chapter, each of which may be identified with a distinct case‐study “type:” typical, diverse, extreme, deviant, influential, crucial, pathway, most‐similar , and most‐different . Table 28.1 summarizes each type, including its general definition, a technique for locating it within a population of potential cases, its uses, and its probable representativeness.

While each of these techniques is normally practiced on one or several cases (the diverse, most‐similar, and most‐different methods require at least two), all may employ additional cases—with the proviso that, at some point, they will no longer offer an opportunity for in‐depth analysis and will thus no longer be “case studies” in the usual sense ( Gerring 2007 , ch. 2 ). It will also be seen that small‐ N case‐selection procedures rest, at least implicitly, upon an analysis of a larger population of potential cases (as does randomization). The case(s) identified for intensive study is chosen from a population and the reasons for this choice hinge upon the way in which it is situated within that population. This is the origin of the terminology—typical, diverse, extreme, et al. It follows that case‐selection procedures in case‐study research may build upon prior cross‐case analysis and that they depend, at the very least, upon certain assumptions about the broader population.

In certain circumstances, the case‐selection procedure may be structured by a quantitative analysis of the larger population. Here, several caveats must be satisfied. First, the inference must pertain to more than a few dozen cases; otherwise, statistical analysis is problematic. Second, relevant data must be available for that population, or a significant sample of that population, on key variables, and the researcher must feel reasonably confident in the accuracy and conceptual validity of these variables. Third, all the standard assumptions of statistical research (e.g. identification, specification, robustness) must be carefully considered, and wherever possible, tested. I shall not dilate further on these familiar issues except to warn the researcher against the unreflective use of statistical techniques. 1 When these requirements are not met, the researcher must employ a qualitative approach to case selection.

The point of this chapter is to elucidate general principles that might guide the process of case selection in case‐study research, building upon earlier work by Harry Eckstein, Arend Lijphart, and others. Sometimes, these principles can be applied in a quantitative framework and sometimes they are limited to a qualitative framework. In either case, the logic of case selection remains quite similar, whether practiced in small‐ N or large‐ N contexts.

Before we begin, a bit of notation is necessary. In this chapter “ N ” refers to cases, not observations. Here, I am concerned primarily with causal inference, rather than inferences that are descriptive or predictive in nature. Thus, all hypotheses involve at least one independent variable ( X ) and one dependent variable ( Y ). For convenience, I shall label the causal factor of special theoretical interest X   1 , and the control variable, or vector of controls (if there are any), X   2 . If the writer is concerned to explain a puzzling outcome, but has no preconceptions about its causes, then the research will be described as Y‐centered . If a researcher is concerned to investigate the effects of a particular cause, with no preconceptions about what these effects might be, the research will be described as X‐centered . If a researcher is concerned to investigate a particular causal relationship, the research will be described as X   1 / Y‐centered , for it connects a particular cause with a particular outcome. 2   X ‐ or Y ‐centered research is exploratory; its purpose is to generate new hypotheses. X   1 / Y‐centered research, by contrast, is confirmatory/disconfirmatory; its purpose is to test an existing hypothesis.

1 Typical Case

In order for a focused case study to provide insight into a broader phenomenon it must be representative of a broader set of cases. It is in this context that one may speak of a typical‐case approach to case selection. The typical case exemplifies what is considered to be a typical set of values, given some general understanding of a phenomenon. By construction, the typical case is also a representative case.

Some typical cases serve an exploratory role. Here, the author chooses a case based upon a set of descriptive characteristics and then probes for causal relationships. Robert and Helen Lynd (1929/1956) selected a single city “to be as representative as possible of contemporary American life.” Specifically, they were looking for a city with

1) a temperate climate; 2) a sufficiently rapid rate of growth to ensure the presence of a plentiful assortment of the growing pains accompanying contemporary social change; 3) an industrial culture with modern, high‐speed machine production; 4) the absence of dominance of the city's industry by a single plant (i.e., not a one‐industry town); 5) a substantial local artistic life to balance its industrial activity …; and 6) the absence of any outstanding peculiarities or acute local problems which would mark the city off from the midchannel sort of American community. ( Lynd and Lynd 1929/1956 , quoted in Yin 2004 , 29–30)

After examining a number of options the Lynds decided that Muncie, Indiana, was more representative than, or at least as representative as, other midsized cities in America, thus qualifying as a typical case.

This is an inductive approach to case selection. Note that typicality may be understood according to the mean, median, or mode on a particular dimension; there may be multiple dimensions (as in the foregoing example); and each may be differently weighted (some dimensions may be more important than others). Where the selection criteria are multidimensional and a large sample of potential cases is in play, some form of factor analysis may be useful in identifying the most‐typical case(s).

However, the more common employment of the typical‐case method involves a causal model of some phenomenon of theoretical interest. Here, the researcher has identified a particular outcome ( Y ), and perhaps a specific X   1 / Y hypothesis, which she wishes to investigate. In order to do so, she looks for a typical example of that causal relationship. Intuitively, one imagines that a case selected according to the mean values of all parameters must be a typical case relative to some causal relationship. However, this is by no means assured.

Suppose that the Lynds were primarily interested in explaining feelings of trust/distrust among members of different social classes (one of the implicit research goals of the Middletown study). This outcome is likely to be affected by many factors, only some of which are included in their six selection criteria. So choosing cases with respect to a causal hypothesis involves, first of all, identifying the relevant parameters. It involves, secondly, the selection of a case that has a “typical” value relative to the overall causal model; it is well explained. Cases with untypical scores on a particular dimension (e.g. very high or very low) may still be typical examples of a causal relationship. Indeed, they may be more typical than cases whose values lie close to the mean. Thus, a descriptive understanding of typicality is quite different from a causal understanding of typicality. Since it is the latter version that is more common, I shall adopt this understanding of typicality in the remainder of the discussion.

From a qualitative perspective, causal typicality involves the selection of a case that conforms to expectations about some general causal relationship. It performs as expected. In a quantitative setting, this notion is measured by the size of a case's residual in a large‐ N cross‐case model. Typical cases lie on or near the regression line; their residuals are small. Insofar as the model is correctly specified, the size of a case's residual (i.e. the number of standard deviations that separate the actual value from the fitted value) provides a helpful clue to how representative that case is likely to be. “Outliers” are unlikely to be representative of the target population.

Of course, just because a case has a low residual does not necessarily mean that it is a representative case (with respect to the causal relationship of interest). Indeed, the issue of case representativeness is an issue that can never be definitively settled. When one refers to a “typical case” one is saying, in effect, that the probability of a case's representativeness is high, relative to other cases. This test of typicality is misleading if the statistical model is mis‐specified. And it provides little insurance against errors that are purely stochastic. A case may lie directly on the regression line but still be, in some important respect, atypical. For example, it might have an odd combination of values; the interaction of variables might be different from other cases; or additional causal mechanisms might be at work. For this reason, it is important to supplement a statistical analysis of cases with evidence drawn from the case in question (the case study itself) and with our deductive knowledge of the world. One should never judge a case solely by its residual. Yet, all other things being equal, a case with a low residual is less likely to be unusual than a case with a high residual, and to this extent the method of case selection outlined here may be a helpful guide to case‐study researchers faced with a large number of potential cases.

By way of conclusion, it should be noted that because the typical case embodies a typical value on some set of causally relevant dimensions, the variance of interest to the researcher must lie within that case. Specifically, the typical case of some phenomenon may be helpful in exploring causal mechanisms and in solving identification problems (e.g. endogeneity between X   1 and Y , an omitted variable that may account for X   1   and Y , or some other spurious causal association). Depending upon the results of the case study, the author may confirm an existing hypothesis, disconfirm that hypothesis, or reframe it in a way that is consistent with the findings of the case study. These are the uses of the typical‐case study.

2 Diverse Cases

A second case‐selection strategy has as its primary objective the achievement of maximum variance along relevant dimensions. I refer to this as a diverse‐case method. For obvious reasons, this method requires the selection of a set of cases—at minimum, two—which are intended to represent the full range of values characterizing X   1 , Y , or some particular X   1 / Y relationship. 3

Where the individual variable of interest is categorical (on/off, red/black/blue, Jewish/Protestant/Catholic), the identification of diversity is readily apparent. The investigator simply chooses one case from each category. For a continuous variable, the choices are not so obvious. However, the researcher usually chooses both extreme values (high and low), and perhaps the mean or median as well. The researcher may also look for break‐points in the distribution that seem to correspond to categorical differences among cases. Or she may follow a theoretical hunch about which threshold values count, i.e. which are likely to produce different values on Y .

Another sort of diverse case takes account of the values of multiple variables (i.e. a vector), rather than a single variable. If these variables are categorical, the identification of causal types rests upon the intersection of each category. Two dichotomous variables produce a matrix with four cells. Three trichotomous variables produce a matrix of eight cells. And so forth. If all variables are deemed relevant to the analysis, the selection of diverse cases mandates the selection of one case drawn from within each cell. Let us say that an outcome is thought to be affected by sex, race (black/white), and marital status. Here, a diverse‐case strategy of case selection would identify one case within each of these intersecting cells—a total of eight cases. Things become slightly more complicated when one or more of the factors is continuous, rather than categorical. Here, the diversity of case values do not fall neatly into cells. Rather, these cells must be created by fiat—e.g. high, medium, low.

It will be seen that where multiple variables are under consideration, the logic of diverse‐case analysis rests upon the logic of typological theorizing—where different combinations of variables are assumed to have effects on an outcome that vary across types ( Elman 2005 ; George and Bennett 2005 , 235; Lazarsfeld and Barton 1951 ). George and Smoke, for example, wish to explore different types of deterrence failure—by “fait accompli,” by “limited probe,” and by “controlled pressure.” Consequently, they wish to find cases that exemplify each type of causal mechanism. 4

Diversity may thus refer to a range of variation on X or Y , or to a particular combination of causal factors (with or without a consideration of the outcome). In each instance, the goal of case selection is to capture the full range of variation along the dimension(s) of interest.

Since diversity can mean many things, its employment in a large‐ N setting is necessarily dependent upon how this key term is defined. If it is understood to pertain only to a single variable ( X   1 or Y ), then the task is fairly simple. A categorical variable mandates the choice of at least one case from each category—two if dichotomous, three if trichotomous, and so forth. A continuous variable suggests the choice of at least one “high” and “low” value, and perhaps one drawn from the mean or median. But other choices might also be justified, according to one's hunch about the underlying causal relationship or according to natural thresholds found in the data, which may be grouped into discrete categories. Single‐variable traits are usually easy to discover in a large‐ N setting through descriptive statistics or through visual inspection of the data.

Where diversity refers to particular combinations of variables, the relevant cross‐ case technique is some version of stratified random sampling (in a probabilistic setting) or Qualitative Comparative Analysis (in a deterministic setting) ( Ragin 2000 ). If the researcher suspects that a causal relationship is affected not only by combinations of factors but also by their sequencing , then the technique of analysis must incorporate temporal elements ( Abbott 2001 ; Abbott and Forrest 1986 ; Abbott and Tsay 2000 ). Thus, the method of identifying causal types rests upon whatever method of identifying causal relationships is employed in the large‐ N sample.

Note that the identification of distinct case types is intended to identify groups of cases that are internally homogeneous (in all respects that might affect the causal relationship of interest). Thus, the choice of cases within each group should not be problematic, and may be accomplished through random sampling or purposive case selection. However, if there is suspected diversity within each category, then measures should be taken to assure that the chosen cases are typical of each category. A case study should not focus on an atypical member of a subgroup.

Indeed, considerations of diversity and typicality often go together. Thus, in a study of globalization and social welfare systems, Duane Swank (2002) first identifies three distinctive groups of welfare states: “universalistic” (social democratic), “corporatist conservative,” and “liberal.” Next, he looks within each group to find the most‐typical cases. He decides that the Nordic countries are more typical of the universalistic model than the Netherlands since the latter has “some characteristics of the occupationally based program structure and a political context of Christian Democratic‐led governments typical of the corporatist conservative nations” ( Swank 2002 , 11; see also Esping‐Andersen 1990 ). Thus, the Nordic countries are chosen as representative cases within the universalistic case type, and are accompanied in the case‐study portion of his analysis by other cases chosen to represent the other welfare state types (corporatist conservative and liberal).

Evidently, when a sample encompasses a full range of variation on relevant parameters one is likely to enhance the representativeness of that sample (relative to some population). This is a distinct advantage. Of course, the inclusion of a full range of variation may distort the actual distribution of cases across this spectrum. If there are more “high” cases than “low” cases in a population and the researcher chooses only one high case and one low case, the resulting sample of two is not perfectly representative. Even so, the diverse‐case method probably has stronger claims to representativeness than any other small‐ N sample (including the standalone typical case). The selection of diverse cases has the additional advantage of introducing variation on the key variables of interest. A set of diverse cases is, by definition, a set of cases that encompasses a range of high and low values on relevant dimensions. There is, therefore, much to recommend this method of case selection. I suspect that these advantages are commonly understood and are applied on an intuitive level by case‐study researchers. However, the lack of a recognizable name—and an explicit methodological defense—has made it difficult for case‐study researchers to utilize this method of case selection, and to do so in an explicit and self‐conscious fashion. Neologism has its uses.

3 Extreme Case

The extreme‐case method selects a case because of its extreme value on an independent ( X   1 ) or dependent ( Y ) variable of interest. Thus, studies of domestic violence may choose to focus on extreme instances of abuse ( Browne 1987 ). Studies of altruism may focus on those rare individuals who risked their lives to help others (e.g. Holocaust resisters) ( Monroe 1996 ). Studies of ethnic politics may focus on the most heterogeneous societies (e.g. Papua New Guinea) in order to better understand the role of ethnicity in a democratic setting ( Reilly 2000–1 ). Studies of industrial policy often focus on the most successful countries (i.e. the NICS) ( Deyo 1987 ). And so forth. 5

Often an extreme case corresponds to a case that is considered to be prototypical or paradigmatic of some phenomena of interest. This is because concepts are often defined by their extremes, i.e. their ideal types. Italian Fascism defines the concept of Fascism, in part, because it offered the most extreme example of that phenomenon. However, the methodological value of this case, and others like it, derives from its extremity (along some dimension of interest), not its theoretical status or its status in the literature on a subject.

The notion of “extreme” may now be defined more precisely. An extreme value is an observation that lies far away from the mean of a given distribution. This may be measured (if there are sufficient observations) by a case's “Z score”—the number of standard deviations between a case and the mean value for that sample. Extreme cases have high Z scores, and for this reason may serve as useful subjects for intensive analysis.

For a continuous variable, the distance from the mean may be in either direction (positive or negative). For a dichotomous variable (present/absent), extremeness may be interpreted as unusual . If most cases are positive along a given dimension, then a negative case constitutes an extreme case. If most cases are negative, then a positive case constitutes an extreme case. It should be clear that researchers are not simply concerned with cases where something “happened,” but also with cases where something did not. It is the rareness of the value that makes a case valuable, in this context, not its positive or negative value. 6 Thus, if one is studying state capacity, a case of state failure is probably more informative than a case of state endurance simply because the former is more unusual. Similarly, if one is interested in incest taboos a culture where the incest taboo is absent or weak is probably more useful than a culture where it is present or strong. Fascism is more important than nonfascism. And so forth. There is a good reason, therefore, why case studies of revolution tend to focus on “revolutionary” cases. Theda Skocpol (1979) had much more to learn from France than from Austro‐Hungary since France was more unusual than Austro‐Hungary within the population of nation states that Skocpol was concerned to explain. The reason is quite simple: There are fewer revolutionary cases than nonrevolutionary cases; thus, the variation that we explore as a clue to causal relationships is encapsulated in these cases, against a background of nonrevolutionary cases.

Note that the extreme‐case method of case selection appears to violate the social science folk wisdom warning us not to “select on the dependent variable.” 7 Selecting cases on the dependent variable is indeed problematic if a number of cases are chosen, all of which lie on one end of a variable's spectrum (they are all positive or negative), and if the researcher then subjects this sample to cross‐case analysis as if it were representative of a population. 8 Results for this sort of analysis would almost assuredly be biased. Moreover, there will be little variation to explain since the values of each case are explicitly constrained.

However, this is not the proper employment of the extreme‐case method. (It is more appropriately labeled an extreme‐ sample method.) The extreme‐case method actually refers back to a larger sample of cases that lie in the background of the analysis and provide a full range of variation as well as a more representative picture of the population. It is a self‐conscious attempt to maximize variance on the dimension of interest, not to minimize it. If this population of cases is well understood— either through the author's own cross‐case analysis, through the work of others, or through common sense—then a researcher may justify the selection of a single case exemplifying an extreme value for within‐case analysis. If not, the researcher may be well advised to follow a diverse‐case method, as discussed above.

By way of conclusion, let us return to the problem of representativeness. It will be seen that an extreme case may be typical or deviant. There is simply no way to tell because the researcher has not yet specified an X   1 / Y causal proposition. Once such a causal proposition has been specified one may then ask whether the case in question is similar to some population of cases in all respects that might affect the X   1 / Y relationship of interest (i.e. unit homogeneous). It is at this point that it becomes possible to say, within the context of a cross‐case statistical model, whether a case lies near to, or far from, the regression line. However, this sort of analysis means that the researcher is no longer pursuing an extreme‐case method. The extreme‐case method is purely exploratory—a way of probing possible causes of Y , or possible effects of X , in an open‐ended fashion. If the researcher has some notion of what additional factors might affect the outcome of interest, or of what relationship the causal factor of interest might have with Y , then she ought to pursue one of the other methods explored in this chapter. This also implies that an extreme‐case method may transform into a different kind of approach as a study evolves; that is, as a more specific hypothesis comes to light. Useful extreme cases at the outset of a study may prove less useful at a later stage of analysis.

4 Deviant Case

The deviant‐case method selects that case(s) which, by reference to some general understanding of a topic (either a specific theory or common sense), demonstrates a surprising value. It is thus the contrary of the typical case. Barbara Geddes (2003) notes the importance of deviant cases in medical science, where researchers are habitually focused on that which is “pathological” (according to standard theory and practice). The New England Journal of Medicine , one of the premier journals of the field, carries a regular feature entitled Case Records of the Massachusetts General Hospital. These articles bear titles like the following: “An 80‐Year‐Old Woman with Sudden Unilateral Blindness” or “A 76‐Year‐Old Man with Fever, Dyspnea, Pulmonary Infiltrates, Pleural Effusions, and Confusion.” 9 Another interesting example drawn from the field of medicine concerns the extensive study now devoted to a small number of persons who seem resistant to the AIDS virus ( Buchbinder and Vittinghoff 1999 ; Haynes, Pantaleo, and Fauci 1996 ). Why are they resistant? What is different about these people? What can we learn about AIDS in other patients by observing people who have built‐in resistance to this disease?

Likewise, in psychology and sociology case studies may be comprised of deviant (in the social sense) persons or groups. In economics, case studies may consist of countries or businesses that overperform (e.g. Botswana; Microsoft) or underperform (e.g. Britain through most of the twentieth century; Sears in recent decades) relative to some set of expectations. In political science, case studies may focus on countries where the welfare state is more developed (e.g. Sweden) or less developed (e.g. the United States) than one would expect, given a set of general expectations about welfare state development. The deviant case is closely linked to the investigation of theoretical anomalies. Indeed, to say deviant is to imply “anomalous.” 10

Note that while extreme cases are judged relative to the mean of a single distribution (the distribution of values along a single variable), deviant cases are judged relative to some general model of causal relations. The deviant‐case method selects cases which, by reference to some (presumably) general relationship, demonstrate a surprising value. They are “deviant” in that they are poorly explained by the multivariate model. The important point is that deviant‐ness can only be assessed relative to the general (quantitative or qualitative) model. This means that the relative deviant‐ness of a case is likely to change whenever the general model is altered. For example, the United States is a deviant welfare state when this outcome is gauged relative to societal wealth. But it is less deviant—and perhaps not deviant at all—when certain additional (political and societal) factors are included in the model, as discussed in the epilogue. Deviance is model dependent. Thus, when discussing the concept of the deviant case it is helpful to ask the following question: Relative to what general model (or set of background factors) is Case A deviant?

Conceptually, we have said that the deviant case is the logical contrary of the typical case. This translates into a directly contrasting statistical measurement. While the typical case is one with a low residual (in some general model of causal relations), a deviant case is one with a high residual. This means, following our previous discussion, that the deviant case is likely to be an un representative case, and in this respect appears to violate the supposition that case‐study samples should seek to reproduce features of a larger population.

However, it must be borne in mind that the primary purpose of a deviant‐case analysis is to probe for new—but as yet unspecified—explanations. (If the purpose is to disprove an extant theory I shall refer to the study as crucial‐case, as discussed below.) The researcher hopes that causal processes identified within the deviant case will illustrate some causal factor that is applicable to other (more or less deviant) cases. This means that a deviant‐case study usually culminates in a general proposition, one that may be applied to other cases in the population. Once this general proposition has been introduced into the overall model, the expectation is that the chosen case will no longer be an outlier. Indeed, the hope is that it will now be typical , as judged by its small residual in the adjusted model. (The exception would be a circumstance in which a case's outcome is deemed to be “accidental,” and therefore inexplicable by any general model.)

This feature of the deviant‐case study should help to resolve questions about its representativeness. Even if it is not possible to measure the new causal factor (and thus to introduce it into a large‐ N cross‐case model), it may still be plausible to assert (based on general knowledge of the phenomenon) that the chosen case is representative of a broader population.

5 Influential Case

Sometimes, the choice of a case is motivated solely by the need to verify the assumptions behind a general model of causal relations. Here, the analyst attempts to provide a rationale for disregarding a problematic case or a set of problematic cases. That is to say, she attempts to show why apparent deviations from the norm are not really deviant, or do not challenge the core of the theory, once the circumstances of the special case or cases are fully understood. A cross‐case analysis may, after all, be marred by several classes of problems including measurement error, specification error, errors in establishing proper boundaries for the inference (the scope of the argument), and stochastic error (fluctuations in the phenomenon under study that are treated as random, given available theoretical resources). If poorly fitting cases can be explained away by reference to these kinds of problems, then the theory of interest is that much stronger. This sort of deviant‐case analysis answers the question, “What about Case A (or cases of type A)? How does that, seemingly disconfirming, case fit the model?”

Because its underlying purpose is different from the usual deviant‐case study, I offer a new term for this method. The influential case is a case that casts doubt upon a theory, and for that reason warrants close inspection. This investigation may reveal, after all, that the theory is validated—perhaps in some slightly altered form. In this guise, the influential case is the “case that proves the rule.” In other instances, the influential‐case analysis may contribute to disconfirming, or reconceptualizing, a theory. The key point is that the value of the case is judged relative to some extant cross‐case model.

A simple version of influential‐case analysis involves the confirmation of a key case's score on some critical dimension. This is essentially a question of measurement. Sometimes cases are poorly explained simply because they are poorly understood. A close examination of a particular context may reveal that an apparently falsifying case has been miscoded. If so, the initial challenge presented by that case to some general theory has been obviated.

However, the more usual employment of the influential‐case method culminates in a substantive reinterpretation of the case—perhaps even of the general model. It is not just a question of measurement. Consider Thomas Ertman's (1997) study of state building in Western Europe, as summarized by Gerardo Munck. This study argues

that the interaction of a) the type of local government during the first period of statebuilding, with b) the timing of increases in geopolitical competition, strongly influences the kind of regime and state that emerge. [Ertman] tests this hypothesis against the historical experience of Europe and finds that most countries fit his predictions. Denmark, however, is a major exception. In Denmark, sustained geopolitical competition began relatively late and local government at the beginning of the statebuilding period was generally participatory, which should have led the country to develop “patrimonial constitutionalism.” But in fact, it developed “bureaucratic absolutism.” Ertman carefully explores the process through which Denmark came to have a bureaucratic absolutist state and finds that Denmark had the early marks of a patrimonial constitutionalist state. However, the country was pushed off this developmental path by the influence of German knights, who entered Denmark and brought with them German institutions of local government. Ertman then traces the causal process through which these imported institutions pushed Denmark to develop bureaucratic absolutism, concluding that this development was caused by a factor well outside his explanatory framework. ( Munck 2004 , 118)

Ertman's overall framework is confirmed insofar as he has been able to show, by an in‐depth discussion of Denmark, that the causal processes stipulated by the general theory hold even in this apparently disconfirming case. Denmark is still deviant, but it is so because of “contingent historical circumstances” that are exogenous to the theory ( Ertman 1997 , 316).

Evidently, the influential‐case analysis is similar to the deviant‐case analysis. Both focus on outliers. However, as we shall see, they focus on different kinds of outliers. Moreover, the animating goals of these two research designs are quite different. The influential‐case study begins with the aim of confirming a general model, while the deviant‐case study has the aim of generating a new hypothesis that modifies an existing general model. The confusion stems from the fact that the same case study may fulfill both objectives—qualifying a general model and, at the same time, confirming its core hypothesis.

Thus, in their study of Roberto Michels's “iron law of oligarchy,” Lipset, Trow, and Coleman (1956) choose to focus on an organization—the International Typographical Union—that appears to violate the central presupposition. The ITU, as noted by one of the authors, has “a long‐term two‐party system with free elections and frequent turnover in office” and is thus anything but oligarchic ( Lipset 1959 , 70). As such, it calls into question Michels's grand generalization about organizational behavior. The authors explain this curious result by the extraordinarily high level of education among the members of this union. Michels's law is shown to be true for most organizations, but not all. It is true, with qualifications. Note that the respecification of the original model (in effect, Lipset, Trow, and Coleman introduce a new control variable or boundary condition) involves the exploration of a new hypothesis. In this instance, therefore, the use of an influential case to confirm an existing theory is quite similar to the use of a deviant case to explore a new theory.

In a quantitative idiom, influential cases are those that, if counterfactually assigned a different value on the dependent variable, would most substantially change the resulting estimates. They may or may not be outliers (high‐residual cases). Two quantitative measures of influence are commonly applied in regression diagnostics ( Belsey, Kuh, and Welsch 2004 ). The first, often referred to as the leverage of a case, derives from what is called the hat matrix . Based solely on each case's scores on the independent variables, the hat matrix tells us how much a change in (or a measurement error on) the dependent variable for that case would affect the overall regression line. The second is Cook's distance , a measure of the extent to which the estimates of all the parameters would change if a given case were omitted from the analysis. Cases with a large leverage or Cook's distance contribute quite a lot to the inferences drawn from a cross‐case analysis. In this sense, such cases are vital for maintaining analytic conclusions. Discovering a significant measurement error on the dependent variable or an important omitted variable for such a case may dramatically revise estimates of the overall relationships. Hence, it may be quite sensible to select influential cases for in‐depth study.

Note that the use of an influential‐case strategy of case selection is limited to instances in which a researcher has reason to be concerned that her results are being driven by one or a few cases. This is most likely to be true in small to moderate‐sized samples. Where N is very large—greater than 1,000, let us say—it is extremely unlikely that a small set of cases (much less an individual case) will play an “influential” role. Of course, there may be influential sets of cases, e.g. countries within a particular continent or cultural region, or persons of Irish extraction. Sets of influential observations are often problematic in a time‐series cross‐section data‐set where each unit (e.g. country) contains multiple observations (through time), and hence may have a strong influence on aggregate results. Still, the general rule is: the larger the sample, the less important individual cases are likely to be and, hence, the less likely a researcher is to use an influential‐case approach to case selection.

6 Crucial Case

Of all the extant methods of case selection perhaps the most storied—and certainly the most controversial—is the crucial‐case method, introduced to the social science world several decades ago by Harry Eckstein. In his seminal essay, Eckstein (1975 , 118) describes the crucial case as one “that must closely fit a theory if one is to have confidence in the theory's validity, or, conversely, must not fit equally well any rule contrary to that proposed.” A case is crucial in a somewhat weaker—but much more common—sense when it is most, or least, likely to fulfill a theoretical prediction. A “most‐likely” case is one that, on all dimensions except the dimension of theoretical interest, is predicted to achieve a certain outcome, and yet does not. It is therefore used to disconfirm a theory. A “least‐likely” case is one that, on all dimensions except the dimension of theoretical interest, is predicted not to achieve a certain outcome, and yet does so. It is therefore used to confirm a theory. In all formulations, the crucial‐case offers a most‐difficult test for an argument, and hence provides what is perhaps the strongest sort of evidence possible in a nonexperimental, single‐case setting.

Since the publication of Eckstein's influential essay, the crucial‐case approach has been claimed in a multitude of studies across several social science disciplines and has come to be recognized as a staple of the case‐study method. 11 Yet the idea of any single case playing a crucial (or “critical”) role is not widely accepted among most methodologists (e.g. Sekhon 2004 ). (Even its progenitor seems to have had doubts.)

Let us begin with the confirmatory (a.k.a. least‐likely) crucial case. The implicit logic of this research design may be summarized as follows. Given a set of facts, we are asked to contemplate the probability that a given theory is true. While the facts matter, to be sure, the effectiveness of this sort of research also rests upon the formal properties of the theory in question. Specifically, the degree to which a theory is amenable to confirmation is contingent upon how many predictions can be derived from the theory and on how “risky” each individual prediction is. In Popper's (1963 , 36) words, “Confirmations should count only if they are the result of risky predictions ; that is to say, if, unenlightened by the theory in question, we should have expected an event which was incompatible with the theory—and event which would have refuted the theory. Every ‘good’ scientific theory is a prohibition; it forbids certain things to happen. The more a theory forbids, the better it is” (see also Popper 1934/1968 ). A risky prediction is therefore one that is highly precise and determinate, and therefore unlikely to be achieved by the product of other causal factors (external to the theory of interest) or through stochastic processes. A theory produces many such predictions if it is fully elaborated, issuing predictions not only on the central outcome of interest but also on specific causal mechanisms, and if it is broad in purview. (The notion of riskiness may also be conceptualized within the Popperian lexicon as degrees of falsifiability .)

These points can also be articulated in Bayesian terms. Colin Howson and Peter Urbach explain: “The degree to which h [a hypothesis] is confirmed by e [a set of evidence] depends … on the extent to which P(eČh) exceeds P (e) , that is, on how much more probable e is relative to the hypothesis and background assumptions than it is relative just to background assumptions.” Again, “confirmation is correlated with how much more probable the evidence is if the hypothesis is true than if it is false” ( Howson and Urlbach 1989 , 86). Thus, the stranger the prediction offered by a theory—relative to what we would normally expect—the greater the degree of confirmation that will be afforded by the evidence. As an intuitive example, Howson and Urbach (1989 , 86) offer the following:

If a soothsayer predicts that you will meet a dark stranger sometime and you do in fact, your faith in his powers of precognition would not be much enhanced: you would probably continue to think his predictions were just the result of guesswork. However, if the prediction also gave the correct number of hairs on the head of that stranger, your previous scepticism would no doubt be severely shaken.

While these Popperian/Bayesian notions 12 are relevant to all empirical research designs, they are especially relevant to case‐study research designs, for in these settings a single case (or, at most, a small number of cases) is required to bear a heavy burden of proof. It should be no surprise, therefore, that Popper's idea of “riskiness” was to be appropriated by case‐study researchers like Harry Eckstein to validate the enterprise of single‐case analysis. (Although Eckstein does not cite Popper the intellectual lineage is clear.) Riskiness, here, is analogous to what is usually referred to as a “most‐ difficult” research design, which in a case‐study research design would be understood as a “least‐likely” case. Note also that the distinction between a “must‐fit” case and a least‐likely case—that, in the event, actually does fit the terms of a theory—is a matter of degree. Cases are more or less crucial for confirming theories. The point is that, in some circumstances, a paucity of empirical evidence may be compensated by the riskiness of the theory.

The crucial‐case research design is, perforce, a highly deductive enterprise; much depends on the quality of the theory under investigation. It follows that the theories most amenable to crucial‐case analysis are those which are lawlike in their precision, degree of elaboration, consistency, and scope. The more a theory attains the status of a causal law, the easier it will be to confirm, or to disconfirm, with a single case. Indeed, risky predictions are common in natural science fields such as physics, which in turn served as the template for the deductive‐nomological (“covering‐law”) model of science that influenced Eckstein and others in the postwar decades (e.g. Hempel 1942 ).

A frequently cited example is the first important empirical demonstration of the theory of relativity, which took the form of a single‐event prediction on the occasion of the May 29, 1919, solar eclipse ( Eckstein 1975 ; Popper 1963 ). Stephen Van Evera (1997 , 66–7) describes the impact of this prediction on the validation of Einstein's theory.

Einstein's theory predicted that gravity would bend the path of light toward a gravity source by a specific amount. Hence it predicted that during a solar eclipse stars near the sun would appear displaced—stars actually behind the sun would appear next to it, and stars lying next to the sun would appear farther from it—and it predicted the amount of apparent displacement. No other theory made these predictions. The passage of this one single‐case‐study test brought the theory wide acceptance because the tested predictions were unique—there was no plausible competing explanation for the predicted result—hence the passed test was very strong.

The strength of this test is the extraordinary fit between the theory and a set of facts found in a single case, and the corresponding lack of fit between all other theories and this set of facts. Einstein offered an explanation of a particular set of anomalous findings that no other existing theory could make sense of. Of course, one must assume that there was no—or limited—measurement error. And one must assume that the phenomenon of interest is largely invariant; light does not bend differently at different times and places (except in ways that can be understood through the theory of relativity). And one must assume, finally, that the theory itself makes sense on other grounds (other than the case of special interest); it is a plausible general theory. If one is willing to accept these a priori assumptions, then the 1919 “case study” provides a very strong confirmation of the theory. It is difficult to imagine a stronger proof of the theory from within an observational (nonexperimental) setting.

In social science settings, by contrast, one does not commonly find single‐case studies offering knockout evidence for a theory. This is, in my view, largely a product of the looseness (the underspecification) of most social science theories. George and Bennett point out that while the thesis of the democratic peace is as close to a “law” as social science has yet seen, it cannot be confirmed (or refuted) by looking at specific causal mechanisms because the causal pathways mandated by the theory are multiple and diverse. Under the circumstances, no single‐case test can offer strong confirmation of the theory ( George and Bennett 2005 , 209).

However, if one adopts a softer version of the crucial‐case method—the least‐likely (most difficult) case—then possibilities abound. Indeed, I suspect that, implicitly , most case‐study work that makes a positive argument focusing on a single case (without a corresponding cross‐case analysis) relies largely on the logic of the least‐ likely case. Rarely is this logic made explicit, except perhaps in a passing phrase or two. Yet the deductive logic of the “risky” prediction is central to the case‐study enterprise. Whether a case study is convincing or not often rests on the reader's evaluation of how strong the evidence for an argument might be, and this in turn—wherever cross‐ case evidence is limited and no manipulated treatment can be devised—rests upon an estimation of the degree of “fit” between a theory and the evidence at hand, as discussed.

Lily Tsai's (2007) investigation of governance at the village level in China employs several in‐depth case studies of villages which are chosen (in part) because of their least‐likely status relative to the theory of interest. Tsai's hypothesis is that villages with greater social solidarity (based on preexisting religious or familial networks) will develop a higher level of social trust and mutual obligation and, as a result, will experience better governance. Crucial cases, therefore, are villages that evidence a high level of social solidarity but which, along other dimensions, would be judged least likely to develop good governance, e.g. they are poor, isolated, and lack democratic institutions or accountability mechanisms from above. “Li Settlement,” in Fujian province, is such a case. The fact that this impoverished village nonetheless boasts an impressive set of infrastructural accomplishments such as paved roads with drainage ditches (a rarity in rural China) suggests that something rather unusual is going on here. Because her case is carefully chosen to eliminate rival explanations, Tsai's conclusions about the special role of social solidarity are difficult to gainsay. How else is one to explain this otherwise anomalous result? This is the strength of the least‐likely case, where all other plausible causal factors for an outcome have been minimized. 13

Jack Levy (2002 , 144) refers to this, evocatively, as a “Sinatra inference:” if it can make it here, it can make it anywhere (see also Khong 1992 , 49; Sagan 1995 , 49; Shafer 1988 , 14–6). Thus, if social solidarity has the hypothesized effect in Li Settlement it should have the same effect in more propitious settings (e.g. where there is greater economic surplus). The same implicit logic informs many case‐study analyses where the intent of the study is to confirm a hypothesis on the basis of a single case.

Another sort of crucial case is employed for the purpose of dis confirming a causal hypothesis. A central Popperian insight is that it is easier to disconfirm an inference than to confirm that same inference. (Indeed, Popper doubted that any inference could be fully confirmed, and for this reason preferred the term “corroborate.”) This is particularly true of case‐study research designs, where evidence is limited to one or several cases. The key proviso is that the theory under investigation must take a consistent (a.k.a. invariant, deterministic) form, even if its predictions are not terrifically precise, well elaborated, or broad.

As it happens, there are a fair number of invariant propositions floating around the social science disciplines (Goertz and Levy forthcoming; Goertz and Starr 2003 ). It used to be argued, for example, that political stability would occur only in countries that are relatively homogeneous, or where existing heterogeneities are mitigated by cross‐cutting cleavages ( Almond 1956 ; Bentley 1908/1967 ; Lipset 1960/1963 ; Truman 1951 ). Arend Lijphart's (1968) study of the Netherlands, a peaceful country with reinforcing social cleavages, is commonly viewed as refuting this theory on the basis of a single in‐depth case analysis. 14

Granted, it may be questioned whether presumed invariant theories are really invariant; perhaps they are better understood as probabilistic. Perhaps, that is, the theory of cross‐cutting cleavages is still true, probabilistically, despite the apparent Dutch exception. Or perhaps the theory is still true, deterministically, within a subset of cases that does not include the Netherlands. (This sort of claim seems unlikely in this particular instance, but it is quite plausible in many others.) Or perhaps the theory is in need of reframing; it is true, deterministically, but applies only to cross‐ cutting ethnic/racial cleavages, not to cleavages that are primarily religious. One can quibble over what it means to “disconfirm” a theory. The point is that the crucial case has, in all these circumstances, provided important updating of a theoretical prior.

Heretofore, I have treated causal factors as dichotomous. Countries have either reinforcing or cross‐cutting cleavages and they have regimes that are either peaceful or conflictual. Evidently, these sorts of parameters are often matters of degree. In this reading of the theory, cases are more or less crucial. Accordingly, the most useful—i.e. most crucial—case for Lijphart's purpose is one that has the most segregated social groups and the most peaceful and democratic track record. In these respects, the Netherlands was a very good choice. Indeed, the degree of disconfirmation offered by this case study is probably greater than the degree of disconfirmation that might have been provided by other cases such as India or Papua New Guinea—countries where social peace has not always been secure. The point is that where variables are continuous rather than dichotomous it is possible to evaluate potential cases in terms of their degree of crucialness .

Note that the crucial‐case method of case‐selection, whether employed in a confirmatory or disconfirmatory mode, cannot be employed in a large‐ N context. This is because an explicit cross‐case model would render the crucial‐case study redundant. Once one identifies the relevant parameters and the scores of all cases on those parameters, one has in effect constructed a cross‐case model that confirms or disconfirms the theory in question. The case study is thenceforth irrelevant, at least as a means of decisive confirmation or disconfirmation. 15 It remains highly relevant as a means of exploring causal mechanisms, of course. Yet, because this objective is quite different from that which is usually associated with the term, I enlist a new term for this technique.

7 Pathway Case

One of the most important functions of case‐study research is the elucidation of causal mechanisms. But which sort of case is most useful for this purpose? Although all case studies presumably shed light on causal mechanisms, not all cases are equally transparent. In situations where a causal hypothesis is clear and has already been confirmed by cross‐case analysis, researchers are well advised to focus on a case where the causal effect of X   1 on Y can be isolated from other potentially confounding factors ( X   2 ). I shall call this a pathway case to indicate its uniquely penetrating insight into causal mechanisms. In contrast to the crucial case, this sort of method is practicable only in circumstances where cross‐case covariational patterns are well studied and where the mechanism linking X   1 and Y remains dim. Because the pathway case builds on prior cross‐case analysis, the problem of case selection must be situated within that sample. There is no standalone pathway case.

The logic of the pathway case is clearest in situations of causal sufficiency—where a causal factor of interest, X   1 , is sufficient by itself (though perhaps not necessary) to account for Y 's value (0 or 1). The other causes of Y , about which we need make no assumptions, are designated as a vector, X   2 .

Note that wherever various causal factors are substitutable for one another, each factor is conceptualized (individually) as sufficient ( Braumoeller 2003 ). Thus, situations of causal equifinality presume causal sufficiency on the part of each factor or set of conjoint factors. An example is provided by the literature on democratization, which stipulates three main avenues of regime change: leadership‐initiated reform, a controlled opening to opposition, or the collapse of an authoritarian regime ( Colomer 1991 ). The case‐study format constrains us to analyze one at a time, so let us limit our scope to the first one—leadership‐initiated reform. So considered, a causal‐pathway case would be one with the following features: (a) democratization, (b) leadership‐initiated reform, (c) no controlled opening to the opposition, (d) no collapse of the previous authoritarian regime, and (e) no other extraneous factors that might affect the process of democratization. In a case of this type, the causal mechanisms by which leadership‐initiated reform may lead to democratization will be easiest to study. Note that it is not necessary to assume that leadership‐initiated reform always leads to democratization; it may or may not be a deterministic cause. But it is necessary to assume that leadership‐initiated reform can sometimes lead to democratization on its own (given certain background features).

Now let us move from these examples to a general‐purpose model. For heuristic purposes, let us presume that all variables in that model are dichotomous (coded as 0 or 1) and that the model is complete (all causes of Y are included). All causal relationships will be coded so as to be positive: X   1 and Y covary as do X   2 and Y . This allows us to visualize a range of possible combinations at a glance.

Recall that the pathway case is always focused, by definition, on a single causal factor, denoted X   1 . (The researcher's focus may shift to other causal factors, but may only focus on one causal factor at a time.) In this scenario, and regardless of how many additional causes of Y there might be (denoted X   2 , a vector of controls), there are only eight relevant case types, as illustrated in Table 28.2 . Identifying these case types is a relatively simple matter, and can be accomplished in a small‐ N sample by the construction of a truth‐table (modeled after Table 28.2 ) or in a large‐ N sample by the use of cross‐tabs.

Notes : X   1 = the variable of theoretical interest. X   2 = a vector of controls (a score of 0 indicates that all control variables have a score of 0, while a score of 1 indicates that all control variables have a score of 1). Y = the outcome of interest. A–H = case types (the N for each case type is indeterminate). G, H = possible pathway cases. Sample size = indeterminate.

Assumptions : (a) all variables can be coded dichotomously (a binary coding of the concept is valid); (b) all independent variables are positively correlated with Y in the general case; ( c ) X   1 is (at least sometimes) a sufficient cause of Y .

Note that the total number of combinations of values depends on the number of control variables, which we have represented with a single vector, X   2 . If this vector consists of a single variable then there are only eight case types. If this vector consists of two variables ( X   2a , X   2b ) then the total number of possible combinations increases from eight (2 3 ) to sixteen (2 4 ). And so forth. However, none of these combinations is relevant for present purposes except those where X   2a and X   2b have the same value (0 or 1). “Mixed” cases are not causal pathway cases, for reasons that should become clear.

The pathway case, following the logic of the crucial case, is one where the causal factor of interest, X   1 , correctly predicts Y while all other possible causes of Y (represented by the vector, X   2 ) make “wrong” predictions. If X   1 is—at least in some circumstances—a sufficient cause of Y , then it is these sorts of cases that should be most useful for tracing causal mechanisms. There are only two such cases in Ta b l e 28.2—G and H. In all other cases, the mechanism running from X   1 to Y would be difficult to discern either because X   1 and Y are not correlated in the usual way (constituting an unusual case, in the terms of our hypothesis) or because other confounding factors ( X   2 ) intrude. In case A, for example, the positive value on Y could be a product of X   1 or X   2 . An in‐depth examination of this case is not likely to be very revealing.

Keep in mind that because the researcher already knows from her cross‐case examination what the general causal relationships are, she knows (prior to the case‐ study investigation) what constitutes a correct or incorrect prediction. In the crucial‐ case method, by contrast, these expectations are deductive rather than empirical. This is what differentiates the two methods. And this is why the causal pathway case is useful principally for elucidating causal mechanisms rather than verifying or falsifying general propositions (which are already more or less apparent from the cross‐case evidence). Of course, we must leave open the possibility that the investigation of causal mechanisms would invalidate a general claim, if that claim is utterly contingent upon a specific set of causal mechanisms and the case study shows that no such mechanisms are present. However, this is rather unlikely in most social science settings. Usually, the result of such a finding will be a reformulation of the causal processes by which X   1 causes Y —or, alternatively, a realization that the case under investigation is aberrant (atypical of the general population of cases).

Sometimes, the research question is framed as a unidirectional cause: one is interested in why 0 becomes 1 (or vice versa) but not in why 1 becomes 0. In our previous example, we asked why democracies fail, not why countries become democratic or authoritarian. So framed, there can be only one type of causal‐pathway case. (Whether regime failure is coded as 0 or 1 is a matter of taste.) Where researchers are interested in bidirectional causality—a movement from 0 to 1 as well as from 1 to 0—there are two possible causal‐pathway cases, G and H. In practice, however, one of these case types is almost always more useful than the other. Thus, it seems reasonable to employ the term “pathway case” in the singular. In order to determine which of these two case types will be more useful for intensive analysis the researcher should look to see whether each case type exhibits desirable features such as: (a) a rare (unusual) value on X   1 or Y (designated “extreme” in our previous discussion), (b) observable temporal variation in X   1 , ( c ) an X   1 / Y relationship that is easier to study (it has more visible features; it is more transparent), or (d) a lower residual (thus indicating a more typical case, within the terms of the general model). Usually, the choice between G and H is intuitively obvious.

Now, let us consider a scenario in which all (or most) variables of concern to the model are continuous, rather than dichotomous. Here, the job of case selection is considerably more complex, for causal “sufficiency” (in the usual sense) cannot be invoked. It is no longer plausible to assume that a given cause can be entirely partitioned, i.e. rival factors eliminated. However, the search for a pathway case may still be viable. What we are looking for in this scenario is a case that satisfies two criteria: (1) it is not an outlier (or at least not an extreme outlier) in the general model and (2) its score on the outcome ( Y ) is strongly influenced by the theoretical variable of interest ( X   1 ), taking all other factors into account ( X   2 ). In this sort of case it should be easiest to “see” the causal mechanisms that lie between X   1 and Y .

Achieving the second desiderata requires a bit of manipulation. In order to determine which (nonoutlier) cases are most strongly affected by X   1 , given all the other parameters in the model, one must compare the size of the residuals for each case in a reduced form model, Y = Constant + X   2 + Res reduced , with the size of the residuals for each case in a full model, Y = Constant + X   2 + X   1 + Res full . The pathway case is that case, or set of cases, which shows the greatest difference between the residual for the reduced‐form model and the full model (ΔResidual). Thus,

Note that the residual for a case must be smaller in the full model than in the reduced‐ form model; otherwise, the addition of the variable of interest ( X   1 ) pulls the case away from the regression line. We want to find a case where the addition of X   1 pushes the case towards the regression line, i.e. it helps to “explain” that case.

As an example, let us suppose that we are interested in exploring the effect of mineral wealth on the prospects for democracy in a society. According to a good deal of work on this subject, countries with a bounty of natural resources—particularly oil—are less likely to democratize (or once having undergone a democratic transition, are more likely to revert to authoritarian rule) ( Barro 1999 ; Humphreys 2005 ; Ross 2001 ). The cross‐country evidence is robust. Yet as is often the case, the causal mechanisms remain rather obscure. In order to better understand this phenomenon it may be worthwhile to exploit the findings of cross‐country regression models in order to identify a country whose regime type (i.e. its democracy “score” on some general index) is strongly affected by its natural‐research wealth, all other things held constant. An analysis of this sort identifies two countries— the United Arab Emirates and Kuwait—with high Δ Residual values and modest residuals in the full model (signifying that these cases are not outliers). Researchers seeking to explore the effect of oil wealth on regime type might do well to focus on these two cases since their patterns of democracy cannot be well explained by other factors—e.g. economic development, religion, European influence, or ethnic fractionalization. The presence of oil wealth in these countries would appear to have a strong independent effect on the prospects for democratization in these cases, an effect that is well modeled by general theory and by the available cross‐case evidence.

To reiterate, the logic of causal “elimination” is much more compelling where variables are dichotomous and where causal sufficiency can be assumed ( X   1 is sufficient by itself, at least in some circumstances, to cause Y ). Where variables are continuous, the strategy of the pathway case is more dubious, for potentially confounding causal factors ( X   2 ) cannot be neatly partitioned. Even so, we have indicated why the selection of a pathway case may be a logical approach to case‐study analysis in many circumstances.

The exceptions may be briefly noted. Sometimes, where all variables in a model are dichotomous, there are no pathway cases, i.e. no cases of type G or H (in Table 28.2 ). This is known as the “empty cell” problem, or a problem of severe causal multicollinearity. The universe of observational data does not always oblige us with cases that allow us to independently test a given hypothesis. Where variables are continuous, the analogous problem is that of a causal variable of interest ( X   1 ) that has only minimal effects on the outcome of interest. That is, its role in the general model is quite minor. In these situations, the only cases that are strongly affected by X   1 —if there are any at all—may be extreme outliers, and these sorts of cases are not properly regarded as providing confirmatory evidence for a proposition, for reasons that are abundantly clear by now.

Finally, it should be clarified that the identification of a causal pathway case does not obviate the utility of exploring other cases. One might, for example, want to compare both sorts of potential pathway cases—G and H—with each other. Many other combinations suggest themselves. However, this sort of multi‐case investigation moves beyond the logic of the causal‐pathway case.

8 Most‐similar Cases

The most‐similar method employs a minimum of two cases. 16 In its purest form, the chosen pair of cases is similar in all respects except the variable(s) of interest. If the study is exploratory (i.e. hypothesis generating), the researcher looks for cases that differ on the outcome of theoretical interest but are similar on various factors that might have contributed to that outcome, as illustrated in Table 28.3 (A) . This is a common form of case selection at the initial stage of research. Often, fruitful analysis begins with an apparent anomaly: two cases are apparently quite similar, and yet demonstrate surprisingly different outcomes. The hope is that intensive study of these cases will reveal one—or at most several—factors that differ across these cases. These differing factors ( X   1 ) are looked upon as putative causes. At this stage, the research may be described by the second diagram in Table 28.3 (B) . Sometimes, a researcher begins with a strong hypothesis, in which case her research design is confirmatory (hypothesis testing) from the get‐go. That is, she strives to identify cases that exhibit different outcomes, different scores on the factor of interest, and similar scores on all other possible causal factors, as illustrated in the second (hypothesis‐testing) diagram in Table 28.3 (B) .

The point is that the purpose of a most‐similar research design, and hence its basic setup, often changes as a researcher moves from an exploratory to a confirmatory mode of analysis. However, regardless of where one begins, the results, when published, look like a hypothesis‐testing research design. Question marks have been removed: (A) becomes (B) in Table 28.3 .

As an example, let us consider Leon Epstein's classic study of party cohesion, which focuses on two “most‐similar” countries, the United States and Canada. Canada has highly disciplined parties whose members vote together on the floor of the House of Commons while the United States has weak, undisciplined parties, whose members often defect on floor votes in Congress. In explaining these divergent outcomes, persistent over many years, Epstein first discusses possible causal factors that are held more or less constant across the two cases. Both the United States and Canada inherited English political cultures, both have large territories and heterogeneous populations, both are federal, and both have fairly loose party structures with strong regional bases and a weak center. These are the “control” variables. Where they differ is in one constitutional feature: Canada is parliamentary while the United States is presidential. And it is this institutional difference that Epstein identifies as the crucial (differentiating) cause. (For further examples of the most‐similar method see Brenner 1976 ; Hamilton 1977 ; Lipset 1968 ; Miguel 2004 ; Moulder 1977 ; Posner 2004 .)

X   1 = the variable of theoretical interest. X   2 = a vector of controls. Y = the outcome of interest.

Several caveats apply to any most‐similar analysis (in addition to the usual set of assumptions applying to all case‐study analysis). First, each causal factor is understood as having an independent and additive effect on the outcome; there are no “interaction” effects. Second, one must code cases dichotomously (high/low, present/absent). This is straightforward if the underlying variables are also dichotomous (e.g. federal/unitary). However, it is often the case that variables of concern in the model are continuous (e.g. party cohesion). In this setting, the researcher must “dichotomize” the scoring of cases so as to simplify the two‐case analysis. (Some flexibility is admissible on the vector of controls ( X   2 ) that are “held constant” across the cases. Nonidentity is tolerable if the deviation runs counter to the predicted hypothesis. For example, Epstein describes both the United States and Canada as having strong regional bases of power, a factor that is probably more significant in recent Canadian history than in recent American history. However, because regional bases of power should lead to weaker parties, rather than stronger parties, this element of nonidentity does not challenge Epstein's conclusions. Indeed, it sets up a most‐difficult research scenario, as discussed above.)

In one respect the requirements for case control are not so stringent. Specifically, it is not usually necessary to measure control variables (at least not with a high degree of precision) in order to control for them. If two countries can be assumed to have similar cultural heritages one needn't worry about constructing variables to measure that heritage. One can simply assert that, whatever they are, they are more or less constant across the two cases. This is similar to the technique employed in a randomized experiment, where the researcher typically does not attempt to measure all the factors that might affect the causal relationship of interest. She assumes, rather, that these unknown factors have been neutralized across the treatment and control groups by randomization or by the choice of a sample that is internally homogeneous.

The most useful statistical tool for identifying cases for in‐depth analysis in a most‐ similar setting is probably some variety of matching strategy—e.g. exact matching, approximate matching, or propensity‐score matching. 17 The product of this procedure is a set of matched cases that can be compared in whatever way the researcher deems appropriate. These are the “most‐similar” cases. Rosenbaum and Silber (2001 , 223) summarize:

Unlike model‐based adjustments, where [individuals] vanish and are replaced by the coefficients of a model, in matching, ostensibly comparable patterns are compared directly, one by one. Modern matching methods involve statistical modeling and combinatorial algorithms, but the end result is a collection of pairs or sets of people who look comparable, at least on average. In matching, people retain their integrity as people, so they can be examined and their stories can be told individually.

Matching, conclude the authors, “facilitates, rather than inhibits, thick description” ( Rosenbaum and Silber 2001 , 223).

In principle, the same matching techniques that have been used successfully in observational studies of medical treatments might also be adapted to the study of nation states, political parties, cities, or indeed any traditional paired cases in the social sciences. Indeed, the current popularity of matching among statisticians—relative, that is, to garden‐variety regression models—rests upon what qualitative researchers would recognize as a “case‐based” approach to causal analysis. If Rosenbaum and Silber are correct, it may be perfectly reasonable to appropriate this large‐ N method of analysis for case‐study purposes.

As with other methods of case selection, the most‐similar method is prone to problems of nonrepresentativeness. If employed in a qualitative fashion (without a systematic cross‐case selection strategy), potential biases in the chosen case must be addressed in a speculative way. If the researcher employs a matching technique of case selection within a large‐ N sample, the problem of potential bias can be addressed by assuring the choice of cases that are not extreme outliers, as judged by their residuals in the full model. Most‐similar cases should also be “typical” cases, though some scope for deviance around the regression line may be acceptable for purposes of finding a good fit among cases.

X   1 = the variable of theoretical interest. X   2a–d = a vector of controls. Y = the outcome of interest.

9 Most‐different Cases

A final case‐selection method is the reverse image of the previous method. Here, variation on independent variables is prized, while variation on the outcome is eschewed. Rather than looking for cases that are most‐similar, one looks for cases that are most‐ different . Specifically, the researcher tries to identify cases where just one independent variable ( X   1 ), as well as the dependent variable ( Y ), covary, while all other plausible factors ( X   2a–d ) show different values. 18

The simplest form of this two‐case comparison is illustrated in Table 28.4 . Cases A and B are deemed “most different,” though they are similar in two essential respects— the causal variable of interest and the outcome.

As an example, I follow Marc Howard's (2003) recent work, which explores the enduring impact of Communism on civil society. 19 Cross‐national surveys show a strong correlation between former Communist regimes and low social capital, controlling for a variety of possible confounders. It is a strong result. Howard wonders why this relationship is so strong and why it persists, and perhaps even strengthens, in countries that are no longer socialist or authoritarian. In order to answer this question, he focuses on two most‐different cases, Russia and East Germany. These two countries were quite different—in all ways other than their Communist experience— prior to the Soviet era, during the Soviet era (since East Germany received substantial subsidies from West Germany), and in the post‐Soviet era, as East Germany was absorbed into West Germany. Yet, they both score near the bottom of various cross‐ national indices intended to measure the prevalence of civic engagement in the current era. Thus, Howard's (2003 , 6–9) case selection procedure meets the requirements of the most‐different research design: Variance is found on all (or most) dimensions aside from the key factor of interest (Communism) and the outcome (civic engagement).

What leverage is brought to the analysis from this approach? Howard's case studies combine evidence drawn from mass surveys and from in‐depth interviews of small, stratified samples of Russians and East Germans. (This is a good illustration, incidentally, of how quantitative and qualitative evidence can be fruitfully combined in the intensive study of several cases.) The product of this analysis is the identification of three causal pathways that, Howard (2003 , 122) claims, help to explain the laggard status of civil society in post‐Communist polities: “the mistrust of communist organizations, the persistence of friendship networks, and the disappointment with post‐communism.” Simply put, Howard (2003 , 145) concludes, “a great number of citizens in Russia and Eastern Germany feel a strong and lingering sense of distrust of any kind of public organization, a general satisfaction with their own personal networks (accompanied by a sense of deteriorating relations within society overall), and disappointment in the developments of post‐communism.”

The strength of this most‐different case analysis is that the results obtained in East Germany and Russia should also apply in other post‐Communist polities (e.g. Lithuania, Poland, Bulgaria, Albania). By choosing a heterogeneous sample, Howard solves the problem of representativeness in his restricted sample. However, this sample is demonstrably not representative across the population of the inference, which is intended to cover all countries of the world.

More problematic is the lack of variation on key causal factors of interest— Communism and its putative causal pathways. For this reason, it is difficult to reach conclusions about the causal status of these factors on the basis of the most‐different analysis alone. It is possible, that is, that the three causal pathways identified by Howard also operate within polities that never experienced Communist rule.

Nor does it seem possible to conclusively eliminate rival hypotheses on the basis of this most‐different analysis. Indeed, this is not Howard's intention. He wishes merely to show that whatever influence on civil society might be attributed to economic, cultural, and other factors does not exhaust this subject.

My considered judgment is that the most‐different research design provides minimal leverage into the problem of why Communist systems appear to suppress civic engagement, years after their disappearance. Fortunately, this is not the only research design employed by Howard in his admirable study. Indeed, the author employs two other small‐ N cross‐case methods, as well as a large‐ N cross‐country statistical analysis. These methods do most of the analytic work. East Germany may be regarded as a causal pathway case (see above). It has all the attributes normally assumed to foster civic engagement (e.g. a growing economy, multiparty competition, civil liberties, a free press, close association with Western European culture and politics), but nonetheless shows little or no improvement on this dimension during the post‐ transition era ( Howard 2003 , 8). It is plausible to attribute this lack of change to its Communist past, as Howard does, in which case East Germany should be a fruitful case for the investigation of causal mechanisms. The contrast between East and West Germany provides a most‐similar analysis since the two polities share virtually everything except a Communist past. This variation is also deftly exploited by Howard.

I do not wish to dismiss the most‐different research method entirely. Surely, Howard's findings are stronger with the intensive analysis of Russia than they would be without. Yet his book would not stand securely on the empirical foundation provided by most‐different analysis alone. If one strips away the pathway‐case (East Germany) and the most‐similar analysis (East/West Germany) there is little left upon which to base an analysis of causal relations (aside from the large‐ N cross‐national analysis). Indeed, most scholars who employ the most‐different method do so in conjunction with other methods. 20 It is rarely, if ever, a standalone method. 21

Generalizing from this discussion of Marc Howard's work, I offer the following summary remarks on the most‐different method of case analysis. (I leave aside issues faced by all case‐study analyses, issues that are explored in Gerring 2007 .)

Let us begin with a methodological obstacle that is faced by both Millean styles of analysis—the necessity of dichotomizing every variable in the analysis. Recall that, as with most‐similar analysis, differences across cases must generally be sizeable enough to be interpretable in an essentially dichotomous fashion (e.g. high/low, present/absent) and similarities must be close enough to be understood as essentially identical (e.g. high/high, present/present). Otherwise the results of a Millean style analysis are not interpretable. The problem of “degrees” is deadly if the variables under consideration are, by nature, continuous (e.g. GDP). This is a particular concern in Howard's analysis, where East Germany scores somewhat higher than Russia in civic engagement; they are both low, but Russia is quite a bit lower. Howard assumes that this divergence is minimal enough to be understood as a difference of degrees rather than of kinds, a judgment that might be questioned. In these respects, most‐different analysis is no more secure—but also no less—than most‐similar analysis.

In one respect, most‐different analysis is superior to most‐similar analysis. If the coding assumptions are sound, the most‐different research design may be quite useful for eliminating necessary causes . Causal factors that do not appear across the chosen cases—e.g. X   2a–d in Table 28.4 —are evidently unnecessary for the production of Y . However, it does not follow that the most‐different method is the best method for eliminating necessary causes. Note that the defining feature of this method is the shared element across cases— X   1 in Table 28.4 . This feature does not help one to eliminate necessary causes. Indeed, if one were focused solely on eliminating necessary causes one would presumably seek out cases that register the same outcomes and have maximum diversity on other attributes. In Table 28.4 , this would be a set of cases that satisfy conditions X   2a–d , but not X   1 . Thus, even the presumed strength of the most‐different analysis is not so strong.

Usually, case‐study analysis is focused on the identification (or clarification) of causal relations, not the elimination of possible causes. In this setting, the most‐ different technique is useful, but only if assumptions of causal uniqueness hold. By “causal uniqueness,” I mean a situation in which a given outcome is the product of only one cause: Y cannot occur except in the presence of X . X is necessary, and in some situations (given certain background conditions) sufficient, to cause Y . 22

Consider the following hypothetical example. Suppose that a new disease, about which little is known, has appeared in Country A. There are hundreds of infected persons across dozens of affected communities in that country. In Country B, located at the other end of the world, several new cases of the disease surface in a single community. In this setting, we can imagine two sorts of Millean analyses. The first examines two similar communities within Country A, one of which has developed the disease and the other of which has not. This is the most‐similar style of case comparison, and focuses accordingly on the identification of a difference between the two cases that might account for variation across the sample. A second approach focuses on communities where the disease has appeared across the two countries and searches for any similarities that might account for these similar outcomes. This is the most‐different research design.

Both are plausible approaches to this particular problem, and we can imagine epidemiologists employing them simultaneously. However, the most‐different design demands stronger assumptions about the underlying factors at work. It supposes that the disease arises from the same cause in any setting. This is often a reasonable operating assumption when one is dealing with natural phenomena, though there are certainly many exceptions. Death, for example, has many causes. For this reason, it would not occur to us to look for most‐different cases of high mortality around the world. In order for the most‐different research design to effectively identify a causal factor at work in a given outcome, the researcher must assume that X   1 —the factor held constant across the diverse cases—is the only possible cause of Y (see Table 28.4 ). This assumption rarely holds in social‐scientific settings. Most outcomes of interest to anthropologists, economists, political scientists, and sociologists have multiple causes. There are many ways to win an election, to build a welfare state, to get into a war, to overthrow a government, or—returning to Marc Howard's work—to build a strong civil society. And it is for this reason that most‐different analysis is rarely applied in social science work and, where applied, is rarely convincing.

If this seems a tad severe, there is a more charitable way of approaching the most‐different method. Arguably, this is not a pure “method” at all but merely a supplement, a way of incorporating diversity in the sub‐sample of cases that provide the unusual outcome of interest. If the unusual outcome is revolutions, one might wish to encompass a wide variety of revolutions in one's analysis. If the unusual outcome is post‐Communist civil society, it seems appropriate to include a diverse set of post‐Communist polities in one's sample of case studies, as Marc Howard does. From this perspective, the most‐different method (so‐called) might be better labeled a diverse‐case method, as explored above.

10 Conclusions

In order to be a case of something broader than itself, the chosen case must be representative (in some respects) of a larger population. Otherwise—if it is purely idiosyncratic (“unique”)—it is uninformative about anything lying outside the borders of the case itself. A study based on a nonrepresentative sample has no (or very little) external validity. To be sure, no phenomenon is purely idiosyncratic; the notion of a unique case is a matter that would be difficult to define. One is concerned, as always, with matters of degree. Cases are more or less representative of some broader phenomenon and, on that score, may be considered better or worse subjects for intensive analysis. (The one exception, as noted, is the influential case.)

Of all the problems besetting case‐study analysis, perhaps the most persistent— and the most persistently bemoaned—is the problem of sample bias ( Achen and Snidal 1989 ; Collier and Mahoney 1996 ; Geddes 1990 ; King, Keohane, and Verba 1994 ; Rohlfing 2004 ; Sekhon 2004 ). Lisa Martin (1992 , 5) finds that the overemphasis of international relations scholars on a few well‐known cases of economic sanctions— most of which failed to elicit any change in the sanctioned country—“has distorted analysts view of the dynamics and characteristics of economic sanctions.” Barbara Geddes (1990) charges that many analyses of industrial policy have focused exclusively on the most successful cases—primarily the East Asian NICs—leading to biased inferences. Anna Breman and Carolyn Shelton (2001) show that case‐study work on the question of structural adjustment is systematically biased insofar as researchers tend to focus on disaster cases—those where structural adjustment is associated with very poor health and human development outcomes. These cases, often located in sub‐Saharan Africa, are by no means representative of the entire population. Consequently, scholarship on the question of structural adjustment is highly skewed in a particular ideological direction (against neoliberalism) (see also Gerring, Thacker, and Moreno 2005) .

These examples might be multiplied many times. Indeed, for many topics the most‐studied cases are acknowledged to be less than representative. It is worth reflecting upon the fact that our knowledge of the world is heavily colored by a few “big” (populous, rich, powerful) countries, and that a good portion of the disciplines of economics, political science, and sociology are built upon scholars' familiarity with the economics, political science, and sociology of one country, the United States. 23 Case‐study work is particularly prone to problems of investigator bias since so much rides on the researcher's selection of one (or a few) cases. Even if the investigator is unbiased, her sample may still be biased simply by virtue of “random” error (which may be understood as measurement error, error in the data‐generation process, or as an underlying causal feature of the universe).

There are only two situations in which a case‐study researcher need not be concerned with the representativeness of her chosen case. The first is the influential case research design, where a case is chosen because of its possible influence on a cross‐case model, and hence is not expected to be representative of a larger sample. The second is the deviant‐case method, where the chosen case is employed to confirm a broader cross‐case argument to which the case stands as an apparent exception. Yet even here the chosen case is expected to be representative of a broader set of cases—those, in particular, that are poorly explained by the extant model.

In all other circumstances, cases must be representative of the population of interest in whatever ways might be relevant to the proposition in question. Note that where a researcher is attempting to disconfirm a deterministic proposition the question of representativeness is perhaps more appropriately understood as a question of classification: Is the chosen case appropriately classified as a member of the designated population? If so, then it is fodder for a disconfirming case study.

If the researcher is attempting to confirm a deterministic proposition, or to make probabilistic arguments about a causal relationship, then the problem of representativeness is of the more usual sort: Is case A unit‐homogeneous relative to other cases in the population? This is not an easy matter to test. However, in a large‐ N context the residual for that case (in whatever model the researcher has greatest confidence in) is a reasonable place to start. Of course, this test is only as good as the model at hand. Any incorrect specifications or incorrect modeling procedures will likely bias the results and give an incorrect assessment of each case's “typicality.” In addition, there is the possibility of stochastic error, errors that cannot be modeled in a general framework. Given the explanatory weight that individual cases are asked to bear in a case‐study analysis, it is wise to consider more than just the residual test of representativeness. Deductive logic and an in‐depth knowledge of the case in question are often more reliable tools than the results of a cross‐case model.

In any case, there is no dispensing with the question. Case studies (with the two exceptions already noted) rest upon an assumed synecdoche: The case should stand for a population. If this is not true, or if there is reason to doubt this assumption, then the utility of the case study is brought severely into question.

Fortunately, there is some safety in numbers. Insofar as case‐study evidence is combined with cross‐case evidence the issue of sample bias is mitigated. Indeed, the suspicion of case‐study work that one finds in the social sciences today is, in my view, a product of a too‐literal interpretation of the case‐study method. A case study tout court is thought to mean a case study tout seul . Insofar as case studies and cross‐case studies can be enlisted within the same investigation (either in the same study or by reference to other studies in the same subfield), problems of representativeness are less worrisome. This is the virtue of cross‐level work, a.k.a. “triangulation.”

11 Ambiguities

Before concluding, I wish to draw attention to two ambiguities in case‐selection strategies in case‐study research. The first concerns the admixture of several case‐ selection strategies. The second concerns the changing status of a case as a study proceeds.

Some case studies follow only one strategy of case selection. They are typical , diverse , extreme , deviant , influential , crucial , pathway , most‐similar , or most‐different research designs, as discussed. However, many case studies mix and match among these case‐selection strategies. Indeed, insofar as all case studies seek representative samples, they are always in search of “typical” cases. Thus, it is common for writers to declare that their case is, for example, both extreme and typical; it has an extreme value on X   1 or Y but is not, in other respects, idiosyncratic. There is not much that one can say about these combinations of strategies except that, where the cases allow for a variety of empirical strategies, there is no reason not to pursue them. And where the same cases can serve several functions at once (without further effort on the researcher's part), there is little cost to a multi‐pronged approach to case analysis.

The second issue that deserves emphasis is the changing status of a case during the course of a researcher's investigation—which may last for years, if not decades. The problem is acute wherever a researcher begins in an exploratory mode and proceeds to hypothesis‐testing (that is, she develops a specific X   1 / Y proposition) or where the operative hypothesis or key control variable changes (a new causal factor is discovered or another outcome becomes the focus of analysis). Things change. And it is the mark of a good researcher to keep her mind open to new evidence and new insights. Too often, methodological discussions give the misleading impression that hypotheses are clear and remain fixed over the course of a study's development. Nothing could be further from the truth. The unofficial transcripts of academia— accessible in informal settings, where researchers let their guards down (particularly if inebriated)—are filled with stories about dead‐ends, unexpected findings, and drastically revised theory chapters. It would be interesting, in this vein, to compare published work with dissertation prospectuses and fellowship applications. I doubt if the correlation between these two stages of research is particularly strong.

Research, after all, is about discovery, not simply the verification or falsification of static hypotheses. That said, it is also true that research on a particular topic should move from hypothesis generating to hypothesis‐testing. This marks the progress of a field, and of a scholar's own work. As a rule, research that begins with an open‐ended ( X ‐ or Y ‐centered) analysis should conclude with a determinate X   1 / Y hypothesis.

The problem is that research strategies that are ideal for exploration are not always ideal for confirmation. The extreme‐case method is inherently exploratory since there is no clear causal hypothesis; the researcher is concerned merely to explore variation on a single dimension ( X or Y ). Other methods can be employed in either an open‐ ended (exploratory) or a hypothesis‐testing (confirmatory/disconfirmatory) mode. The difficulty is that once the researcher has arrived at a determinate hypothesis the originally chosen research design may no longer appear to be so well designed.

This is unfortunate, but inevitable. One cannot construct the perfect research design until (a) one has a specific hypothesis and (b) one is reasonably certain about what one is going to find “out there” in the empirical world. This is particularly true of observational research designs, but it also applies to many experimental research designs: Usually, there is a “good” (informative) finding, and a finding that is less insightful. In short, the perfect case‐study research design is usually apparent only ex post facto .

There are three ways to handle this. One can explain, straightforwardly, that the initial research was undertaken in an exploratory fashion, and therefore not constructed to test the specific hypothesis that is—now—the primary argument. Alternatively, one can try to redesign the study after the new (or revised) hypothesis has been formulated. This may require additional field research or perhaps the integration of additional cases or variables that can be obtained through secondary sources or through consultation of experts. A final approach is to simply jettison, or de‐emphasize, the portion of research that no longer addresses the (revised) key hypothesis. A three‐case study may become a two‐case study, and so forth. Lost time and effort are the costs of this downsizing.

In the event, practical considerations will probably determine which of these three strategies, or combinations of strategies, is to be followed. (They are not mutually exclusive.) The point to remember is that revision of one's cross‐case research design is normal and perhaps to be expected. Not all twists and turns on the meandering trail of truth can be anticipated.

12 Are There Other Methods of Case Selection?

At the outset of this chapter I summarized the task of case selection as a matter of achieving two objectives: representativeness (typicality) and variation (causal leverage). Evidently, there are other objectives as well. For example, one wishes to identify cases that are independent of each other. If chosen cases are affected by each other (sometimes known as Galton's problem or a problem of diffusion), this problem must be corrected before analysis can take place. I have neglected this issue because it is usually apparent to the researcher and, in any case, there are no simple techniques that might be utilized to correct for such biases. (For further discussion of this and other factors impinging upon case selection see Gerring 2001 , 178–81.)

I have also disregarded pragmatic/logistical issues that might affect case selection. Evidently, case selection is often influenced by a researcher's familiarity with the language of a country, a personal entrée into that locale, special access to important data, or funding that covers one archive rather than another. Pragmatic considerations are often—and quite rightly—decisive in the case‐selection process.

A final consideration concerns the theoretical prominence of a particular case within the literature on a subject. Researchers are sometimes obliged to study cases that have received extensive attention in previous studies. These are sometimes referred to as “paradigmatic” cases or “exemplars” ( Flyvbjerg 2004 , 427).

However, neither pragmatic/logistical utility nor theoretical prominence qualifies as a methodological factor in case selection. That is, these features of a case have no bearing on the validity of the findings stemming from a study. As such, it is appropriate to grant these issues a peripheral status in this chapter.

One final caveat must be issued. While it is traditional to distinguish among the tasks of case selection and case analysis, a close look at these processes shows them to be indistinct and overlapping. One cannot choose a case without considering the sort of analysis that it might be subjected to, and vice versa. Thus, the reader should consider choosing cases by employing the nine techniques laid out in this chapter along with any considerations that might be introduced by virtue of a case's quasi‐experimental qualities, a topic taken up elsewhere ( Gerring 2007 , ch. 6 ).

Abadie, A. , Drukker, D. , Herr, J. L. , and Imbens, G. W.   2001 . Implementing matching estimators for average treatment effects in Stata.   Stata Journal , 1: 1–18.

Google Scholar

Abbott, A.   2001 . Time Matters: On Theory and Method . Chicago: University of Chicago Press.

Google Preview

——  and Tsay, A.   2000 . Sequence analysis and optimal matching methods in sociology.   Sociological Methods and Research , 29: 3–33. 10.1177/0049124100029001001

——  and Forrest, J.   1986 . Optimal matching methods for historical sequences.   Journal of Interdisciplinary History , 16: 471–94. 10.2307/204500

Achen, C. H. , and Snidal, D.   1989 . Rational deterrence theory and comparative case studies.   World Politics , 41: 143–69. 10.2307/2010405

Allen, W. S.   1965 . The Nazi Seizure of Power: The Experience of a Single German Town, 1930–1935 . New York: Watts.

Almond, G. A.   1956 . Comparative political systems.   Journal of Politics , 18: 391–409.

Amenta, E.   1991 . Making the most of a case study: theories of the welfare state and the American experience. Pp. 172–94 in Issues and Alternatives in Comparative Social Research ed. C. C. Ragin . Leiden: E. J. Brill.

Barro, R. J.   1999 . Determinants of democracy.   Journal of Political Economy , 107: 158–83. 10.1086/250107

Belsey, D. A. , Kuh, E. , and Welsch, R. E.   2004 . Regression Diagnostics: Identifying Influential Data and Sources of Collinearity . New York: Wiley.

Bennett, A. , Lepgold, J. , and Unger, D.   1994 . Burden‐sharing in the Persian Gulf War.   International Organization , 48: 39–75. 10.1017/S0020818300000813

Bentley, A. 1908/ 1967 . The Process of Government . Cambridge, Mass.: Harvard University Press.

Brady, H. E. , and Collier, D. (eds.) 2004 . Rethinking Social Inquiry: Diverse Tools, Shared Standards . Lanham, Md.: Rowman and Littlefield.

Braumoeller, B. F.   2003 . Causal complexity and the study of politics.   Political Analysis , 11: 209–33. 10.1093/pan/mpg012

Breman, A. , and Shelton, C. 2001. Structural adjustment and health: a literature review of the debate, its role‐players and presented empirical evidence. CMH Working Paper Series, Paper No. WG6: 6. WHO, Commission on Macroeconomics and Health.

Brenner, R.   1976 . Agrarian class structure and economic development in pre‐industrial Europe.   Past and Present , 70: 30–75. 10.1093/past/70.1.30

Browne, A.   1987 . When Battered Women Kill . New York: Free Press.

Buchbinder, S. , and Vittinghoff, E.   1999 . HIV‐infected long‐term nonprogressors: epidemiology, mechanisms of delayed progression, and clinical and research implications.   Microbes Infect , 1: 1113–20. 10.1016/S1286-4579(99)00204-X

Cohen, M. R. , and Nagel, E.   1934 . An Introduction to Logic and Scientific Method . New York: Harcourt, Brace and Company.

Collier, D. , and Mahoney, J.   1996 . Insights and pitfalls: selection bias in qualitative research.   World Politics , 49: 56–91. 10.1353/wp.1996.0023

Collier, R. B. , and Collier, D. 1991/ 2002 . Shaping the Political Arena: Critical Junctures, the Labor Movement, and Regime Dynamics in Latin America . Notre Dame, Ind.: University of Notre Dame Press.

Colomer, J. M.   1991 . Transitions by agreement: modeling the Spanish way.   American Political Science Review , 85: 1283–302. 10.2307/1963946

Converse, P. E. , and Dupeux, G.   1962 . Politicization of the electorate in France and the United States.   Public Opinion Quarterly , 16: 1–23. 10.1086/267067

Coppedge, M. J. 2004. The conditional impact of the economy on democracy in Latin America. Presented at the conference “Democratic Advancements and Setbacks: What Have We Learnt?”, Uppsala University, June 11–13.

De Felice, E. G.   1986 . Causal inference and comparative methods.   Comparative Political Studies , 19: 415–37. 10.1177/0010414086019003005

Desch, M. C.   2002 . Democracy and victory: why regime type hardly matters.   International Security , 27: 5–47. 10.1162/016228802760987815

Deyo, F. (ed.) 1987 . The Political Economy of the New Asian Industrialism . Ithaca, NY: Cornell University Press.

Dion, D.   1998 . Evidence and inference in the comparative case study.   Comparative Politics , 30: 127–45. 10.2307/422284

Eckstein, H.   1975 . Case studies and theory in political science. In Handbook of Political Science , vii: Political Science: Scope and Theory , ed. F. I. Greenstein and N. W. Polsby . Reading, Mass.: Addison‐Wesley.

Eggan, F.   1954 . Social anthropology and the method of controlled comparison.   American Anthropologist , 56: 743–63. 10.1525/aa.1954.56.5.02a00020

Elman, C.   2003 . Lessons from Lakatos. In Progress in International Relations Theory: Appraising the Field , ed. C. Elman and M. F. Elman . Cambridge, Mass.: MIT Press.

——  2005 . Explanatory typologies in qualitative studies of international politics.   International Organization , 59: 293–326.

Emigh, R.   1997 . The power of negative thinking: the use of negative case methodology in the development of sociological theory.   Theory and Society , 26: 649–84. 10.1023/A:1006896217647

Epstein, L. D.   1964 . A comparative study of Canadian parties.   American Political Science Review , 58: 46–59. 10.2307/1952754

Ertman, T.   1997 . Birth of the Leviathan: Building States and Regimes in Medieval and Early Modern Europe . Cambridge: Cambridge University Press.

Esping‐Andersen, G.   1990 . The Three Worlds of Welfare Capitalism . Princeton, NJ: Princeton University Press.

Flyvbjerg, B.   2004 . Five misunderstandings about case‐study research. Pp. 420–34 in Qualitative Research Practice , ed. C. Seale , G. Gobo , J. F. Gubrium , and D. Silverman . London: Sage.

Geddes, B.   1990 . How the cases you choose affect the answers you get: selection bias in comparative politics. In Political Analysis , vol. ii, ed. J. A. Stimson . Ann Arbor: University of Michigan Press.

——  2003 . Paradigms and Sand Castles: Theory Building and Research Design in Comparative Politics . Ann Arbor: University of Michigan Press.

George, A. L. , and Bennett, A.   2005 . Case Studies and Theory Development . Cambridge, Mass.: MIT Press.

——  and Smoke, R.   1974 . Deterrence in American Foreign Policy: Theory and Practice . New York: Columbia University Press.

Gerring, J.   2001 . Social Science Methodology: A Criterial Framework . Cambridge: Cambridge University Press.

——  2007 . Case Study Research: Principles and Practices . Cambridge: Cambridge University Press.

——  Thacker, S. and Moreno, C. 2005. Do neoliberal policies save lives? Unpublished manuscript.

Goertz, G. and Starr, H. (eds.) 2003 . Necessary Conditions: Theory, Methodology and Applications . New York: Rowman and Littlefield.

——  and Levy, J. (eds.) forthcoming. Causal explanations, necessary conditions, and case studies: World War I and the end of the Cold War. Manuscript.

Goodin, R. E. and Smitsman, A.   2000 . Placing welfare states: the Netherlands as a crucial test case.   Journal of Comparative Policy Analysis , 2: 39–64. 10.1080/13876980008412635

Gujarati, D. N.   2003 . Basic Econometrics , 4th edn. New York: McGraw‐Hill.

Hamilton, G. G.   1977 . Chinese consumption of foreign commodities: a comparative perspective.   American Sociological Review , 42: 877–91. 10.2307/2094574

Haynes, B. F.   Pantaleo, G. and Fauci, A. S.   1996 . Toward an understanding of the correlates of protective immunity to HIV infection.   Science , 271: 324–8. 10.1126/science.271.5247.324

Hempel, C. G.   1942 . The function of general laws in history.   Journal of Philosophy , 39: 35–48. 10.2307/2017635

Ho, D. E.   Imai, K.   King, G. and Stuart, E. A. 2004. Matching as nonparametric preprocessing for reducing model dependence in parametric causal inference. Manuscript.

Howard, M. M.   2003 . The Weakness of Civil Society in Post‐Communist Europe . Cambridge: Cambridge University Press.

Howson, C. and Urbach, P.   1989 . Scientific Reasoning: The Bayesian Approach . La Salle, Ill.: Open Court.

Humphreys, M.   2005 . Natural resources, conflict, and conflict resolution: uncovering the mechanisms.   Journal of Conflict Resolution , 49: 508–37. 10.1177/0022002705277545

Jenicek, M.   2001 . Clinical Case Reporting in Evidence‐Based Medicine , 2nd edn. Oxford: Oxford University Press.

Karl, T. L.   1997 . The Paradox of Plenty: Oil Booms and Petro‐states . Berkeley: University of California Press.

Kazancigil, A.   1994 . The deviant case in comparative analysis: high stateness in comparative analysis. Pp. 213–38 in Comparing Nations: Concepts, Strategies, Substance , ed. M. Dogan and A. Kazancigil . Cambridge: Blackwell.

Kemp, K. A.   1986 . Race, ethnicity, class and urban spatial conflict: Chicago as a crucial case   Urban Studies , 23: 197–208. 10.1080/00420988620080231

Kendall, P. L. and Wolf, K. M. 1949/ 1955 . The analysis of deviant cases in communications research. In Communications Research, 1948–1949 , ed. P. F. Lazarsfeld and F. N. Stanton. New York: Harper and Brothers. Reprinted as pp. 167–70 in The Language of Social Research , ed. P. F. Lazarsfeld and M. Rosenberg . New York: Free Press.

Kennedy, C. H.   2005 . Single‐case Designs for Educational Research . Boston: Allyn and Bacon.

Kennedy, P.   2003 . A Guide to Econometrics , 5th edn. Cambridge, Mass.: MIT Press.

Khong, Y. F.   1992 . Analogies at War: Korea, Munich, Dien Bien Phu, and the Vietnam Decisions of 1965 . Princeton, NJ: Princeton University Press.

King, G.   Keohane, R. O. and Verba, S.   1994 . Designing Social Inquiry: Scientific Inference in Qualitative Research . Princeton, NJ: Princeton University Press.

Lakatos, I.   1978 . The Methodology of Scientific Research Programmes . Cambridge: Cambridge University Press.

Lazarsfeld, P. F. and Barton, A. H.   1951 . Qualitative measurement in the social sciences: classification, typologies, and indices. In The Policy Sciences , ed. D. Lerner and H. D. Lass‐ well. Stanford, Calif.: Stanford University Press.

Levy, J. S.   2002 . Qualitative methods in international relations. In Evaluating Methodology in International Studies , ed. F. P. Harvey and M. Brecher. Ann Arbor: University of Michigan Press.

Lijphart, A.   1968 . The Politics of Accommodation: Pluralism and Democracy in the Netherlands . Berkeley: University of California Press.

——  1969 . Consociational democracy.   World Politics , 21: 207–25. 10.2307/2009820

——  1971 . Comparative politics and the comparative method. American Political Science Review , 65: 682–93.

——  1975 . The comparable cases strategy in comparative research.   Comparative Political Studies , 8: 158–77.

Lipset, S. M.   1959 . Some social requisites of democracy: economic development and political development.   American Political Science Review , 53: 69–105. 10.2307/1951731

——  1960/ 1963 . Political Man: The Social Bases of Politics . Garden City, NY: Anchor.

——  1968 . Agrarian Socialism: The Cooperative Commonwealth Federation in Saskatchewan. A Study in Political Sociology . Garden City, NY: Doubleday.

——  Trow, M. A. and Coleman, J. S.   1956 . Union Democracy: The Internal Politics of the International Typographical Union . New York: Free Press.

Lynd, R. S. and Lynd, H. M. 1929/ 1956 . Middletown: A Study in American Culture . New York: Harcourt, Brace.

Mahoney, J. and Goertz, G.   2004 . The possibility principle: choosing negative cases in comparative research.   American Political Science Review , 98: 653–69.

Martin, L. L.   1992 . Coercive Cooperation: Explaining Multilateral Economic Sanctions .Princeton, NJ: Princeton University Press.

Mayo, D. G.   1996 . Error and the Growth of Experimental Knowledge . Chicago: University of Chicago Press.

Meckstroth, T.   1975 . “Most different systems” and “most similar systems:” a study in the logic of comparative inquiry.   Comparative Political Studies , 8: 133–77.

Miguel, E.   2004 . Tribe or nation: nation‐building and public goods in Kenya versus Tanzania.   World Politics , 56: 327–62. 10.1353/wp.2004.0018

Mill, J. S. 1843/ 1872 . The System of Logic , 8th edn. London: Longmans, Green.

Monroe, K. R.   1996 . The Heart of Altruism: Perceptions of a Common Humanity . Princeton, NJ: Princeton University Press.

Moore, B., Jr.   1966 . Social Origins of Dictatorship and Democracy: Lord and Peasant in the Making of the Modern World . Boston: Beacon Press.

Morgan, S. L. and Harding, D. J. 2005. Matching estimators of causal effects: from stratification and weighting to practical data analysis routines. Manuscript.

Moulder, F. V.   1977 . Japan, China and the Modern World Economy: Toward a Reinterpretation of East Asian Development ca. 1600 to ca. 1918 . Cambridge: Cambridge University Press.

Munck, G. L.   2004 . Tools for qualitative research. Pp. 105–21 in Rethinking Social Inquiry: Diverse Tools, Shared Standards , ed. H. E. Brady and D. Collier . Lanham, Md. : Rowman and Littlefield.

Njolstad, O.   1990 . Learning from history? Case studies and the limits to theory‐building. Pp. 220–46 in Arms Races: Technological and Political Dynamics , ed. O. Njolstad . Thousand Oaks, Calif.: Sage.

Patton, M. Q.   2002 . Qualitative Evaluation and Research Methods . Newbury Park, Calif.: Sage.

Popper, K. 1934/ 1968 . The Logic of Scientific Discovery . New York: Harper and Row.

——  1963 . Conjectures and Refutations . London: Routledge and Kegan Paul.

Posner, D.   2004 . The political salience of cultural difference: why Chewas and Tumbukas are allies in Zambia and adversaries in Malawi.   American Political Science Review , 98: 529–46.

Przeworski, A. and Teune, H.   1970 . The Logic of Comparative Social Inquiry . New York: John Wiley.

Queen, S.   1928 . Round table on the case study in sociological research.   Publications of the American Sociological Society, Papers and Proceedings , 22: 225–7.

Ragin, C. C.   2000 . Fuzzy‐set Social Science . Chicago: University of Chicago Press.

——  2004 . Turning the tables. Pp. 123–38 in Rethinking Social Inquiry: Diverse Tools, Shared Standards , ed. H. E. Brady and D. Collier.   Lanham, Md. : Rowman and Littlefield.

Reilly, B.   2000 –1. Democracy, ethnic fragmentation, and internal conflict: confused theories, faulty data, and the “crucial case” of Papua New Guinea.   International Security , 25: 162–85. 10.1162/016228800560552

——  and Phillpot, R.   2003 . “Making democracy work” in Papua New Guinea: social capital and provincial development in an ethnically fragmented society.   Asian Survey , 42: 906–27. 10.1525/as.2002.42.6.906

Rogowski, R.   1995 . The role of theory and anomaly in social‐scientific inference.   American Political Science Review , 89: 467–70. 10.2307/2082443

Rohlfing, I. 2004. Have you chosen the right case? Uncertainty in case selection for single case studies. Working Paper, International University, Bremen.

Rosenbaum, P. R.   2004 . Matching in observational studies. In Applied Bayesian Modeling and Causal Inference from an Incomplete‐data Perspective , ed. A. Gelman and X.‐L. Meng . New York: John Wiley.

——  and Silber, J. H.   2001 . Matching and thick description in an observational study of mortality after surgery.   Biostatistics , 2: 217–32. 10.1093/biostatistics/2.2.217

Ross, M.   2001 . Does oil hinder democracy?   World Politics , 53: 325–61. 10.1353/wp.2001.0011

Sagan, S. D.   1995 . Limits of Safety: Organizations, Accidents, and Nuclear Weapons . Princeton, NJ: Princeton University Press.

Sekhon, J. S.   2004 . Quality meets quantity: case studies, conditional probability and counter‐ factuals.   Perspectives in Politics , 2: 281–93.

Shafer, M. D.   1988 . Deadly Paradigms: The Failure of U.S. Counterinsurgency Policy . Princeton, NJ: Princeton University Press.

Skocpol, T.   1979 . States and Social Revolutions: A Comparative Analysis of France, Russia, and China . Cambridge: Cambridge University Press.

——  and Somers, M.   1980 . The uses of comparative history in macrosocial inquiry.   Comparative Studies in Society and History , 22: 147–97.

Stinchcombe, A. L.   1968 . Constructing Social Theories . New York: Harcourt, Brace.

Swank, D. H.   2002 . Global Capital, Political Institutions, and Policy Change in Developed Welfare States . Cambridge: Cambridge University Press.

Tendler, J.   1997 . Good Government in the Tropics . Baltimore: Johns Hopkins University Press.

Truman, D. B.   1951 . The Governmental Process . New York: Alfred A. Knopf.

Tsai, L.   2007 . Accountability without Democracy: How Solidary Groups Provide Public Goods in Rural China . Cambridge: Cambridge University Press.

Van Evera, S.   1997 . Guide to Methods for Students of Political Science . Ithaca, NY: Cornell University Press.

Wahlke, J. C.   1979 . Pre‐behavioralism in political science. American Political Science Review , 73: 9–31. 10.2307/1954728

Yashar, D. J.   2005 . Contesting Citizenship in Latin America: The Rise of Indigenous Movements and the Postliberal Challenge . Cambridge: Cambridge University Press.

Yin, R. K.   2004 . Case Study Anthology . Thousand Oaks, Calif.: Sage.

Gujarati (2003) ; Kennedy (2003) . Interestingly, the potential of cross‐case statistics in helping to choose cases for in‐depth analysis is recognized in some of the earliest discussions of the case‐study method (e.g. Queen 1928 , 226).

This expands on Mill (1843/1872 , 253), who wrote of scientific enquiry as twofold: “either inquiries into the cause of a given effect or into the effects or properties of a given cause.”

This method has not received much attention on the part of qualitative methodologists; hence, the absence of a generally recognized name. It bears some resemblance to J. S. Mill's Joint Method of Agreement and Difference ( Mill 1843/1872 ), which is to say a mixture of most‐similar and most‐different analysis, as discussed below. Patton (2002 , 234) employs the concept of “maximum variation (heterogeneity) sampling.”

More precisely, George and Smoke (1974 , 534, 522–36, ch. 18 ; see also discussion in Collier and Mahoney 1996 , 78) set out to investigate causal pathways and discovered, through the course of their investigation of many cases, these three causal types. Yet, for our purposes what is important is that the final sample includes at least one representative of each “type.”

For further examples see Collier and Mahoney (1996) ; Geddes (1990) ; Tendler (1997) .

Traditionally, methodologists have conceptualized cases as having “positive” or “negative” values (e.g. Emigh 1997 ; Mahoney and Goertz 2004 ; Ragin 2000 , 60; 2004 , 126).

Geddes (1990) ; King, Keohane, and Verba (1994) . See also discussion in Brady and Collier (2004) ; Collier and Mahoney (1996) ; Rogowski (1995) .

The exception would be a circumstance in which the researcher intends to disprove a deterministic argument ( Dion 1998 ).

Geddes (2003 , 131). For other examples of casework from the annals of medicine see “Clinical reports” in the Lancet , “Case studies” in Canadian Medical Association Journal , and various issues of the Journal of Obstetrics and Gynecology , often devoted to clinical cases (discussed in Jenicek 2001 , 7). For examples from the subfield of comparative politics see Kazancigil (1994) .

For a discussion of the important role of anomalies in the development of scientific theorizing see Elman (2003) ; Lakatos (1978) . For examples of deviant‐case research designs in the social sciences see Amenta (1991) ; Coppedge (2004) ; Eckstein (1975) ; Emigh (1997) ; Kendall and Wolf (1949/1955) .

For examples of the crucial‐case method see Bennett, Lepgold, and Unger (1994) ; Desch (2002) ; Goodin and Smitsman (2000) ; Kemp (1986) ; Reilly and Phillpot (2003) . For general discussion see George and Bennett (2005) ; Levy (2002) ; Stinchcombe (1968 , 24–8).

A third position, which purports to be neither Popperian or Bayesian, has been articulated by Mayo (1996 , ch. 6 ). From this perspective, the same idea is articulated as a matter of “severe tests.”

It should be noted that Tsai's conclusions do not rest solely on this crucial case. Indeed, she employs a broad range of methodological tools, encompassing case‐study and cross‐case methods.

See also the discussion in Eckstein (1975) and Lijphart (1969) . For additional examples of case studies disconfirming general propositions of a deterministic nature see Allen (1965); Lipset, Trow, and Coleman (1956) ; Njolstad (1990) ; Reilly (2000–1) ; and discussion in Dion (1998) ; Rogowski (1995) .

Granted, insofar as case‐study analysis provides a window into causal mechanisms, and causal mechanisms are integral to a given theory, a single case may be enlisted to confirm or disconfirm a proposition. However, if the case study upholds a posited pattern of X/Y covariation, and finds fault only with the stipulated causal mechanism, it would be more accurate to say that the study forces the reformulation of a given theory, rather than its confirmation or disconfirmation. See further discussion in the following section.

Sometimes, the most‐similar method is known as the “method of difference,” after its inventor ( Mill 1843/1872 ). For later treatments see Cohen and Nagel (1934) ; Eggan (1954) ; Gerring (2001 , ch. 9 ); Lijphart (1971 ; 1975) ; Meckstroth (1975) ; Przeworski and Teune (1970) ; Skocpol and Somers (1980) .

For good introductions see Ho et al. (2004) ; Morgan and Harding (2005) ; Rosenbaum (2004) ; Rosenbaum and Silber (2001) . For a discussion of matching procedures in Stata see Abadie et al. (2001) .

The most‐different method is also sometimes referred to as the “method of agreement,” following its inventor, J. S. Mill (1843/1872) . See also De Felice (1986) ; Gerring (2001 , 212–14); Lijphart (1971 ; 1975) ; Meckstroth (1975) ; Przeworski and Teune (1970) ; Skocpol and Somers (1980) . For examples of this method see Collier and Collier (1991/2002) ; Converse and Dupeux (1962) ; Karl (1997) ; Moore (1966) ; Skocpol (1979) ; Yashar (2005 , 23). However, most of these studies are described as combining most‐similar and most‐different methods.

In the following discussion I treat the terms social capital, civil society, and civic engagement interchangeably.

E.g. Collier and Collier (1991/2002) ; Karl (1997) ; Moore (1966) ; Skocpol (1979) ; Yashar (2005 , 23). Karl (1997) , which affects to be a most‐different system analysis (20), is a particularly clear example of this. Her study, focused ostensibly on petro‐states (states with large oil reserves), makes two sorts of inferences. The first concerns the (usually) obstructive role of oil in political and economic development. The second sort of inference concerns variation within the population of petro‐states, showing that some countries (e.g. Norway, Indonesia) manage to avoid the pathologies brought on elsewhere by oil resources. When attempting to explain the constraining role of oil on petro‐states, Karl usually relies on contrasts between petro‐states and nonpetro‐states (e.g. ch. 10 ). Only when attempting to explain differences among petro‐states does she restrict her sample to petro‐states. In my opinion, very little use is made of the most‐different research design.

This was recognized, at least implicitly, by Mill (1843/1872 , 258–9). Skepticism has been echoed by methodologists in the intervening years (e.g. Cohen and Nagel 1934 , 251–6; Gerring 2001 ; Skocpol and Somers 1980 ). Indeed, explicit defenses of the most‐different method are rare (but see De Felice 1986 ).

Another way of stating this is to say that X is a “nontrivial necessary condition” of Y .

Wahlke (1979 , 13) writes of the failings of the “behavioralist” mode of political science analysis: “It rarely aims at generalization; research efforts have been confined essentially to case studies of single political systems, most of them dealing …with the American system.”

• About Oxford Academic
• Publish journals with us
• University press partners
• What we publish
• New features  
• Open access
• Institutional account management
• Rights and permissions
• Get help with access
• Accessibility
• Advertising
• Media enquiries
• Oxford University Press
• Oxford Languages
• University of Oxford

Oxford University Press is a department of the University of Oxford. It furthers the University's objective of excellence in research, scholarship, and education by publishing worldwide

• Copyright © 2024 Oxford University Press
• Cookie settings
• Cookie policy
• Privacy policy
• Legal notice

This Feature Is Available To Subscribers Only

Sign In or Create an Account

This PDF is available to Subscribers Only

For full access to this pdf, sign in to an existing account, or purchase an annual subscription.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Int J Qual Stud Health Well-being

Methodology or method? A critical review of qualitative case study reports

Despite on-going debate about credibility, and reported limitations in comparison to other approaches, case study is an increasingly popular approach among qualitative researchers. We critically analysed the methodological descriptions of published case studies. Three high-impact qualitative methods journals were searched to locate case studies published in the past 5 years; 34 were selected for analysis. Articles were categorized as health and health services ( n= 12), social sciences and anthropology ( n= 7), or methods ( n= 15) case studies. The articles were reviewed using an adapted version of established criteria to determine whether adequate methodological justification was present, and if study aims, methods, and reported findings were consistent with a qualitative case study approach. Findings were grouped into five themes outlining key methodological issues: case study methodology or method, case of something particular and case selection, contextually bound case study, researcher and case interactions and triangulation, and study design inconsistent with methodology reported. Improved reporting of case studies by qualitative researchers will advance the methodology for the benefit of researchers and practitioners.

Case study research is an increasingly popular approach among qualitative researchers (Thomas, 2011 ). Several prominent authors have contributed to methodological developments, which has increased the popularity of case study approaches across disciplines (Creswell, 2013b ; Denzin & Lincoln, 2011b ; Merriam, 2009 ; Ragin & Becker, 1992 ; Stake, 1995 ; Yin, 2009 ). Current qualitative case study approaches are shaped by paradigm, study design, and selection of methods, and, as a result, case studies in the published literature vary. Differences between published case studies can make it difficult for researchers to define and understand case study as a methodology.

Experienced qualitative researchers have identified case study research as a stand-alone qualitative approach (Denzin & Lincoln, 2011b ). Case study research has a level of flexibility that is not readily offered by other qualitative approaches such as grounded theory or phenomenology. Case studies are designed to suit the case and research question and published case studies demonstrate wide diversity in study design. There are two popular case study approaches in qualitative research. The first, proposed by Stake ( 1995 ) and Merriam ( 2009 ), is situated in a social constructivist paradigm, whereas the second, by Yin ( 2012 ), Flyvbjerg ( 2011 ), and Eisenhardt ( 1989 ), approaches case study from a post-positivist viewpoint. Scholarship from both schools of inquiry has contributed to the popularity of case study and development of theoretical frameworks and principles that characterize the methodology.

The diversity of case studies reported in the published literature, and on-going debates about credibility and the use of case study in qualitative research practice, suggests that differences in perspectives on case study methodology may prevent researchers from developing a mutual understanding of practice and rigour. In addition, discussion about case study limitations has led some authors to query whether case study is indeed a methodology (Luck, Jackson, & Usher, 2006 ; Meyer, 2001 ; Thomas, 2010 ; Tight, 2010 ). Methodological discussion of qualitative case study research is timely, and a review is required to analyse and understand how this methodology is applied in the qualitative research literature. The aims of this study were to review methodological descriptions of published qualitative case studies, to review how the case study methodological approach was applied, and to identify issues that need to be addressed by researchers, editors, and reviewers. An outline of the current definitions of case study and an overview of the issues proposed in the qualitative methodological literature are provided to set the scene for the review.

Definitions of qualitative case study research

Case study research is an investigation and analysis of a single or collective case, intended to capture the complexity of the object of study (Stake, 1995 ). Qualitative case study research, as described by Stake ( 1995 ), draws together “naturalistic, holistic, ethnographic, phenomenological, and biographic research methods” in a bricoleur design, or in his words, “a palette of methods” (Stake, 1995 , pp. xi–xii). Case study methodology maintains deep connections to core values and intentions and is “particularistic, descriptive and heuristic” (Merriam, 2009 , p. 46).

As a study design, case study is defined by interest in individual cases rather than the methods of inquiry used. The selection of methods is informed by researcher and case intuition and makes use of naturally occurring sources of knowledge, such as people or observations of interactions that occur in the physical space (Stake, 1998 ). Thomas ( 2011 ) suggested that “analytical eclecticism” is a defining factor (p. 512). Multiple data collection and analysis methods are adopted to further develop and understand the case, shaped by context and emergent data (Stake, 1995 ). This qualitative approach “explores a real-life, contemporary bounded system (a case ) or multiple bounded systems (cases) over time, through detailed, in-depth data collection involving multiple sources of information … and reports a case description and case themes ” (Creswell, 2013b , p. 97). Case study research has been defined by the unit of analysis, the process of study, and the outcome or end product, all essentially the case (Merriam, 2009 ).

The case is an object to be studied for an identified reason that is peculiar or particular. Classification of the case and case selection procedures informs development of the study design and clarifies the research question. Stake ( 1995 ) proposed three types of cases and study design frameworks. These include the intrinsic case, the instrumental case, and the collective instrumental case. The intrinsic case is used to understand the particulars of a single case, rather than what it represents. An instrumental case study provides insight on an issue or is used to refine theory. The case is selected to advance understanding of the object of interest. A collective refers to an instrumental case which is studied as multiple, nested cases, observed in unison, parallel, or sequential order. More than one case can be simultaneously studied; however, each case study is a concentrated, single inquiry, studied holistically in its own entirety (Stake, 1995 , 1998 ).

Researchers who use case study are urged to seek out what is common and what is particular about the case. This involves careful and in-depth consideration of the nature of the case, historical background, physical setting, and other institutional and political contextual factors (Stake, 1998 ). An interpretive or social constructivist approach to qualitative case study research supports a transactional method of inquiry, where the researcher has a personal interaction with the case. The case is developed in a relationship between the researcher and informants, and presented to engage the reader, inviting them to join in this interaction and in case discovery (Stake, 1995 ). A postpositivist approach to case study involves developing a clear case study protocol with careful consideration of validity and potential bias, which might involve an exploratory or pilot phase, and ensures that all elements of the case are measured and adequately described (Yin, 2009 , 2012 ).

Current methodological issues in qualitative case study research

The future of qualitative research will be influenced and constructed by the way research is conducted, and by what is reviewed and published in academic journals (Morse, 2011 ). If case study research is to further develop as a principal qualitative methodological approach, and make a valued contribution to the field of qualitative inquiry, issues related to methodological credibility must be considered. Researchers are required to demonstrate rigour through adequate descriptions of methodological foundations. Case studies published without sufficient detail for the reader to understand the study design, and without rationale for key methodological decisions, may lead to research being interpreted as lacking in quality or credibility (Hallberg, 2013 ; Morse, 2011 ).

There is a level of artistic license that is embraced by qualitative researchers and distinguishes practice, which nurtures creativity, innovation, and reflexivity (Denzin & Lincoln, 2011b ; Morse, 2009 ). Qualitative research is “inherently multimethod” (Denzin & Lincoln, 2011a , p. 5); however, with this creative freedom, it is important for researchers to provide adequate description for methodological justification (Meyer, 2001 ). This includes paradigm and theoretical perspectives that have influenced study design. Without adequate description, study design might not be understood by the reader, and can appear to be dishonest or inaccurate. Reviewers and readers might be confused by the inconsistent or inappropriate terms used to describe case study research approach and methods, and be distracted from important study findings (Sandelowski, 2000 ). This issue extends beyond case study research, and others have noted inconsistencies in reporting of methodology and method by qualitative researchers. Sandelowski ( 2000 , 2010 ) argued for accurate identification of qualitative description as a research approach. She recommended that the selected methodology should be harmonious with the study design, and be reflected in methods and analysis techniques. Similarly, Webb and Kevern ( 2000 ) uncovered inconsistencies in qualitative nursing research with focus group methods, recommending that methodological procedures must cite seminal authors and be applied with respect to the selected theoretical framework. Incorrect labelling using case study might stem from the flexibility in case study design and non-directional character relative to other approaches (Rosenberg & Yates, 2007 ). Methodological integrity is required in design of qualitative studies, including case study, to ensure study rigour and to enhance credibility of the field (Morse, 2011 ).

Case study has been unnecessarily devalued by comparisons with statistical methods (Eisenhardt, 1989 ; Flyvbjerg, 2006 , 2011 ; Jensen & Rodgers, 2001 ; Piekkari, Welch, & Paavilainen, 2009 ; Tight, 2010 ; Yin, 1999 ). It is reputed to be the “the weak sibling” in comparison to other, more rigorous, approaches (Yin, 2009 , p. xiii). Case study is not an inherently comparative approach to research. The objective is not statistical research, and the aim is not to produce outcomes that are generalizable to all populations (Thomas, 2011 ). Comparisons between case study and statistical research do little to advance this qualitative approach, and fail to recognize its inherent value, which can be better understood from the interpretive or social constructionist viewpoint of other authors (Merriam, 2009 ; Stake, 1995 ). Building on discussions relating to “fuzzy” (Bassey, 2001 ), or naturalistic generalizations (Stake, 1978 ), or transference of concepts and theories (Ayres, Kavanaugh, & Knafl, 2003 ; Morse et al., 2011 ) would have more relevance.

Case study research has been used as a catch-all design to justify or add weight to fundamental qualitative descriptive studies that do not fit with other traditional frameworks (Merriam, 2009 ). A case study has been a “convenient label for our research—when we ‘can't think of anything ‘better”—in an attempt to give it [qualitative methodology] some added respectability” (Tight, 2010 , p. 337). Qualitative case study research is a pliable approach (Merriam, 2009 ; Meyer, 2001 ; Stake, 1995 ), and has been likened to a “curious methodological limbo” (Gerring, 2004 , p. 341) or “paradigmatic bridge” (Luck et al., 2006 , p. 104), that is on the borderline between postpositivist and constructionist interpretations. This has resulted in inconsistency in application, which indicates that flexibility comes with limitations (Meyer, 2001 ), and the open nature of case study research might be off-putting to novice researchers (Thomas, 2011 ). The development of a well-(in)formed theoretical framework to guide a case study should improve consistency, rigour, and trust in studies published in qualitative research journals (Meyer, 2001 ).

Assessment of rigour

The purpose of this study was to analyse the methodological descriptions of case studies published in qualitative methods journals. To do this we needed to develop a suitable framework, which used existing, established criteria for appraising qualitative case study research rigour (Creswell, 2013b ; Merriam, 2009 ; Stake, 1995 ). A number of qualitative authors have developed concepts and criteria that are used to determine whether a study is rigorous (Denzin & Lincoln, 2011b ; Lincoln, 1995 ; Sandelowski & Barroso, 2002 ). The criteria proposed by Stake ( 1995 ) provide a framework for readers and reviewers to make judgements regarding case study quality, and identify key characteristics essential for good methodological rigour. Although each of the factors listed in Stake's criteria could enhance the quality of a qualitative research report, in Table I we present an adapted criteria used in this study, which integrates more recent work by Merriam ( 2009 ) and Creswell ( 2013b ). Stake's ( 1995 ) original criteria were separated into two categories. The first list of general criteria is “relevant for all qualitative research.” The second list, “high relevance to qualitative case study research,” was the criteria that we decided had higher relevance to case study research. This second list was the main criteria used to assess the methodological descriptions of the case studies reviewed. The complete table has been preserved so that the reader can determine how the original criteria were adapted.

Framework for assessing quality in qualitative case study research.

Adapted from Stake ( 1995 , p. 131).

Study design

The critical review method described by Grant and Booth ( 2009 ) was used, which is appropriate for the assessment of research quality, and is used for literature analysis to inform research and practice. This type of review goes beyond the mapping and description of scoping or rapid reviews, to include “analysis and conceptual innovation” (Grant & Booth, 2009 , p. 93). A critical review is used to develop existing, or produce new, hypotheses or models. This is different to systematic reviews that answer clinical questions. It is used to evaluate existing research and competing ideas, to provide a “launch pad” for conceptual development and “subsequent testing” (Grant & Booth, 2009 , p. 93).

Qualitative methods journals were located by a search of the 2011 ISI Journal Citation Reports in Social Science, via the database Web of Knowledge (see m.webofknowledge.com). No “qualitative research methods” category existed in the citation reports; therefore, a search of all categories was performed using the term “qualitative.” In Table II , we present the qualitative methods journals located, ranked by impact factor. The highest ranked journals were selected for searching. We acknowledge that the impact factor ranking system might not be the best measure of journal quality (Cheek, Garnham, & Quan, 2006 ); however, this was the most appropriate and accessible method available.

International Journal of Qualitative Studies on Health and Well-being.

Search strategy

In March 2013, searches of the journals, Qualitative Health Research , Qualitative Research , and Qualitative Inquiry were completed to retrieve studies with “case study” in the abstract field. The search was limited to the past 5 years (1 January 2008 to 1 March 2013). The objective was to locate published qualitative case studies suitable for assessment using the adapted criterion. Viewpoints, commentaries, and other article types were excluded from review. Title and abstracts of the 45 retrieved articles were read by the first author, who identified 34 empirical case studies for review. All authors reviewed the 34 studies to confirm selection and categorization. In Table III , we present the 34 case studies grouped by journal, and categorized by research topic, including health sciences, social sciences and anthropology, and methods research. There was a discrepancy in categorization of one article on pedagogy and a new teaching method published in Qualitative Inquiry (Jorrín-Abellán, Rubia-Avi, Anguita-Martínez, Gómez-Sánchez, & Martínez-Mones, 2008 ). Consensus was to allocate to the methods category.

Outcomes of search of qualitative methods journals.

In Table III , the number of studies located, and final numbers selected for review have been reported. Qualitative Health Research published the most empirical case studies ( n= 16). In the health category, there were 12 case studies of health conditions, health services, and health policy issues, all published in Qualitative Health Research . Seven case studies were categorized as social sciences and anthropology research, which combined case study with biography and ethnography methodologies. All three journals published case studies on methods research to illustrate a data collection or analysis technique, methodological procedure, or related issue.

The methodological descriptions of 34 case studies were critically reviewed using the adapted criteria. All articles reviewed contained a description of study methods; however, the length, amount of detail, and position of the description in the article varied. Few studies provided an accurate description and rationale for using a qualitative case study approach. In the 34 case studies reviewed, three described a theoretical framework informed by Stake ( 1995 ), two by Yin ( 2009 ), and three provided a mixed framework informed by various authors, which might have included both Yin and Stake. Few studies described their case study design, or included a rationale that explained why they excluded or added further procedures, and whether this was to enhance the study design, or to better suit the research question. In 26 of the studies no reference was provided to principal case study authors. From reviewing the description of methods, few authors provided a description or justification of case study methodology that demonstrated how their study was informed by the methodological literature that exists on this approach.

The methodological descriptions of each study were reviewed using the adapted criteria, and the following issues were identified: case study methodology or method; case of something particular and case selection; contextually bound case study; researcher and case interactions and triangulation; and, study design inconsistent with methodology. An outline of how the issues were developed from the critical review is provided, followed by a discussion of how these relate to the current methodological literature.

Case study methodology or method

A third of the case studies reviewed appeared to use a case report method, not case study methodology as described by principal authors (Creswell, 2013b ; Merriam, 2009 ; Stake, 1995 ; Yin, 2009 ). Case studies were identified as a case report because of missing methodological detail and by review of the study aims and purpose. These reports presented data for small samples of no more than three people, places or phenomenon. Four studies, or “case reports” were single cases selected retrospectively from larger studies (Bronken, Kirkevold, Martinsen, & Kvigne, 2012 ; Coltart & Henwood, 2012 ; Hooghe, Neimeyer, & Rober, 2012 ; Roscigno et al., 2012 ). Case reports were not a case of something, instead were a case demonstration or an example presented in a report. These reports presented outcomes, and reported on how the case could be generalized. Descriptions focussed on the phenomena, rather than the case itself, and did not appear to study the case in its entirety.

Case reports had minimal in-text references to case study methodology, and were informed by other qualitative traditions or secondary sources (Adamson & Holloway, 2012 ; Buzzanell & D'Enbeau, 2009 ; Nagar-Ron & Motzafi-Haller, 2011 ). This does not suggest that case study methodology cannot be multimethod, however, methodology should be consistent in design, be clearly described (Meyer, 2001 ; Stake, 1995 ), and maintain focus on the case (Creswell, 2013b ).

To demonstrate how case reports were identified, three examples are provided. The first, Yeh ( 2013 ) described their study as, “the examination of the emergence of vegetarianism in Victorian England serves as a case study to reveal the relationships between boundaries and entities” (p. 306). The findings were a historical case report, which resulted from an ethnographic study of vegetarianism. Cunsolo Willox, Harper, Edge, ‘My Word’: Storytelling and Digital Media Lab, and Rigolet Inuit Community Government (2013) used “a case study that illustrates the usage of digital storytelling within an Inuit community” (p. 130). This case study reported how digital storytelling can be used with indigenous communities as a participatory method to illuminate the benefits of this method for other studies. This “case study was conducted in the Inuit community” but did not include the Inuit community in case analysis (Cunsolo Willox et al., 2013 , p. 130). Bronken et al. ( 2012 ) provided a single case report to demonstrate issues observed in a larger clinical study of aphasia and stroke, without adequate case description or analysis.

Case study of something particular and case selection

Case selection is a precursor to case analysis, which needs to be presented as a convincing argument (Merriam, 2009 ). Descriptions of the case were often not adequate to ascertain why the case was selected, or whether it was a particular exemplar or outlier (Thomas, 2011 ). In a number of case studies in the health and social science categories, it was not explicit whether the case was of something particular, or peculiar to their discipline or field (Adamson & Holloway, 2012 ; Bronken et al., 2012 ; Colón-Emeric et al., 2010 ; Jackson, Botelho, Welch, Joseph, & Tennstedt, 2012 ; Mawn et al., 2010 ; Snyder-Young, 2011 ). There were exceptions in the methods category ( Table III ), where cases were selected by researchers to report on a new or innovative method. The cases emerged through heuristic study, and were reported to be particular, relative to the existing methods literature (Ajodhia-Andrews & Berman, 2009 ; Buckley & Waring, 2013 ; Cunsolo Willox et al., 2013 ; De Haene, Grietens, & Verschueren, 2010 ; Gratton & O'Donnell, 2011 ; Sumsion, 2013 ; Wimpenny & Savin-Baden, 2012 ).

Case selection processes were sometimes insufficient to understand why the case was selected from the global population of cases, or what study of this case would contribute to knowledge as compared with other possible cases (Adamson & Holloway, 2012 ; Bronken et al., 2012 ; Colón-Emeric et al., 2010 ; Jackson et al., 2012 ; Mawn et al., 2010 ). In two studies, local cases were selected (Barone, 2010 ; Fourie & Theron, 2012 ) because the researcher was familiar with and had access to the case. Possible limitations of a convenience sample were not acknowledged. Purposeful sampling was used to recruit participants within the case of one study, but not of the case itself (Gallagher et al., 2013 ). Random sampling was completed for case selection in two studies (Colón-Emeric et al., 2010 ; Jackson et al., 2012 ), which has limited meaning in interpretive qualitative research.

To demonstrate how researchers provided a good justification for the selection of case study approaches, four examples are provided. The first, cases of residential care homes, were selected because of reported occurrences of mistreatment, which included residents being locked in rooms at night (Rytterström, Unosson, & Arman, 2013 ). Roscigno et al. ( 2012 ) selected cases of parents who were admitted for early hospitalization in neonatal intensive care with a threatened preterm delivery before 26 weeks. Hooghe et al. ( 2012 ) used random sampling to select 20 couples that had experienced the death of a child; however, the case study was of one couple and a particular metaphor described only by them. The final example, Coltart and Henwood ( 2012 ), provided a detailed account of how they selected two cases from a sample of 46 fathers based on personal characteristics and beliefs. They described how the analysis of the two cases would contribute to their larger study on first time fathers and parenting.

Contextually bound case study

The limits or boundaries of the case are a defining factor of case study methodology (Merriam, 2009 ; Ragin & Becker, 1992 ; Stake, 1995 ; Yin, 2009 ). Adequate contextual description is required to understand the setting or context in which the case is revealed. In the health category, case studies were used to illustrate a clinical phenomenon or issue such as compliance and health behaviour (Colón-Emeric et al., 2010 ; D'Enbeau, Buzzanell, & Duckworth, 2010 ; Gallagher et al., 2013 ; Hooghe et al., 2012 ; Jackson et al., 2012 ; Roscigno et al., 2012 ). In these case studies, contextual boundaries, such as physical and institutional descriptions, were not sufficient to understand the case as a holistic system, for example, the general practitioner (GP) clinic in Gallagher et al. ( 2013 ), or the nursing home in Colón-Emeric et al. ( 2010 ). Similarly, in the social science and methods categories, attention was paid to some components of the case context, but not others, missing important information required to understand the case as a holistic system (Alexander, Moreira, & Kumar, 2012 ; Buzzanell & D'Enbeau, 2009 ; Nairn & Panelli, 2009 ; Wimpenny & Savin-Baden, 2012 ).

In two studies, vicarious experience or vignettes (Nairn & Panelli, 2009 ) and images (Jorrín-Abellán et al., 2008 ) were effective to support description of context, and might have been a useful addition for other case studies. Missing contextual boundaries suggests that the case might not be adequately defined. Additional information, such as the physical, institutional, political, and community context, would improve understanding of the case (Stake, 1998 ). In Boxes 1 and 2 , we present brief synopses of two studies that were reviewed, which demonstrated a well bounded case. In Box 1 , Ledderer ( 2011 ) used a qualitative case study design informed by Stake's tradition. In Box 2 , Gillard, Witt, and Watts ( 2011 ) were informed by Yin's tradition. By providing a brief outline of the case studies in Boxes 1 and 2 , we demonstrate how effective case boundaries can be constructed and reported, which may be of particular interest to prospective case study researchers.

Article synopsis of case study research using Stake's tradition

Ledderer ( 2011 ) used a qualitative case study research design, informed by modern ethnography. The study is bounded to 10 general practice clinics in Denmark, who had received federal funding to implement preventative care services based on a Motivational Interviewing intervention. The researcher question focussed on “why is it so difficult to create change in medical practice?” (Ledderer, 2011 , p. 27). The study context was adequately described, providing detail on the general practitioner (GP) clinics and relevant political and economic influences. Methodological decisions are described in first person narrative, providing insight on researcher perspectives and interaction with the case. Forty-four interviews were conducted, which focussed on how GPs conducted consultations, and the form, nature and content, rather than asking their opinion or experience (Ledderer, 2011 , p. 30). The duration and intensity of researcher immersion in the case enhanced depth of description and trustworthiness of study findings. Analysis was consistent with Stake's tradition, and the researcher provided examples of inquiry techniques used to challenge assumptions about emerging themes. Several other seminal qualitative works were cited. The themes and typology constructed are rich in narrative data and storytelling by clinic staff, demonstrating individual clinic experiences as well as shared meanings and understandings about changing from a biomedical to psychological approach to preventative health intervention. Conclusions make note of social and cultural meanings and lessons learned, which might not have been uncovered using a different methodology.

Article synopsis of case study research using Yin's tradition

Gillard et al. ( 2011 ) study of camps for adolescents living with HIV/AIDs provided a good example of Yin's interpretive case study approach. The context of the case is bounded by the three summer camps of which the researchers had prior professional involvement. A case study protocol was developed that used multiple methods to gather information at three data collection points coinciding with three youth camps (Teen Forum, Discover Camp, and Camp Strong). Gillard and colleagues followed Yin's ( 2009 ) principles, using a consistent data protocol that enhanced cross-case analysis. Data described the young people, the camp physical environment, camp schedule, objectives and outcomes, and the staff of three youth camps. The findings provided a detailed description of the context, with less detail of individual participants, including insight into researcher's interpretations and methodological decisions throughout the data collection and analysis process. Findings provided the reader with a sense of “being there,” and are discovered through constant comparison of the case with the research issues; the case is the unit of analysis. There is evidence of researcher immersion in the case, and Gillard reports spending significant time in the field in a naturalistic and integrated youth mentor role.

This case study is not intended to have a significant impact on broader health policy, although does have implications for health professionals working with adolescents. Study conclusions will inform future camps for young people with chronic disease, and practitioners are able to compare similarities between this case and their own practice (for knowledge translation). No limitations of this article were reported. Limitations related to publication of this case study were that it was 20 pages long and used three tables to provide sufficient description of the camp and program components, and relationships with the research issue.

Researcher and case interactions and triangulation

Researcher and case interactions and transactions are a defining feature of case study methodology (Stake, 1995 ). Narrative stories, vignettes, and thick description are used to provoke vicarious experience and a sense of being there with the researcher in their interaction with the case. Few of the case studies reviewed provided details of the researcher's relationship with the case, researcher–case interactions, and how these influenced the development of the case study (Buzzanell & D'Enbeau, 2009 ; D'Enbeau et al., 2010 ; Gallagher et al., 2013 ; Gillard et al., 2011 ; Ledderer, 2011 ; Nagar-Ron & Motzafi-Haller, 2011 ). The role and position of the researcher needed to be self-examined and understood by readers, to understand how this influenced interactions with participants, and to determine what triangulation is needed (Merriam, 2009 ; Stake, 1995 ).

Gillard et al. ( 2011 ) provided a good example of triangulation, comparing data sources in a table (p. 1513). Triangulation of sources was used to reveal as much depth as possible in the study by Nagar-Ron and Motzafi-Haller ( 2011 ), while also enhancing confirmation validity. There were several case studies that would have benefited from improved range and use of data sources, and descriptions of researcher–case interactions (Ajodhia-Andrews & Berman, 2009 ; Bronken et al., 2012 ; Fincham, Scourfield, & Langer, 2008 ; Fourie & Theron, 2012 ; Hooghe et al., 2012 ; Snyder-Young, 2011 ; Yeh, 2013 ).

Study design inconsistent with methodology

Good, rigorous case studies require a strong methodological justification (Meyer, 2001 ) and a logical and coherent argument that defines paradigm, methodological position, and selection of study methods (Denzin & Lincoln, 2011b ). Methodological justification was insufficient in several of the studies reviewed (Barone, 2010 ; Bronken et al., 2012 ; Hooghe et al., 2012 ; Mawn et al., 2010 ; Roscigno et al., 2012 ; Yeh, 2013 ). This was judged by the absence, or inadequate or inconsistent reference to case study methodology in-text.

In six studies, the methodological justification provided did not relate to case study. There were common issues identified. Secondary sources were used as primary methodological references indicating that study design might not have been theoretically sound (Colón-Emeric et al., 2010 ; Coltart & Henwood, 2012 ; Roscigno et al., 2012 ; Snyder-Young, 2011 ). Authors and sources cited in methodological descriptions were inconsistent with the actual study design and practices used (Fourie & Theron, 2012 ; Hooghe et al., 2012 ; Jorrín-Abellán et al., 2008 ; Mawn et al., 2010 ; Rytterström et al., 2013 ; Wimpenny & Savin-Baden, 2012 ). This occurred when researchers cited Stake or Yin, or both (Mawn et al., 2010 ; Rytterström et al., 2013 ), although did not follow their paradigmatic or methodological approach. In 26 studies there were no citations for a case study methodological approach.

The findings of this study have highlighted a number of issues for researchers. A considerable number of case studies reviewed were missing key elements that define qualitative case study methodology and the tradition cited. A significant number of studies did not provide a clear methodological description or justification relevant to case study. Case studies in health and social sciences did not provide sufficient information for the reader to understand case selection, and why this case was chosen above others. The context of the cases were not described in adequate detail to understand all relevant elements of the case context, which indicated that cases may have not been contextually bounded. There were inconsistencies between reported methodology, study design, and paradigmatic approach in case studies reviewed, which made it difficult to understand the study methodology and theoretical foundations. These issues have implications for methodological integrity and honesty when reporting study design, which are values of the qualitative research tradition and are ethical requirements (Wager & Kleinert, 2010a ). Poorly described methodological descriptions may lead the reader to misinterpret or discredit study findings, which limits the impact of the study, and, as a collective, hinders advancements in the broader qualitative research field.

The issues highlighted in our review build on current debates in the case study literature, and queries about the value of this methodology. Case study research can be situated within different paradigms or designed with an array of methods. In order to maintain the creativity and flexibility that is valued in this methodology, clearer descriptions of paradigm and theoretical position and methods should be provided so that study findings are not undervalued or discredited. Case study research is an interdisciplinary practice, which means that clear methodological descriptions might be more important for this approach than other methodologies that are predominantly driven by fewer disciplines (Creswell, 2013b ).

Authors frequently omit elements of methodologies and include others to strengthen study design, and we do not propose a rigid or purist ideology in this paper. On the contrary, we encourage new ideas about using case study, together with adequate reporting, which will advance the value and practice of case study. The implications of unclear methodological descriptions in the studies reviewed were that study design appeared to be inconsistent with reported methodology, and key elements required for making judgements of rigour were missing. It was not clear whether the deviations from methodological tradition were made by researchers to strengthen the study design, or because of misinterpretations. Morse ( 2011 ) recommended that innovations and deviations from practice are best made by experienced researchers, and that a novice might be unaware of the issues involved with making these changes. To perpetuate the tradition of case study research, applications in the published literature should have consistencies with traditional methodological constructions, and deviations should be described with a rationale that is inherent in study conduct and findings. Providing methodological descriptions that demonstrate a strong theoretical foundation and coherent study design will add credibility to the study, while ensuring the intrinsic meaning of case study is maintained.

The value of this review is that it contributes to discussion of whether case study is a methodology or method. We propose possible reasons why researchers might make this misinterpretation. Researchers may interchange the terms methods and methodology, and conduct research without adequate attention to epistemology and historical tradition (Carter & Little, 2007 ; Sandelowski, 2010 ). If the rich meaning that naming a qualitative methodology brings to the study is not recognized, a case study might appear to be inconsistent with the traditional approaches described by principal authors (Creswell, 2013a ; Merriam, 2009 ; Stake, 1995 ; Yin, 2009 ). If case studies are not methodologically and theoretically situated, then they might appear to be a case report.

Case reports are promoted by university and medical journals as a method of reporting on medical or scientific cases; guidelines for case reports are publicly available on websites ( http://www.hopkinsmedicine.org/institutional_review_board/guidelines_policies/guidelines/case_report.html ). The various case report guidelines provide a general criteria for case reports, which describes that this form of report does not meet the criteria of research, is used for retrospective analysis of up to three clinical cases, and is primarily illustrative and for educational purposes. Case reports can be published in academic journals, but do not require approval from a human research ethics committee. Traditionally, case reports describe a single case, to explain how and what occurred in a selected setting, for example, to illustrate a new phenomenon that has emerged from a larger study. A case report is not necessarily particular or the study of a case in its entirety, and the larger study would usually be guided by a different research methodology.

This description of a case report is similar to what was provided in some studies reviewed. This form of report lacks methodological grounding and qualities of research rigour. The case report has publication value in demonstrating an example and for dissemination of knowledge (Flanagan, 1999 ). However, case reports have different meaning and purpose to case study, which needs to be distinguished. Findings of our review suggest that the medical understanding of a case report has been confused with qualitative case study approaches.

In this review, a number of case studies did not have methodological descriptions that included key characteristics of case study listed in the adapted criteria, and several issues have been discussed. There have been calls for improvements in publication quality of qualitative research (Morse, 2011 ), and for improvements in peer review of submitted manuscripts (Carter & Little, 2007 ; Jasper, Vaismoradi, Bondas, & Turunen, 2013 ). The challenging nature of editor and reviewers responsibilities are acknowledged in the literature (Hames, 2013 ; Wager & Kleinert, 2010b ); however, review of case study methodology should be prioritized because of disputes on methodological value.

Authors using case study approaches are recommended to describe their theoretical framework and methods clearly, and to seek and follow specialist methodological advice when needed (Wager & Kleinert, 2010a ). Adequate page space for case study description would contribute to better publications (Gillard et al., 2011 ). Capitalizing on the ability to publish complementary resources should be considered.

Limitations of the review

There is a level of subjectivity involved in this type of review and this should be considered when interpreting study findings. Qualitative methods journals were selected because the aims and scope of these journals are to publish studies that contribute to methodological discussion and development of qualitative research. Generalist health and social science journals were excluded that might have contained good quality case studies. Journals in business or education were also excluded, although a review of case studies in international business journals has been published elsewhere (Piekkari et al., 2009 ).

The criteria used to assess the quality of the case studies were a set of qualitative indicators. A numerical or ranking system might have resulted in different results. Stake's ( 1995 ) criteria have been referenced elsewhere, and was deemed the best available (Creswell, 2013b ; Crowe et al., 2011 ). Not all qualitative studies are reported in a consistent way and some authors choose to report findings in a narrative form in comparison to a typical biomedical report style (Sandelowski & Barroso, 2002 ), if misinterpretations were made this may have affected the review.

Case study research is an increasingly popular approach among qualitative researchers, which provides methodological flexibility through the incorporation of different paradigmatic positions, study designs, and methods. However, whereas flexibility can be an advantage, a myriad of different interpretations has resulted in critics questioning the use of case study as a methodology. Using an adaptation of established criteria, we aimed to identify and assess the methodological descriptions of case studies in high impact, qualitative methods journals. Few articles were identified that applied qualitative case study approaches as described by experts in case study design. There were inconsistencies in methodology and study design, which indicated that researchers were confused whether case study was a methodology or a method. Commonly, there appeared to be confusion between case studies and case reports. Without clear understanding and application of the principles and key elements of case study methodology, there is a risk that the flexibility of the approach will result in haphazard reporting, and will limit its global application as a valuable, theoretically supported methodology that can be rigorously applied across disciplines and fields.

Conflict of interest and funding

The authors have not received any funding or benefits from industry or elsewhere to conduct this study.

• Adamson S, Holloway M. Negotiating sensitivities and grappling with intangibles: Experiences from a study of spirituality and funerals. Qualitative Research. 2012; 12 (6):735–752. doi: 10.1177/1468794112439008. [ CrossRef ] [ Google Scholar ]
• Ajodhia-Andrews A, Berman R. Exploring school life from the lens of a child who does not use speech to communicate. Qualitative Inquiry. 2009; 15 (5):931–951. doi: 10.1177/1077800408322789. [ CrossRef ] [ Google Scholar ]
• Alexander B. K, Moreira C, Kumar H. S. Resisting (resistance) stories: A tri-autoethnographic exploration of father narratives across shades of difference. Qualitative Inquiry. 2012; 18 (2):121–133. doi: 10.1177/1077800411429087. [ CrossRef ] [ Google Scholar ]
• Austin W, Park C, Goble E. From interdisciplinary to transdisciplinary research: A case study. Qualitative Health Research. 2008; 18 (4):557–564. doi: 10.1177/1049732307308514. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Ayres L, Kavanaugh K, Knafl K. A. Within-case and across-case approaches to qualitative data analysis. Qualitative Health Research. 2003; 13 (6):871–883. doi: 10.1177/1049732303013006008. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Barone T. L. Culturally sensitive care 1969–2000: The Indian Chicano Health Center. Qualitative Health Research. 2010; 20 (4):453–464. doi: 10.1177/1049732310361893. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Bassey M. A solution to the problem of generalisation in educational research: Fuzzy prediction. Oxford Review of Education. 2001; 27 (1):5–22. doi: 10.1080/03054980123773. [ CrossRef ] [ Google Scholar ]
• Bronken B. A, Kirkevold M, Martinsen R, Kvigne K. The aphasic storyteller: Coconstructing stories to promote psychosocial well-being after stroke. Qualitative Health Research. 2012; 22 (10):1303–1316. doi: 10.1177/1049732312450366. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Broyles L. M, Rodriguez K. L, Price P. A, Bayliss N. K, Sevick M. A. Overcoming barriers to the recruitment of nurses as participants in health care research. Qualitative Health Research. 2011; 21 (12):1705–1718. doi: 10.1177/1049732311417727. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Buckley C. A, Waring M. J. Using diagrams to support the research process: Examples from grounded theory. Qualitative Research. 2013; 13 (2):148–172. doi: 10.1177/1468794112472280. [ CrossRef ] [ Google Scholar ]
• Buzzanell P. M, D'Enbeau S. Stories of caregiving: Intersections of academic research and women's everyday experiences. Qualitative Inquiry. 2009; 15 (7):1199–1224. doi: 10.1177/1077800409338025. [ CrossRef ] [ Google Scholar ]
• Carter S. M, Little M. Justifying knowledge, justifying method, taking action: Epistemologies, methodologies, and methods in qualitative research. Qualitative Health Research. 2007; 17 (10):1316–1328. doi: 10.1177/1049732307306927. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Cheek J, Garnham B, Quan J. What's in a number? Issues in providing evidence of impact and quality of research(ers) Qualitative Health Research. 2006; 16 (3):423–435. doi: 10.1177/1049732305285701. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Colón-Emeric C. S, Plowman D, Bailey D, Corazzini K, Utley-Smith Q, Ammarell N, et al. Regulation and mindful resident care in nursing homes. Qualitative Health Research. 2010; 20 (9):1283–1294. doi: 10.1177/1049732310369337. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Coltart C, Henwood K. On paternal subjectivity: A qualitative longitudinal and psychosocial case analysis of men's classed positions and transitions to first-time fatherhood. Qualitative Research. 2012; 12 (1):35–52. doi: 10.1177/1468794111426224. [ CrossRef ] [ Google Scholar ]
• Creswell J. W. Five qualitative approaches to inquiry. In: Creswell J. W, editor. Qualitative inquiry and research design: Choosing among five approaches. 3rd ed. Thousand Oaks, CA: Sage; 2013a. pp. 53–84. [ Google Scholar ]
• Creswell J. W. Qualitative inquiry and research design: Choosing among five approaches. 3rd ed. Thousand Oaks, CA: Sage; 2013b. [ Google Scholar ]
• Crowe S, Cresswell K, Robertson A, Huby G, Avery A, Sheikh A. The case study approach. BMC Medical Research Methodology. 2011; 11 (1):1–9. doi: 10.1186/1471-2288-11-100. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Cunsolo Willox A, Harper S. L, Edge V. L, ‘My Word’: Storytelling and Digital Media Lab, & Rigolet Inuit Community Government Storytelling in a digital age: Digital storytelling as an emerging narrative method for preserving and promoting indigenous oral wisdom. Qualitative Research. 2013; 13 (2):127–147. doi: 10.1177/1468794112446105. [ CrossRef ] [ Google Scholar ]
• De Haene L, Grietens H, Verschueren K. Holding harm: Narrative methods in mental health research on refugee trauma. Qualitative Health Research. 2010; 20 (12):1664–1676. doi: 10.1177/1049732310376521. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• D'Enbeau S, Buzzanell P. M, Duckworth J. Problematizing classed identities in fatherhood: Development of integrative case studies for analysis and praxis. Qualitative Inquiry. 2010; 16 (9):709–720. doi: 10.1177/1077800410374183. [ CrossRef ] [ Google Scholar ]
• Denzin N. K, Lincoln Y. S. Introduction: Disciplining the practice of qualitative research. In: Denzin N. K, Lincoln Y. S, editors. The SAGE handbook of qualitative research. 4th ed. Thousand Oaks, CA: Sage; 2011a. pp. 1–6. [ Google Scholar ]
• Denzin N. K, Lincoln Y. S, editors. The SAGE handbook of qualitative research. 4th ed. Thousand Oaks, CA: Sage; 2011b. [ Google Scholar ]
• Edwards R, Weller S. Shifting analytic ontology: Using I-poems in qualitative longitudinal research. Qualitative Research. 2012; 12 (2):202–217. doi: 10.1177/1468794111422040. [ CrossRef ] [ Google Scholar ]
• Eisenhardt K. M. Building theories from case study research. The Academy of Management Review. 1989; 14 (4):532–550. doi: 10.2307/258557. [ CrossRef ] [ Google Scholar ]
• Fincham B, Scourfield J, Langer S. The impact of working with disturbing secondary data: Reading suicide files in a coroner's office. Qualitative Health Research. 2008; 18 (6):853–862. doi: 10.1177/1049732307308945. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Flanagan J. Public participation in the design of educational programmes for cancer nurses: A case report. European Journal of Cancer Care. 1999; 8 (2):107–112. doi: 10.1046/j.1365-2354.1999.00141.x. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Flyvbjerg B. Five misunderstandings about case-study research. Qualitative Inquiry. 2006; 12 (2):219–245. doi: 10.1177/1077800405284.363. [ CrossRef ] [ Google Scholar ]
• Flyvbjerg B. Case study. In: Denzin N. K, Lincoln Y. S, editors. The SAGE handbook of qualitative research. 4th ed. Thousand Oaks, CA: Sage; 2011. pp. 301–316. [ Google Scholar ]
• Fourie C. L, Theron L. C. Resilience in the face of fragile X syndrome. Qualitative Health Research. 2012; 22 (10):1355–1368. doi: 10.1177/1049732312451871. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Gallagher N, MacFarlane A, Murphy A. W, Freeman G. K, Glynn L. G, Bradley C. P. Service users’ and caregivers’ perspectives on continuity of care in out-of-hours primary care. Qualitative Health Research. 2013; 23 (3):407–421. doi: 10.1177/1049732312470521. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Gerring J. What is a case study and what is it good for? American Political Science Review. 2004; 98 (2):341–354. doi: 10.1017/S0003055404001182. [ CrossRef ] [ Google Scholar ]
• Gillard A, Witt P. A, Watts C. E. Outcomes and processes at a camp for youth with HIV/AIDS. Qualitative Health Research. 2011; 21 (11):1508–1526. doi: 10.1177/1049732311413907. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Grant M, Booth A. A typology of reviews: An analysis of 14 review types and associated methodologies. Health Information and Libraries Journal. 2009; 26 :91–108. doi: 10.1111/j.1471-1842.2009.00848.x. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Gratton M.-F, O'Donnell S. Communication technologies for focus groups with remote communities: A case study of research with First Nations in Canada. Qualitative Research. 2011; 11 (2):159–175. doi: 10.1177/1468794110394068. [ CrossRef ] [ Google Scholar ]
• Hallberg L. Quality criteria and generalization of results from qualitative studies. International Journal of Qualitative Studies on Health and Wellbeing. 2013; 8 :1. doi: 10.3402/qhw.v8i0.20647. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Hames I. Committee on Publication Ethics, 1. 2013, March. COPE Ethical guidelines for peer reviewers. Retrieved April 7, 2013, from http://publicationethics.org/resources/guidelines . [ Google Scholar ]
• Hooghe A, Neimeyer R. A, Rober P. “Cycling around an emotional core of sadness”: Emotion regulation in a couple after the loss of a child. Qualitative Health Research. 2012; 22 (9):1220–1231. doi: 10.1177/1049732312449209. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Jackson C. B, Botelho E. M, Welch L. C, Joseph J, Tennstedt S. L. Talking with others about stigmatized health conditions: Implications for managing symptoms. Qualitative Health Research. 2012; 22 (11):1468–1475. doi: 10.1177/1049732312450323. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Jasper M, Vaismoradi M, Bondas T, Turunen H. Validity and reliability of the scientific review process in nursing journals—time for a rethink? Nursing Inquiry. 2013 doi: 10.1111/nin.12030. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Jensen J. L, Rodgers R. Cumulating the intellectual gold of case study research. Public Administration Review. 2001; 61 (2):235–246. doi: 10.1111/0033-3352.00025. [ CrossRef ] [ Google Scholar ]
• Jorrín-Abellán I. M, Rubia-Avi B, Anguita-Martínez R, Gómez-Sánchez E, Martínez-Mones A. Bouncing between the dark and bright sides: Can technology help qualitative research? Qualitative Inquiry. 2008; 14 (7):1187–1204. doi: 10.1177/1077800408318435. [ CrossRef ] [ Google Scholar ]
• Ledderer L. Understanding change in medical practice: The role of shared meaning in preventive treatment. Qualitative Health Research. 2011; 21 (1):27–40. doi: 10.1177/1049732310377451. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Lincoln Y. S. Emerging criteria for quality in qualitative and interpretive research. Qualitative Inquiry. 1995; 1 (3):275–289. doi: 10.1177/107780049500100301. [ CrossRef ] [ Google Scholar ]
• Luck L, Jackson D, Usher K. Case study: A bridge across the paradigms. Nursing Inquiry. 2006; 13 (2):103–109. doi: 10.1111/j.1440-1800.2006.00309.x. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Mawn B, Siqueira E, Koren A, Slatin C, Devereaux Melillo K, Pearce C, et al. Health disparities among health care workers. Qualitative Health Research. 2010; 20 (1):68–80. doi: 10.1177/1049732309355590. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Merriam S. B. Qualitative research: A guide to design and implementation. 3rd ed. San Francisco, CA: Jossey-Bass; 2009. [ Google Scholar ]
• Meyer C. B. A case in case study methodology. Field Methods. 2001; 13 (4):329–352. doi: 10.1177/1525822x0101300402. [ CrossRef ] [ Google Scholar ]
• Morse J. M. Mixing qualitative methods. Qualitative Health Research. 2009; 19 (11):1523–1524. doi: 10.1177/1049732309349360. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Morse J. M. Molding qualitative health research. Qualitative Health Research. 2011; 21 (8):1019–1021. doi: 10.1177/1049732311404706. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Morse J. M, Dimitroff L. J, Harper R, Koontz A, Kumra S, Matthew-Maich N, et al. Considering the qualitative–quantitative language divide. Qualitative Health Research. 2011; 21 (9):1302–1303. doi: 10.1177/1049732310392386. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Nagar-Ron S, Motzafi-Haller P. “My life? There is not much to tell”: On voice, silence and agency in interviews with first-generation Mizrahi Jewish women immigrants to Israel. Qualitative Inquiry. 2011; 17 (7):653–663. doi: 10.1177/1077800411414007. [ CrossRef ] [ Google Scholar ]
• Nairn K, Panelli R. Using fiction to make meaning in research with young people in rural New Zealand. Qualitative Inquiry. 2009; 15 (1):96–112. doi: 10.1177/1077800408318314. [ CrossRef ] [ Google Scholar ]
• Nespor J. The afterlife of “teachers’ beliefs”: Qualitative methodology and the textline. Qualitative Inquiry. 2012; 18 (5):449–460. doi: 10.1177/1077800412439530. [ CrossRef ] [ Google Scholar ]
• Piekkari R, Welch C, Paavilainen E. The case study as disciplinary convention: Evidence from international business journals. Organizational Research Methods. 2009; 12 (3):567–589. doi: 10.1177/1094428108319905. [ CrossRef ] [ Google Scholar ]
• Ragin C. C, Becker H. S. What is a case?: Exploring the foundations of social inquiry. Cambridge: Cambridge University Press; 1992. [ Google Scholar ]
• Roscigno C. I, Savage T. A, Kavanaugh K, Moro T. T, Kilpatrick S. J, Strassner H. T, et al. Divergent views of hope influencing communications between parents and hospital providers. Qualitative Health Research. 2012; 22 (9):1232–1246. doi: 10.1177/1049732312449210. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Rosenberg J. P, Yates P. M. Schematic representation of case study research designs. Journal of Advanced Nursing. 2007; 60 (4):447–452. doi: 10.1111/j.1365-2648.2007.04385.x. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Rytterström P, Unosson M, Arman M. Care culture as a meaning- making process: A study of a mistreatment investigation. Qualitative Health Research. 2013; 23 :1179–1187. doi: 10.1177/1049732312470760. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Sandelowski M. Whatever happened to qualitative description? Research in Nursing & Health. 2000; 23 (4):334–340. doi: 10.1002/1098-240X. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Sandelowski M. What's in a name? Qualitative description revisited. Research in Nursing & Health. 2010; 33 (1):77–84. doi: 10.1002/nur.20362. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Sandelowski M, Barroso J. Reading qualitative studies. International Journal of Qualitative Methods. 2002; 1 (1):74–108. [ Google Scholar ]
• Snyder-Young D. “Here to tell her story”: Analyzing the autoethnographic performances of others. Qualitative Inquiry. 2011; 17 (10):943–951. doi: 10.1177/1077800411425149. [ CrossRef ] [ Google Scholar ]
• Stake R. E. The case study method in social inquiry. Educational Researcher. 1978; 7 (2):5–8. [ Google Scholar ]
• Stake R. E. The art of case study research. Thousand Oaks, CA: Sage; 1995. [ Google Scholar ]
• Stake R. E. Case studies. In: Denzin N. K, Lincoln Y. S, editors. Strategies of qualitative inquiry. Thousand Oaks, CA: Sage; 1998. pp. 86–109. [ Google Scholar ]
• Sumsion J. Opening up possibilities through team research: Investigating infants’ experiences of early childhood education and care. Qualitative Research. 2013; 14 (2):149–165. doi: 10.1177/1468794112468471.. [ CrossRef ] [ Google Scholar ]
• Thomas G. Doing case study: Abduction not induction, phronesis not theory. Qualitative Inquiry. 2010; 16 (7):575–582. doi: 10.1177/1077800410372601. [ CrossRef ] [ Google Scholar ]
• Thomas G. A typology for the case study in social science following a review of definition, discourse, and structure. Qualitative Inquiry. 2011; 17 (6):511–521. doi: 10.1177/1077800411409884. [ CrossRef ] [ Google Scholar ]
• Tight M. The curious case of case study: A viewpoint. International Journal of Social Research Methodology. 2010; 13 (4):329–339. doi: 10.1080/13645570903187181. [ CrossRef ] [ Google Scholar ]
• Wager E, Kleinert S. Responsible research publication: International standards for authors. A position statement developed at the 2nd World Conference on Research Integrity, Singapore, July 22–24, 2010. In: Mayer T, Steneck N, editors. Promoting research integrity in a global environment. Singapore: Imperial College Press/World Scientific; 2010a. pp. 309–316. [ Google Scholar ]
• Wager E, Kleinert S. Responsible research publication: International standards for editors. A position statement developed at the 2nd World Conference on Research Integrity, Singapore, July 22–24, 2010. In: Mayer T, Steneck N, editors. Promoting research integrity in a global environment. Singapore: Imperial College Press/World Scientific; 2010b. pp. 317–328. [ Google Scholar ]
• Webb C, Kevern J. Focus groups as a research method: A critique of some aspects of their use in nursing research. Journal of Advanced Nursing. 2000; 33 (6):798–805. doi: 10.1046/j.1365-2648.2001.01720.x. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Wimpenny K, Savin-Baden M. Exploring and implementing participatory action synthesis. Qualitative Inquiry. 2012; 18 (8):689–698. doi: 10.1177/1077800412452854. [ CrossRef ] [ Google Scholar ]
• Yeh H.-Y. Boundaries, entities, and modern vegetarianism: Examining the emergence of the first vegetarian organization. Qualitative Inquiry. 2013; 19 (4):298–309. doi: 10.1177/1077800412471516. [ CrossRef ] [ Google Scholar ]
• Yin R. K. Enhancing the quality of case studies in health services research. Health Services Research. 1999; 34 (5 Pt 2):1209–1224. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Yin R. K. Case study research: Design and methods. 4th ed. Thousand Oaks, CA: Sage; 2009. [ Google Scholar ]
• Yin R. K. Applications of case study research. 3rd ed. Thousand Oaks, CA: Sage; 2012. [ Google Scholar ]

Understanding Research Questions: Quantitative vs Qualitative

By Divya Bhansali

Columbia University; Biomedical Engineering PhD candidate

3 minute read

Research is like being a detective, trying to uncover the mysteries of the world. In the world of research, one of the first and most crucial decisions you'll make is whether to ask quantitative or qualitative method questions. But what's the difference between quantitative and qualitative research, and why does it matter? Let's dive in and find out!

Quantitative Research Questions

Quantitative research involves numbers, statistics, and hard data. It's like counting beans in a jar. Quantitative research questions aim to answer "how much," "how many," or "to what extent" questions. When understanding how to write research paper , quantitative research questions can provide clear, measurable data to support your findings.

A proven college admissions edge

Polygence alumni had a 92% admissions rate to R1 universities in 2023. Polygence provides high schoolers a personalized, flexible research experience proven to boost your admission odds. Get matched to a mentor now!"

Examples of Quantitative Research Questions

1. How many high school students use smartphones for over four hours a day?

This research question can be answered with precise numbers - a certain percentage of students may fall into this category.

2. What is the average GPA of students in our school?

You'll get a specific number, like 3.5, as an answer to this question.

3. How much has the average temperature increased over the last decade?

In this case, you're looking for a specific temperature change in degrees Celsius or Fahrenheit.

Considerations for Quantitative Research

Data Collection Methods : To answer quantitative research questions, you'll often use structured surveys, experiments, or observations with predefined variables. These methods help you collect precise, quantifiable data.

Data Analysis : Quantitative research involves statistical analysis, where you'll use mathematical tools to identify patterns and relationships in the data. Understanding how to write a research paper outline can help you organize these methods effectively.

Generalizability : Quantitative research often aims for generalizability, meaning you can draw conclusions that apply to a larger population.

Qualitative Research Questions

On the other hand, the qualitative research method is more about words, descriptions, and understanding the "whys" and "hows" of a phenomenon. It's like exploring the stories behind the beans in the jar. Qualitative analysis questions aim to answer questions about experiences, feelings, and behaviors.

Examples of Qualitative Research Questions

How do high school students feel about using smartphones for extended periods of time?

This question invites students to share their thoughts, emotions, and personal experiences.

2. What are the main challenges that students face when it comes to maintaining a high GPA?

This question prompts students to talk about their struggles, motivations, and strategies.

3. In what ways has climate change affected the daily lives of people in our community?

This question encourages people to share their stories about how they've been impacted.

Considerations for Qualitative Research

Data Collection Methods : Qualitative research methods often involve open-ended interviews, observations, or content analysis. These methods allow you to collect rich, descriptive data. 

Data Analysis : Qualitative research method requires a more interpretive approach. You'll analyze text or visual data to identify themes, patterns, and any unique insight.

In-Depth Understanding : Qualitative research delves deep into the experiences and perceptions of individuals, providing a nuanced understanding of a specific topic.

Knowing how to write an introduction for a research paper can be particularly important when presenting qualitative research. A compelling introduction sets the stage for the rich, descriptive data that follows.

If your study involves STEM subjects, having a solid stem research paper outline will be beneficial. Additionally, knowing how to write a thesis statement for a research paper is crucial for establishing a clear argument or hypothesis.

Do your own research through Polygence!

Polygence pairs you with an expert mentor in your area of passion. Together, you work to create a high quality research project that is uniquely your own.

Which One to Choose?

The choice between qualitative and quantitative research questions depends on what you want to discover and the nature of your study. Here are some key factors to consider:

Nature of the Research : Is your research more about numbers and statistical analysis, or is it about having a deeper understanding the human experience? Choose the approach that aligns with your research goals.

Data Collection : Think about how you'll gather information. Surveys and experiments often lead to quantitative data, while interviews and observations typically provide qualitative data.

Time and Resources : Consider the time and resources you have. Quantitative research can often be quicker and require fewer resources than in-depth qualitative studies.

Research Participants : The preferences and characteristics of your research participants matter. Some may prefer answering surveys with numeric options, while others may enjoy sharing their stories.

When you are ready to start your study, make sure to also understand how to write a research paper abstract for summarizing your work effectively.

Whether you choose to ask quantitative or qualitative survey questions, remember that both approaches are valuable and have their unique strengths. The key is to match your research goals with the right approach, ensuring that you gather the most relevant and meaningful data.

So, high school detectives, the choice is yours: will you count the beans or explore the stories behind them? Happy researching!

6 Qualitative data examples for thorough market researchers

Types of qualitative data in market research, 6 qualitative data examples, get nuanced insights from qualitative market research.

There are plenty of ways to gather consumer insights for fresh campaigns and better products, but qualitative research is up there with the best sources of insight.

This guide is packed with examples of how to turn qualitative data into actionable insights, to spark your creativity and sharpen your research strategy. You’ll see how qualitative data, especially through surveys, opens doors to deeper understanding by inviting consumers to share their experiences and thoughts freely, in their own words — and how qualitative data can transform your brand.

Before we dig into some examples of how qualitative data can empower your teams to make focused, confident and quick decisions on anything from product to marketing, let’s go back to basics. We can categorize qualitative data into roughly three categories: binary, nominal and ordinal data. Here’s how each of them is used in qualitative data analysis.

Binary data

Binary data represents a choice between two distinct options, like ‘yes’ or ‘no’. In market research, this type of qualitative data is useful for filtering responses or making clear distinctions in consumer preferences.

Binary data in qualitative research is great for straightforward insights, but has its limits. Here’s a quick guide on when to use it and when to opt for qualitative data that is more detailed:

Binary data is great for:

• Quick Yes/No questions : like “Have you used our app? Yes or No.”
• Initial screening : to quickly sort participants for further studies.
• Clear-cut answers : absolute factors, such as ownership or usage.

Avoid binary data for:

• Understanding motivations : it lacks the depth to explore why behind actions.
• Measuring intensity : can’t show how much someone likes or uses something.
• Detail needed for product development : misses the nuanced feedback necessary for innovations.

Nominal data

Nominal data categorizes responses without implying any order. For example, when survey respondents choose their favorite brand from a list, the data collected is nominal, offering insights into brand preferences among different demographics.

Some other examples of qualitative data that can be qualified as nominal are asking participants to name their primary information source about products in categories like social media, friends, or online reviews. Or in focus groups, discussing brand perceptions could classify brands into categories such as luxury, budget-friendly, or eco-conscious, based on participant descriptions.

Nominal data is great for:

• Categorizing responses : such as types of consumer complaints (product quality, customer service, delivery issues).
• Identifying preferences : like favorite product categories (beverages, electronics, apparel).
• Segmentation : grouping participants based on attributes (first-time buyers, loyal customers).

Nominal data is not for:

• Measuring quantities : it can’t quantify how much more one category is preferred over another.
• Ordering or ranking responses : it doesn’t indicate which category is higher or lower in any hierarchy.
• Detailed behavioral analysis : While it can group behaviors, it doesn’t delve into the frequency or intensity of those behaviors.

Ordinal data

Ordinal data introduces a sense of order, ranking preferences or satisfaction levels. In qualitative analysis, it’s particularly useful for understanding how consumers prioritize features or products, giving researchers a clearer picture of market trends.

Other examples of qualitative data analyses that use ordinal data, are for instance a study on consumer preferences for coffee flavors, participants might rank flavors in order of preference, providing insights into flavor trends. You can also get ordinal data from focus groups on things like customer satisfaction surveys or app usability, by asking users to rate their ease of use or happiness on an ordinal scale.

Ordinal data is great for:

• Ranking preferences : asking participants to rank product features from most to least important.
• Measuring satisfaction levels : using scales like “very satisfied,” “satisfied,” “neutral,” “dissatisfied,” “very dissatisfied.”
• Assessing Agreement : with statements on a scale from “strongly agree” to “strongly disagree.”

Ordinal data is not for:

• Quantifying differences : it doesn’t show how much more one rank is preferred over another, just the order.
• Precise measurements : can’t specify the exact degree of satisfaction or agreement, only relative positions.

This mix of qualitative and quantitative data will give you a well-rounded view of participant attitudes and preferences.

The things you can do with qualitative data are endless. But this article shouldn’t turn into a work of literature, so we’ll highlight six ways to collect qualitative data and give you examples of how to use these qualitative research methods to get actionable results.

How to get qual insights with Attest

You can get to the heart of what your target customers think, with reliable qualitative insights from Attest Video Responses

1. Highlighting brand loyalty drivers with open-ended surveys and questionnaires

Open-ended surveys and questionnaires are great at finding out what makes customers choose and stick with a brand. Here’s why this qualitative data analysis tool is so good for gathering qualitative data on things like brand loyalty and customer experience:

Straight from the source

Open-ended survey responses show the actual thoughts and feelings of your target audience in their own words, while still giving you structure in your data analysis.

Understanding ‘why’

Numbers can show us how many customers are loyal; open-ended survey responses explain why they are. You can also easily add thematic analysis to the mix by counting certain keywords or phrases.

Guiding decisions

The insights from these surveys can help a brand decide where to focus its efforts, from making sure their marketing highlights what customers love most to improving parts of their product.

Surveys are one of the most versatile and efficient qualitative data collection methods out there. We want to bring the power of qualitative data analysis to every business and make it easy to gather qualitative data from the people who matter most to your brand. Check out our survey templates to hit the ground running. And you’re not limited to textual data as your only data source — we also enable you to gather video responses to get additional context from non verbal cues and more.

2. Trend identification with observation notes

Observation notes are a powerful qualitative data analysis tool for spotting trends as they naturally unfold in real-world settings. Here’s why they’re particularly valuable insights and effective for identifying new trends:

Real behavior

Observing people directly shows us how they actually interact with products or services, not just how they say they do. This can highlight emerging trends in consumer behavior or preferences before people can even put into words what they are doing and why.

Immediate insights

By watching how people engage with different products, we can quickly spot patterns or changes in behavior. This immediate feedback is invaluable for catching trends as they start.

Context matters

Observations give you context. You can see not just what people do, but where and how they do it. This context can be key to understanding why a trend is taking off.

Unprompted reactions

Since people don’t know they’re being observed for these purposes, their actions are genuine. This leads to authentic insights about what’s really catching on.

3. Understanding consumer sentiments through semi-structured interviews

Semi-structured interviews for qualitative data analysis are an effective method for data analysts to get a deep understanding of consumer sentiments. It provides a structured yet flexible approach to gather in-depth insights. Here’s why they’re particularly useful for this type of research question:

Personal connection

These interviews create a space for a real conversation, allowing consumers to share their feelings, experiences, and opinions about a brand or product in a more personal setting.

Flexibility

The format lets the interviewer explore interesting points that come up during the conversation, diving deeper into unexpected areas of discussion. This flexibility uncovers richer insights than strictly structured interviews.

Depth of understanding

By engaging in detailed discussions, brands can understand not just what consumers think but why they think that way and what stations their train of thought passes by.

Structure and surprise

Semi-structured interviews can be tailored to explore specific areas of interest while still allowing for new insights to emerge.

4. Using focus groups for informing market entry strategies

Using a focus group to inform market entry strategies provides a dynamic way to discover your potential customers’ needs, preferences, and perceptions before launching a product or entering a new market. Here’s how focus groups can be particularly effective for this kind of research goal:

Real conversations

Focus groups allow for real-time, interactive discussions, giving you a front-row seat to hear what your potential customers think and feel about your product or service idea.

Diverse Perspectives

By bringing together people from various backgrounds, a focus group can offer a wide range of views and insights, highlighting different consumer needs and contextual information that you might miss out on in a survey.

Spotting opportunities and challenges

The dynamic nature of focus groups can help uncover unique market opportunities or potential challenges that might not be evident through other research methods, like cultural nuances.

Testing ideas

A focus group is a great way to test and compare reactions to different market entry strategies, from pricing models to distribution channels, providing clear direction on what approach might work best.

5. Case studies to gain a nuanced understanding of consumers on a broad level

Case studies in qualitative research zoom in on specific stories from customers or groups using a product or service, great for gaining a nuanced understanding of consumers at a broad level. Here’s why case studies are a particularly effective qualitative data analysis tool for this type of research goal:

In-depth analysis

Case studies can provide a 360-degree look at the consumer experience, from initial awareness to post-purchase feelings.

This depth of insight reveals not just what consumers do, but why they do it, uncovering motivations, influences, and decision-making processes.

Longitudinal insight

Case studies can track changes in consumer behavior or satisfaction over time, offering a dynamic view of how perceptions evolve.

This longitudinal perspective is crucial for giving context to the lifecycle of consumer engagement with a brand.

Storytelling power

The narrative nature of case studies — when done right — makes them powerful tools for communicating complex consumer insights in an accessible and engaging way, which can be especially useful for internal strategy discussions or external marketing communications.

6. Driving product development with diary studies

Diary studies are a unique qualitative research method that involves participants recording their thoughts, experiences, or behaviors over a period of time, related to using a product or service. This qualitative data analysis method is especially valuable for driving product development for several reasons:

Real-time insights

Diary studies capture real-time user experiences and feedback as they interact with a product in their daily lives.

This ongoing documentation provides a raw, unfiltered view of how a product fits into the user’s routine, highlighting usability issues or unmet needs that might not be captured in a one-time survey or interview.

Realistic user journey mapping

By analyzing diary entries, you can map out the entire user journey, identifying critical touch points where users feel delighted, frustrated, or indifferent.

This then enables you to implement targeted improvements and innovations at the moments that matter most.

Identifying patterns

Over the course of a diary study, patterns in behavior, preferences, and challenges can emerge, which is great for thematic analysis.

It can guide product developers to prioritize features or fixes that will have the most significant impact on user satisfaction, which is especially great if they don’t know what areas to focus on first.

Qualitative research brings your consumers’ voices directly to your strategy table. The examples we’ve explored show how qualitative data analysis methods like surveys, interviews, and case studies illuminate the ‘why’ behind consumer choices, guiding more informed decisions. Using these insights means crafting products and messages that resonate deeply, ensuring your brand not only meets but exceeds consumer expectations.

Customer Research Manager 

Related articles

Making it personal – using tech to build connections with consumers | diageo, panel discussion – future-proofing your brand with consumer insights, attest product release: multi-market | attest’s alyssa stringer, subscribe to our newsletter.

Fill in your email and we’ll drop fresh insights and events info into your inbox each week.

* I agree to receive communications from Attest. Privacy Policy .

You're now subscribed to our mailing list to receive exciting news, reports, and other updates!

You are using an outdated browser. Please upgrade your browser to improve your experience.

Qualitative and Quantitative Case Study Research Method on Social Science: Accounting Perspective

• Bubaker F. Shareia

Description

The main aim of this paper is to set the parameters within which the study is to be conducted, specifically justifying the use of qualitative research, informed by theory. This paper argues that the social world is subjective in nature and may be accessed through the interpretive approach provided by the people involved in the context of the study. The paper defines and distinguishes between qualitative and quantitative research methodologies, explores Burrell and Morgan's framework for social research, and presents the study's adopted methodology and methods, with the rationale for these choices.

10005863.pdf

Files (182.4 kb), additional details.

• Bakar, M. and A. Russell (2003). The development of Accounting Education and Practice in Libya. Research in Accounting in Emerging Economies. R. S. O. Wallace, J. M. Samuels, R. J. Briston and S. M. Saudagaran. Oxford, Elsevier Science 5: 197-236.
• Bryman, A. (1995). Quantity and quality in social research. London Routledge.
• Burrell, G. and G. Morgan (1979). Sociological Paradigms and Organisational Analysis. London, Heinemann Education Books
• Cavana, R. Y., U. Sekaran, et al. (2001). Applied business research: qualitative and quantitative methods. Milton, Qld., John Wiley & Sons Australia.
• Creswell, J. W. (1994). Research design: qualitative & quantitative approaches. Thousand Oaks, Calif., Sage Publications.
• Creswell, J. W. (2003). Research design: qualitative, quantitative, and mixed methods approaches. Thousand Oaks, Calif., Sage Publications.
• Denzin, N. K. and Y. S. Lincoln, Eds. (2003). Collecting and interpreting qualitative materials. Thousand Oaks, Calif.; London, Sage Pub.
• Eisenhardt, K. M. (1989). "Building Theories from Case Study Research." The Academy of Management Review 14(4): 532-550.
• Ezzamel, M., H. Willmott, et al. (2004). "Accounting and management-labour relations: the politics of production in the 'factory with a problem'." Accounting, Organizations and Society 29(3-4): 269-302. [10] Flick, U. (2002). An Introduction to Qualitative Research London, Sage [11] Glesne, C. and A. Peshkin (1992). Becoming Qualitative Researchers an Introduction London Longman. [12] Hartley, J. (1994). Case studies in organisational research. Qualitative Methods in Organisational Research. C. Cassell and G. Symon. London, Sage: 208-229. [13] Hartley, J. (2004). Case Study Research Essential Guide to Qualitative Methods in Organizational Research C. Cassell and G. Symon. London Sage [14] Hussey, J. and R. Hussey (1997). Business Research: A practical guide for undergraduate and postgraduate students. London, Antony Rowe. [15] Irvine, H. and H. Deo (2006). "The power of the lens, A comparative analysis of two views of the Fiji development bank " Accounting, Auditing & Accountability Journal 19(2): 205-227. [16] Irvine, H. and M. Gaffikin (2006). "Methodological Insights Getting in, Getting on and Getting out: Reflections on Qualitative Research Project " Accounting, Auditing & Accountability Journal 19(1): 115-145. [17] Jones, I. and C. Gratton (2003). Research methods for sport studies. London; New York, Routledge. [18] Kilani, A. K. (1988). The Evolution and Status of Accounting in Libya. Department of Accounting. Hull, University of Hull. Unpublished PhD thesis. [19] King, N. (1994). The qualitative research interview. Qualitative Method in Organisational Research: A Practical Guide. C. Cassell and G. Symon. London, Sage: 14-36. [20] King, N. (2004). Using Interviews in Qualitative Research Essential Guide to Qualitative Methods in Organizational Research C. Cassell and G. Symon. London Sage 388. [21] Llewellyn (2003). "Methodological Issues What counts as "theory" in qualitative management and accounting research? Introducing five levels of theorizing." Accounting, Auditing & Accountability Journal 16(4): 662-708. [22] Miles, M. B. and A. M. Huberman (1994). Qualitative data analysis: an expanded sourcebook. Thousand Oaks, Sage Publications. [23] Neuman, W. L. (2000). Social research methods: qualitative and quantitative approaches. Boston, Allyn and Bacon. [24] Saunders, M., P. Lewis, et al. (2003). Research methods for business students Upper Saddle River, Prentice Hall. [25] Scapens, R. W. (1990). "Researching management accounting practice: The role of case study method." British Accounting Review 22: 259-281. [26] Scapens, R. W. (1991). Management accounting: a review of contemporary developments. Basingstoke, Hampshire, Macmillan Education. [27] Sekaran, U. (1992). Research Methods for Business: Askill-Building Approach. New York, John Wiley & Sons, INC. [28] Shareia, B (2015), The Qualitative Case Study Research in Accounting: Challenges and Prospects- The Libyan case study., International Journal of Social, Management, Economics, and Business Engineering, 9(12): 3873- 3877. [29] Shareia, B., B. Parasuraman, et al. (2005). The Qualitative Case Study Research in Africa and Asia: Challenges and Prospects. Third International Qualitative Research Convention, Senai, Johor; Malaysia. [30] Silverman, D. (1985). Qualitative methodology and sociology: describing the social world Aldershot, Gower. [31] Silverman, D. (2006). Interpreting qualitative data: methods for analyzing talk, text and interaction London, Sage [32] Stake, R. E. (1995). The art of case study research. Thousand Oaks, Sage Publications. [33] Strauss, A. and J. Corbin (1998). Basics of Qualitative Research, Techniques and Procedures for developing Grounded Theory London Sage [34] Tashakkori, A. and C. Teddlie (1998). Mixed methodology: combining qualitative and quantitative approaches. Thousand Oaks, Calif., Sage. [35] Van der Velde, M., N. Anderson, et al. (2004). Guide to management research methods. Oxford, Blackwell. [36] Witte, R. S. and J. S. Witte (1997). Statistics. London, Harcourt Barace College. [37] Yin, R. K. (2003). Case study research: design and methods. Thousand Oaks, Calif., Sage Publications.

This site uses cookies. Find out more on how we use cookies

A framework and process for community-engaged, mixed-methods cancer needs assessments

• Original Paper
• Open access
• Published: 29 May 2024

Cite this article

You have full access to this open access article

• Todd Burus 1 , 12 ,
• Jessica R. Thompson 1 ,
• Caree R. McAfee 1 ,
• Lovoria B. Williams 1 , 2 ,
• Jennifer Redmond Knight 3 , 4 ,
• Bin Huang 5 , 6 ,
• Sarojini Kanotra 7 ,
• Natalie P. Wilhite 1 ,
• Elaine Russell 1 , 3 ,
• Melinda Rogers 8 ,
• Connie L. Sorrell 9 ,
• Christine Stroebel 1 ,
• Rachael King 10 &
• Pamela C. Hull 1 , 11  

204 Accesses

1 Altmetric

Explore all metrics

Community health needs assessments are required for most state and local public health agencies and non-profit hospitals. Typically based on community health improvement planning models, these assessments encompass overall community health and multiple diseases to inform program planning. National Cancer Institute (NCI)-designated Cancer Centers and community-based cancer-focused programs share the goal of reducing cancer burden in the catchment areas they serve. However, to date, no published models exist to guide cancer-specific needs assessments for a determined geographic area that can inform both public health and research initiatives. The purpose of this article is to outline a cancer needs assessment (CNA) framework and community-engaged, mixed-methods process, along with a case study of how we applied it in Kentucky.

We convened a steering committee of key organizational partners to provide input throughout the process. We developed a conceptual framework of multi-level determinants affecting cancer-related outcomes. We incorporated both quantitative and qualitative data gathered through a variety of means, including a novel application of group concept mapping to guide definition of priorities.

The resulting CNA has helped guide strategic planning and priorities for Kentucky’s Cancer Action Plan, Markey Cancer Center, state agencies, and community-based organizations.

This framework and process can be used collaboratively by cancer center Community Outreach and Engagement offices, public health agencies, oncology programs, and community partners to plan impactful cancer control programs and research in their catchment areas. Universities can also use them to inform the planning of community engagement and health equity research efforts.

Similar content being viewed by others

A practical method for integrating community priorities in planning and implementing cancer control programs

Twenty years of capacity building across the cancer prevention and control research network

Engaging community members in cancer research: an assessment of an NCI-designated cancer center

Avoid common mistakes on your manuscript.

Introduction

Needs assessments arose in the mid-twentieth century as a useful tool for performing program planning and evaluation. Their purpose is to, first, identify needs (the gap between the current and future desired conditions) and then prioritize how to address them [ 1 ]. State and local public health agencies were early adopters of community health needs assessments (CHNA), and since 2011, the national Public Health Accreditation Board has required CHNAs [ 2 ]. In recent years, needs assessments have taken a more prominent role in healthcare with the requirement that nonprofit hospitals perform a CHNA every 3 years in accordance with the Patient Protection and Affordable Care Act (ACA) of 2010 [ 3 ]. Hospitals are also required to develop a corresponding implementation plan to act on priorities identified through their CHNAs. Combined, this process is meant as an accountability measure for facilities receiving federal funding, pushing them to focus on health outcome improvements for individuals living within their service areas. In 2012 the Commission on Cancer (CoC) under the American College of Surgeons initiated new requirements for CoC-accredited oncology treatment facilities to perform a CHNA once every 3 years [ 4 ]. While the 2020 CoC standards no longer require CHNAs, their implementation is encouraged to guide outreach and psychosocial programs that address barriers to cancer care [ 5 ].

To promote local accountability and community benefit, in 2013, the National Cancer Institute (NCI) introduced the concept of catchment areas in the Cancer Center Support Grant (CCSG) funding requirements for NCI-Designated Cancer Centers. A cancer center catchment area is a population- or geographically based area in which the cancer center does or desires to serve patients and perform research to reduce the cancer burden [ 6 ]. The subsequent 2016 CCSG guidelines expanded on prior requirements for cancer prevention and control by establishing a new Community Outreach and Engagement (COE) component [ 7 ]. The COE component was charged with continuously generating a comprehensive profile of the catchment area’s unique cancer needs (i.e., factors influencing poor cancer-related health outcomes and disparities) and opportunities for improving them. This profile should catalyze both cancer control activities and basic, clinical, translational, and population research priorities to address identified needs in collaboration with community partners [ 8 ]. Conducting a needs assessment can help cancer center COE offices accomplish these objectives [ 9 , 10 ].

Public health agencies typically follow CHNA processes outlined by community health improvement planning (CHIP) models, which emphasize involving local residents and partner organizations to assess community health status; identify health priorities; and plan, implement, and evaluate city- or statewide health improvement initiatives [ 11 , 12 , 13 ]. In contrast, the non-profit hospital CHNA requirement did not come with much guidance and, as such, has produced varied results during its first decade of existence [ 14 ]. Furthermore, CHIP-based CHNA processes are designed to encompass the entire spectrum of community health and select priorities among multiple diseases, with a narrow focus on planning public health agency programs. Additionally, the National Comprehensive Cancer Control Program requires funded states, tribes and territories and their cancer coalitions to review cancer-related data and develop a jurisdiction-wide cancer plan [ 15 ]. No formal process is in place to fulfill this requirement, and it varies considerably among states and jurisdictions [ 16 ]. To date, no published models exist to guide cancer-specific needs assessments that can inform public health initiatives, cancer plans, and research.

To fill this gap, the objectives of this article are to: (1) present a cancer needs assessment (CNA) framework and community-engaged, mixed-methods process to guide the identification of priorities for both community-based cancer control activities and research agendas aimed at reducing cancer burden and disparities in a specific catchment area, and (2) illustrate a case study of how we applied the CNA framework and process in Kentucky through a partnership among the University of Kentucky Markey Cancer Center (UKMCC) COE team and a steering committee comprised of academic, public health, and community partners. While separate articles will detail the methods and results of specific components of the Kentucky CNA in greater depth, this article provides a high-level overview of how to conduct a community-engaged, multi-method CNA that integrates various sources and types of data.

Conceptual framework

We drew on several existing models to develop a conceptual framework of social and individual factors that influence cancer outcomes to guide the collection, analysis, interpretation, and presentation of data in Cancer Needs Assessments (CNA). For example, Rodriguez et al. adapted McLeroy’s Social-Ecological Model and illustrated how nested interactions at the patient, community, and policy levels impact patient health outcomes and health disparities [ 17 ]. Hiatt and Breen went a step further by considering the direct influence different levels of analysis, including social determinants of health (SDOH), have on the cancer care continuum, and where opportunities for intervention exist [ 18 , 19 ]. Additionally, the American Association for Cancer Research put forward a model which recognized the roles both personal and population evidence-based actions can play in effective cancer prevention [ 20 ]. Finally, Alcaraz et al. created a framework for advancing cancer health equity through understanding and addressing SDOH [ 21 , 22 ]. The Alcaraz et al. model introduced the idea of an upstream/downstream intervention orientation in which they argue that “[to] achieve cancer health equity, more focused efforts are needed upstream to address social factors for population-level impact.”

The UKMCC COE synthesized these concepts to develop a draft CNA conceptual framework, the Multilevel Determinants of Cancer-related Outcomes Framework, and the steering committee gave input to refine it (Fig.  1 ). This framework organizes several intertwined levels of consideration for assessing cancer needs and identifying potential interventions to reduce cancer burden and disparities. On the left, the SDOH reflect the overall context in which an individual lives. We classified SDOH as falling into three levels—society, environment, and community (see Fig.  1 ), which influence one another in a complex, circular manner over time [ 23 ]. In the middle, under individual-level factors, one’s personal characteristics and behaviors interact with their body’s internal biological mechanisms, which could potentially create somatic mutations and result in cancer. The complex interplay of contextual SDOH factors and individual factors interact with cancer-related outcomes along the cancer care continuum, shown on the right. On the bottom left, the green to red continuum indicates varying degrees of health equity in contextual SDOH factors for a given geographic area, which influence the risk of exposures and behaviors for individuals in different social groups (e.g., gender, race/ethnicity, rural/urban). The degree of health equity in SDOH impacts the degree of health disparities in population-level cancer-related outcomes across social groups, as indicated on the bottom right. Along the top of the figure, we indicate the continuum of upstream actions directed toward SDOH to downstream actions targeting the individual level.

Multilevel determinants of cancer-related outcomes framework. Society includes social context (e.g., culture; social norms; the meaning of socially constructed concepts like race, ethnicity, and gender), economic conditions (e.g., median income, unemployment), institutions (e.g., education system, criminal justice system), and policies (e.g., Affordable Care Act, Medicaid, institutional policies). Environment includes the “natural environment” (e.g., water, air, soil) and substances (natural or man-made) that individuals are exposed to through these means, and the “built” environment constructed by humans (e.g., roads, sidewalks, parks, buildings). Community includes the local health care system (e.g., primary care, hospitals, cancer clinics, and other facilities) and interpersonal relationships (e.g., family, neighborhoods)

Cancer needs assessment mixed method process

Mixed methods research brings together the strengths of quantitative breadth and qualitative depth of data within the same study to gain greater insights [ 24 ]. In needs assessments, mixed methods provide a comprehensive picture of community health needs and issues, adding lived experience to identified quantitative patterns [ 25 ]. Considering the availability of existing cancer-related quantitative data and important gaps that need to be filled with qualitative data, we recommend the process illustrated in Fig.  2 , which represents a community-engaged, explanatory sequential mixed methods approach. The first phase of the CNA process includes assessment of patterns in existing quantitative data plus simultaneous gathering of qualitative community insights through focus groups and review of hospital CHNAs. While extensive secondary quantitative data exists in publicly available sources, groups may also choose to perform additional primary quantitative data collection (e.g., population health surveys), if they have sufficient financial and personnel resources [ 8 , 26 ]. The second phase is a group concept mapping process with quantitative and qualitative steps to generate consensus on priorities. The final phase involves summarizing and disseminating CNA findings. A CNA steering committee of key stakeholders should ideally provide input throughout all phases. Below we describe the methodology of each component and illustrate the case study of how we applied this process for the Kentucky CNA (KY CNA).

Process workflow for performing a community-engaged, mixed-methods cancer needs assessment

Community partner input

An essential preliminary step for a community-engaged CNA process is identifying a collaborative steering committee, which could be a new or existing committee, task force, or community advisory board. This group is typically comprised of key stakeholders within the catchment area representing a variety of points of view, such as local non-profit organizations, government agencies, universities, healthcare providers, policy makers, and other community leaders or members [ 25 ]. The committee’s responsibility is to provide input throughout the CNA process on design, recruitment, data collection, interpretation of findings, and dissemination. This collaborative approach grounds the CNA process in existing community insights and assets, ensures representation, and builds credibility and investment in the results among stakeholder organizations.

Given that the catchment area for the 2021 KY CNA encompassed the entire state, the UKMCC COE formed a KY CNA steering committee consisting of organizations and programs with a statewide reach, a focus that includes cancer, access to data sources or networks of organizations and community members, and a common interest in a statewide CNA. The 27-member steering committee included representatives of the UKMCC, Kentucky Cancer Consortium (KCC, the state cancer coalition), Kentucky Cancer Program (KCP, a statewide outreach program), Kentucky Cancer Registry (KCR), Kentucky Department for Public Health, American Cancer Society, Foundation for a Healthy Kentucky, and University of Louisville. From March 2021 to March 2022, through all steps of the process, UKMCC COE convened monthly steering committee meetings and also sought their input through email. In addition, UKMCC COE regularly updated and gathered feedback during the process from the UKMCC Community Advisory Board, comprised of 12 lay community members and community organizations.

Quantitative methods

In performing a CNA, the quantitative methods provide a numerical view of the target catchment area. This assists in, among other things, measuring the observed degree of cancer burden, inequities in SDOH, and disparities in cancer-related outcomes that exists among certain population subgroups. For the 2021 KY CNA we chose to only gather secondary quantitative data due to practical considerations surrounding the large size of our catchment area, the ongoing COVID-19 pandemic at the time, and to focus available resources on gathering needed qualitative data.

Gathering secondary data

While developing the conceptual framework for the KY CNA, we began to brainstorm available quantitative data from secondary sources that correspond to components of the framework. We collected these ideas in an indicator list with notes about available sources and the additional demographic information each contained. Subsequently, we organized indicators based on the conceptual framework. Throughout the needs assessment process, we regularly revisited this list. By the end, this list contained over 100 quantitative variables to consider beyond just cancer incidence and mortality (Suppl. Table S1 ). We compiled this data, developed summaries, and analyzed it for the presence of significant disparities between various subgroups of the population. In a parallel project, the UKMCC COE streamlined the process of gathering of multiple publicly available data sources through the creation of a catchment area data collection software called Cancer InFocus [ 26 ].

Geographic levels

Given that the scope of our needs assessment was the entire state, we processed all of the data collected at the state, county, and/or census tract level (as available) to aid in creating geographic visualizations. Kentucky has a unique level of administrative boundaries utilized in this process known as Area Development Districts (ADDs). ADDs combine several counties into larger sub-state regions, making it possible to report on certain variables that would otherwise need to be suppressed at the smaller county level. Depending on the makeup of the area being assessed, it may be useful to collect available data for other geographic levels (such as public use microdata areas) or non-standard administrative regions.

Visualizations

We constructed numerous tables, charts, and maps with the quantitative data we collected. We shared these visualizations with our steering committee during monthly meetings to get feedback on ease of comprehension and how to best capture the story being told by the data. Consistent colors were used across visualizations to assist in developing a visual narrative that was easy to follow with minimal need for written text in the final document.

Dissemination follow-up survey

To evaluate the reach and impact of our CNA, a follow-up survey was sent to individuals who volunteered to be recontacted when downloading a digital copy of resources from the 2021 KY CNA website ( www.kycancerneeds.org ). The survey was constructed in REDCap and consisted of multiple questions intended to assess the usage of needs assessment findings throughout the state [ 27 ].

Data processing

Data processing was performed in the R statistical programming language, and visualizations were created in R (version 4.2.1, R Core Team, Vienna, Austria), Tableau (version 2021.3, Tableau, Seattle, WA), and ArcGIS Pro (version 2.9, Environmental Systems Research Institute, Redlands, California). When formal statistical tests were performed to compare values, a p -value of 0.05 was used to assess statistical significance. Otherwise, statistically significant differences were noted using non-overlapping 95% confidence intervals.

Qualitative methods

The qualitative aspect of a mixed methods CNA allows researchers to examine the perceived needs in the community and add additional context to the inequities and disparities observed using quantitative methods. Importantly, qualitative methods allow for the inclusion of voices in needs assessments from underrepresented portions of the population whose experiences may not be captured by solely relying on quantitative population-level data [ 25 ]. Additionally, through open-ended questions, qualitative data can provide details on the ‘how’ and ‘why’ certain quantitative trends exist in order to develop the community-specific strategies necessary to address identified needs [ 24 ].

Hospital community health needs assessments scoping review

We conducted a scoping review of hospital CHNAs from all non-profit hospitals across the state of Kentucky to improve our understanding of statewide priority health needs and implementation strategies and to identify where priority needs aligned with the cancer care continuum. The CHNA review process included: (1) Conducting a literature review of the CHNA process; (2) Collecting the most recent CHNA and implementation strategy reports for Kentucky hospitals; (3) Documenting information about hospital catchment areas from CHNAs and building hospital profiles; (4) Building a glossary of terms and definitions, and grouping them into categories under priority health needs and implementation strategies; (5) Training staff on how to review and code priority health needs and implementation; (6) Performing two reviews per CHNA; and (7) Reconciling discrepancies in categorization between initial reviews with a third reviewer.

Focus groups

To incorporate more community participation beyond our steering committee, we conducted online focus groups with adult (age 18+) residents of Kentucky who did not work in a healthcare profession ( n  = 51). From July to September 2021, we recruited participants utilizing our existing KCC and KCP partnership networks and ResearchMatch, an online volunteer research registry [ 28 ]. Our study team contacted eligible participants, gathered study consent, and assigned participants to a specific scheduled focus group. As we sought to identify needs by race and ethnicity, geographic area, and sexual orientation and gender identity, we intentionally stratified groups by these characteristics based on a brief demographic questionnaire, resulting in 11 focus group sessions.

The focus group discussions lasted approximately two hours and took place via Zoom. Discussions were led by a trained facilitator and supported by a research assistant. Our semi-structured questions focused on participant awareness of existing resources and needs across the cancer care continuum. We supported this discussion using a visual graphic displaying the various areas of the cancer care continuum (risk reduction, screening, treatment, and follow-up/survivorship). Participants received a $50 e-gift card for their participation. Each focus group discussion was recorded and transcribed for qualitative coding analysis. The facilitator and research assistant conducted a line-by-line review of the transcripts and double-coded 20% of the transcripts, resulting in over 90% agreement. Any discrepancies were resolved through consensus.

Concept mapping

After focus group data collection, we designed a concept mapping project to prioritize the wide variety of identified cancer needs and strategies. A participatory mixed method, concept mapping utilizes a series of survey-based activities and group discussions to generate consensus on a particular topic of interest [ 29 , 30 , 31 ]. From September to December 2021, we recruited community members (adults, non-health professionals, and Kentucky residents) and community organization employees who work in cancer services in Kentucky to participate in concept mapping remotely ( n  = 162). The community member participants were individuals who previously participated in the KY CNA focus groups. Community organizations were identified by KCC and KCP and included representatives from health departments, nonprofit organizations, advocacy groups, insurance companies, health systems, and educational organizations.

We recruited participants by email and provided a link to the online concept mapping activities utilizing the Groupwisdom concept mapping platform [ 32 ]. Prior to beginning the first set of activities, a working group of COE, KCP, and KCC members compiled a list of 80 items from the previous quantitative findings, qualitative focus group discussion themes, and concerns raised at partner meetings. The survey-based activities asked participants to sort these 80 items into thematic groups and to rate each item on two five-point Likert-type scales: (1) importance for Kentucky communities and (2) how easy it would be to address in Kentucky communities (i.e., feasibility). With these data, the study team generated concept maps (e.g., point and cluster maps) using cluster analysis and rating assessments (e.g., correlational comparisons), which were shown to participants in group discussions. We conducted six discussion groups via Zoom, including three with community members and three with organizational partners. The community member participants received up to $60 in e-gift cards for their participation ($30 for the online survey-based activities and $30 for the group discussion).

Dissemination

We used a multi-pronged strategy for disseminating the KY CNA report and district profiles across Kentucky. This approach included distributing physical copies of the summary report and district profiles and creating a website with the digital summary report, district profiles, and interactive data dashboard. In addition, we hosted meetings and webinars on the findings of the report and how they could be used to inform research, guide strategic planning and outreach, and impact the community.

Institutional review board statement

UK Institutional Review Board (IRB) approved the research procedures under three protocols: Kentucky Behavioral Risk Factor Survey analysis (#69894), focus groups (#65451), and concept mapping (#73420). The IRB issued a non-human research determination for using the publicly available, aggregated secondary data and the CNA dissemination follow-up survey.

Quantitative results

We gathered secondary data from over 18 different sources including KCR, a population-based central cancer registry for the Commonwealth of Kentucky and member of the NCI Surveillance, Epidemiology, and End Results program since 2000 (Table  1 ). KCR provided cancer incidence and mortality data for 2014–2018 for all primary cancer sites stratified by combinations of sex, race, rurality, Appalachian residence, and county or ADD. They also calculated new combined incidence and mortality rates for three groups of cancer sites associated with the major cancer risk factors of tobacco, obesity, and human papillomavirus (HPV) [ 33 , 34 , 35 ]. Twelve of the top 20 incidence rate cancers, and 12 of the top 20 mortality rate cancers, were observed to have significantly higher rates in Kentucky than in the rest of the U.S. (Suppl. Table S2; Fig. S1 ). Of particular interest, we found that Kentucky lung cancer incidence rates were 78.7% higher than U.S. rates, and lung cancer mortality rates were 81.9% higher. Kentucky also experienced significantly higher rates in incidence and mortality for the groupings of tobacco-, obesity-, and HPV-related cancers.

Likewise, we noted significant disparities among several behavioral risk factors and socioeconomic factors, though Kentucky’s rates for cancer screening were comparable to U.S. rates (Suppl. Table S3; Fig. S2). KCR provided relative survival rates for cancer patients for the years of 2012–2018, defined as the percentage of patients with a particular cancer diagnosis surviving 5-years after being diagnosed compared to a similar population of individuals without a cancer diagnosis. This revealed that Black Kentucky females with a breast cancer diagnosis had a significant 10% lower 5-year survival rate than White Kentucky females.

Qualitative results

Community health needs assessments scoping review.

One-hundred and ten non-profit hospitals in the state of Kentucky met the criteria for needing to have performed a CHNA at the time of our review. Research staff successfully located CHNAs for 73 of these hospitals from websites and requesting copies. We analyzed these CHNAs for content on their strategic priorities and implementation strategies (Fig.  3 ). While 60.3% of them noted tobacco/smoking cessation as a strategic priority, only about a quarter (27.4%) specifically highlighted cancer. Lung cancer screening was a priority for 24.7% of hospitals reviewed, breast cancer screening for 13.7%, and colorectal cancer screening for 12.3% (no hospital highlighted a priority for cervical cancer screening despite Kentucky ranking first among U.S. states in cervical cancer incidence). Mention of focusing on SDOHs appeared on 28.8% of CHNAs reviewed, while health equity was only found in 1 of the 73.

Bar chart showing the results of the Kentucky non-profit hospitals Community Health Needs Assessments (CHNA) review, indicating the percentage of CHNAs that include these items as a priority and/or strategy

Using an online format, we uncovered a wide variety of factors affecting Kentuckians across the cancer care continuum. For example, continued novel approaches are needed to make information accessible and to utilize messaging that will not be interpreted as blaming or shaming. Likewise, screening efforts need to continue reaching individuals where they are and include messaging from individuals who engender trust. Continued efforts are also needed to address practical concerns for both screening and treatment, such as cost and transportation as well as lack of knowledge of which screening tools and treatments are covered by insurance. Participants expressed unique concerns based on their race and ethnicity, rurality, sexual orientation and gender identity, and age. These differences suggest uncovering ways to promote positive, understanding communication between patients and providers and to create safe care spaces that consider ways cultural norms affect cancer care to fight stigma and to improve health equity [ 36 ].

Concept mapping results

The concept mapping process successfully grouped the 80 items, which span factors across the cancer care continuum, into six thematic clusters. We also identified potential community-driven strategies to improve cancer risk reduction, treatment, and follow-up in Kentucky (Fig.  4 ). Specifically, participants commonly identified key areas for continued efforts, such as lung cancer screening, smoking cessation, HPV vaccination, and disparities driven by social determinants among rural, Appalachian, Black, and Hispanic Kentuckians. Community member and partner-driven strategies to affect these areas include a continued focus on health communication strategies, supports for treatment navigation, ways to overcome barriers to accessing care, and methods for increasing trust in patient–provider relationships. Moving forward, healthcare professionals dedicated to improving cancer in Kentucky can consider ways to build upon these strategies.

Community prioritization of needs and strategies from the 2021 Kentucky Cancer Needs Assessment

Final synthesis

Results from the quantitative and qualitative data collection were synthesized into a final KY CNA report entitled, “2021 Kentucky Cancer Needs Assessment: The Story of Cancer in Kentucky.” This 59-page report was broken down into five sections, including an executive summary of findings. Emphasis was placed on presenting information in a visual format with minimal prose where possible. Additional one-page district profiles were created using data from the report for each of Kentucky’s 15 ADDs.

Dissemination results

The KY CNA website ( www.kycancerneeds.org ) launched in April 2022 with a downloadable version of the summary report, along with two media toolkits breaking down the report visualizations into individual images for use in presentations and grant proposals. To facilitate tracking, we asked users to complete a request form to access the downloads, with the option of sharing contact information for a follow-up survey to be sent approximately 2 months later. Between April 2022 and January 2023, 304 people submitted the request form and 650+ report documents were downloaded. We sent 148 follow-up surveys. Among respondents ( n  = 64), the top uses of the CNA reported were to guide program and strategic planning (48.4%), to inform grant applications (29.7%), and using the web portal to get local data (23.4%) (Fig.  5 ). Use of the KY CNA was spread broadly across individuals in healthcare professions, university settings, state and local government, and cancer-focused non-profit organizations.

Bar chart showing how individuals used the 2021 Kentucky Cancer Needs Assessment ( n  = 64)

In addition, 750 physical copies of the KY CNA report were distributed throughout Kentucky, including 138 given to the current members of Kentucky’s state legislature. Over 2,000 copies of the district profiles were printed for dissemination by KCP Regional Cancer Control Specialists to local partners. Report findings were shared by UKMCC COE staff to over 800+ persons across 26 oral presentations.

We sought to outline a conceptual framework and process that cancer centers, cancer coalitions and other organizations can follow to perform community-engaged CNAs in their catchment areas. In particular, this project provided important methodological insight on how to triangulate quantitative and qualitative data and gather community feedback on setting research and outreach priorities. Applying this iterative process in the KY CNA, we were able to incorporate the perspectives of diverse community members across the state. Moreover, the contributions from focus groups, concept mapping, and our steering committee helped ensure that we could maximally assess the assets, opportunities, and barriers for cancer prevention and control. The KY CNA represented a significant advancement in understanding the story of cancer in Kentucky and laid the groundwork for future efforts to reduce the cancer burden in Kentucky.

Previous research has highlighted the importance of engaging community stakeholders in needs assessments and identification of priorities [ 12 , 37 ]. However, in a review of CHNAs among Texas non-profit hospitals, Pennel et al. observed that 18% made no attempt to engage community members, and only four out of 76 involved community members in strategy selection [ 14 ]. Within the KY CNA steering committee, the Kentucky Cancer Program—which already operates within Kentucky communities—played an important role in facilitating the involvement of lay community members. These individuals represented key population subgroups in the state and provided crucial feedback in setting priorities and strategies—reflecting an “interpretive approach” to defining community as highlighted by Franz et al. [ 38 ]. Although a simple look at incidence and mortality rates reveals the disproportionate burden of cancer in Kentucky, the KY CNA findings illustrate that the causes of this problem and the avenues for addressing it require a much deeper analysis. This finding supports the argument of Pennel et al. that grappling with the broader social determinants of health is necessary to achieve a beneficial impact on cancer outcomes and the improvement of health equity [ 39 ].

Our CNA process also adds several methodological contributions to the identification and prioritization of catchment area needs through community-engaged and mixed method approaches. Specifically, our novel approach includes: (1) the use of virtual focus groups to engage a wide-range of community members throughout our catchment area, including oversampling populations most at risk (e.g., rural, Black, sexual, and gender minorities); (2) the employment of concept mapping with both community members and statewide organizational partners to prioritize the wide array of identified cancer needs in our catchment area; and (3) the combination of several quantitative and qualitative data sources to ultimately inform new strategic plans, both across our catchment area and for local healthcare systems and communities. The CNA results were subsequently used to inform a new Kentucky Cancer Action Plan. Our use of concept mapping provided an opportunity for resulting strategic plans to incorporate the practical considerations of local organizational partners and the lived experiences of those in the community. Although concept mapping has been utilized for program planning, evaluation, and community needs assessments previously, to our knowledge this is the first use of the approach by a cancer center in a cancer-focused needs assessment capturing needs in a statewide catchment area [ 31 , 40 ]. Similar to other research conducted during the COVID-19 pandemic, we successfully adapted to pandemic restrictions by using virtual data collection for both the focus group and concept mapping elements [ 41 , 42 ]. Our ability to capture perspectives from diverse lay community members in over 40 different Kentucky counties suggests that virtual qualitative data collection is a viable CNA method for cancer centers and other organizations seeking to broadly capture needs in diverse and/or large catchment areas.

Although this project makes important contributions to the literature, it contains certain limitations that we acknowledge. The available secondary quantitative data sources to assess cancer rates, social determinants of health, and behavioral risk factors have varying time delays. To moderate this impact, we used data from the most recent years available for each source. Additionally, our analysis is subject to all of the limitations that come with the available secondary sources—in particular, lack of information about sexual orientation, gender identity, and disabilities. Though collecting new (primary) survey data on individuals in our catchment area could have potentially contributed additional quantitative insights unavailable in the secondary sources, the large size of our catchment area and cost made doing so impractical. For focus groups and concept mapping, the steering committee expressed concerns regarding technological literacy among participants due to the use of virtual formats. We sought to address this issue by asking potential participants about their comfort level with the technologies used and, when requested, providing them trainings to increase participation. Our goal was to broadly capture diverse perspectives across the state, which we achieved. However, although we oversampled vulnerable populations, we did not include a sufficient number of participants from individual groups to form specific recommendations by demographic group. The qualitative methods used in this project were not intended to be generalizable, but were designed with the intent of gaining depth of understanding on needs and general consensus on priorities with respect to cancer specifically. Finally, we did not gather extensive process metrics beyond dissemination data to evaluate the process. However, the steering committee expressed satisfaction with the process throughout each phase. Future applications of this framework and process could gather additional process metrics to demonstrate efficacy.

Performing periodic CNAs should be adopted as standard practice for cancer center COE offices and other organizations wishing to improve impact within their catchment area. The CNA framework aids in understanding the multilevel determinants of cancer-related outcomes to guide CNA planning and interpretation of findings. The CNA process employs a mixed-methods design with a variety of community input throughout to complement the quantitative data with people’s lived experiences. The resulting KY CNA report and dissemination not only informed the cancer center’s institutional research and strategic efforts, but also the efforts of state-level and community-based cancer organizations across Kentucky. This CNA framework and process can be replicated by COE offices at NCI-Designated Cancer Centers, public health agencies, cancer coalitions, oncology programs and community organizations. In addition, they can be used by universities for institution-level efforts focused on community engagement, health equity, and cancer research that spans across multiple research projects.

Data availability

Inquiries about data availability should be directed to the authors.

Altschuld JW, Watkins R (2014) A primer on needs assessment: more than 40 years of research and practice: a primer on needs assessment: more than 40 years of research and practice. New Dir Eval 2014:5–18. https://doi.org/10.1002/ev.20099

Article   Google Scholar  

Public Health Accreditation Board Standards & Measures for Initial Accreditation, Version 2022

Community Health Needs Assessment for Charitable Hospital Organizations—Section 501(r)(3). https://www.irs.gov/charities-non-profits/community-health-needs-assessment-for-charitable-hospital-organizations-section-501r3

Commission on Cancer, American College of Surgeons. Cancer Programs Standards 2012: Ensuring Patient-Centered Care. Version 1.2.1. https://apos-society.org/wp-content/uploads/2016/06/CoCStandards-1.pdf . Published 2012. Accessed 14 Sep 2023.

The GW Cancer Center. Implementing the Commission on Cancer Standard 8: Addressing Barriers to Care. https://cancercontroltap.smhs.gwu.edu/sites/g/files/zaskib661/files/2021-09/30%20-%20CoC%20Standard%208.1%20Road%20Map%20FINAL%20508.pdf . Published 2021. Accessed 12 Sep 2023.

Cancer Center Support Grants (CCSGs) for NCI-designated Cancer Centers. https://grants.nih.gov/grants/guide/pa-files/PAR-21-321.html

Paskett ED, Hiatt RA (2018) Catchment areas and community outreach and engagement: the new mandate for NCI-designated cancer centers. Cancer Epidemiol Biomark Prev 27:517–519. https://doi.org/10.1158/1055-9965.EPI-17-1050

Tai CG, Hiatt RA (2017) The population burden of cancer: research driven by the catchment area of a cancer center. Epidemiol Rev 39:108–122. https://doi.org/10.1093/epirev/mxx001

Article   PubMed   Google Scholar  

Hiatt RA, Kobetz EN, Paskett ED (2022) Catchment areas, community outreach and engagement revisited: the 2021 guidelines for cancer center support grants from the National Cancer Institute. Cancer Prev Res 15:349–354. https://doi.org/10.1158/1940-6207.CAPR-22-0034

Vadaparampil ST, Tiro JA (2022) Catchment area: an opportunity for collective impact, strategic collaboration, and complementary focus. Cancer Epidemiol Biomark Prev 31:952–954. https://doi.org/10.1158/1055-9965.EPI-22-0173

Durch J, Bailey LA, Stoto MA (1997) Improving health in the community: a role for performance monitoring. National Academy Press, Washington, D.C.

Google Scholar  

Pennel CL, Burdine JN, Prochaska JD, McLeroy KR (2017) Common and critical components among community health assessment and community health improvement planning models. J Public Health Manag Pract 23:S14–S21. https://doi.org/10.1097/PHH.0000000000000588

Burdine JN, Felix MRJ (2017) A personal history of the evolution of health status assessment. J Public Health Manag Pract 23:S9–S13. https://doi.org/10.1097/PHH.0000000000000577

Pennel CL, McLeroy KR, Burdine JN, Matarrita-Cascante D, Wang J (2016) Community health needs assessment: potential for population health improvement. Popul Health Manag 19:178–186. https://doi.org/10.1089/pop.2015.0075

National Comprehensive Cancer Control Program. https://www.cdc.gov/cancer/ncccp/index.htm . Accessed 27 Sept 2023

Hager P, Given L, Redmond J, Rogers K (2010) Revision of comprehensive cancer control plans: experiences shared by three states. Cancer Causes Control 21:2005–2013. https://doi.org/10.1007/s10552-010-9662-y

Rodriguez SD, Vanderford NL, Huang B, Vanderpool RC (2018) A social-ecological review of cancer disparities in Kentucky. South Med J 111:213–219. https://doi.org/10.14423/SMJ.0000000000000794

Article   PubMed   PubMed Central   Google Scholar  

Hiatt RA, Breen N (2008) The social determinants of cancer. Am J Prev Med 35:S141–S150. https://doi.org/10.1016/j.amepre.2008.05.006

Institute of Medicine (2013) Delivering high-quality cancer care: charting a new course for a system in crisis. The National Academies Press, Washington, D.C.

Lippman SM, Abate-Shen C, Colbert Maresso KL, Colditz GA, Dannenberg AJ, Davidson NE, Disis ML, DuBois RN, Szabo E, Giuliano AR, Hait WN, Lee JJ, Kensler TW, Kramer BS, Limburg P, Maitra A, Martinez ME, Rebbeck TR, Schmitz KH, Vilar E, Hawk ET (2018) AACR white paper: shaping the future of cancer prevention—a roadmap for advancing science and public health. Cancer Prev Res 11:735–778. https://doi.org/10.1158/1940-6207.CAPR-18-0421

Alcaraz KI, Wiedt TL, Daniels EC, Yabroff KR, Guerra CE, Wender RC (2020) Understanding and addressing social determinants to advance cancer health equity in the United States: a blueprint for practice, research, and policy. CA Cancer J Clin 70:31–46. https://doi.org/10.3322/caac.21586

Warnecke RB, Oh A, Breen N, Gehlert S, Paskett E, Tucker KL, Lurie N, Rebbeck T, Goodwin J, Flack J, Srinivasan S, Kerner J, Heurtin-Roberts S, Abeles R, Tyson FL, Patmios G, Hiatt RA (2008) Approaching health disparities from a population perspective: the National Institutes of Health Centers for Population Health and Health Disparities. Am J Public Health 98:1608–1615. https://doi.org/10.2105/AJPH.2006.102525

Portrait of Promise: The California Statewide Plan to Promote Health and Mental Health Equity

Tariq S, Woodman J (2013) Using mixed methods in health research. JRSM Short Rep 4:204253331347919. https://doi.org/10.1177/2042533313479197

Ravaghi H, Guisset A-L, Elfeky S, Nasir N, Khani S, Ahmadnezhad E, Abdi Z (2023) A scoping review of community health needs and assets assessment: concepts, rationale, tools and uses. BMC Health Serv Res 23:44. https://doi.org/10.1186/s12913-022-08983-3

Burus JT, Park L, McAfee CR, Wilhite NP, Hull PC (2023) Cancer InFocus: tools for cancer center catchment area geographic data collection and visualization. Cancer Epidemiol Biomark Prev 32:889–893. https://doi.org/10.1158/1055-9965.EPI-22-1319

Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG (2009) Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 42:377–381. https://doi.org/10.1016/j.jbi.2008.08.010

Harris PA, Scott KW, Lebo L, Hassan N, Lightner C, Pulley J (2012) ResearchMatch: a national registry to recruit volunteers for clinical research. Acad Med 87:66–73. https://doi.org/10.1097/ACM.0b013e31823ab7d2

Burke JG, O’Campo P, Peak GL, Gielen AC, McDonnell KA, Trochim WMK (2005) An introduction to concept mapping as a participatory public health research method. Qual Health Res 15:1392–1410. https://doi.org/10.1177/1049732305278876

Kan M, Trochim WMK (2007) Concept mapping for planning and evaluation. Sage Publications, Inc.

Thompson JR, Burke JG (2020) Increasing community participation in public health research: applications for concept mapping methodology. Prog Community Health Partnersh 14:243–250. https://doi.org/10.1353/cpr.2020.0025

The Concept System® groupwisdom™ (Build 2022.30.10) [Web-based Platform]. (2022). Ithaca, NY. Available from https://www.groupwisdom.tech .

IARC Working Group on the Evaluation of Carcinogenic Risks to Humans, World Health Organization, International Agency for Research on Cancer (2010) Alcohol consumption and ethyl carbamate. International Agency for Research on Cancer; Distributed by WHO Press, Lyon, France: Geneva

Viens LJ, Henley SJ, Watson M, Markowitz LE, Thomas CC, Thompson TD, Razzaghi H, Saraiya M (2016) Human papillomavirus-associated cancers—United States, 2008–2012. MMWR Morb Mortal Wkly Rep 65:661–666. https://doi.org/10.15585/mmwr.mm6526a1

Gallaway MS, Henley SJ, Steele CB, Momin B, Thomas CC, Jamal A, Trivers KF, Singh SD, Stewart SL (2018) Surveillance for Cancers Associated with Tobacco Use—United States, 2010–2014. MMWR Surveill Summ 67:1–42. https://doi.org/10.15585/mmwr.ss6712a1

Vanderpool RC, Huang B, Deng Y, Bear TM, Chen Q, Johnson MF, Paskett ED, Robertson LB, Young GS, Iachan R (2019) Cancer-related beliefs and perceptions in Appalachia: findings from 3 states. J Rural Health 35:176–188. https://doi.org/10.1111/jrh.12359

Rosenbaum S (2013) Principles to Consider for the Implementation of a Community Health Needs Assessment Process. Washington, DC : The George Washington University, School of Public Health and Health Services. Dep Health Policy

Franz BA, Skinner D, Murphy JW (2018) Defining “Community” in community health evaluation: perspectives from a sample of nonprofit Appalachian hospitals. Am J Eval 39:237–256. https://doi.org/10.1177/1098214017722857

Pennel CL, McLeroy KR, Burdine JN, Matarrita-Cascante D (2015) Nonprofit hospitals’ approach to community health needs assessment. Am J Public Health 105:e103–e113. https://doi.org/10.2105/AJPH.2014.302286

Velonis AJ, Molnar A, Lee-Foon N, Rahim A, Boushel M, O’Campo P (2018) “One program that could improve health in this neighbourhood is ____?” using concept mapping to engage communities as part of a health and human services needs assessment. BMC Health Serv Res 18:150. https://doi.org/10.1186/s12913-018-2936-x

Dos Santos Marques IC, Theiss LM, Johnson CY, McLin E, Ruf BA, Vickers SM, Fouad MN, Scarinci IC, Chu DI (2021) Implementation of virtual focus groups for qualitative data collection in a global pandemic. Am J Surg 221:918–922. https://doi.org/10.1016/j.amjsurg.2020.10.009

Keen S, Lomeli-Rodriguez M, Joffe H (2022) From challenge to opportunity: virtual qualitative research during COVID-19 and beyond. Int J Qual Methods 21:160940692211050. https://doi.org/10.1177/16094069221105075

Download references

Acknowledgments

This work as supported in part by the National Institutes of Health (NIH) (Grant Numbers P30CA177558 and UL1TR001998), the Center for Disease Control and Prevention (CDC) (Grant Numbers NU58DP006313-04-02, NU58DP007144-01-00, NU58DP006058-05-00), and the William Stamps Farish Endowed Chair in Cancer Research. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH or CDC. We wish to acknowledge the valuable input of the members of the Kentucky Cancer Needs Assessment Steering Committee, some of whom contributed to this article as co-authors, along with the additional members who represented Kentucky Cancer Registry (Jaclyn McDowell), Kentucky Cancer Program (Elizabeth Westbrook, Jamie Smith), Kentucky Cabinet for Health and Family Services (Janie Cambron, Vivian Lasley-Bibbs, Emily Messerli, Carissa Adams, Carrie Conia, Elizabeth Owen), American Cancer Society (Elizabeth Holtsclaw, Julie Waters), Foundation for a Healthy Kentucky (Allison Adams), and the University of Louisville (Stephanie Boone). We also wish to thank Dr. Eric Durbin, Director of the Kentucky Cancer Registry for his feedback and support of this effort.

P.C. Hull, B. Huang and C.R. McAfee were supported by the National Institutes of Health (NIH) grant P30CA177558. P.C. Hull and J.R. Thompson received support from the William Stamps Farish Endowed Chair in Cancer Research. P.C. Hull, E. Russell, J. Redmond Knight, and B. Huang were supported by the Center for Disease Control and Prevention (CDC) Grants NU58DP006313-04-02 and NU58DP007144-01-00. B, Huang received support from the NIH SEER program contract HHSN26100001. S. Kanotra was supported in part by funds from the CDC Cooperative Agreement Number 5 NU58DP006058-05-00. This work was supported by the Cancer Research Informatics Shared Resource Facility and the Biostatistics and Bioinformatics Shared Resource Facility of the University of Kentucky Markey Cancer Center (P30CA177558) and by the NIH National Center for Advancing Translational Sciences through Grant Number UL1TR001998. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH or CDC.

Author information

Authors and affiliations.

Markey Cancer Center, University of Kentucky, Lexington, KY, USA

Todd Burus, Jessica R. Thompson, Caree R. McAfee, Lovoria B. Williams, Natalie P. Wilhite, Elaine Russell, Christine Stroebel & Pamela C. Hull

College of Nursing, University of Kentucky, Lexington, KY, USA

Lovoria B. Williams

Kentucky Cancer Consortium, Lexington, KY, USA

Jennifer Redmond Knight & Elaine Russell

Department of Health Management and Policy, College of Public Health, University of Kentucky, Lexington, KY, USA

Jennifer Redmond Knight

Division of Cancer Biostatistics, College of Medicine, University of Kentucky, Lexington, KY, USA

Kentucky Cancer Registry, Markey Cancer Center, University of Kentucky, Lexington, KY, USA

Kentucky Department for Public Health, Frankfort, KY, USA

Sarojini Kanotra

Kentucky Cancer Program, Markey Cancer Center, University of Kentucky, Lexington, KY, USA

Melinda Rogers

Kentucky Cancer Program, Brown Cancer Center, University of Louisville, Louisville, KY, USA

Connie L. Sorrell

American Cancer Society, Kennesaw, GA, USA

Rachael King

Department of Behavioral Science, College of Medicine, University of Kentucky, Lexington, KY, USA

Pamela C. Hull

University of Kentucky Markey Cancer Center, 760 Press Avenue, Suite 460, Lexington, KY, 40536, USA

You can also search for this author in PubMed   Google Scholar

Contributions

All authors contributed to the study conception and design. Project administration was performed by TB, JT, CM, LW, and PH. Data collection, data curation and analysis were performed by TB, JT, BH, SK, and CS. Visualizations were prepared by TB and NW. The first draft of the manuscript was written by TB, JT, CM, and PH. All authors provided critical review and comments on drafts for revisions of the manuscript, and all authors read and approved the final manuscript.

Corresponding author

Correspondence to Todd Burus .

Ethics declarations

Competing interests.

The authors have no relevant financial or non-financial interests to disclose.

Ethical approval

The University of Kentucky Institutional Review Board (IRB) approved the research procedures under three protocols: Kentucky Behavioral Risk Factor Survey analysis (#69894), focus groups (#65451), and concept mapping (#73420). The IRB issued a non-human research determination for using the publicly available, aggregated secondary data and the CNA dissemination follow-up survey.

Consent to participate

Informed consent was obtained from all individual participants included in the study.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (PPTX 702 kb)

Rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Burus, T., Thompson, J.R., McAfee, C.R. et al. A framework and process for community-engaged, mixed-methods cancer needs assessments. Cancer Causes Control (2024). https://doi.org/10.1007/s10552-024-01892-2

Download citation

Received : 16 October 2023

Accepted : 17 May 2024

Published : 29 May 2024

DOI : https://doi.org/10.1007/s10552-024-01892-2

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Catchment area
• Needs assessment
• Qualitative research
• Quantitative evaluation
• Community outreach

Advertisement

• Find a journal
• Publish with us
• Track your research

• Study protocol
• Open access
• Published: 04 June 2024

Understanding the processes underpinning IMPlementing IMProved Asthma self-management as RouTine (IMP 2 ART) in primary care: study protocol for a process evaluation within a cluster randomised controlled implementation trial

• J. Sheringham   ORCID: orcid.org/0000-0003-3468-129X 1 ,
• L. Steed 2 ,
• K. McClatchey 3 ,
• B. Delaney 4 ,
• A. Barat 2 ,
• V. Hammersley 5 ,
• V. Marsh 5 ,
• N. J. Fulop 1 ,
• S. J. C. Taylor 2 &
• H. Pinnock 5  

Trials volume  25 , Article number:  359 ( 2024 ) Cite this article

Metrics details

Providing supported self-management for people with asthma can reduce the burden on patients, health services and wider society. Implementation, however, remains poor in routine clinical practice. IMPlementing IMProved Asthma self-management as RouTine (IMP 2 ART) is a UK-wide cluster randomised implementation trial that aims to test the impact of a whole-systems implementation strategy, embedding supported asthma self-management in primary care compared with usual care. To maximise opportunities for sustainable implementation beyond the trial, it is necessary to understand how and why the IMP 2 ART trial achieved its clinical and implementation outcomes.

A mixed-methods process evaluation nested within the IMP 2 ART trial will be undertaken to understand how supported self-management was implemented (or not) by primary care practices, to aid interpretation of trial findings and to inform scaling up and sustainability. Data and analysis strategies have been informed by mid-range and programme-level theory. Quantitative data will be collected across all practices to describe practice context, IMP 2 ART delivery (including fidelity and adaption) and practice response. Case studies undertaken in three to six sites, supplemented by additional interviews with practice staff and stakeholders, will be undertaken to gain an in-depth understanding of the interaction of practice context, delivery, and response. Synthesis, informed by theory, will combine analyses of both qualitative and quantitative data. Finally, implications for the scale up of asthma self-management implementation strategies to other practices in the UK will be explored through workshops with stakeholders.

This mixed-methods, theoretically informed, process evaluation seeks to provide insights into the delivery and response to a whole-systems approach to the implementation of supported self-management in asthma care in primary care. It is underway at a time of significant change in primary care in the UK. The methods have, therefore, been developed to be adaptable to this changing context and to capture the impact of these changes on the delivery and response to research and implementation processes.

Peer Review reports

Asthma places a major burden on patients, health services and wider society. Providing self-management education to people with asthma, supported by a personalised action plan and regular review, can reduce this burden, by preventing unscheduled healthcare use and improving asthma control [ 1 ]. Several studies have demonstrated that the implementation of supported self-management in primary care, however, remains low [ 2 ]. Enhancing the implementation of supported self-management in primary care requires a whole-systems approach—i.e. a combination of patient education, professional training and organisational support [ 3 ].

IMP 2 ART is a whole-systems, evidence-based implementation strategy developed to help primary care practices to implement supported self-management for asthma patients [ 4 , 5 , 6 , 7 , 8 ]. Evaluations of such implementation strategies are complex and require consideration of the clinical effectiveness, the implementation success and the process by which such outcomes are achieved. Process evaluations play a particularly crucial role in this understanding. They unpick the ‘black box’ of interventions by understanding who received what, how and the process through which it impacted (or not) outcomes and inform potential mechanisms for sustainability. There is plentiful guidance that process evaluations should use mid-range theory in their design and delivery [ 9 ]. There is increasing recognition that process evaluations could and should seek to develop these mid-range theories to improve the design and evaluation of future implementation studies [ 10 ].

This paper describes the protocol for the process evaluation taking place alongside a cluster randomised trial of the IMPlementing IMProved Asthma self-management as RouTine (IMP 2 ART) strategy in UK primary care practices [ref: ISRCTN15448074]. It sets out how we seek to measure the delivery and response to IMP 2 ART, how we seek to understand the trial’s effectiveness findings and how it may contribute to the development of theory.

IMP 2 ART cluster randomised trial

IMP 2 ART is a UK-wide cluster randomised implementation trial that aims to test the impact of a whole-systems implementation strategy embedding supported asthma self-management in primary care compared with usual care on both clinical and implementation outcomes. The main trial protocol and the IMP 2 ART strategy are described in McClatchey et al. [ 5 ].

Programme theory

IMP 2 ART combined the mid-range implementation and behaviour change theories of iPARIHS [ 9 ] and capability, opportunity, and motivation required for behaviour change (COM-B) [ 11 ] to develop a programme-level theory of how IMP 2 ART can support practices to implement supported self-management in asthma [ 12 ]. The programme theory states the central hypothesis of IMP 2 ART, i.e. that facilitation plays a critical role in implementation success. Facilitation encompasses both the inputs of a trained facilitator and the delivery of IMP 2 ART implementation strategies. Facilitation is expected to achieve its impacts through increasing staff capability, motivation and opportunity towards supported self-management. It is also expected that tailoring by facilitators to take account of practice context, particularly capacity, culture and leadership will be an important aspect of their interactions with practices. Moreover, we expect that the relationship between the facilitation and practice response to IMP 2 ART will be influenced by practice context.

IMP 2 ART strategies

The IMP 2 ART strategies comprise multiple components directed at patients, professionals and the organisation, supported by expert nurse facilitation for 12 months, summarised in a logic model (Fig.  1 ) and described in greater detail in McClatchey et al. (2023) [ 5 ].

IMP2ART’s logic mode [ 12 ]. Facilitation through trained specialist asthma nurses acts as a catalyst for the use of MP2ART strategies in practices, which in turn aims to optimise professionals’ capacity, motivation and opportunity to deliver supported self-management to patients with asthma. The nature of the delivery of IMP2ART and the response to facilitation and the IMP2ART strategies will vary between practices and will be strongly influenced through practice context

While all strategies will be made available to all implementation practices, it is expected that practices will choose which IMP 2 ART strategies they use and how they adapt them to their own context. In all implementation practices, the following ‘core’ strategies will be delivered: facilitation and ongoing support from a facilitator (12 months), monthly audit and feedback reports highlighting practice-supported self-management performance (e.g. number of action plans provided) [ 4 ], and access to education modules (a team education module to highlight the whole team role in supported self-management; an individual module for clinicians delivering asthma reviews) [ 5 ] and access to a ‘Living with Asthma’ website with practice and patient resources [ 8 ].

The cluster randomised controlled trial takes place in England and Scotland with a target recruitment of 144 practices.

An internal pilot of IMP 2 ART was conducted in 2021 with the first 12 practices recruited to the trial, primarily to optimise trial design as the feasibility of the components of the implementation strategy had already been tested and refined [ 4 , 5 , 6 ]. The pilot also provided an opportunity to test and refine some of the bespoke data collection and analysis approaches for the process evaluation [ 13 ].

Aims and objectives

The IMP 2 ART process evaluation has two primary aims:

To explain the IMP 2 ART trial’s clinical and implementation outcome findings

To identify learning, in relation to IMP 2 ART outcomes, to inform the design, scaling up and sustainability of implementation strategies to improve supported self-management of asthma in primary care

The evaluation is structured to achieve five interrelated objectives, with associated research questions, see Table  1 .

The process evaluation is based on the Medical Research Council guidelines for the process evaluation of complex interventions [ 14 ], also drawing on Grant et al.’s process evaluation framework for cluster randomised trials [ 15 ]. Aligned with the Medical Research Council recommendations for the evaluation at different stages of development, an internal pilot process evaluation has already been conducted and will be reported elsewhere [ 13 ]. The process evaluation incorporates learning from the pilot and focuses on three key dimensions: implementation strategy delivery (including fidelity and adaptation), practice response and practice context.

As shown in Fig.  2 , we will primarily use quantitative analysis to achieve objectives 1 and 2 and an in-depth qualitative approach to achieve objective 3, culminating in a mixed-methods synthesis supported by additional interviews to achieve objectives 4 and 5. In line with Fetters et al.’s description of an interactive approach, we will iterate data collection and analysis through the use of interim (formative) analyses and discussion of emerging findings during the process evaluation [ 16 ]. We will adopt a critical realist perspective, which is in keeping with our aim to derive causative explanations for IMP 2 ART’s findings, whilst acknowledging that we can only capture aspects of reality [ 17 ].

Relationship between objectives in IMP2ART’s process evaluation

There are no specific reporting guidelines for process evaluations, so we have drawn on suggestions from Moore et al. [ 14 ] StaRI guidelines for reporting implementation studies [ 18 ] and used the TRIPLE C reporting principles for case study evaluations as a guide to the case study element [ 19 ].

Data collection and collation

The process evaluation will use quantitative data from all 144 recruited practices (control and implementation), with a focus on the practices allocated to the implementation arm. A subset of practices will be invited to take part in further qualitative data collection, as case studies or as one-off interviews. Additional qualitative interview data will be collected from IMP 2 ART’s facilitators and national stakeholders, described in more detail below.

Data for the process evaluation comes from a range of sources, which are summarised in Fig.  3 , and described in more detail below. Table 2 summarises their links to our research questions.

IMP2ART process evaluation data collection timeline. *Months refer to IMP2ART trial duration. All practices will participate in the trial for 24 months. Practices assigned to the implementation arm will receive 12 months of active facilitation and 12 months of follow-up

Researcher logs

Researchers will keep detailed notes about all practices approached, practices showing interest and the proportion who agree to participate. Where available, reasons for participation and non-participation will be noted to inform the potential for scaling up ( aim 2 ). Reasons for withdrawal (if available) will be noted. For those practices randomised to the trial, researchers will continue to log contacts and delivery of IMP2ART tools such as audit and feedback reports sent and templates uploaded.

Facilitator logs and facilitator/practice contact recordings

Facilitators will keep logs of all practice contacts, including duration and nature of contact (e.g. email, phone or video call). The practice introductory workshop and the end of the facilitation meeting, both conducted via video calls, will be video recorded, and data from the call (duration, attendees and comments) will be downloaded. In addition, the facilitator will complete an observation form shortly after to structure impressions of the practice.

Publicly available data and practice profile survey (n  =  144)

Details on all trial practices (control and implementation) will be obtained where possible from publicly available sources at the start of the trial on practice characteristics (e.g. population size, ethnicity, deprivation), supplemented with a survey of all practices to obtain information on their model of asthma management. In line with StaRI, we will examine both initial and changes in context [ 18 ]. To obtain changes in the practice context, key details of this profile will be reviewed in discussion with practices at the end of the trial meeting.

Educational online module usage data (n  ~  72)

Practice completion of the online educational modules will be logged automatically. We will store data on logins and completion of both modules at the practice level. The use of the team module will be recorded for each practice as it is designed to be used and discussed in groups.

Practice response survey (n ~ 72)

We will collect self-reported implementation of IMP 2 ART strategies at 12 months (end of facilitation) and 24 months (end of trial). We have drawn on Proctor et al.’s taxonomy of implementation outcomes to identify different aspects of implementation relevant to IMP 2 ART and will focus on different outcomes at different points during the trial, e.g. acceptability and adoption in initial measures, adaptation at the mid-point and sustainability towards the end of the trial [ 21 ].

Facilitator group interviews (n  <  8)

IMP 2 ART’s four trained asthma nurse facilitators will be interviewed as a group at four points in the trial. Interviews will focus on their evolving experiences and learning as they progress from delivering their initial IMP 2 ART workshops and becoming experienced facilitators to their experiences of ending the facilitation process with practices. These interviews will serve a formative purpose, to provide an alert of any problems in the delivery of IMP 2 ART and to inform data collection strategies (e.g. to highlight practices that have engaged in IMP 2 ART in specific ways for non-case study interviews). They will also serve a summative purpose in providing insights into the IMP 2 ART delivery, particularly on the evolving interaction between delivery and practice context.

Full case studies (n  ≤  4) and mini-case studies (n  ≤  4)

We will seek to explore from multiple perspectives how IMP 2 ART fits with a practice’s culture and routines, absorptive capacity and leadership over the 2 years in which practices participate in the trial.

A case study methodology, as described by Yin, is applicable where an in-depth investigation of a contemporary phenomenon is needed within a real-life context and where the boundaries between context and the phenomenon are not clear [ 22 ]. This fits with the fact that context and intervention are intentionally very closely interlinked in IMP 2 ART because the strategies have been designed to be adapted in response to practice context.

We have designed flexibility into our case study methodology, recognising that general practices are under significant pressure and may not be able to commit to 2 years of data collection. We have therefore developed a mini-case study adaptation of our full method, as a bridge between one-off interviews and full case studies.

Case study selection: From the implementation group practices, we will approach case studies to try and reflect diversity in baseline asthma management and population characteristics.

Data collection: For full case studies, we will seek to collect several sources of bespoke data at key intervals in the practice’s participation in the trial (Table  2 ).

Interviews ( n  ≤ 12/case study) with key actors will be conducted at early, mid- and later stages of the trial to track how implementation changes and changes in practice context and to explore the potential for sustainability. Key actors may be individuals who deliver supported self-management in the practice (usually nurses, but may also be healthcare assistants, pharmacists, GPs); people in an administrative role who contribute to, or might be affected by, the implementation of supported self-management (e.g. practice managers, prescribing clerks, receptionists); people making decisions about self-management (e.g. GP partners, practice managers); and lay members of a practice user group if they are involved in the IMP 2 ART initiative (see example topic guide, Supplementary data file 1).

These will often be repeat interviews with the same individuals (or sometimes new individuals who have taken over a role of a previous participant). Researchers will tailor topic guides to the stage of the practice’s participation in IMP 2 ART, i.e. focusing on context and expectations in early interviews and asking for reflections on IMP 2 ART and sustainability at later interviews.

Observation of activities ( n  ≤ 20 h/case), e.g. training sessions, practice meetings, facilitator visits, shadowing practice staff and routine review consultations. Observation field notes will focus on the practice context, the processes by which practices implement self-management and evidence of adopting/adapting the IMP. 2 ART implementation strategy (see example observation guide, Supplementary data file 1)

Documentary analysis ( n  ≤ 40/case), e.g. anonymised personalised asthma action plans, meeting minutes, asthma review procedures and policies

Audio-recording of asthma clinics n  > 3 asthma clinics/case study practice

For mini-case studies, we will seek at least one face-to-face interview and an observation and draw on data already collected at other trial time points (e.g. end-of-facilitation meeting). Data collection will be influenced by the stage at which the practice has reached by the time of recruitment (Table  3 ).

Data collection tools are based on the IMP 2 ART programme theory, reflecting particularly the mid-range i-PARIHS and COM-B frameworks. Each tool is designed to be tailored to ensure data gathering is aligned with participants’ roles and the stage of trial participation (see Supplementary data file 1 for examples of topic guides and observation tools).

Non-case study interviews (n  ≤  12)

Up to 12 non-case study interviews will be undertaken informed by preliminary findings from other data sources (e.g. facilitator feedback) and learning from prior IMP 2 ART research [ 23 ], so that we can explore the applicability of our emerging themes in a range of contexts. From the pool of non-case study practices, we will recruit a key informant (GP, nurse or practice manager) able to discuss the implementation of supported self-management in their practice in a semi-structured interview. Practices will be selected because they offer a contrasting context to our case study practices, use novel approaches to implementation, or are outliers in terms of outcomes/processes (e.g. where facilitator notes suggest very low or very high engagement with IMP 2 ART or exhibit innovative adaption). Interviews will be informed by the ongoing process analysis and will seek views on emerging themes.

Stakeholder interviews (n  ≤  10)

We will arrange focussed interviews with stakeholders to explore the generalisability of emerging themes and/or policy perspectives. These may represent national or regional opinion leaders in asthma care or healthcare management with whom we can test out emerging themes. They may also include IMP 2 ART collaborators who can give a view on the feasibility and value of embedding IMP 2 ART approaches beyond the trial if they are found to be effective.

Data management and analysis

Many of the sources will include data that could be analysed qualitatively and quantitatively. For example, the researcher logs will include dates of key activities for all practices, but for some practices, there will also be researcher field notes (e.g. practice feedback). Key variables will be extracted from the data sources and analysed quantitatively for all practices. We will also select data to be analysed qualitatively alongside case study and additional interview data to supplement the case studies or where data provide evidence that contributes to the programme theory.

Quantitative

Data management.

The main quantitative analysis of IMP 2 ART will be conducted at the practice level so a core dataset will be formed at the practice level from all the sources and imported into Excel.

Quantitative descriptive analysis will be conducted on data from all implementation practices to answer our objective 1 research questions related to IMP 2 ART delivery, practice response and summary practice characteristics.

Fidelity and adaptation

The concept of fidelity has been variably defined and interpreted [ 24 ]. In some conceptualisations, fidelity is synonymous with adherence, with maximising adherence being a goal of intervention delivery [ 25 ]. In implementation research, however, adaptation of a strategy—changes in its content, format or delivery—to align the innovation with important characteristics of local context is often critical [ 26 ]. This is reflected in the StaRI guidelines which recommend reporting of both fidelity (in terms of core strategies to be delivered) and also adaptations made [ 18 ]. In IMP 2 ART, we therefore measure both fidelity of delivery for core strategies, whilst also seeking to capture adaptations and the rationale for these.

The dimensions of fidelity measured relate to the five recommended key domains of the NIH Behaviour Change Consortium (BCC) Treatment Fidelity Framework—treatment design, training, delivery, receipt and enactment [ 27 ] (with a focus on the delivery of IMP 2 ART), receipt (practice response to IMP 2 ART) and enactment (delivery of supported self-management with patients).

To allow for a more in-depth understanding of the delivery of facilitation and its potential mechanisms of action, a sub-sample of video-recorded introductory facilitation and end-of-facilitation workshops will be coded to understand the activities and processes of facilitation used. A study-specific tool has been developed for this purpose. A sample of at least 10% ( n  = 7) of practices will be selected at random stratified to ensure at least one workshop per facilitator. In addition, all workshops from case studies will be coded [ 28 ]. Each workshop will be independently coded by two individuals following training to ensure consistency in the application of the tool.

Qualitative

All interviews will be audio-recorded, transcribed verbatim and anonymised. Researchers will take structured field notes for all case study observations which will be stored in a central repository. The research team will work together to build consistency in the process of both data collection and analysis. Activities to enhance consistency include all researchers watching and writing field notes on an initial IMP 2 ART workshop recording from a case study site and then discussing the similarities and differences in their observations.

As advised by Yin for explanatory case studies [ 22 ], our analytic strategy for both case study data and the additional interviews will be in part guided by the theoretical propositions we have already set out, and in part inductive, guided by the data [ 12 ]. These propositions have informed the data analysis tools we have developed to organise and redescribe the data for each case. In line with critical realist theory, we also recognise the fallibility of theory and remain open to drawing on other theories or frameworks to support data interpretation [ 17 ].

Mid-range and programme theories have guided an initial coding framework, which we will iterate following researcher discussions in light of the data (see Fig.  4 , stage 1). We will use the coded data to identify themes, and with reference to evidence from quantitative data sources (see mixed methods synthesis), we will produce both standardised descriptions of each case and a timeline. These documents will be used for cross-case comparisons to identify similarities and differences between cases (see Fig.  4 , stages 2–3). As shown in step 4, wider interviews will be coded using the initial coding framework, with refinements carried out as described in step 1. The coding of these interviews will be used to identify further themes.

Analytical stages for qualitative and mixed methods analysis

Mixed methods synthesis and interpretation

In the mixed methods synthesis, we will seek to address objectives 4 and 5 providing explanation for results and making recommendations for future practice.

We will integrate quantitative data analysis (objectives 1 and 2), case study findings (objective 3) and the themes from additional interview data from group interviews with IMP 2 ART’s four facilitators; non-case study practice staff ( n  ≤ 12); and wider stakeholders ( n  ≤ 10). We will use these data to interrogate and refine the explanations derived from the case studies about which practice contexts and how IMP 2 ART might work best (objective 4).

As shown in Figs. 1 and 4 , we will conduct the analysis iteratively through frequent sharing of interim analyses, to enable us to draw on learning from each data source to refine the analysis and interpretation of the data as a whole. For example, the qualitative analysis may help us select variables to examine quantitative associations between contextual characteristics and delivery/response to IMP 2 ART.

We will use integrated analysis to produce a narrative synthesis across all the data sources that critically assess the evidence for the initial programme theory [ 12 ]. If possible, we will look to understand the mechanisms and moderators that enable the use of the IMP 2 ART implementation strategies to increase supported self-management of asthma in general practice.

Preliminary findings will be shared at three end-of-programme workshops in each of the main research sites (London, Sheffield and Lothian) to share preliminary findings from the IMP 2 ART programme, gauge the application of our findings to a broader range of contexts and explore the potential for scale-up and transferability of the findings to other practices in England and Scotland.

This protocol describes a nested process evaluation to aid the interpretation of the IMP 2 ART trial results and—in its own right—seeks to provide insights into the delivery and response to a whole-systems approach to the implementation of supported self-management in asthma care in general practice.

Methodological considerations

This process evaluation has been informed by a comprehensive pilot evaluation across the first twelve practices to participate in the trial and substantial research on strategies to implement supported self-management in primary care [ 7 , 23 , 29 ]. The pilot experience in IMP 2 ART indicated that the implementation strategy was feasible and acceptable and enabled testing and refinement of core elements of the process evaluation methods [ 13 ]. It is important to note that conditions in pilots may be different from those in the main trial [ 20 ], which necessitates exploration of some of the pilot themes with a wider sample. By conducting the process evaluation in parallel with the trial, there is the potential for interim process evaluation findings to influence peripheral content and delivery to maximise the chance of a successful outcome [ 28 ].

The inclusion of a multisite case study within the process evaluation is also now more widely recognised as a significant opportunity to provide evidence about context and transferability, and also to support elucidating causal inferences, particularly in trials like IMP 2 ART where the pathway from intervention delivery to impact is likely to be non-linear [ 28 ].

Limitations and challenges

IMP 2 ART is taking place at a time of considerable change and uncertainty in general practice. This uncertainty may affect both practice engagement with the implementation strategy and their participation in the process evaluation data collection. We have sought to minimise the burden for practices through using publicly available data where possible to describe characteristics and routine data for many of the measurements. Whilst maintaining consistency, we have also sought to make the process evaluation methods flexible so we can adapt to changing circumstances. For example, we have introduced an adapted case study design in recognition that many IMP 2 ART practices are not able to commit to longitudinal case studies over the entire 2 years of their trial participation. Whilst this may reduce the depth of our case studies, it will enable us to explore a wider range of contexts. We have also conducted an interim analysis of process evaluation data to identify where our methods need additional iteration.

The process evaluation is highly complex, both in terms of its combinations of theory and in its involvement of a large multidisciplinary research team split across multiple sites. Additional complexity can arise because of the differences in philosophical perspectives across the different disciplines contributing to IMP 2 ART. In order to build consistency across a large team working with these multiple perspectives, we have worked to create common understandings of all the theoretical perspectives on which IMP 2 ART is drawn [ 12 ]. We have also worked to maximise alignments between philosophical and methodological perspectives. For example, we recognise the possible philosophical misalignment between a trial, which implicitly takes a positivist perspective, and the process evaluation’s critical realist perspective, which considers that research can, at best, only capture a small part of reality [ 17 ]. Our choice of a critical realist approach, however, allows us to recognise the relevance of the trial’s findings to the reality of primary care practice and recommends the use of theory as a way of getting closer to identifying causal explanations for the trial’s findings.

It is not feasible to examine every process or outcome to the same depth. Our focus in the process evaluation is on IMP 2 ART’s main focus, helping general practices to implement supported self-management. It means, however, that while trial outcomes will be at the patient level, the process evaluation has limited opportunities to understand the impacts of IMP 2 ART on patients. We will seek to address this gap through some future additional linked projects, for example, to interview patients in some of our sites (Supplementary data file 2).

Study status

This protocol of the process evaluation is version 2.0. Version 1 was approved by ethics in 2019. Significant changes to the process evaluation since version 1 include the addition of adapted—mini-case studies—alongside full case studies.

Recruitment for the process evaluation began in July 2022. At the time of protocol submission (January 2024), the process evaluation was part way through the recruitment of case study sites (4/6 recruited) and had not started recruitment of non-case study interviews. Data collection is anticipated to be complete by 31 December 2024.

Availability of data and materials

Availability of data will depend on the data source. Due to the confidentiality of NHS routine data, trial and other data extracted from practices will not be made available. Other quantitative data may be made available via the University of Edinburgh DataShare if the practices are non-identifiable. By its nature, it will not be possible to anonymise the qualitative interview data for public availability. Requests for secondary use should be directed to the trial manager.

Abbreviations

Capability, Opportunity, Motivation – as determinants of Behaviour

General practitioner

Health Service and Delivery Research

IMPlementing IMProved Asthma self-management as RouTine

Integrated-Promoting Action on Research Implementation in Health Services

National Institute for Health Research

National Health Service

Randomised controlled trial

Research Ethics Committee

United Kingdom

Pinnock H, Parke HL, Panagioti M, Daines L, Pearce G, Epiphaniou E, Bower P, Sheikh A, Griffiths CJ, Taylor SJC, et al. Systematic meta-review of supported self-management for asthma: a healthcare perspective. BMC Med. 2017;15(1):64. https://doi.org/10.1186/s12916-017-0823-7 .

Article   PubMed   PubMed Central   Google Scholar  

Wiener-Ogilvie S, Pinnock H, Huby G, Sheikh A, Partridge MR, Gillies J. Do practices comply with key recommendations of the British Asthma Guideline? If not, why not? Prim Care Respir J. 2007;16(6):369–77. https://doi.org/10.3132/pcrj.2007.00074 .

Taylor SJC, Pinnock H, Epiphaniou E, Pearce G, Parke H. A rapid synthesis of the evidence on interventions supporting self-management for people with long-term conditions. (PRISMS Practical Systematic Review of Self-Management Support for long-term conditions) Health Serv Deliv Res. 2014;2:54.

McClatchey K, Sheldon A, Steed L, Sheringham J, Holmes S, Preston M, Appiagyei F, Price D, Taylor SJC, Pinnock H, et al. Development of theoretically informed audit and feedback: an exemplar from a complex implementation strategy to improve asthma self-management in UK primary care. J Eval Clin Pract. 2023. https://doi.org/10.1111/jep.13895 .

Article   PubMed   Google Scholar  

McClatchey K, Marsh V, Steed L, Holmes S, Taylor SJC, Wiener-Ogilvie S, Neal J, Last R, Saxon A, Pinnock H, et al. Developing a theoretically informed education programme within the context of a complex implementation strategy in UK primary care: an exemplar from the IMP2ART trial. Trials. 2022;23(1):350. https://doi.org/10.1186/s13063-022-06147-6 .

McClatchey K, Sheldon A, Steed L, Sheringham J, Appiagyei F, Price D, Hammersley V, Taylor S, Pinnock H. Development of a patient-centred electronic review template to support self-management in primary care: a mixed-methods study. BJGP Open 2023, 7 (2), BJGPO.2022.0165. https://doi.org/10.3399/bjgpo.2022.0165 .

Morrissey M, Shepherd E, Kinley E, McClatchey K, Pinnock H. Effectiveness and perceptions of using templates in long-term condition reviews: a systematic synthesis of quantitative and qualitative studies. Br J Gen Pract. 2021;71(710):e652–9. https://doi.org/10.3399/bjgp.2020.0963 .

Harvey G, Kitson A. PARIHS revisited: from heuristic to integrated framework for the successful implementation of knowledge into practice. Implement Sci. 2016;11:33. https://doi.org/10.1186/s13012-016-0398-2 .

Kislov R, Pope C, Martin GP, Wilson PM. Harnessing the power of theorising in implementation science. Implement Sci. 2019;14(1):103. https://doi.org/10.1186/s13012-019-0957-4 .

Michie S, van Stralen MM, West R. The behaviour change wheel: a new method for characterising and designing behaviour change interventions. Implement Sci. 2011;6:42. https://doi.org/10.1186/1748-5908-6-42 .

Steed L, Sheringham J, McClatchey K, Hammersley V, Marsh V, Morgan N, Jackson T, Holmes S, Taylor S, Pinnock H. IMP(2)ART: development of a multi-level programme theory integrating the COM-B model and the iPARIHS framework, to enhance implementation of supported self-management of asthma in primary care. Implement Sci Commun. 2023;4(1):136. https://doi.org/10.1186/s43058-023-00515-2 .

McClatchey K, Sheringham J, Barat A, Delaney B, Searle B, Marsh V, Hammersley V, Taylor S, Pinnock H. IMPlementing IMProved Asthma self-management as RouTine (IMP2ART) in primary care: internal pilot for a cluster randomised controlled trial. Eur Respir J. 2022;60(suppl 66):394. https://doi.org/10.1183/13993003.congress-2022.394 .

Article   Google Scholar  

Moore GF, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, Moore L, O’Cathain A, Tinati T, Wight D, et al. Process evaluation of complex interventions: Medical Research Council guidance. BMJ 2015, 350. https://doi.org/10.1136/bmj.h1258 .

Grant A, Treweek S, Dreischulte T, Foy R, Guthrie B. Process evaluations for cluster-randomised trials of complex interventions: a proposed framework for design and reporting. Trials. 2013;14:15. https://doi.org/10.1186/1745-6215-14-15 .

Fetters MD, Curry LA, Creswell JW. Achieving integration in mixed methods designs-principles and practices. Health Serv Res. 2013;48(6 Pt 2):2134–56. https://doi.org/10.1111/1475-6773.12117 .

Pinnock H, Barwick M, Carpenter CR, Eldridge S, Grandes G, Griffiths CJ, Rycroft-Malone J, Meissner P, Murray E, Patel A, et al. Standards for Reporting Implementation Studies (StaRI) Statement. BMJ 2017, 356. https://doi.org/10.1136/bmj.i6795 .

Shaw SE, Paparini S, Murdoch J, Green J, Greenhalgh T, Hanckel B, James HM, Petticrew M, Wood GW, Papoutsi C. TRIPLE C reporting principles for case study evaluations of the role of context in complex interventions. BMC Med Res Methodol. 2023;23(1):115. https://doi.org/10.1186/s12874-023-01888-7 .

Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, Griffey R, Hensley M. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health. 2011;38(2):65–76. https://doi.org/10.1007/s10488-010-0319-7 .

Yin RK. Case study research and applications; Thousand Oaks. California: SAGE Publications Inc; 2018.

Google Scholar  

Fingertips: Public Health Data - National General Practice Profiles. Office for Health Improvement and Disparities. https://fingertips.phe.org.uk/profile/general-practice . Accessed 4 Jan 2024.

Morrow S, Daines L, Wiener-Ogilvie S, Steed L, McKee L, Caress A-L, Taylor SJC, Pinnock H. Exploring the perspectives of clinical professionals and support staff on implementing supported self-management for asthma in UK general practice: an IMP2ART qualitative study. Primary Care Respiratory Medicine. 2017;27(1):45. https://doi.org/10.1038/s41533-017-0041-y .

Carroll C, Patterson M, Wood S, Booth A, Rick J, Balain S. A conceptual framework for implementation fidelity. Implement Sci. 2007;2(1):40. https://doi.org/10.1186/1748-5908-2-40 .

Borrelli B. The assessment, monitoring, and enhancement of treatment fidelity in public health clinical trials. J Public Health Dent. 2011;71(s1):S52-s63. https://doi.org/10.1111/j.1752-7325.2011.00233.x .

McCleary N, Andrews A, Buelo A, Captieux M, Morrow S, Wiener-Ogilvie S, Fletcher M, Steed L, Taylor SJC, Pinnock H. IMP2ART systematic review of education for healthcare professionals implementing supported self-management for asthma. npj Primary Care Respiratory Medicine 2018, 28 (1), 42. https://doi.org/10.1038/s41533-018-0108-4 .

Moore GF, Evans RE, Hawkins J, Littlecott H, Melendez-Torres GJ, Bonell C, Murphy S. From complex social interventions to interventions in complex social systems: future directions and unresolved questions for intervention development and evaluation. Evaluation. 2019;25(1):23–45. https://doi.org/10.1177/1356389018803219 .

Grant A, Bugge C, Wells M. Designing process evaluations using case study to explore the context of complex interventions evaluated in trials. Trials. 2020;21(1):982. https://doi.org/10.1186/s13063-020-04880-4 .

McIntyre SA, Francis JJ, Gould NJ, Lorencatto F. The use of theory in process evaluations conducted alongside randomized trials of implementation interventions: a systematic review. Translational Behavioral Medicine. 2018;10(1):168–78. https://doi.org/10.1093/tbm/iby110(acccessed6/28/2022) .

Paparini S, Papoutsi C, Murdoch J, Green J, Petticrew M, Greenhalgh T, Shaw SE. Evaluating complex interventions in context: systematic, meta-narrative review of case study approaches. BMC Med Res Methodol. 2021;21(1):225. https://doi.org/10.1186/s12874-021-01418-3 .

Fletcher AJ. Applying critical realism in qualitative research: methodology meets method. Int J Soc Res Methodol. 2017;20(2):181–94. https://doi.org/10.1080/13645579.2016.1144401 .

Download references

Acknowledgements

We are grateful for the contributions of the Asthma UK Centre for Applied Research patient panel and IMP 2 ART Programme Group for their input into the design of the process evaluation by providing feedback on plans for the process evaluation as it developed.

The National Institute for Health and Care Research (NIHR) Programme Grants for Applied Research (reference number RP-PG-1016–20008). JS and ST are supported by NIHR ARC North Thames. The views expressed are those of the authors and not necessarily those of the NIHR or the Department of Health and Social Care. The funder had no role in the design of this study and will not have any role during its execution, analyses, interpretation of the data or decision to publish. The Asthma UK Centre for Applied Research (reference Asthma UK: AC-2012–01) funded some pre-grant work on the theoretical development of the implementation strategy. Education for Health developed the education modules, and Optimum Patient Care developed the templates and audit and feedback components of the implementation strategy.

Author information

Authors and affiliations.

Institute of Epidemiology and Health Care, UCL, London, WC1E 6BT, UK

J. Sheringham & N. J. Fulop

Wolfson Institute of Population Health, Queen Mary University of London, London, UK

L. Steed, A. Barat & S. J. C. Taylor

University of Dundee, Dundee, UK

K. McClatchey

School of Health and Related Research, The University of Sheffield, Sheffield, UK

Asthma UK Centre for Applied Research, Usher Institute, University of Edinburgh, Edinburgh, UK

V. Hammersley, V. Marsh & H. Pinnock

You can also search for this author in PubMed   Google Scholar

Contributions

HP, SJCT, KM, VM, VH, NJF, JS and LS contributed to the design of the study. JS drafted the manuscript. LS, HP, SJCT, KM, VH, VM, NJF, AB and BD contributed to the drafts of the manuscript. All authors read, commented on and approved the final manuscript.

Corresponding author

Correspondence to J. Sheringham .

Ethics declarations

Consent for publication.

Not applicable. This protocol contains no baseline or pilot data.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

13063_2024_8179_moesm1_esm.docx.

Additional file 1. Data collection tools. a. Practice staff interview schedule (case study – early). b. General observation form (case study). c. Post-workshop facilitator observation form.

13063_2024_8179_MOESM2_ESM.docx

Additional file 2. Examples of potential allied process evaluation projects to enable additional exploration of IMP2ART’s delivery, response and context.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Sheringham, J., Steed, L., McClatchey, K. et al. Understanding the processes underpinning IMPlementing IMProved Asthma self-management as RouTine (IMP 2 ART) in primary care: study protocol for a process evaluation within a cluster randomised controlled implementation trial. Trials 25 , 359 (2024). https://doi.org/10.1186/s13063-024-08179-6

Download citation

Received : 11 February 2024

Accepted : 16 May 2024

Published : 04 June 2024

DOI : https://doi.org/10.1186/s13063-024-08179-6

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Process evaluation
• Implementation
• Primary care
• Self-management

ISSN: 1745-6215

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]