research design about qualitative research

Chapter 2. Research Design

Getting started.

When I teach undergraduates qualitative research methods, the final product of the course is a “research proposal” that incorporates all they have learned and enlists the knowledge they have learned about qualitative research methods in an original design that addresses a particular research question. I highly recommend you think about designing your own research study as you progress through this textbook. Even if you don’t have a study in mind yet, it can be a helpful exercise as you progress through the course. But how to start? How can one design a research study before they even know what research looks like? This chapter will serve as a brief overview of the research design process to orient you to what will be coming in later chapters. Think of it as a “skeleton” of what you will read in more detail in later chapters. Ideally, you will read this chapter both now (in sequence) and later during your reading of the remainder of the text. Do not worry if you have questions the first time you read this chapter. Many things will become clearer as the text advances and as you gain a deeper understanding of all the components of good qualitative research. This is just a preliminary map to get you on the right road.

Research Design Steps

Before you even get started, you will need to have a broad topic of interest in mind. [1] . In my experience, students can confuse this broad topic with the actual research question, so it is important to clearly distinguish the two. And the place to start is the broad topic. It might be, as was the case with me, working-class college students. But what about working-class college students? What’s it like to be one? Why are there so few compared to others? How do colleges assist (or fail to assist) them? What interested me was something I could barely articulate at first and went something like this: “Why was it so difficult and lonely to be me?” And by extension, “Did others share this experience?”

Once you have a general topic, reflect on why this is important to you. Sometimes we connect with a topic and we don’t really know why. Even if you are not willing to share the real underlying reason you are interested in a topic, it is important that you know the deeper reasons that motivate you. Otherwise, it is quite possible that at some point during the research, you will find yourself turned around facing the wrong direction. I have seen it happen many times. The reason is that the research question is not the same thing as the general topic of interest, and if you don’t know the reasons for your interest, you are likely to design a study answering a research question that is beside the point—to you, at least. And this means you will be much less motivated to carry your research to completion.

Researcher Note

Why do you employ qualitative research methods in your area of study? What are the advantages of qualitative research methods for studying mentorship?

Qualitative research methods are a huge opportunity to increase access, equity, inclusion, and social justice. Qualitative research allows us to engage and examine the uniquenesses/nuances within minoritized and dominant identities and our experiences with these identities. Qualitative research allows us to explore a specific topic, and through that exploration, we can link history to experiences and look for patterns or offer up a unique phenomenon. There’s such beauty in being able to tell a particular story, and qualitative research is a great mode for that! For our work, we examined the relationships we typically use the term mentorship for but didn’t feel that was quite the right word. Qualitative research allowed us to pick apart what we did and how we engaged in our relationships, which then allowed us to more accurately describe what was unique about our mentorship relationships, which we ultimately named liberationships ( McAloney and Long 2021) . Qualitative research gave us the means to explore, process, and name our experiences; what a powerful tool!

How do you come up with ideas for what to study (and how to study it)? Where did you get the idea for studying mentorship?

Coming up with ideas for research, for me, is kind of like Googling a question I have, not finding enough information, and then deciding to dig a little deeper to get the answer. The idea to study mentorship actually came up in conversation with my mentorship triad. We were talking in one of our meetings about our relationship—kind of meta, huh? We discussed how we felt that mentorship was not quite the right term for the relationships we had built. One of us asked what was different about our relationships and mentorship. This all happened when I was taking an ethnography course. During the next session of class, we were discussing auto- and duoethnography, and it hit me—let’s explore our version of mentorship, which we later went on to name liberationships ( McAloney and Long 2021 ). The idea and questions came out of being curious and wanting to find an answer. As I continue to research, I see opportunities in questions I have about my work or during conversations that, in our search for answers, end up exposing gaps in the literature. If I can’t find the answer already out there, I can study it.

—Kim McAloney, PhD, College Student Services Administration Ecampus coordinator and instructor

When you have a better idea of why you are interested in what it is that interests you, you may be surprised to learn that the obvious approaches to the topic are not the only ones. For example, let’s say you think you are interested in preserving coastal wildlife. And as a social scientist, you are interested in policies and practices that affect the long-term viability of coastal wildlife, especially around fishing communities. It would be natural then to consider designing a research study around fishing communities and how they manage their ecosystems. But when you really think about it, you realize that what interests you the most is how people whose livelihoods depend on a particular resource act in ways that deplete that resource. Or, even deeper, you contemplate the puzzle, “How do people justify actions that damage their surroundings?” Now, there are many ways to design a study that gets at that broader question, and not all of them are about fishing communities, although that is certainly one way to go. Maybe you could design an interview-based study that includes and compares loggers, fishers, and desert golfers (those who golf in arid lands that require a great deal of wasteful irrigation). Or design a case study around one particular example where resources were completely used up by a community. Without knowing what it is you are really interested in, what motivates your interest in a surface phenomenon, you are unlikely to come up with the appropriate research design.

These first stages of research design are often the most difficult, but have patience . Taking the time to consider why you are going to go through a lot of trouble to get answers will prevent a lot of wasted energy in the future.

There are distinct reasons for pursuing particular research questions, and it is helpful to distinguish between them.  First, you may be personally motivated.  This is probably the most important and the most often overlooked.   What is it about the social world that sparks your curiosity? What bothers you? What answers do you need in order to keep living? For me, I knew I needed to get a handle on what higher education was for before I kept going at it. I needed to understand why I felt so different from my peers and whether this whole “higher education” thing was “for the likes of me” before I could complete my degree. That is the personal motivation question. Your personal motivation might also be political in nature, in that you want to change the world in a particular way. It’s all right to acknowledge this. In fact, it is better to acknowledge it than to hide it.

There are also academic and professional motivations for a particular study.  If you are an absolute beginner, these may be difficult to find. We’ll talk more about this when we discuss reviewing the literature. Simply put, you are probably not the only person in the world to have thought about this question or issue and those related to it. So how does your interest area fit into what others have studied? Perhaps there is a good study out there of fishing communities, but no one has quite asked the “justification” question. You are motivated to address this to “fill the gap” in our collective knowledge. And maybe you are really not at all sure of what interests you, but you do know that [insert your topic] interests a lot of people, so you would like to work in this area too. You want to be involved in the academic conversation. That is a professional motivation and a very important one to articulate.

Practical and strategic motivations are a third kind. Perhaps you want to encourage people to take better care of the natural resources around them. If this is also part of your motivation, you will want to design your research project in a way that might have an impact on how people behave in the future. There are many ways to do this, one of which is using qualitative research methods rather than quantitative research methods, as the findings of qualitative research are often easier to communicate to a broader audience than the results of quantitative research. You might even be able to engage the community you are studying in the collecting and analyzing of data, something taboo in quantitative research but actively embraced and encouraged by qualitative researchers. But there are other practical reasons, such as getting “done” with your research in a certain amount of time or having access (or no access) to certain information. There is nothing wrong with considering constraints and opportunities when designing your study. Or maybe one of the practical or strategic goals is about learning competence in this area so that you can demonstrate the ability to conduct interviews and focus groups with future employers. Keeping that in mind will help shape your study and prevent you from getting sidetracked using a technique that you are less invested in learning about.

STOP HERE for a moment

I recommend you write a paragraph (at least) explaining your aims and goals. Include a sentence about each of the following: personal/political goals, practical or professional/academic goals, and practical/strategic goals. Think through how all of the goals are related and can be achieved by this particular research study . If they can’t, have a rethink. Perhaps this is not the best way to go about it.

You will also want to be clear about the purpose of your study. “Wait, didn’t we just do this?” you might ask. No! Your goals are not the same as the purpose of the study, although they are related. You can think about purpose lying on a continuum from “ theory ” to “action” (figure 2.1). Sometimes you are doing research to discover new knowledge about the world, while other times you are doing a study because you want to measure an impact or make a difference in the world.

Basic research involves research that is done for the sake of “pure” knowledge—that is, knowledge that, at least at this moment in time, may not have any apparent use or application. Often, and this is very important, knowledge of this kind is later found to be extremely helpful in solving problems. So one way of thinking about basic research is that it is knowledge for which no use is yet known but will probably one day prove to be extremely useful. If you are doing basic research, you do not need to argue its usefulness, as the whole point is that we just don’t know yet what this might be.

Researchers engaged in basic research want to understand how the world operates. They are interested in investigating a phenomenon to get at the nature of reality with regard to that phenomenon. The basic researcher’s purpose is to understand and explain ( Patton 2002:215 ).

Basic research is interested in generating and testing hypotheses about how the world works. Grounded Theory is one approach to qualitative research methods that exemplifies basic research (see chapter 4). Most academic journal articles publish basic research findings. If you are working in academia (e.g., writing your dissertation), the default expectation is that you are conducting basic research.

Applied research in the social sciences is research that addresses human and social problems. Unlike basic research, the researcher has expectations that the research will help contribute to resolving a problem, if only by identifying its contours, history, or context. From my experience, most students have this as their baseline assumption about research. Why do a study if not to make things better? But this is a common mistake. Students and their committee members are often working with default assumptions here—the former thinking about applied research as their purpose, the latter thinking about basic research: “The purpose of applied research is to contribute knowledge that will help people to understand the nature of a problem in order to intervene, thereby allowing human beings to more effectively control their environment. While in basic research the source of questions is the tradition within a scholarly discipline, in applied research the source of questions is in the problems and concerns experienced by people and by policymakers” ( Patton 2002:217 ).

Applied research is less geared toward theory in two ways. First, its questions do not derive from previous literature. For this reason, applied research studies have much more limited literature reviews than those found in basic research (although they make up for this by having much more “background” about the problem). Second, it does not generate theory in the same way as basic research does. The findings of an applied research project may not be generalizable beyond the boundaries of this particular problem or context. The findings are more limited. They are useful now but may be less useful later. This is why basic research remains the default “gold standard” of academic research.

Evaluation research is research that is designed to evaluate or test the effectiveness of specific solutions and programs addressing specific social problems. We already know the problems, and someone has already come up with solutions. There might be a program, say, for first-generation college students on your campus. Does this program work? Are first-generation students who participate in the program more likely to graduate than those who do not? These are the types of questions addressed by evaluation research. There are two types of research within this broader frame; however, one more action-oriented than the next. In summative evaluation , an overall judgment about the effectiveness of a program or policy is made. Should we continue our first-gen program? Is it a good model for other campuses? Because the purpose of such summative evaluation is to measure success and to determine whether this success is scalable (capable of being generalized beyond the specific case), quantitative data is more often used than qualitative data. In our example, we might have “outcomes” data for thousands of students, and we might run various tests to determine if the better outcomes of those in the program are statistically significant so that we can generalize the findings and recommend similar programs elsewhere. Qualitative data in the form of focus groups or interviews can then be used for illustrative purposes, providing more depth to the quantitative analyses. In contrast, formative evaluation attempts to improve a program or policy (to help “form” or shape its effectiveness). Formative evaluations rely more heavily on qualitative data—case studies, interviews, focus groups. The findings are meant not to generalize beyond the particular but to improve this program. If you are a student seeking to improve your qualitative research skills and you do not care about generating basic research, formative evaluation studies might be an attractive option for you to pursue, as there are always local programs that need evaluation and suggestions for improvement. Again, be very clear about your purpose when talking through your research proposal with your committee.

Action research takes a further step beyond evaluation, even formative evaluation, to being part of the solution itself. This is about as far from basic research as one could get and definitely falls beyond the scope of “science,” as conventionally defined. The distinction between action and research is blurry, the research methods are often in constant flux, and the only “findings” are specific to the problem or case at hand and often are findings about the process of intervention itself. Rather than evaluate a program as a whole, action research often seeks to change and improve some particular aspect that may not be working—maybe there is not enough diversity in an organization or maybe women’s voices are muted during meetings and the organization wonders why and would like to change this. In a further step, participatory action research , those women would become part of the research team, attempting to amplify their voices in the organization through participation in the action research. As action research employs methods that involve people in the process, focus groups are quite common.

If you are working on a thesis or dissertation, chances are your committee will expect you to be contributing to fundamental knowledge and theory ( basic research ). If your interests lie more toward the action end of the continuum, however, it is helpful to talk to your committee about this before you get started. Knowing your purpose in advance will help avoid misunderstandings during the later stages of the research process!

The Research Question

Once you have written your paragraph and clarified your purpose and truly know that this study is the best study for you to be doing right now , you are ready to write and refine your actual research question. Know that research questions are often moving targets in qualitative research, that they can be refined up to the very end of data collection and analysis. But you do have to have a working research question at all stages. This is your “anchor” when you get lost in the data. What are you addressing? What are you looking at and why? Your research question guides you through the thicket. It is common to have a whole host of questions about a phenomenon or case, both at the outset and throughout the study, but you should be able to pare it down to no more than two or three sentences when asked. These sentences should both clarify the intent of the research and explain why this is an important question to answer. More on refining your research question can be found in chapter 4.

Chances are, you will have already done some prior reading before coming up with your interest and your questions, but you may not have conducted a systematic literature review. This is the next crucial stage to be completed before venturing further. You don’t want to start collecting data and then realize that someone has already beaten you to the punch. A review of the literature that is already out there will let you know (1) if others have already done the study you are envisioning; (2) if others have done similar studies, which can help you out; and (3) what ideas or concepts are out there that can help you frame your study and make sense of your findings. More on literature reviews can be found in chapter 9.

In addition to reviewing the literature for similar studies to what you are proposing, it can be extremely helpful to find a study that inspires you. This may have absolutely nothing to do with the topic you are interested in but is written so beautifully or organized so interestingly or otherwise speaks to you in such a way that you want to post it somewhere to remind you of what you want to be doing. You might not understand this in the early stages—why would you find a study that has nothing to do with the one you are doing helpful? But trust me, when you are deep into analysis and writing, having an inspirational model in view can help you push through. If you are motivated to do something that might change the world, you probably have read something somewhere that inspired you. Go back to that original inspiration and read it carefully and see how they managed to convey the passion that you so appreciate.

At this stage, you are still just getting started. There are a lot of things to do before setting forth to collect data! You’ll want to consider and choose a research tradition and a set of data-collection techniques that both help you answer your research question and match all your aims and goals. For example, if you really want to help migrant workers speak for themselves, you might draw on feminist theory and participatory action research models. Chapters 3 and 4 will provide you with more information on epistemologies and approaches.

Next, you have to clarify your “units of analysis.” What is the level at which you are focusing your study? Often, the unit in qualitative research methods is individual people, or “human subjects.” But your units of analysis could just as well be organizations (colleges, hospitals) or programs or even whole nations. Think about what it is you want to be saying at the end of your study—are the insights you are hoping to make about people or about organizations or about something else entirely? A unit of analysis can even be a historical period! Every unit of analysis will call for a different kind of data collection and analysis and will produce different kinds of “findings” at the conclusion of your study. [2]

Regardless of what unit of analysis you select, you will probably have to consider the “human subjects” involved in your research. [3] Who are they? What interactions will you have with them—that is, what kind of data will you be collecting? Before answering these questions, define your population of interest and your research setting. Use your research question to help guide you.

Let’s use an example from a real study. In Geographies of Campus Inequality , Benson and Lee ( 2020 ) list three related research questions: “(1) What are the different ways that first-generation students organize their social, extracurricular, and academic activities at selective and highly selective colleges? (2) how do first-generation students sort themselves and get sorted into these different types of campus lives; and (3) how do these different patterns of campus engagement prepare first-generation students for their post-college lives?” (3).

Note that we are jumping into this a bit late, after Benson and Lee have described previous studies (the literature review) and what is known about first-generation college students and what is not known. They want to know about differences within this group, and they are interested in ones attending certain kinds of colleges because those colleges will be sites where academic and extracurricular pressures compete. That is the context for their three related research questions. What is the population of interest here? First-generation college students . What is the research setting? Selective and highly selective colleges . But a host of questions remain. Which students in the real world, which colleges? What about gender, race, and other identity markers? Will the students be asked questions? Are the students still in college, or will they be asked about what college was like for them? Will they be observed? Will they be shadowed? Will they be surveyed? Will they be asked to keep diaries of their time in college? How many students? How many colleges? For how long will they be observed?

Recommendation

Take a moment and write down suggestions for Benson and Lee before continuing on to what they actually did.

Have you written down your own suggestions? Good. Now let’s compare those with what they actually did. Benson and Lee drew on two sources of data: in-depth interviews with sixty-four first-generation students and survey data from a preexisting national survey of students at twenty-eight selective colleges. Let’s ignore the survey for our purposes here and focus on those interviews. The interviews were conducted between 2014 and 2016 at a single selective college, “Hilltop” (a pseudonym ). They employed a “purposive” sampling strategy to ensure an equal number of male-identifying and female-identifying students as well as equal numbers of White, Black, and Latinx students. Each student was interviewed once. Hilltop is a selective liberal arts college in the northeast that enrolls about three thousand students.

How did your suggestions match up to those actually used by the researchers in this study? It is possible your suggestions were too ambitious? Beginning qualitative researchers can often make that mistake. You want a research design that is both effective (it matches your question and goals) and doable. You will never be able to collect data from your entire population of interest (unless your research question is really so narrow to be relevant to very few people!), so you will need to come up with a good sample. Define the criteria for this sample, as Benson and Lee did when deciding to interview an equal number of students by gender and race categories. Define the criteria for your sample setting too. Hilltop is typical for selective colleges. That was a research choice made by Benson and Lee. For more on sampling and sampling choices, see chapter 5.

Benson and Lee chose to employ interviews. If you also would like to include interviews, you have to think about what will be asked in them. Most interview-based research involves an interview guide, a set of questions or question areas that will be asked of each participant. The research question helps you create a relevant interview guide. You want to ask questions whose answers will provide insight into your research question. Again, your research question is the anchor you will continually come back to as you plan for and conduct your study. It may be that once you begin interviewing, you find that people are telling you something totally unexpected, and this makes you rethink your research question. That is fine. Then you have a new anchor. But you always have an anchor. More on interviewing can be found in chapter 11.

Let’s imagine Benson and Lee also observed college students as they went about doing the things college students do, both in the classroom and in the clubs and social activities in which they participate. They would have needed a plan for this. Would they sit in on classes? Which ones and how many? Would they attend club meetings and sports events? Which ones and how many? Would they participate themselves? How would they record their observations? More on observation techniques can be found in both chapters 13 and 14.

At this point, the design is almost complete. You know why you are doing this study, you have a clear research question to guide you, you have identified your population of interest and research setting, and you have a reasonable sample of each. You also have put together a plan for data collection, which might include drafting an interview guide or making plans for observations. And so you know exactly what you will be doing for the next several months (or years!). To put the project into action, there are a few more things necessary before actually going into the field.

First, you will need to make sure you have any necessary supplies, including recording technology. These days, many researchers use their phones to record interviews. Second, you will need to draft a few documents for your participants. These include informed consent forms and recruiting materials, such as posters or email texts, that explain what this study is in clear language. Third, you will draft a research protocol to submit to your institutional review board (IRB) ; this research protocol will include the interview guide (if you are using one), the consent form template, and all examples of recruiting material. Depending on your institution and the details of your study design, it may take weeks or even, in some unfortunate cases, months before you secure IRB approval. Make sure you plan on this time in your project timeline. While you wait, you can continue to review the literature and possibly begin drafting a section on the literature review for your eventual presentation/publication. More on IRB procedures can be found in chapter 8 and more general ethical considerations in chapter 7.

Once you have approval, you can begin!

Research Design Checklist

Before data collection begins, do the following:

• Write a paragraph explaining your aims and goals (personal/political, practical/strategic, professional/academic).
• Define your research question; write two to three sentences that clarify the intent of the research and why this is an important question to answer.
• Review the literature for similar studies that address your research question or similar research questions; think laterally about some literature that might be helpful or illuminating but is not exactly about the same topic.
• Find a written study that inspires you—it may or may not be on the research question you have chosen.
• Consider and choose a research tradition and set of data-collection techniques that (1) help answer your research question and (2) match your aims and goals.
• Define your population of interest and your research setting.
• Define the criteria for your sample (How many? Why these? How will you find them, gain access, and acquire consent?).
• If you are conducting interviews, draft an interview guide.
•  If you are making observations, create a plan for observations (sites, times, recording, access).
• Acquire any necessary technology (recording devices/software).
• Draft consent forms that clearly identify the research focus and selection process.
• Create recruiting materials (posters, email, texts).
• Apply for IRB approval (proposal plus consent form plus recruiting materials).
• Block out time for collecting data.
• At the end of the chapter, you will find a " Research Design Checklist " that summarizes the main recommendations made here ↵
• For example, if your focus is society and culture , you might collect data through observation or a case study. If your focus is individual lived experience , you are probably going to be interviewing some people. And if your focus is language and communication , you will probably be analyzing text (written or visual). ( Marshall and Rossman 2016:16 ). ↵
• You may not have any "live" human subjects. There are qualitative research methods that do not require interactions with live human beings - see chapter 16 , "Archival and Historical Sources." But for the most part, you are probably reading this textbook because you are interested in doing research with people. The rest of the chapter will assume this is the case. ↵

One of the primary methodological traditions of inquiry in qualitative research, ethnography is the study of a group or group culture, largely through observational fieldwork supplemented by interviews. It is a form of fieldwork that may include participant-observation data collection. See chapter 14 for a discussion of deep ethnography. 

A methodological tradition of inquiry and research design that focuses on an individual case (e.g., setting, institution, or sometimes an individual) in order to explore its complexity, history, and interactive parts.  As an approach, it is particularly useful for obtaining a deep appreciation of an issue, event, or phenomenon of interest in its particular context.

The controlling force in research; can be understood as lying on a continuum from basic research (knowledge production) to action research (effecting change).

In its most basic sense, a theory is a story we tell about how the world works that can be tested with empirical evidence.  In qualitative research, we use the term in a variety of ways, many of which are different from how they are used by quantitative researchers.  Although some qualitative research can be described as “testing theory,” it is more common to “build theory” from the data using inductive reasoning , as done in Grounded Theory .  There are so-called “grand theories” that seek to integrate a whole series of findings and stories into an overarching paradigm about how the world works, and much smaller theories or concepts about particular processes and relationships.  Theory can even be used to explain particular methodological perspectives or approaches, as in Institutional Ethnography , which is both a way of doing research and a theory about how the world works.

Research that is interested in generating and testing hypotheses about how the world works.

A methodological tradition of inquiry and approach to analyzing qualitative data in which theories emerge from a rigorous and systematic process of induction.  This approach was pioneered by the sociologists Glaser and Strauss (1967).  The elements of theory generated from comparative analysis of data are, first, conceptual categories and their properties and, second, hypotheses or generalized relations among the categories and their properties – “The constant comparing of many groups draws the [researcher’s] attention to their many similarities and differences.  Considering these leads [the researcher] to generate abstract categories and their properties, which, since they emerge from the data, will clearly be important to a theory explaining the kind of behavior under observation.” (36).

An approach to research that is “multimethod in focus, involving an interpretative, naturalistic approach to its subject matter.  This means that qualitative researchers study things in their natural settings, attempting to make sense of, or interpret, phenomena in terms of the meanings people bring to them.  Qualitative research involves the studied use and collection of a variety of empirical materials – case study, personal experience, introspective, life story, interview, observational, historical, interactional, and visual texts – that describe routine and problematic moments and meanings in individuals’ lives." ( Denzin and Lincoln 2005:2 ). Contrast with quantitative research .

Research that contributes knowledge that will help people to understand the nature of a problem in order to intervene, thereby allowing human beings to more effectively control their environment.

Research that is designed to evaluate or test the effectiveness of specific solutions and programs addressing specific social problems.  There are two kinds: summative and formative .

Research in which an overall judgment about the effectiveness of a program or policy is made, often for the purpose of generalizing to other cases or programs.  Generally uses qualitative research as a supplement to primary quantitative data analyses.  Contrast formative evaluation research .

Research designed to improve a program or policy (to help “form” or shape its effectiveness); relies heavily on qualitative research methods.  Contrast summative evaluation research

Research carried out at a particular organizational or community site with the intention of affecting change; often involves research subjects as participants of the study.  See also participatory action research .

Research in which both researchers and participants work together to understand a problematic situation and change it for the better.

The level of the focus of analysis (e.g., individual people, organizations, programs, neighborhoods).

The large group of interest to the researcher.  Although it will likely be impossible to design a study that incorporates or reaches all members of the population of interest, this should be clearly defined at the outset of a study so that a reasonable sample of the population can be taken.  For example, if one is studying working-class college students, the sample may include twenty such students attending a particular college, while the population is “working-class college students.”  In quantitative research, clearly defining the general population of interest is a necessary step in generalizing results from a sample.  In qualitative research, defining the population is conceptually important for clarity.

A fictional name assigned to give anonymity to a person, group, or place.  Pseudonyms are important ways of protecting the identity of research participants while still providing a “human element” in the presentation of qualitative data.  There are ethical considerations to be made in selecting pseudonyms; some researchers allow research participants to choose their own.

A requirement for research involving human participants; the documentation of informed consent.  In some cases, oral consent or assent may be sufficient, but the default standard is a single-page easy-to-understand form that both the researcher and the participant sign and date.   Under federal guidelines, all researchers "shall seek such consent only under circumstances that provide the prospective subject or the representative sufficient opportunity to consider whether or not to participate and that minimize the possibility of coercion or undue influence. The information that is given to the subject or the representative shall be in language understandable to the subject or the representative.  No informed consent, whether oral or written, may include any exculpatory language through which the subject or the representative is made to waive or appear to waive any of the subject's rights or releases or appears to release the investigator, the sponsor, the institution, or its agents from liability for negligence" (21 CFR 50.20).  Your IRB office will be able to provide a template for use in your study .

An administrative body established to protect the rights and welfare of human research subjects recruited to participate in research activities conducted under the auspices of the institution with which it is affiliated. The IRB is charged with the responsibility of reviewing all research involving human participants. The IRB is concerned with protecting the welfare, rights, and privacy of human subjects. The IRB has the authority to approve, disapprove, monitor, and require modifications in all research activities that fall within its jurisdiction as specified by both the federal regulations and institutional policy.

Introduction to Qualitative Research Methods Copyright © 2023 by Allison Hurst is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License , except where otherwise noted.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Neurol Res Pract

How to use and assess qualitative research methods

Loraine busetto.

1 Department of Neurology, Heidelberg University Hospital, Im Neuenheimer Feld 400, 69120 Heidelberg, Germany

Wolfgang Wick

2 Clinical Cooperation Unit Neuro-Oncology, German Cancer Research Center, Heidelberg, Germany

Christoph Gumbinger

Associated data.

Not applicable.

This paper aims to provide an overview of the use and assessment of qualitative research methods in the health sciences. Qualitative research can be defined as the study of the nature of phenomena and is especially appropriate for answering questions of why something is (not) observed, assessing complex multi-component interventions, and focussing on intervention improvement. The most common methods of data collection are document study, (non-) participant observations, semi-structured interviews and focus groups. For data analysis, field-notes and audio-recordings are transcribed into protocols and transcripts, and coded using qualitative data management software. Criteria such as checklists, reflexivity, sampling strategies, piloting, co-coding, member-checking and stakeholder involvement can be used to enhance and assess the quality of the research conducted. Using qualitative in addition to quantitative designs will equip us with better tools to address a greater range of research problems, and to fill in blind spots in current neurological research and practice.

The aim of this paper is to provide an overview of qualitative research methods, including hands-on information on how they can be used, reported and assessed. This article is intended for beginning qualitative researchers in the health sciences as well as experienced quantitative researchers who wish to broaden their understanding of qualitative research.

What is qualitative research?

Qualitative research is defined as “the study of the nature of phenomena”, including “their quality, different manifestations, the context in which they appear or the perspectives from which they can be perceived” , but excluding “their range, frequency and place in an objectively determined chain of cause and effect” [ 1 ]. This formal definition can be complemented with a more pragmatic rule of thumb: qualitative research generally includes data in form of words rather than numbers [ 2 ].

Why conduct qualitative research?

Because some research questions cannot be answered using (only) quantitative methods. For example, one Australian study addressed the issue of why patients from Aboriginal communities often present late or not at all to specialist services offered by tertiary care hospitals. Using qualitative interviews with patients and staff, it found one of the most significant access barriers to be transportation problems, including some towns and communities simply not having a bus service to the hospital [ 3 ]. A quantitative study could have measured the number of patients over time or even looked at possible explanatory factors – but only those previously known or suspected to be of relevance. To discover reasons for observed patterns, especially the invisible or surprising ones, qualitative designs are needed.

While qualitative research is common in other fields, it is still relatively underrepresented in health services research. The latter field is more traditionally rooted in the evidence-based-medicine paradigm, as seen in " research that involves testing the effectiveness of various strategies to achieve changes in clinical practice, preferably applying randomised controlled trial study designs (...) " [ 4 ]. This focus on quantitative research and specifically randomised controlled trials (RCT) is visible in the idea of a hierarchy of research evidence which assumes that some research designs are objectively better than others, and that choosing a "lesser" design is only acceptable when the better ones are not practically or ethically feasible [ 5 , 6 ]. Others, however, argue that an objective hierarchy does not exist, and that, instead, the research design and methods should be chosen to fit the specific research question at hand – "questions before methods" [ 2 , 7 – 9 ]. This means that even when an RCT is possible, some research problems require a different design that is better suited to addressing them. Arguing in JAMA, Berwick uses the example of rapid response teams in hospitals, which he describes as " a complex, multicomponent intervention – essentially a process of social change" susceptible to a range of different context factors including leadership or organisation history. According to him, "[in] such complex terrain, the RCT is an impoverished way to learn. Critics who use it as a truth standard in this context are incorrect" [ 8 ] . Instead of limiting oneself to RCTs, Berwick recommends embracing a wider range of methods , including qualitative ones, which for "these specific applications, (...) are not compromises in learning how to improve; they are superior" [ 8 ].

Research problems that can be approached particularly well using qualitative methods include assessing complex multi-component interventions or systems (of change), addressing questions beyond “what works”, towards “what works for whom when, how and why”, and focussing on intervention improvement rather than accreditation [ 7 , 9 – 12 ]. Using qualitative methods can also help shed light on the “softer” side of medical treatment. For example, while quantitative trials can measure the costs and benefits of neuro-oncological treatment in terms of survival rates or adverse effects, qualitative research can help provide a better understanding of patient or caregiver stress, visibility of illness or out-of-pocket expenses.

How to conduct qualitative research?

Given that qualitative research is characterised by flexibility, openness and responsivity to context, the steps of data collection and analysis are not as separate and consecutive as they tend to be in quantitative research [ 13 , 14 ]. As Fossey puts it : “sampling, data collection, analysis and interpretation are related to each other in a cyclical (iterative) manner, rather than following one after another in a stepwise approach” [ 15 ]. The researcher can make educated decisions with regard to the choice of method, how they are implemented, and to which and how many units they are applied [ 13 ]. As shown in Fig.  1 , this can involve several back-and-forth steps between data collection and analysis where new insights and experiences can lead to adaption and expansion of the original plan. Some insights may also necessitate a revision of the research question and/or the research design as a whole. The process ends when saturation is achieved, i.e. when no relevant new information can be found (see also below: sampling and saturation). For reasons of transparency, it is essential for all decisions as well as the underlying reasoning to be well-documented.

Iterative research process

While it is not always explicitly addressed, qualitative methods reflect a different underlying research paradigm than quantitative research (e.g. constructivism or interpretivism as opposed to positivism). The choice of methods can be based on the respective underlying substantive theory or theoretical framework used by the researcher [ 2 ].

Data collection

The methods of qualitative data collection most commonly used in health research are document study, observations, semi-structured interviews and focus groups [ 1 , 14 , 16 , 17 ].

Document study

Document study (also called document analysis) refers to the review by the researcher of written materials [ 14 ]. These can include personal and non-personal documents such as archives, annual reports, guidelines, policy documents, diaries or letters.

Observations

Observations are particularly useful to gain insights into a certain setting and actual behaviour – as opposed to reported behaviour or opinions [ 13 ]. Qualitative observations can be either participant or non-participant in nature. In participant observations, the observer is part of the observed setting, for example a nurse working in an intensive care unit [ 18 ]. In non-participant observations, the observer is “on the outside looking in”, i.e. present in but not part of the situation, trying not to influence the setting by their presence. Observations can be planned (e.g. for 3 h during the day or night shift) or ad hoc (e.g. as soon as a stroke patient arrives at the emergency room). During the observation, the observer takes notes on everything or certain pre-determined parts of what is happening around them, for example focusing on physician-patient interactions or communication between different professional groups. Written notes can be taken during or after the observations, depending on feasibility (which is usually lower during participant observations) and acceptability (e.g. when the observer is perceived to be judging the observed). Afterwards, these field notes are transcribed into observation protocols. If more than one observer was involved, field notes are taken independently, but notes can be consolidated into one protocol after discussions. Advantages of conducting observations include minimising the distance between the researcher and the researched, the potential discovery of topics that the researcher did not realise were relevant and gaining deeper insights into the real-world dimensions of the research problem at hand [ 18 ].

Semi-structured interviews

Hijmans & Kuyper describe qualitative interviews as “an exchange with an informal character, a conversation with a goal” [ 19 ]. Interviews are used to gain insights into a person’s subjective experiences, opinions and motivations – as opposed to facts or behaviours [ 13 ]. Interviews can be distinguished by the degree to which they are structured (i.e. a questionnaire), open (e.g. free conversation or autobiographical interviews) or semi-structured [ 2 , 13 ]. Semi-structured interviews are characterized by open-ended questions and the use of an interview guide (or topic guide/list) in which the broad areas of interest, sometimes including sub-questions, are defined [ 19 ]. The pre-defined topics in the interview guide can be derived from the literature, previous research or a preliminary method of data collection, e.g. document study or observations. The topic list is usually adapted and improved at the start of the data collection process as the interviewer learns more about the field [ 20 ]. Across interviews the focus on the different (blocks of) questions may differ and some questions may be skipped altogether (e.g. if the interviewee is not able or willing to answer the questions or for concerns about the total length of the interview) [ 20 ]. Qualitative interviews are usually not conducted in written format as it impedes on the interactive component of the method [ 20 ]. In comparison to written surveys, qualitative interviews have the advantage of being interactive and allowing for unexpected topics to emerge and to be taken up by the researcher. This can also help overcome a provider or researcher-centred bias often found in written surveys, which by nature, can only measure what is already known or expected to be of relevance to the researcher. Interviews can be audio- or video-taped; but sometimes it is only feasible or acceptable for the interviewer to take written notes [ 14 , 16 , 20 ].

Focus groups

Focus groups are group interviews to explore participants’ expertise and experiences, including explorations of how and why people behave in certain ways [ 1 ]. Focus groups usually consist of 6–8 people and are led by an experienced moderator following a topic guide or “script” [ 21 ]. They can involve an observer who takes note of the non-verbal aspects of the situation, possibly using an observation guide [ 21 ]. Depending on researchers’ and participants’ preferences, the discussions can be audio- or video-taped and transcribed afterwards [ 21 ]. Focus groups are useful for bringing together homogeneous (to a lesser extent heterogeneous) groups of participants with relevant expertise and experience on a given topic on which they can share detailed information [ 21 ]. Focus groups are a relatively easy, fast and inexpensive method to gain access to information on interactions in a given group, i.e. “the sharing and comparing” among participants [ 21 ]. Disadvantages include less control over the process and a lesser extent to which each individual may participate. Moreover, focus group moderators need experience, as do those tasked with the analysis of the resulting data. Focus groups can be less appropriate for discussing sensitive topics that participants might be reluctant to disclose in a group setting [ 13 ]. Moreover, attention must be paid to the emergence of “groupthink” as well as possible power dynamics within the group, e.g. when patients are awed or intimidated by health professionals.

Choosing the “right” method

As explained above, the school of thought underlying qualitative research assumes no objective hierarchy of evidence and methods. This means that each choice of single or combined methods has to be based on the research question that needs to be answered and a critical assessment with regard to whether or to what extent the chosen method can accomplish this – i.e. the “fit” between question and method [ 14 ]. It is necessary for these decisions to be documented when they are being made, and to be critically discussed when reporting methods and results.

Let us assume that our research aim is to examine the (clinical) processes around acute endovascular treatment (EVT), from the patient’s arrival at the emergency room to recanalization, with the aim to identify possible causes for delay and/or other causes for sub-optimal treatment outcome. As a first step, we could conduct a document study of the relevant standard operating procedures (SOPs) for this phase of care – are they up-to-date and in line with current guidelines? Do they contain any mistakes, irregularities or uncertainties that could cause delays or other problems? Regardless of the answers to these questions, the results have to be interpreted based on what they are: a written outline of what care processes in this hospital should look like. If we want to know what they actually look like in practice, we can conduct observations of the processes described in the SOPs. These results can (and should) be analysed in themselves, but also in comparison to the results of the document analysis, especially as regards relevant discrepancies. Do the SOPs outline specific tests for which no equipment can be observed or tasks to be performed by specialized nurses who are not present during the observation? It might also be possible that the written SOP is outdated, but the actual care provided is in line with current best practice. In order to find out why these discrepancies exist, it can be useful to conduct interviews. Are the physicians simply not aware of the SOPs (because their existence is limited to the hospital’s intranet) or do they actively disagree with them or does the infrastructure make it impossible to provide the care as described? Another rationale for adding interviews is that some situations (or all of their possible variations for different patient groups or the day, night or weekend shift) cannot practically or ethically be observed. In this case, it is possible to ask those involved to report on their actions – being aware that this is not the same as the actual observation. A senior physician’s or hospital manager’s description of certain situations might differ from a nurse’s or junior physician’s one, maybe because they intentionally misrepresent facts or maybe because different aspects of the process are visible or important to them. In some cases, it can also be relevant to consider to whom the interviewee is disclosing this information – someone they trust, someone they are otherwise not connected to, or someone they suspect or are aware of being in a potentially “dangerous” power relationship to them. Lastly, a focus group could be conducted with representatives of the relevant professional groups to explore how and why exactly they provide care around EVT. The discussion might reveal discrepancies (between SOPs and actual care or between different physicians) and motivations to the researchers as well as to the focus group members that they might not have been aware of themselves. For the focus group to deliver relevant information, attention has to be paid to its composition and conduct, for example, to make sure that all participants feel safe to disclose sensitive or potentially problematic information or that the discussion is not dominated by (senior) physicians only. The resulting combination of data collection methods is shown in Fig.  2 .

Possible combination of data collection methods

Attributions for icons: “Book” by Serhii Smirnov, “Interview” by Adrien Coquet, FR, “Magnifying Glass” by anggun, ID, “Business communication” by Vectors Market; all from the Noun Project

The combination of multiple data source as described for this example can be referred to as “triangulation”, in which multiple measurements are carried out from different angles to achieve a more comprehensive understanding of the phenomenon under study [ 22 , 23 ].

Data analysis

To analyse the data collected through observations, interviews and focus groups these need to be transcribed into protocols and transcripts (see Fig.  3 ). Interviews and focus groups can be transcribed verbatim , with or without annotations for behaviour (e.g. laughing, crying, pausing) and with or without phonetic transcription of dialects and filler words, depending on what is expected or known to be relevant for the analysis. In the next step, the protocols and transcripts are coded , that is, marked (or tagged, labelled) with one or more short descriptors of the content of a sentence or paragraph [ 2 , 15 , 23 ]. Jansen describes coding as “connecting the raw data with “theoretical” terms” [ 20 ]. In a more practical sense, coding makes raw data sortable. This makes it possible to extract and examine all segments describing, say, a tele-neurology consultation from multiple data sources (e.g. SOPs, emergency room observations, staff and patient interview). In a process of synthesis and abstraction, the codes are then grouped, summarised and/or categorised [ 15 , 20 ]. The end product of the coding or analysis process is a descriptive theory of the behavioural pattern under investigation [ 20 ]. The coding process is performed using qualitative data management software, the most common ones being InVivo, MaxQDA and Atlas.ti. It should be noted that these are data management tools which support the analysis performed by the researcher(s) [ 14 ].

From data collection to data analysis

Attributions for icons: see Fig. ​ Fig.2, 2 , also “Speech to text” by Trevor Dsouza, “Field Notes” by Mike O’Brien, US, “Voice Record” by ProSymbols, US, “Inspection” by Made, AU, and “Cloud” by Graphic Tigers; all from the Noun Project

How to report qualitative research?

Protocols of qualitative research can be published separately and in advance of the study results. However, the aim is not the same as in RCT protocols, i.e. to pre-define and set in stone the research questions and primary or secondary endpoints. Rather, it is a way to describe the research methods in detail, which might not be possible in the results paper given journals’ word limits. Qualitative research papers are usually longer than their quantitative counterparts to allow for deep understanding and so-called “thick description”. In the methods section, the focus is on transparency of the methods used, including why, how and by whom they were implemented in the specific study setting, so as to enable a discussion of whether and how this may have influenced data collection, analysis and interpretation. The results section usually starts with a paragraph outlining the main findings, followed by more detailed descriptions of, for example, the commonalities, discrepancies or exceptions per category [ 20 ]. Here it is important to support main findings by relevant quotations, which may add information, context, emphasis or real-life examples [ 20 , 23 ]. It is subject to debate in the field whether it is relevant to state the exact number or percentage of respondents supporting a certain statement (e.g. “Five interviewees expressed negative feelings towards XYZ”) [ 21 ].

How to combine qualitative with quantitative research?

Qualitative methods can be combined with other methods in multi- or mixed methods designs, which “[employ] two or more different methods [ …] within the same study or research program rather than confining the research to one single method” [ 24 ]. Reasons for combining methods can be diverse, including triangulation for corroboration of findings, complementarity for illustration and clarification of results, expansion to extend the breadth and range of the study, explanation of (unexpected) results generated with one method with the help of another, or offsetting the weakness of one method with the strength of another [ 1 , 17 , 24 – 26 ]. The resulting designs can be classified according to when, why and how the different quantitative and/or qualitative data strands are combined. The three most common types of mixed method designs are the convergent parallel design , the explanatory sequential design and the exploratory sequential design. The designs with examples are shown in Fig.  4 .

Three common mixed methods designs

In the convergent parallel design, a qualitative study is conducted in parallel to and independently of a quantitative study, and the results of both studies are compared and combined at the stage of interpretation of results. Using the above example of EVT provision, this could entail setting up a quantitative EVT registry to measure process times and patient outcomes in parallel to conducting the qualitative research outlined above, and then comparing results. Amongst other things, this would make it possible to assess whether interview respondents’ subjective impressions of patients receiving good care match modified Rankin Scores at follow-up, or whether observed delays in care provision are exceptions or the rule when compared to door-to-needle times as documented in the registry. In the explanatory sequential design, a quantitative study is carried out first, followed by a qualitative study to help explain the results from the quantitative study. This would be an appropriate design if the registry alone had revealed relevant delays in door-to-needle times and the qualitative study would be used to understand where and why these occurred, and how they could be improved. In the exploratory design, the qualitative study is carried out first and its results help informing and building the quantitative study in the next step [ 26 ]. If the qualitative study around EVT provision had shown a high level of dissatisfaction among the staff members involved, a quantitative questionnaire investigating staff satisfaction could be set up in the next step, informed by the qualitative study on which topics dissatisfaction had been expressed. Amongst other things, the questionnaire design would make it possible to widen the reach of the research to more respondents from different (types of) hospitals, regions, countries or settings, and to conduct sub-group analyses for different professional groups.

How to assess qualitative research?

A variety of assessment criteria and lists have been developed for qualitative research, ranging in their focus and comprehensiveness [ 14 , 17 , 27 ]. However, none of these has been elevated to the “gold standard” in the field. In the following, we therefore focus on a set of commonly used assessment criteria that, from a practical standpoint, a researcher can look for when assessing a qualitative research report or paper.

Assessors should check the authors’ use of and adherence to the relevant reporting checklists (e.g. Standards for Reporting Qualitative Research (SRQR)) to make sure all items that are relevant for this type of research are addressed [ 23 , 28 ]. Discussions of quantitative measures in addition to or instead of these qualitative measures can be a sign of lower quality of the research (paper). Providing and adhering to a checklist for qualitative research contributes to an important quality criterion for qualitative research, namely transparency [ 15 , 17 , 23 ].

Reflexivity

While methodological transparency and complete reporting is relevant for all types of research, some additional criteria must be taken into account for qualitative research. This includes what is called reflexivity, i.e. sensitivity to the relationship between the researcher and the researched, including how contact was established and maintained, or the background and experience of the researcher(s) involved in data collection and analysis. Depending on the research question and population to be researched this can be limited to professional experience, but it may also include gender, age or ethnicity [ 17 , 27 ]. These details are relevant because in qualitative research, as opposed to quantitative research, the researcher as a person cannot be isolated from the research process [ 23 ]. It may influence the conversation when an interviewed patient speaks to an interviewer who is a physician, or when an interviewee is asked to discuss a gynaecological procedure with a male interviewer, and therefore the reader must be made aware of these details [ 19 ].

Sampling and saturation

The aim of qualitative sampling is for all variants of the objects of observation that are deemed relevant for the study to be present in the sample “ to see the issue and its meanings from as many angles as possible” [ 1 , 16 , 19 , 20 , 27 ] , and to ensure “information-richness [ 15 ]. An iterative sampling approach is advised, in which data collection (e.g. five interviews) is followed by data analysis, followed by more data collection to find variants that are lacking in the current sample. This process continues until no new (relevant) information can be found and further sampling becomes redundant – which is called saturation [ 1 , 15 ] . In other words: qualitative data collection finds its end point not a priori , but when the research team determines that saturation has been reached [ 29 , 30 ].

This is also the reason why most qualitative studies use deliberate instead of random sampling strategies. This is generally referred to as “ purposive sampling” , in which researchers pre-define which types of participants or cases they need to include so as to cover all variations that are expected to be of relevance, based on the literature, previous experience or theory (i.e. theoretical sampling) [ 14 , 20 ]. Other types of purposive sampling include (but are not limited to) maximum variation sampling, critical case sampling or extreme or deviant case sampling [ 2 ]. In the above EVT example, a purposive sample could include all relevant professional groups and/or all relevant stakeholders (patients, relatives) and/or all relevant times of observation (day, night and weekend shift).

Assessors of qualitative research should check whether the considerations underlying the sampling strategy were sound and whether or how researchers tried to adapt and improve their strategies in stepwise or cyclical approaches between data collection and analysis to achieve saturation [ 14 ].

Good qualitative research is iterative in nature, i.e. it goes back and forth between data collection and analysis, revising and improving the approach where necessary. One example of this are pilot interviews, where different aspects of the interview (especially the interview guide, but also, for example, the site of the interview or whether the interview can be audio-recorded) are tested with a small number of respondents, evaluated and revised [ 19 ]. In doing so, the interviewer learns which wording or types of questions work best, or which is the best length of an interview with patients who have trouble concentrating for an extended time. Of course, the same reasoning applies to observations or focus groups which can also be piloted.

Ideally, coding should be performed by at least two researchers, especially at the beginning of the coding process when a common approach must be defined, including the establishment of a useful coding list (or tree), and when a common meaning of individual codes must be established [ 23 ]. An initial sub-set or all transcripts can be coded independently by the coders and then compared and consolidated after regular discussions in the research team. This is to make sure that codes are applied consistently to the research data.

Member checking

Member checking, also called respondent validation , refers to the practice of checking back with study respondents to see if the research is in line with their views [ 14 , 27 ]. This can happen after data collection or analysis or when first results are available [ 23 ]. For example, interviewees can be provided with (summaries of) their transcripts and asked whether they believe this to be a complete representation of their views or whether they would like to clarify or elaborate on their responses [ 17 ]. Respondents’ feedback on these issues then becomes part of the data collection and analysis [ 27 ].

Stakeholder involvement

In those niches where qualitative approaches have been able to evolve and grow, a new trend has seen the inclusion of patients and their representatives not only as study participants (i.e. “members”, see above) but as consultants to and active participants in the broader research process [ 31 – 33 ]. The underlying assumption is that patients and other stakeholders hold unique perspectives and experiences that add value beyond their own single story, making the research more relevant and beneficial to researchers, study participants and (future) patients alike [ 34 , 35 ]. Using the example of patients on or nearing dialysis, a recent scoping review found that 80% of clinical research did not address the top 10 research priorities identified by patients and caregivers [ 32 , 36 ]. In this sense, the involvement of the relevant stakeholders, especially patients and relatives, is increasingly being seen as a quality indicator in and of itself.

How not to assess qualitative research

The above overview does not include certain items that are routine in assessments of quantitative research. What follows is a non-exhaustive, non-representative, experience-based list of the quantitative criteria often applied to the assessment of qualitative research, as well as an explanation of the limited usefulness of these endeavours.

Protocol adherence

Given the openness and flexibility of qualitative research, it should not be assessed by how well it adheres to pre-determined and fixed strategies – in other words: its rigidity. Instead, the assessor should look for signs of adaptation and refinement based on lessons learned from earlier steps in the research process.

Sample size

For the reasons explained above, qualitative research does not require specific sample sizes, nor does it require that the sample size be determined a priori [ 1 , 14 , 27 , 37 – 39 ]. Sample size can only be a useful quality indicator when related to the research purpose, the chosen methodology and the composition of the sample, i.e. who was included and why.

Randomisation

While some authors argue that randomisation can be used in qualitative research, this is not commonly the case, as neither its feasibility nor its necessity or usefulness has been convincingly established for qualitative research [ 13 , 27 ]. Relevant disadvantages include the negative impact of a too large sample size as well as the possibility (or probability) of selecting “ quiet, uncooperative or inarticulate individuals ” [ 17 ]. Qualitative studies do not use control groups, either.

Interrater reliability, variability and other “objectivity checks”

The concept of “interrater reliability” is sometimes used in qualitative research to assess to which extent the coding approach overlaps between the two co-coders. However, it is not clear what this measure tells us about the quality of the analysis [ 23 ]. This means that these scores can be included in qualitative research reports, preferably with some additional information on what the score means for the analysis, but it is not a requirement. Relatedly, it is not relevant for the quality or “objectivity” of qualitative research to separate those who recruited the study participants and collected and analysed the data. Experiences even show that it might be better to have the same person or team perform all of these tasks [ 20 ]. First, when researchers introduce themselves during recruitment this can enhance trust when the interview takes place days or weeks later with the same researcher. Second, when the audio-recording is transcribed for analysis, the researcher conducting the interviews will usually remember the interviewee and the specific interview situation during data analysis. This might be helpful in providing additional context information for interpretation of data, e.g. on whether something might have been meant as a joke [ 18 ].

Not being quantitative research

Being qualitative research instead of quantitative research should not be used as an assessment criterion if it is used irrespectively of the research problem at hand. Similarly, qualitative research should not be required to be combined with quantitative research per se – unless mixed methods research is judged as inherently better than single-method research. In this case, the same criterion should be applied for quantitative studies without a qualitative component.

The main take-away points of this paper are summarised in Table ​ Table1. 1 . We aimed to show that, if conducted well, qualitative research can answer specific research questions that cannot to be adequately answered using (only) quantitative designs. Seeing qualitative and quantitative methods as equal will help us become more aware and critical of the “fit” between the research problem and our chosen methods: I can conduct an RCT to determine the reasons for transportation delays of acute stroke patients – but should I? It also provides us with a greater range of tools to tackle a greater range of research problems more appropriately and successfully, filling in the blind spots on one half of the methodological spectrum to better address the whole complexity of neurological research and practice.

Take-away-points

Acknowledgements

Abbreviations, authors’ contributions.

LB drafted the manuscript; WW and CG revised the manuscript; all authors approved the final versions.

no external funding.

Availability of data and materials

Ethics approval and consent to participate, consent for publication, competing interests.

The authors declare no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Qualitative Research Design: Start

Qualitative Research Design

What is Qualitative research design?

Qualitative research is a type of research that explores and provides deeper insights into real-world problems. Instead of collecting numerical data points or intervening or introducing treatments just like in quantitative research, qualitative research helps generate hypotheses as well as further investigate and understand quantitative data. Qualitative research gathers participants' experiences, perceptions, and behavior. It answers the hows and whys instead of how many or how much . It could be structured as a stand-alone study, purely relying on qualitative data or it could be part of mixed-methods research that combines qualitative and quantitative data.

Qualitative research involves collecting and analyzing non-numerical data (e.g., text, video, or audio) to understand concepts, opinions, or experiences. It can be used to gather in-depth insights into a problem or generate new ideas for research. Qualitative research is the opposite of quantitative research, which involves collecting and analyzing numerical data for statistical analysis. Qualitative research is commonly used in the humanities and social sciences, in subjects such as anthropology, sociology, education, health sciences, history, etc.

While qualitative and quantitative approaches are different, they are not necessarily opposites, and they are certainly not mutually exclusive. For instance, qualitative research can help expand and deepen understanding of data or results obtained from quantitative analysis. For example, say a quantitative analysis has determined that there is a correlation between length of stay and level of patient satisfaction, but why does this correlation exist? This dual-focus scenario shows one way in which qualitative and quantitative research could be integrated together.

Research Paradigms 

• Positivist versus Post-Positivist
• Social Constructivist (this paradigm/ideology mostly birth qualitative studies)

Events Relating to the Qualitative Research and Community Engagement Workshops @ CMU Libraries

CMU Libraries is committed to helping members of our community become data experts. To that end, CMU is offering public facing workshops that discuss Qualitative Research, Coding, and Community Engagement best practices.

The following workshops are a part of a broader series on using data. Please follow the links to register for the events. 

Qualitative Coding

Using Community Data to improve Outcome (Grant Writing)

Survey Design  

Upcoming Event: March 21st, 2024 (12:00pm -1:00 pm)

Community Engagement and Collaboration Event 

Join us for an event to improve, build on and expand the connections between Carnegie Mellon University resources and the Pittsburgh community. CMU resources such as the Libraries and Sustainability Initiative can be leveraged by users not affiliated with the university, but barriers can prevent them from fully engaging.

The conversation features representatives from CMU departments and local organizations about the community engagement efforts currently underway at CMU and opportunities to improve upon them. Speakers will highlight current and ongoing projects and share resources to support future collaboration.

Event Moderators:

Taiwo Lasisi, CLIR Postdoctoral Fellow in Community Data Literacy,  Carnegie Mellon University Libraries

Emma Slayton, Data Curation, Visualization, & GIS Specialist,  Carnegie Mellon University Libraries

Nicky Agate , Associate Dean for Academic Engagement, Carnegie Mellon University Libraries

Chelsea Cohen , The University’s Executive fellow for community engagement, Carnegie Mellon University

Sarah Ceurvorst , Academic Pathways Manager, Program Director, LEAP (Leadership, Excellence, Access, Persistence) Carnegie Mellon University

Julia Poeppibg , Associate Director of Partnership Development, Information Systems, Carnegie Mellon University 

Scott Wolovich , Director of New Sun Rising, Pittsburgh 

Additional workshops and events will be forthcoming. Watch this space for updates. 

Workshop Organizer

Qualitative Research Methods

What are Qualitative Research methods?

Qualitative research adopts numerous methods or techniques including interviews, focus groups, and observation. Interviews may be unstructured, with open-ended questions on a topic and the interviewer adapts to the responses. Structured interviews have a predetermined number of questions that every participant is asked. It is usually one-on-one and is appropriate for sensitive topics or topics needing an in-depth exploration. Focus groups are often held with 8-12 target participants and are used when group dynamics and collective views on a topic are desired. Researchers can be participant observers to share the experiences of the subject or non-participant or detached observers.

What constitutes a good research question? Does the question drive research design choices?

According to Doody and Bailey (2014);

 We can only develop a good research question by consulting relevant literature, colleagues, and supervisors experienced in the area of research. (inductive interactions).

Helps to have a directed research aim and objective.

Researchers should not be “ research trendy” and have enough evidence. This is why research objectives are important. It helps to take time, and resources into consideration.

Research questions can be developed from theoretical knowledge, previous research or experience, or a practical need at work (Parahoo 2014). They have numerous roles, such as identifying the importance of the research and providing clarity of purpose for the research, in terms of what the research intends to achieve in the end.

Qualitative Research Questions

What constitutes a good Qualitative research question?

A good qualitative question answers the hows and whys instead of how many or how much. It could be structured as a stand-alone study, purely relying on qualitative data or it could be part of mixed-methods research that combines qualitative and quantitative data. Qualitative research gathers participants' experiences, perceptions and behavior.

Examples of good Qualitative Research Questions:

What are people's thoughts on the new library? 

How does it feel to be a first-generation student attending college?

Difference example (between Qualitative and Quantitative research questions):

How many college students signed up for the new semester? (Quan) 

How do college students feel about the new semester? What are their experiences so far? (Qual)

• Qualitative Research Design Workshop Powerpoint

Foley G, Timonen V. Using Grounded Theory Method to Capture and Analyze Health Care Experiences. Health Serv Res. 2015 Aug;50(4):1195-210. [ PMC free article: PMC4545354 ] [ PubMed: 25523315 ]

Devers KJ. How will we know "good" qualitative research when we see it? Beginning the dialogue in health services research. Health Serv Res. 1999 Dec;34(5 Pt 2):1153-88. [ PMC free article: PMC1089058 ] [ PubMed: 10591278 ]

Huston P, Rowan M. Qualitative studies. Their role in medical research. Can Fam Physician. 1998 Nov;44:2453-8. [ PMC free article: PMC2277956 ] [ PubMed: 9839063 ]

Corner EJ, Murray EJ, Brett SJ. Qualitative, grounded theory exploration of patients' experience of early mobilisation, rehabilitation and recovery after critical illness. BMJ Open. 2019 Feb 24;9(2):e026348. [ PMC free article: PMC6443050 ] [ PubMed: 30804034 ]

Moser A, Korstjens I. Series: Practical guidance to qualitative research. Part 3: Sampling, data collection and analysis. Eur J Gen Pract. 2018 Dec;24(1):9-18. [ PMC free article: PMC5774281 ] [ PubMed: 29199486 ]

Houghton C, Murphy K, Meehan B, Thomas J, Brooker D, Casey D. From screening to synthesis: using nvivo to enhance transparency in qualitative evidence synthesis. J Clin Nurs. 2017 Mar;26(5-6):873-881. [ PubMed: 27324875 ]

Soratto J, Pires DEP, Friese S. Thematic content analysis using ATLAS.ti software: Potentialities for researchs in health. Rev Bras Enferm. 2020;73(3):e20190250. [ PubMed: 32321144 ]

Zamawe FC. The Implication of Using NVivo Software in Qualitative Data Analysis: Evidence-Based Reflections. Malawi Med J. 2015 Mar;27(1):13-5. [ PMC free article: PMC4478399 ] [ PubMed: 26137192 ]

Korstjens I, Moser A. Series: Practical guidance to qualitative research. Part 4: Trustworthiness and publishing. Eur J Gen Pract. 2018 Dec;24(1):120-124. [ PMC free article: PMC8816392 ] [ PubMed: 29202616 ]

Saldaña, J. (2021). The coding manual for qualitative researchers. The coding manual for qualitative researchers, 1-440.

O'Brien BC, Harris IB, Beckman TJ, Reed DA, Cook DA. Standards for reporting qualitative research: a synthesis of recommendations. Acad Med. 2014 Sep;89(9):1245-51. [ PubMed: 24979285 ]

Palermo C, King O, Brock T, Brown T, Crampton P, Hall H, Macaulay J, Morphet J, Mundy M, Oliaro L, Paynter S, Williams B, Wright C, E Rees C. Setting priorities for health education research: A mixed methods study. Med Teach. 2019 Sep;41(9):1029-1038. [ PubMed: 31141390 ]

• Last Updated: Feb 14, 2024 4:25 PM
• URL: https://guides.library.cmu.edu/c.php?g=1346006

• Types of qualitative research designs

Last updated

20 February 2023

Reviewed by

Jean Kaluza

Researchers often conduct these studies to gain a detailed understanding of a particular topic through a small, focused sample. Qualitative research methods delve into understanding why something is happening in a larger quantitative study. 

To determine whether qualitative research is the best choice for your study, let’s look at the different types of qualitative research design.

Analyze all your qualitative research

Analyze qualitative data faster and surface more actionable insights

• What are qualitative research designs?

Qualitative research designs are research methods that collect and analyze non-numerical data. The research uncovers why or how a particular behavior or occurrence takes place. The information is usually subjective and in a written format instead of numerical.

Researchers may use interviews, focus groups , case studies , journaling, and open-ended questions to gather in-depth information. Qualitative research designs can determine users' concepts, develop a hypothesis , or add context to data from a quantitative study.

• Characteristics of qualitative research design

Most often, qualitative data answers how or why something occurs. Certain characteristics are usually present in all qualitative research designs to ensure accurate data. 

The most common characteristics of qualitative research design include the following:

Natural environment

It’s best to collect qualitative research as close to the subject’s original environment as possible to encourage natural behavior and accurate insights.

Empathy is key

Qualitative researchers collect the best data when they’re in sync with their users’ concerns and motivations. They can play into natural human psychology by combining open-ended questioning and subtle cues.

They may mimic body language, adopt the users’ terminology, and use pauses or trailing sentences to encourage their participants to fill in the blanks. The more empathic the interviewer, the purer the data.

Participant selection

Qualitative research depends on the meaning obtained from participants instead of the meaning conveyed in similar research or studies. To increase research accuracy, you choose participants randomly from carefully chosen groups of potential participants.

Different research methods or multiple data sources

To gain in-depth knowledge, qualitative research designs often rely on multiple research methods within the same group. 

Emergent design

Qualitative research constantly evolves, meaning the initial study plan might change after you collect data. This evolution might result in changes in research methods or the introduction of a new research problem.

Inductive reasoning

Since qualitative research seeks in-depth meaning, you need complex reasoning to get the right results. Qualitative researchers build categories, patterns, and themes from separate data sets to form a complete conclusion.

Interpretive data

Once you collect the data, you need to read between the lines rather than just noting what your participant said. Qualitative research is unique as we can attach actions to feedback. 

If a user says they love the look of your design but haven’t completed any tasks, it’s up to you to interpret this as a failed test, even with their positive sentiments.  

Holistic account

To paint a large picture of an issue and potential solutions, a qualitative researcher works to develop a complex description of the research problem. You can avoid a narrow cause-and-effect perspective by describing the problem’s wider perspectives. 

• When to use qualitative research design

Qualitative research aims to get a detailed understanding of a particular topic. To accomplish this, you’ll typically use small focus groups to gather in-depth data from varied perspectives. 

This approach is only effective for some types of study. For instance, a qualitative approach wouldn’t work for a study that seeks to understand a statistically relevant finding.

When determining if a qualitative research design is appropriate, remember the goal of qualitative research is understanding the “ why .” 

Qualitative research design gathers in-depth information that stands on its own. It can also answer the “why” of a quantitative study or be a precursor to forming a hypothesis. 

You can use qualitative research in these situations:

Developing a hypothesis for testing in a quantitative study

Identifying customer needs

Developing a new feature

Adding context to the results of a quantitative study

Understanding the motivations, values, and pain points that guide behavior

Difference between qualitative and quantitative research design

Qualitative and quantitative research designs gather data, but that's where the similarities end. Consider the difference between quality and quantity. Both are useful in different ways.

Qualitative research gathers in-depth information to answer how or why . It uses subjective data from detailed interviews, observations, and open-ended questions. Most often, qualitative data is thoughts, experiences, and concepts.

In contrast, quantitative research designs gather large amounts of objective data that you can quantify mathematically. You typically express quantitative data in numbers or graphs, and you use it to test or confirm hypotheses.

Qualitative research designs generally have the same goals. However, there are various ways to achieve these goals. Researchers may use one or more of these approaches in qualitative research.

Historical study

This is where you use extensive information about people and events in the past to draw conclusions about the present and future.

Phenomenology

Phenomenology investigates a phenomenon, activity, or event using data from participants' perspectives. Often, researchers use a combination of methods.

Grounded theory

Grounded theory uses interviews and existing data to build a theory inductively.

Ethnography

Researchers immerse themselves in the target participant's environments to understand goals, cultures, challenges, and themes with ethnography .

A case study is where you use multiple data sources to examine a person, group, community, or institution. Participants must share a connection to the research question you’re studying.

• Advantages and disadvantages of qualitative research

All qualitative research design types share the common goal of obtaining in-depth information. Achieving this goal generally requires extensive data collection methods that can be time-consuming. As such, qualitative research has advantages and disadvantages. 

Natural settings

Since you can collect data closer to an authentic environment, it offers more accurate results.  

The ability to paint a picture with data

Quantitative studies don't always reveal the full picture. With multiple data collection methods, you can expose the motivations and reasons behind data.

Flexibility

Analysis processes aren't set in stone, so you can adapt the process as ideas or patterns emerge.

Generation of new ideas

Using open-ended responses can uncover new opportunities or solutions that weren't part of your original research plan.

Small sample sizes

You can generate meaningful results with small groups.

Disadvantages

Potentially unreliable.

A natural setting can be a double-edged sword. The inability to attach findings to anything statistically relevant can make data more difficult to quantify. 

Subjectivity

Since the researcher plays a vital role in collecting and interpreting data, qualitative research is subject to the researcher's skills. For example, they may miss a cue that changes some of the context of the quotes they collected.

Labor-intensive

You generally collect qualitative data through manual processes like extensive interviews, open-ended questions, and case studies.

Qualitative research designs allow researchers to provide an in-depth analysis of why specific behavior or events occur. It can offer fresh insights, generate new ideas, or add context to statistics from quantitative studies. Depending on your needs, qualitative data might be a great way to gain the information your organization needs to move forward.

Get started today

Go from raw data to valuable insights with a flexible research platform

Editor’s picks

Last updated: 21 December 2023

Last updated: 16 December 2023

Last updated: 6 October 2023

Last updated: 25 November 2023

Last updated: 12 May 2023

Last updated: 15 February 2024

Last updated: 11 March 2024

Last updated: 12 December 2023

Last updated: 18 May 2023

Last updated: 6 March 2024

Last updated: 10 April 2023

Last updated: 20 December 2023

Latest articles

Related topics, log in or sign up.

Get started for free

• Tools and Resources
• Customer Services
• Original Language Spotlight
• Alternative and Non-formal Education 
• Cognition, Emotion, and Learning
• Curriculum and Pedagogy
• Education and Society
• Education, Change, and Development
• Education, Cultures, and Ethnicities
• Education, Gender, and Sexualities
• Education, Health, and Social Services
• Educational Administration and Leadership
• Educational History
• Educational Politics and Policy
• Educational Purposes and Ideals
• Educational Systems
• Educational Theories and Philosophies
• Globalization, Economics, and Education
• Languages and Literacies
• Professional Learning and Development
• Research and Assessment Methods
• Technology and Education
• Share This Facebook LinkedIn Twitter

Article contents

Qualitative design research methods.

• Michael Domínguez Michael Domínguez San Diego State University
• https://doi.org/10.1093/acrefore/9780190264093.013.170
• Published online: 19 December 2017

Emerging in the learning sciences field in the early 1990s, qualitative design-based research (DBR) is a relatively new methodological approach to social science and education research. As its name implies, DBR is focused on the design of educational innovations, and the testing of these innovations in the complex and interconnected venue of naturalistic settings. As such, DBR is an explicitly interventionist approach to conducting research, situating the researcher as a part of the complex ecology in which learning and educational innovation takes place.

With this in mind, DBR is distinct from more traditional methodologies, including laboratory experiments, ethnographic research, and large-scale implementation. Rather, the goal of DBR is not to prove the merits of any particular intervention, or to reflect passively on a context in which learning occurs, but to examine the practical application of theories of learning themselves in specific, situated contexts. By designing purposeful, naturalistic, and sustainable educational ecologies, researchers can test, extend, or modify their theories and innovations based on their pragmatic viability. This process offers the prospect of generating theory-developing, contextualized knowledge claims that can complement the claims produced by other forms of research.

Because of this interventionist, naturalistic stance, DBR has also been the subject of ongoing debate concerning the rigor of its methodology. In many ways, these debates obscure the varied ways DBR has been practiced, the varied types of questions being asked, and the theoretical breadth of researchers who practice DBR. With this in mind, DBR research may involve a diverse range of methods as researchers from a variety of intellectual traditions within the learning sciences and education research design pragmatic innovations based on their theories of learning, and document these complex ecologies using the methodologies and tools most applicable to their questions, focuses, and academic communities.

DBR has gained increasing interest in recent years. While it remains a popular methodology for developmental and cognitive learning scientists seeking to explore theory in naturalistic settings, it has also grown in importance to cultural psychology and cultural studies researchers as a methodological approach that aligns in important ways with the participatory commitments of liberatory research. As such, internal tension within the DBR field has also emerged. Yet, though approaches vary, and have distinct genealogies and commitments, DBR might be seen as the broad methodological genre in which Change Laboratory, design-based implementation research (DBIR), social design-based experiments (SDBE), participatory design research (PDR), and research-practice partnerships might be categorized. These critically oriented iterations of DBR have important implications for educational research and educational innovation in historically marginalized settings and the Global South.

• design-based research
• learning sciences
• social-design experiment
• qualitative research
• research methods

Educational research, perhaps more than many other disciplines, is a situated field of study. Learning happens around us every day, at all times, in both formal and informal settings. Our worlds are replete with complex, dynamic, diverse communities, contexts, and institutions, many of which are actively seeking guidance and support in the endless quest for educational innovation. Educational researchers—as a source of potential expertise—are necessarily implicated in this complexity, linked to the communities and institutions through their very presence in spaces of learning, poised to contribute with possible solutions, yet often positioned as separate from the activities they observe, creating dilemmas of responsibility and engagement.

So what are educational scholars and researchers to do? These tensions invite a unique methodological challenge for the contextually invested researcher, begging them to not just produce knowledge about learning, but to participate in the ecology, collaborating on innovations in the complex contexts in which learning is taking place. In short, for many educational researchers, our backgrounds as educators, our connections to community partners, and our sociopolitical commitments to the process of educational innovation push us to ensure that our work is generative, and that our theories and ideas—our expertise—about learning and education are made pragmatic, actionable, and sustainable. We want to test what we know outside of laboratories, designing, supporting, and guiding educational innovation to see if our theories of learning are accurate, and useful to the challenges faced in schools and communities where learning is messy, collaborative, and contested. Through such a process, we learn, and can modify our theories to better serve the real needs of communities. It is from this impulse that qualitative design-based research (DBR) emerged as a new methodological paradigm for education research.

Qualitative design-based research will be examined, documenting its origins, the major tenets of the genre, implementation considerations, and methodological issues, as well as variance within the paradigm. As a relatively new methodology, much tension remains in what constitutes DBR, and what design should mean, and for whom. These tensions and questions, as well as broad perspectives and emergent iterations of the methodology, will be discussed, and considerations for researchers looking toward the future of this paradigm will be considered.

The Origins of Design-Based Research

Qualitative design-based research (DBR) first emerged in the learning sciences field among a group of scholars in the early 1990s, with the first articulation of DBR as a distinct methodological construct appearing in the work of Ann Brown ( 1992 ) and Allan Collins ( 1992 ). For learning scientists in the 1970s and 1980s, the traditional methodologies of laboratory experiments, ethnographies, and large-scale educational interventions were the only methods available. During these decades, a growing community of learning science and educational researchers (e.g., Bereiter & Scardamalia, 1989 ; Brown, Campione, Webber, & McGilley, 1992 ; Cobb & Steffe, 1983 ; Cole, 1995 ; Scardamalia & Bereiter, 1991 ; Schoenfeld, 1982 , 1985 ; Scribner & Cole, 1978 ) interested in educational innovation and classroom interventions in situated contexts began to find the prevailing methodologies insufficient for the types of learning they wished to document, the roles they wished to play in research, and the kinds of knowledge claims they wished to explore. The laboratory, or laboratory-like settings, where research on learning was at the time happening, was divorced from the complexity of real life, and necessarily limiting. Alternatively, most ethnographic research, while more attuned to capturing these complexities and dynamics, regularly assumed a passive stance 1 and avoided interceding in the learning process, or allowing researchers to see what possibility for innovation existed from enacting nascent learning theories. Finally, large-scale interventions could test innovations in practice but lost sight of the nuance of development and implementation in local contexts (Brown, 1992 ; Collins, Joseph, & Bielaczyc, 2004 ).

Dissatisfied with these options, and recognizing that in order to study and understand learning in the messiness of socially, culturally, and historically situated settings, new methods were required, Brown ( 1992 ) proposed an alternative: Why not involve ourselves in the messiness of the process, taking an active, grounded role in disseminating our theories and expertise by becoming designers and implementers of educational innovations? Rather than observing from afar, DBR researchers could trace their own iterative processes of design, implementation, tinkering, redesign, and evaluation, as it unfolded in shared work with teachers, students, learners, and other partners in lived contexts. This premise, initially articulated as “design experiments” (Brown, 1992 ), would be variously discussed over the next decade as “design research,” (Edelson, 2002 ) “developmental research,” (Gravemeijer, 1994 ), and “design-based research,” (Design-Based Research Collective, 2003 ), all of which reflect the original, interventionist, design-oriented concept. The latter term, “design-based research” (DBR), is used here, recognizing this as the prevailing terminology used to refer to this research approach at present. 2

Regardless of the evolving moniker, the prospects of such a methodology were extremely attractive to researchers. Learning scientists acutely aware of various aspects of situated context, and interested in studying the applied outcomes of learning theories—a task of inquiry into situated learning for which canonical methods were rather insufficient—found DBR a welcome development (Bell, 2004 ). As Barab and Squire ( 2004 ) explain: “learning scientists . . . found that they must develop technological tools, curriculum, and especially theories that help them systematically understand and predict how learning occurs” (p. 2), and DBR methodologies allowed them to do this in proactive, hands-on ways. Thus, rather than emerging as a strict alternative to more traditional methodologies, DBR was proposed to fill a niche that other methodologies were ill-equipped to cover.

Effectively, while its development is indeed linked to an inherent critique of previous research paradigms, neither Brown nor Collins saw DBR in opposition to other forms of research. Rather, by providing a bridge from the laboratory to the real world, where learning theories and proposed innovations could interact and be implemented in the complexity of lived socio-ecological contexts (Hoadley, 2004 ), new possibilities emerged. Learning researchers might “trace the evolution of learning in complex, messy classrooms and schools, test and build theories of teaching and learning, and produce instructional tools that survive the challenges of everyday practice” (Shavelson, Phillips, Towne, & Feuer, 2003 , p. 25). Thus, DBR could complement the findings of laboratory, ethnographic, and large-scale studies, answering important questions about the implementation, sustainability, limitations, and usefulness of theories, interventions, and learning when introduced as innovative designs into situated contexts of learning. Moreover, while studies involving these traditional methodologies often concluded by pointing toward implications—insights subsequent studies would need to take up—DBR allowed researchers to address implications iteratively and directly. No subsequent research was necessary, as emerging implications could be reflexively explored in the context of the initial design, offering considerable insight into how research is translated into theory and practice.

Since its emergence in 1992 , DBR as a methodological approach to educational and learning research has quickly grown and evolved, used by researchers from a variety of intellectual traditions in the learning sciences, including developmental and cognitive psychology (e.g., Brown & Campione, 1996 , 1998 ; diSessa & Minstrell, 1998 ), cultural psychology (e.g., Cole, 1996 , 2007 ; Newman, Griffin, & Cole, 1989 ; Gutiérrez, Bien, Selland, & Pierce, 2011 ), cultural anthropology (e.g., Barab, Kinster, Moore, Cunningham, & the ILF Design Team, 2001 ; Polman, 2000 ; Stevens, 2000 ; Suchman, 1995 ), and cultural-historical activity theory (e.g., Engeström, 2011 ; Espinoza, 2009 ; Espinoza & Vossoughi, 2014 ; Gutiérrez, 2008 ; Sannino, 2011 ). Given this plurality of epistemological and theoretical fields that employ DBR, it might best be understood as a broad methodology of educational research, realized in many different, contested, heterogeneous, and distinct iterations, and engaging a variety of qualitative tools and methods (Bell, 2004 ). Despite tensions among these iterations, and substantial and important variances in the ways they employ design-as-research in community settings, there are several common, methodological threads that unite the broad array of research that might be classified as DBR under a shared, though pluralistic, paradigmatic umbrella.

The Tenets of Design-Based Research

Why design-based research.

As we turn to the core tenets of the design-based research (DBR) paradigm, it is worth considering an obvious question: Why use DBR as a methodology for educational research? To answer this, it is helpful to reflect on the original intentions for DBR, particularly, that it is not simply the study of a particular, isolated intervention. Rather, DBR methodologies were conceived of as the complete, iterative process of designing, modifying, and assessing the impact of an educational innovation in a contextual, situated learning environment (Barab & Kirshner, 2001 ; Brown, 1992 ; Cole & Engeström, 2007 ). The design process itself—inclusive of the theory of learning employed, the relationships among participants, contextual factors and constraints, the pedagogical approach, any particular intervention, as well as any changes made to various aspects of this broad design as it proceeds—is what is under study.

Considering this, DBR offers a compelling framework for the researcher interested in having an active and collaborative hand in designing for educational innovation, and interested in creating knowledge about how particular theories of learning, pedagogical or learning practices, or social arrangements function in a context of learning. It is a methodology that can put the researcher in the position of engineer , actively experimenting with aspects of learning and sociopolitical ecologies to arrive at new knowledge and productive outcomes, as Cobb, Confrey, diSessa, Lehrer, and Schauble ( 2003 ) explain:

Prototypically, design experiments entail both “engineering” particular forms of learning and systematically studying those forms of learning within the context defined by the means of supporting them. This designed context is subject to test and revision, and the successive iterations that result play a role similar to that of systematic variation in experiment. (p. 9)

This being said, how directive the engineering role the researcher takes on varies considerably among iterations of DBR. Indeed, recent approaches have argued strongly for researchers to take on more egalitarian positionalities with respect to the community partners with whom they work (e.g., Zavala, 2016 ), acting as collaborative designers, rather than authoritative engineers.

Method and Methodology in Design-Based Research

Now, having established why we might use DBR, a recurring question that has faced the DBR paradigm is whether DBR is a methodology at all. Given the variety of intellectual and ontological traditions that employ it, and thus the pluralism of methods used in DBR to enact the “engineering” role (whatever shape that may take) that the researcher assumes, it has been argued that DBR is not, in actuality a methodology at all (Kelly, 2004 ). The proliferation and diversity of approaches, methods, and types of analysis purporting to be DBR have been described as a lack of coherence that shows there is no “argumentative grammar” or methodology present in DBR (Kelly, 2004 ).

Now, the conclusions one will eventually draw in this debate will depend on one’s orientations and commitments, but it is useful to note that these demands for “coherence” emerge from previous paradigms in which methodology was largely marked by a shared, coherent toolkit for data collection and data analysis. These previous paradigmatic rules make for an odd fit when considering DBR. Yet, even if we proceed—within the qualitative tradition from which DBR emerges—defining methodology as an approach to research that is shaped by the ontological and epistemological commitments of the particular researcher, and methods as the tools for research, data collection, and analysis that are chosen by the researcher with respect to said commitments (Gutiérrez, Engeström, & Sannino, 2016 ), then a compelling case for DBR as a methodology can be made (Bell, 2004 ).

Effectively, despite the considerable variation in how DBR has been and is employed, and tensions within the DBR field, we might point to considerable, shared epistemic common ground among DBR researchers, all of whom are invested in an approach to research that involves engaging actively and iteratively in the design and exploration of learning theory in situated, natural contexts. This common epistemic ground, even in the face of pluralistic ideologies and choices of methods, invites in a new type of methodological coherence, marked by “intersubjectivity without agreement” (Matusov, 1996 ), that links DBR from traditional developmental and cognitive psychology models of DBR (e.g., Brown, 1992 ; Brown & Campione, 1998 ; Collins, 1992 ), to more recent critical and sociocultural manifestations (e.g., Bang & Vossoughi, 2016 ; Engeström, 2011 ; Gutiérrez, 2016 ), and everything in between.

Put in other terms, even as DBR researchers may choose heterogeneous methods for data collection, data analysis, and reporting results complementary to the ideological and sociopolitical commitments of the particular researcher and the types of research questions that are under examination (Bell, 2004 ), a shared epistemic commitment gives the methodology shape. Indeed, the common commitment toward design innovation emerges clearly across examples of DBR methodological studies ranging in method from ethnographic analyses (Salvador, Bell, & Anderson, 1999 ) to studies of critical discourse within a design (Kärkkäinen, 1999 ), to focused examinations of metacognition of individual learners (White & Frederiksen, 1998 ), and beyond. Rather than indicating a lack of methodology, or methodological weakness, the use of varying qualitative methods for framing data collection and retrospective analyses within DBR, and the tensions within the epistemic common ground itself, simply reflects the scope of its utility. Learning in context is complex, contested, and messy, and the plurality of methods present across DBR allow researchers to dynamically respond to context as needed, employing the tools that fit best to consider the questions that are present, or may arise.

All this being the case, it is useful to look toward the coherent elements—the “argumentative grammar” of DBR, if you will—that can be identified across the varied iterations of DBR. Understanding these shared features, in the context and terms of the methodology itself, help us to appreciate what is involved in developing robust and thorough DBR research, and how DBR seeks to make strong, meaningful claims around the types of research questions it takes up.

Coherent Features of Design-Based Research

Several scholars have provided comprehensive overviews and listings of what they see as the cross-cutting features of DBR, both in the context of more traditional models of DBR (e.g., Cobb et al., 2003 ; Design-Based Research Collective, 2003 ), and in regards to newer iterations (e.g., Gutiérrez & Jurow, 2016 ; Bang & Vossoughi, 2016 ). Rather than try to offer an overview of each of these increasingly pluralistic classifications, the intent here is to attend to three broad elements that are shared across articulations of DBR and reflect the essential elements that constitute the methodological approach DBR offers to educational researchers.

Design research is concerned with the development, testing, and evolution of learning theory in situated contexts

This first element is perhaps most central to what DBR of all types is, anchored in what Brown ( 1992 ) was initially most interested in: testing the pragmatic validity of theories of learning by designing interventions that engaged with, or proposed, entire, naturalistic, ecologies of learning. Put another way, while DBR studies may have various units of analysis, focuses, and variables, and may organize learning in many different ways, it is the theoretically informed design for educational innovation that is most centrally under evaluation. DBR actively and centrally exists as a paradigm that is engaged in the development of theory, not just the evaluation of aspects of its usage (Bell, 2004 ; Design-Based Research Collective, 2003 ; Lesh & Kelly, 2000 ; van den Akker, 1999 ).

Effectively, where DBR is taking place, theory as a lived possibility is under examination. Specifically, in most DBR, this means a focus on “intermediate-level” theories of learning, rather than “grand” ones. In essence, DBR does not contend directly with “grand” learning theories (such as developmental or sociocultural theory writ large) (diSessa, 1991 ). Rather, DBR seeks to offer constructive insights by directly engaging with particular learning processes that flow from these theories on a “grounded,” “intermediate” level. This is not, however, to say DBR is limited in what knowledge it can produce; rather, tinkering in this “intermediate” realm can produce knowledge that informs the “grand” theory (Gravemeijer, 1994 ). For example, while cognitive and motivational psychology provide “grand” theoretical frames, interest-driven learning (IDL) is an “intermediate” theory that flows from these and can be explored in DBR to both inform the development of IDL designs in practice and inform cognitive and motivational psychology more broadly (Joseph, 2004 ).

Crucially, however, DBR entails putting the theory in question under intense scrutiny, or, “into harm’s way” (Cobb et al., 2003 ). This is an especially core element to DBR, and one that distinguishes it from the proliferation of educational-reform or educational-entrepreneurship efforts that similarly take up the discourse of “design” and “innovation.” Not only is the reflexive, often participatory element of DBR absent from such efforts—that is, questioning and modifying the design to suit the learning needs of the context and partners—but the theory driving these efforts is never in question, and in many cases, may be actively obscured. Indeed, it is more common to see educational-entrepreneur design innovations seek to modify a context—such as the way charter schools engage in selective pupil recruitment and intensive disciplinary practices (e.g., Carnoy et al., 2005 ; Ravitch, 2010 ; Saltman, 2007 )—rather than modify their design itself, and thus allow for humility in their theory. Such “innovations” and “design” efforts are distinct from DBR, which must, in the spirit of scientific inquiry, be willing to see the learning theory flail and struggle, be modified, and evolve.

This growth and evolution of theory and knowledge is of course central to DBR as a rigorous research paradigm; moving it beyond simply the design of local educational programs, interventions, or innovations. As Barab and Squire ( 2004 ) explain:

Design-based research requires more than simply showing a particular design works but demands that the researcher (move beyond a particular design exemplar to) generate evidence-based claims about learning that address contemporary theoretical issues and further the theoretical knowledge of the field. (pp. 5–6)

DBR as a research paradigm offers a design process through which theories of learning can be tested; they can be modified, and by allowing them to operate with humility in situated conditions, new insights and knowledge, even new theories, may emerge that might inform the field, as well as the efforts and directions of other types of research inquiry. These productive, theory-developing outcomes, or “ontological innovations” (diSessa & Cobb, 2004 ), represent the culmination of an effective program of DBR—the production of new ways to understand, conceptualize, and enact learning as a lived, contextual process.

Design research works to understand learning processes, and the design that supports them in situated contexts

As a research methodology that operates by tinkering with “grounded” learning theories, DBR is itself grounded, and seeks to develop its knowledge claims and designs in naturalistic, situated contexts (Brown, 1992 ). This is, again, a distinguishing element of DBR—setting it apart from laboratory research efforts involving design and interventions in closed, controlled environments. Rather than attempting to focus on singular variables, and isolate these from others, DBR is concerned with the multitude of variables that naturally occur across entire learning ecologies, and present themselves in distinct ways across multiple planes of possible examination (Rogoff, 1995 ; Collins, Joseph, & Bielaczyc, 2004 ). Certainly, specific variables may be identified as dependent, focal units of analysis, but identifying (while not controlling for) the variables beyond these, and analyzing their impact on the design and learning outcomes, is an equally important process in DBR (Collins et al., 2004 ; Barab & Kirshner, 2001 ). In practice, this of course varies across iterations in its depth and breadth. Traditional models of developmental or cognitive DBR may look to account for the complexity and nuance of a setting’s social, developmental, institutional, and intellectual characteristics (e.g., Brown, 1992 ; Cobb et al., 2003 ), while more recent, critical iterations will give increased attention to how historicity, power, intersubjectivity, and culture, among other things, influence and shape a setting, and the learning that occurs within it (e.g., Gutiérrez, 2016 ; Vakil, de Royston, Nasir, & Kirshner, 2016 ).

Beyond these variations, what counts as “design” in DBR varies widely, and so too will what counts as a naturalistic setting. It has been well documented that learning occurs all the time, every day, and in every space imaginable, both formal and informal (Leander, Phillips, & Taylor, 2010 ), and in ways that span strictly defined setting boundaries (Engeström, Engeström, & Kärkkäinen, 1995 ). DBR may take place in any number of contexts, based on the types of questions asked, and the learning theories and processes that a researcher may be interested in exploring. DBR may involve one-to-one tutoring and learning settings, single classrooms, community spaces, entire institutions, or even holistically designed ecologies (Design-Based Research Collective, 2003 ; Engeström, 2008 ; Virkkunen & Newnham, 2013 ). In all these cases, even the most completely designed experimental ecology, the setting remains naturalistic and situated because DBR actively embraces the uncontrollable variables that participants bring with them to the learning process for and from their situated worlds, lives, and experiences—no effort is made to control for these complicated influences of life, simply to understand how they operate in a given ecology as innovation is attempted. Thus, the extent of the design reflects a broader range of qualitative and theoretical study, rather than an attempt to control or isolate some particular learning process from outside influence.

While there is much variety in what design may entail, where DBR takes place, what types of learning ecologies are under examination, and what methods are used, situated ecologies are always the setting of this work. In this way, conscious of naturalistic variables, and the influences that culture, historicity, participation, and context have on learning, researchers can use DBR to build on prior research, and extend knowledge around the learning that occurs in the complexity of situated contexts and lived practices (Collins et al., 2004 ).

Design based research is iterative; it changes, grows, and evolves to meet the needs and emergent questions of the context, and this tinkering process is part of the research

The final shared element undergirding models of DBR is that it is an iterative, active, and interventionist process, interested in and focused on producing educational innovation by actually and actively putting design innovations into practice (Brown, 1992 , Collins, 1992 ; Gutiérrez, 2008 ). Given this interventionist, active stance, tinkering with the design and the theory of learning informing the design is as much a part of the research process as the outcome of the intervention or innovation itself—we learn what impacts learning as much, if not more, than we learn what was learned. In this sense, DBR involves a focus on analyzing the theory-driven design itself, and its implementation as an object of study (Edelson, 2002 ; Penuel, Fishman, Cheng, & Sabelli, 2011 ), and is ultimately interested in the improvement of the design—of how it unfolds, how it shifts, how it is modified, and made to function productively for participants in their contexts and given their needs (Kirshner & Polman, 2013 ).

While DBR is iterative and contextual as a foundational methodological principle, what this means varies across conceptions of DBR. For instance, in more traditional models, Brown and Campione ( 1996 ) pointed out the dangers of “lethal mutation” in which a design, introduced into a context, may become so warped by the influence, pressures, incomplete implementation, or misunderstanding of participants in the local context, that it no longer reflects or tests the theory under study. In short, a theory-driven intervention may be put in place, and then subsumed to such a degree by participants based on their understanding and needs, that it remains the original innovative design in name alone. The assertion here is that in these cases, the research ceases to be DBR in the sense that the design is no longer central, actively shaping learning. We cannot, they argue, analyze a design—and the theory it was meant to reflect—as an object of study when it has been “mutated,” and it is merely a banner under which participants are enacting their idiosyncratic, pragmatic needs.

While the ways in which settings and individuals might disrupt designs intended to produce robust learning is certainly a tension to be cautious of in DBR, it is also worth noting that in many critical approaches to DBR, such mutations—whether “lethal” to the original design or not—are seen as compelling and important moments. Here, where collaboration and community input is more central to the design process, iterative is understood differently. Thus, a “mutation” becomes a point where reflexivity, tension, and contradiction might open the door for change, for new designs, for reconsiderations of researcher and collaborative partner positionalities, or for ethnographic exploration into how a context takes up, shapes, and ultimately engages innovations in a particular sociocultural setting. In short, accounting for and documenting changes in design is a vital part of the DBR process, allowing researchers to respond to context in a variety of ways, always striving for their theories and designs to act with humility, and in the interest of usefulness .

With this in mind, the iterative nature of DBR means that the relationships researchers have with other design partners (educators and learners) in the ecology are incredibly important, and vital to consider (Bang et al., 2016 ; Engeström, 2007 ; Engeström, Sannino, & Virkkunen, 2014 ). Different iterations of DBR might occur in ways in which the researcher is more or less intimately involved in the design and implementation process, both in terms of actual presence and intellectual ownership of the design. Regarding the former, in some cases, a researcher may hand a design off to others to implement, periodically studying and modifying it, while in other contexts or designs, the researcher may be actively involved, tinkering in every detail of the implementation and enactment of the design. With regard to the latter, DBR might similarly range from a somewhat prescribed model, in which the researcher is responsible for the original design, and any modifications that may occur based on their analyses, without significant input from participants (e.g., Collins et al., 2004 ), to incredibly participatory models, in which all parties (researchers, educators, learners) are part of each step of the design-creation, modification, and research process (e.g., Bang, Faber, Gurneau, Marin, & Soto, 2016 ; Kirshner, 2015 ).

Considering the wide range of ideological approaches and models for DBR, we might acknowledge that DBR can be gainfully conducted through many iterations of “openness” to the design process. However, the strength of the research—focused on analyzing the design itself as a unit of study reflective of learning theory—will be bolstered by thoughtfully accounting for how involved the researcher will be, and how open to participation the modification process is. These answers should match the types of questions, and conceptual or ideological framing, with which researchers approach DBR, allowing them to tinker with the process of learning as they build on prior research to extend knowledge and test theory (Barab & Kirshner, 2001 ), while thoughtfully documenting these changes in the design as they go.

Implementation and Research Design

As with the overarching principles of design-based research (DBR), even amid the pluralism of conceptual frameworks of DBR researchers, it is possible, and useful, to trace the shared contours in how DBR research design is implemented. Though texts provide particular road maps for undertaking various iterations of DBR consistent with the specific goals, types of questions, and ideological orientations of these scholarly communities (e.g., Cole & Engeström, 2007 ; Collins, Joseph, & Bielaczyc, 2004 ; Fishman, Penuel, Allen, Cheng, & Sabelli, 2013 ; Gutiérrez & Jurow, 2016 ; Virkkunen & Newnham, 2013 ), certain elements, realized differently, can be found across all of these models, and may be encapsulated in five broad methodological phases.

Considering the Design Focus

DBR begins by considering what the focus of the design, the situated context, and the units of analysis for research will be. Prospective DBR researchers will need to consider broader research in regard to the “grand” theory of learning with which they work to determine what theoretical questions they have, or identify “intermediate” aspects of the theories that might be studied and strengthened by a design process in situated contexts, and what planes of analysis (Rogoff, 1995 ) will be most suitable for examination. This process allows for the identification of the critical theoretical elements of a design, and articulation of initial research questions.

Given the conceptual framework, theoretical and research questions, and sociopolitical interests at play, researchers may undertake this, and subsequent steps in the process, on their own, or in close collaboration with the communities and individuals in the situated contexts in which the design will unfold. As such, across iterations of DBR, and with respect to the ways DBR researchers choose to engage with communities, the origin of the design will vary, and might begin in some cases with theoretical questions, or arise in others as a problem of practice (Coburn & Penuel, 2016 ), though as has been noted, in either case, theory and practice are necessarily linked in the research.

Creating and Implementing a Designed Innovation

From the consideration and identification of the critical elements, planned units of analysis, and research questions that will drive a design, researchers can then actively create (either on their own or in conjunction with potential design partners) a designed intervention reflecting these critical elements, and the overarching theory.

Here, the DBR researcher should consider what partners exist in the process and what ownership exists around these partnerships, determine exactly what the pragmatic features of the intervention/design will be and who will be responsible for them, and consider when checkpoints for modification and evaluation will be undertaken, and by whom. Additionally, researchers should at this stage consider questions of timeline and of recruiting participants, as well as what research materials will be needed to adequately document the design, its implementation, and its outcomes, and how and where collected data will be stored.

Once a design (the planned, theory-informed innovative intervention) has been produced, the DBR researcher and partners can begin the implementation process, putting the design into place and beginning data collection and documentation.

Assessing the Impact of the Design on the Learning Ecology

Chronologically, the next two methodological steps happen recursively in the iterative process of DBR. The researcher must assess the impact of the design, and then, make modifications as necessary, before continuing to assess the impact of these modifications. In short, these next two steps are a cycle that continues across the life and length of the research design.

Once a design has been created and implemented, the researcher begins to observe and document the learning, the ecology, and the design itself. Guided by and in conversation with the theory and critical elements, the researcher should periodically engage in ongoing data analysis, assessing the success of the design, and of learning, paying equal attention to the design itself, and how its implementation is working in the situated ecology.

Within the realm of qualitative research, measuring or assessing variables of learning and assessing the design may look vastly different, require vastly different data-collection and data-analysis tools, and involve vastly different research methods among different researchers.

Modifying the Design

Modification, based on ongoing assessment of the design, is what makes DBR iterative, helping the researcher extend the field’s knowledge about the theory, design, learning, and the context under examination.

Modification of the design can take many forms, from complete changes in approach or curriculum, to introducing an additional tool or mediating artifact into a learning ecology. Moreover, how modification unfolds involves careful reflection from the researcher and any co-designing participants, deciding whether modification will be an ongoing, reflexive, tinkering process, or if it will occur only at predefined checkpoints, after formal evaluation and assessment. Questions of ownership, issues of resource availability, technical support, feasibility, and communication are all central to the work of design modification, and answers will vary given the research questions, design parameters, and researchers’ epistemic commitments.

Each moment of modification indicates a new phase in a DBR project, and a new round of assessing—through data analysis—the impact of the design on the learning ecology, either to guide continued or further modification, report the results of the design, or in some cases, both.

Reporting the Results of the Design

The final step in DBR methodology is to report on the results of the designed intervention, how it contributed to understandings of theory, and how it impacted the local learning ecology or context. The format, genre, and final data analysis methods used in reporting data and research results will vary across iterations of DBR. However, it is largely understood that to avoid methodological confusion, DBR researchers should clearly situate themselves in the DBR paradigm by clearly describing and detailing the design itself; articulating the theory, central elements, and units of analysis under scrutiny, what modifications occurred and what precipitated these changes, and what local effects were observed; and exploring any potential contributions to learning theory, while accounting for the context and their interventionist role and positionality in the design. As such, careful documentation of pragmatic and design decisions for retrospective data analysis, as well as research findings, should be done at each stage of this implementation process.

Methodological Issues in the Design-Based Research Paradigm

Because of its pluralistic nature, its interventionist, nontraditional stance, and the fact that it remains in its conceptual infancy, design-based research (DBR) is replete with ongoing methodological questions and challenges, both from external and internal sources. While there are many more that may exist, addressed will be several of the most pressing the prospective DBR researcher may encounter, or want to consider in understanding the paradigm and beginning a research design.

Challenges to Rigor and Validity

Perhaps the place to begin this reflection on tensions in the DBR paradigm is the recurrent and ongoing challenge to the rigor and validity of DBR, which has asked: Is DBR research at all? Given the interventionist and activist way in which DBR invites the researcher to participate, and the shift in orientation from long-accepted research paradigms, such critiques are hardly surprising, and fall in line with broader challenges to the rigor and objectivity of qualitative social science research in general. Historically, such complaints about DBR are linked to decades of critique of any research that does not adhere to the post-positivist approach set out as the U.S. Department of Education began to prioritize laboratory and large-scale randomized control-trial experimentation as the “gold standard” of research design (e.g., Mosteller & Boruch, 2002 ).

From the outset, DBR, as an interventionist, local, situated, non-laboratory methodology, was bound to run afoul of such conservative trends. While some researchers involved in (particularly traditional developmental and cognitive) DBR have found broader acceptance within these constraints, the rigor of DBR remains contested. It has been suggested that DBR is under-theorized and over-methologized, a haphazard way for researchers to do activist work without engaging in the development of robust knowledge claims about learning (Dede, 2004 ), and an approach lacking in coherence that sheltered interventionist projects of little impact to developing learning theory and allowed researchers to make subjective, pet claims through selective analysis of large bodies of collected data (Kelly, 2003 , 2004 ).

These critiques, however, impose an external set of criteria on DBR, desiring it to fit into the molds of rigor and coherence as defined by canonical methodologies. Bell ( 2004 ) and Bang and Vossoughi ( 2016 ) have made compelling cases for the wide variety of methods and approaches present in DBR not as a fracturing, but as a generative proliferation of different iterations that can offer powerful insights around the different types of questions that exist about learning in the infinitely diverse settings in which it occurs. Essentially, researchers have argued that within the DBR paradigm, and indeed within educational research more generally, the practical impact of research on learning, context, and practices should be a necessary component of rigor (Gutiérrez & Penuel, 2014 ), and the pluralism of methods and approaches available in DBR ensures that the practical impacts and needs of the varied contexts in which the research takes place will always drive the design and research tools.

These moves are emblematic of the way in which DBR is innovating and pushing on paradigms of rigor in educational research altogether, reflecting how DBR fills a complementary niche with respect to other methodologies and attends to elements and challenges of learning in lived, real environments that other types of research have consistently and historically missed. Beyond this, Brown ( 1992 ) was conscious of the concerns around data collection, validity, rigor, and objectivity from the outset, identifying this dilemma—the likelihood of having an incredible amount of data collected in a design only a small fraction of which can be reported and shared, thus leading potentially to selective data analysis and use—as the Bartlett Effect (Brown, 1992 ). Since that time, DBR researchers have been aware of this challenge, actively seeking ways to mitigate this threat to validity by making data sets broadly available, documenting their design, tinkering, and modification processes, clearly situating and describing disconfirming evidence and their own position in the research, and otherwise presenting the broad scope of human and learning activity that occurs within designs in large learning ecologies as comprehensively as possible.

Ultimately, however, these responses are likely to always be insufficient as evidence of rigor to some, for the root dilemma is around what “counts” as education science. While researchers interested and engaged in DBR ought rightly to continue to push themselves to ensure the methodological rigor of their work and chosen methods, it is also worth noting that DBR should seek to hold itself to its own criteria of assessment. This reflects broader trends in qualitative educational research that push back on narrow constructions of what “counts” as science, recognizing the ways in which new methodologies and approaches to research can help us examine aspects of learning, culture, and equity that have continued to be blind spots for traditional education research; invite new voices and perspectives into the process of achieving rigor and validity (Erickson & Gutiérrez, 2002 ); bolster objectivity by bringing it into conversation with the positionality of the researcher (Harding, 1993 ); and perhaps most important, engage in axiological innovation (Bang, Faber, Gurneau, Marin, & Soto, 2016 ), or the exploration of and design for what is, “good right, true, and beautiful . . . in cultural ecologies” (p. 2).

Questions of Generalizability and Usefulness

The generalizability of research results in DBR has been an ongoing and contentious issue in the development of the paradigm. Indeed, by the standards of canonical methods (e.g., laboratory experimentation, ethnography), these local, situated interventions should lack generalizability. While there is reason to discuss and question the merit of generalizability as a goal of qualitative research at all, researchers in the DBR paradigm have long been conscious of this issue. Understanding the question of generalizability around DBR, and how the paradigm has responded to it, can be done in two ways.

First, by distinguishing questions specific to a particular design from the generalizability of the theory. Cole’s (Cole & Underwood, 2013 ) 5th Dimension work, and the nationwide network of linked, theoretically similar sites, operating nationwide with vastly different designs, is a powerful example of this approach to generalizability. Rather than focus on a single, unitary, potentially generalizable design, the project is more interested in variability and sustainability of designs across local contexts (e.g., Cole, 1995 ; Gutiérrez, Bien, Selland, & Pierce, 2011 ; Jurow, Tracy, Hotchkiss, & Kirshner, 2012 ). Through attention to sustainable, locally effective innovations, conscious of the wide variation in culture and context that accompanies any and all learning processes, 5th Dimension sites each derive their idiosyncratic structures from sociocultural theory, sharing some elements, but varying others, while seeking their own “ontological innovations” based on the affordances of their contexts. This pattern reflects a key element of much of the DBR paradigm: that questions of generalizability in DBR may be about the generalizability of the theory of learning, and the variability of learning and design in distinct contexts, rather than the particular design itself.

A second means of addressing generalizability in DBR has been to embrace the pragmatic impacts of designing innovations. This response stems from Messick ( 1992 ) and Schoenfeld’s ( 1992 ) arguments early on in the development of DBR that the consequentialness and validity of DBR efforts as potentially generalizable research depend on the “ usefulness ” of the theories and designs that emerge. Effectively, because DBR is the examination of situated theory, a design must be able to show pragmatic impact—it must succeed at showing the theory to be useful . If there is evidence of usefulness to both the context in which it takes place, and the field of educational research more broadly, then the DBR researcher can stake some broader knowledge claims that might be generalizable. As a result, the DBR paradigm tends to “treat changes in [local] contexts as necessary evidence for the viability of a theory” (Barab & Squire, 2004 , p. 6). This of course does not mean that DBR is only interested in successful efforts. A design that fails or struggles can provide important information and knowledge to the field. Ultimately, though, DBR tends to privilege work that proves the usefulness of designs, whose pragmatic or theoretical findings can then be generalized within the learning science and education research fields.

With this said, the question of usefulness is not always straightforward, and is hardly unitary. While many DBR efforts—particularly those situated in developmental and cognitive learning science traditions—are interested in the generalizability of their useful educational designs (Barab & Squire, 2004 ; Cobb, Confrey, diSessa, Lehrer, & Schauble, 2003 ; Joseph, 2004 ; Steffe & Thompson, 2000 ), not all are. Critical DBR researchers have noted that if usefulness remains situated in the extant sociopolitical and sociocultural power-structures—dominant conceptual and popular definitions of what useful educational outcomes are—the result will be a bar for research merit that inexorably bends toward the positivist spectrum (Booker & Goldman, 2016 ; Dominguez, 2015 ; Zavala, 2016 ). This could potentially, and likely, result in excluding the non-normative interventions and innovations that are vital for historically marginalized communities, but which might have vastly different-looking outcomes, that are nonetheless useful in the sociopolitical context they occur in. Alternative framings to this idea of usefulness push on and extend the intention, and seek to involve the perspectives and agency of situated community partners and their practices in what “counts” as generative and rigorous research outcomes (Gutiérrez & Penuel, 2014 ). An example in this regard is the idea of consequential knowledge (Hall & Jurow, 2015 ; Jurow & Shea, 2015 ), which suggests outcomes that are consequential will be taken up by participants in and across their networks, and over-time—thus a goal of consequential knowledge certainly meets the standard of being useful , but it also implicates the needs and agency of communities in determining the success and merit of a design or research endeavor in important ways that strict usefulness may miss.

Thus, the bar of usefulness that characterizes the DBR paradigm should not be approached without critical reflection. Certainly designs that accomplish little for local contexts should be subject to intense questioning and critique, but considering the sociopolitical and systemic factors that might influence what “counts” as useful in local contexts and education science more generally, should be kept firmly in mind when designing, choosing methods, and evaluating impacts (Zavala, 2016 ). Researchers should think deeply about their goals, whether they are reaching for generalizability at all, and in what ways they are constructing contextual definitions of success, and be clear about these ideologically influenced answers in their work, such that generalizability and the usefulness of designs can be adjudicated based on and in conversation with the intentions and conceptual framework of the research and researcher.

Ethical Concerns of Sustainability, Participation, and Telos

While there are many external challenges to rigor and validity of DBR, another set of tensions comes from within the DBR paradigm itself. Rather than concerns about rigor or validity, these internal critiques are not unrelated to the earlier question of the contested definition of usefulness , and more accurately reflect questions of research ethics and grow from ideological concerns with how an intentional, interventionist stance is taken up in research as it interacts with situated communities.

Given that the nature of DBR is to design and implement some form of educational innovation, the DBR researcher will in some way be engaging with an individual or community, becoming part of a situated learning ecology, complete with a sociopolitical and cultural history. As with any research that involves providing an intervention or support, the question of what happens when the research ends is as much an ethical as a methodological one. Concerns then arise given how traditional models of DBR seem intensely focused on creating and implementing a “complete” cycle of design, but giving little attention to what happens to the community and context afterward (Engeström, 2011 ). In contrast to this privileging of “completeness,” sociocultural and critical approaches to DBR have suggested that if research is actually happening in naturalistic, situated contexts that authentically recognize and allow social and cultural dimensions to function (i.e., avoid laboratory-type controls to mitigate independent variables), there can never be such a thing as “complete,” for the design will, and should, live on as part of the ecology of the space (Cole, 2007 ; Engeström, 2000 ). Essentially, these internal critiques push DBR to consider sustainability, and sustainable scale, as equally important concerns to the completeness of an innovation. Not only are ethical questions involved, but accounting for the unbounded and ongoing nature of learning as a social and cultural activity can help strengthen the viability of knowledge claims made, and what degree of generalizability is reasonably justified.

Related to this question of sustainability are internal concerns regarding the nature and ethics of participation in DBR, whether partners in a design are being adequately invited to engage in the design and modification processes that will unfold in their situated contexts and lived communities (Bang et al., 2016 ; Engeström, 2011 ). DBR has actively sought to examine multiple planes of analysis in learning that might be occurring in a learning ecology but has rarely attended to the subject-subject dynamics (Bang et al., 2016 ), or “relational equity” (DiGiacomo & Gutiérrez, 2015 ) that exists between researchers and participants as a point of focus. Participatory design research (PDR) (Bang & Vossoughi, 2016 ) models have recently emerged as a way to better attend to these important dimensions of collective participation (Engeström, 2007 ), power (Vakil et al., 2016 ), positionality (Kirshner, 2015 ), and relational agency (Edwards, 2007 , 2009 ; Sannino & Engeström, 2016 ) as they unfold in DBR.

Both of these ethical questions—around sustainability and participation—reflect challenges to what we might call the telos —or direction—that DBR takes to innovation and research. These are questions related to whose voices are privileged, in what ways, for what purposes, and toward what ends. While DBR, like many other forms of educational research, has involved work with historically marginalized communities, it has, like many other forms of educational research, not always done so in humanizing ways. Put another way, there are ethical and political questions surrounding whether the designs, goals, and standards of usefulness we apply to DBR efforts should be purposefully activist, and have explicitly liberatory ends. To this point, critical and decolonial perspectives have pushed on the DBR paradigm, suggesting that DBR should situate itself as being a space of liberatory innovation and potential, in which communities and participants can become designers and innovators of their own futures (Gutiérrez, 2005 ). This perspective is reflected in the social design experiment (SDE) approach to DBR (Gutiérrez, 2005 , 2008 ; Gutierréz & Vossoughi, 2010 ; Gutiérrez, 2016 ; Gutiérrez & Jurow, 2016 ), which begins in participatory fashion, engaging a community in identifying its own challenges and desires, and reflecting on the historicity of learning practices, before proleptic design efforts are undertaken that ensure that research is done with , not on , communities of color (Arzubiaga, Artiles, King, & Harris-Murri, 2008 ), and intentionally focused on liberatory goals.

Global Perspectives and Unique Iterations

While design-based research (DBR) has been a methodology principally associated with educational research in the United States, its development is hardly limited to the U.S. context. Rather, while DBR emerged in U.S. settings, similar methods of situated, interventionist research focused on design and innovation were emerging in parallel in European contexts (e.g., Gravemeijer, 1994 ), most significantly in the work of Vygotskian scholars both in Europe and the United States (Cole, 1995 ; Cole & Engeström, 1993 , 2007 ; Engeström, 1987 ).

Particularly, where DBR began in the epistemic and ontological terrain of developmental and cognitive psychology, this vein of design-based research work began deeply grounded in cultural-historical activity theory (CHAT). This ontological and epistemic grounding meant that the approach to design that was taken was more intensively conscious of context, historicity, hybridity, and relational factors, and framed around understanding learning as a complex, collective activity system that, through design, could be modified and transformed (Cole & Engeström, 2007 ). The models of DBR that emerged in this context abroad were the formative intervention (Engeström, 2011 ; Engeström, Sannino, & Virkkunen, 2014 ), which relies heavily on Vygotskian double-stimulation to approach learning in nonlinear, unbounded ways, accounting for the role of learner, educator, and researcher in a collective process, shifting and evolving and tinkering with the design as the context needs and demands; and the Change Laboratory (Engeström, 2008 ; Virkkunen & Newnham, 2013 ), which similarly relies on the principle of double stimulation, while presenting holistic way to approach transforming—or changing—entire learning activity systems in fundamental ways through designs that encourage collective “expansive learning” (Engeström, 2001 ), through which participants can produce wholly new activity systems as the object of learning itself.

Elsewhere in the United States, still parallel to the developmental- or cognitive-oriented DBR work that was occurring, American researchers employing CHAT began to leverage the tools and aims of expansive learning in conversation with the tensions and complexity of the U.S. context (Cole, 1995 ; Gutiérrez, 2005 ; Gutiérrez & Rogoff, 2003 ). Like the CHAT design research of the European context, there was a focus on activity systems, historicity, nonlinear and unbounded learning, and collective learning processes and outcomes. Rather than a simple replication, however, these researchers put further attention on questions of equity, diversity, and justice in this work, as Gutiérrez, Engeström, and Sannino ( 2016 ) note:

The American contribution to a cultural historical activity theoretic perspective has been its attention to diversity, including how we theorize, examine, and represent individuals and their communities. (p. 276)

Effectively, CHAT scholars in parts of the United States brought critical and decolonial perspectives to bear on their design-focused research, focusing explicitly on the complex cultural, racial, and ethnic terrain in which they worked, and ensuring that diversity, equity, justice, and non-dominant perspectives would become central principles to the types of design research conducted. The result was the emergence of the aforementioned social design experiments (e.g., Gutiérrez, 2005 , 2016 ), and participatory design research (Bang & Vossoughi, 2016 ) models, which attend intentionally to historicity and relational equity, tailor their methods to the liberation of historically marginalized communities, aim intentionally for liberatory outcomes as key elements of their design processes, and seek to produce outcomes in which communities of learners become designers of new community futures (Gutiérrez, 2016 ). While these approaches emerged in the United States, their origins reflect ontological and ideological perspectives quite distinct from more traditional learning science models of DBR, and dominant U.S. ontologies in general. Indeed, these iterations of DBR are linked genealogically to the ontologies, ideologies, and concerns of peoples in the Global South, offering some promise for the method in those regions, though DBR has yet to broadly take hold among researchers beyond the United States and Europe.

There is, of course, much more nuance to these models, and each of these models (formative interventions, Change Laboratories, social design experiments, and participatory design research) might itself merit independent exploration and review well beyond the scope here. Indeed, there is some question as to whether all adherents of these CHAT design-based methodologies, with their unique genealogies and histories, would even consider themselves under the umbrella of DBR. Yet, despite significant ontological divergences, these iterations share many of the same foundational tenets of the traditional models (though realized differently), and it is reasonable to argue that they do indeed share the same, broad methodological paradigm (DBR), or at the very least, are so intimately related that any discussion of DBR, particularly one with a global view, should consider the contributions CHAT iterations have made to the DBR methodology in the course of their somewhat distinct, but parallel, development.

Possibilities and Potentials for Design-Based Research

Since its emergence in 1992 , the DBR methodology for educational research has continued to grow in popularity, ubiquity, and significance. Its use has begun to expand beyond the confines of the learning sciences, taken up by researchers in a variety of disciplines, and across a breadth of theoretical and intellectual traditions. While still not as widely recognized as more traditional and well-established research methodologies, DBR as a methodology for rigorous research is unquestionably here to stay.

With this in mind, the field ought to still be cautious of the ways in which the discourse of design is used. Not all design is DBR, and preserving the integrity, rigor, and research ethics of the paradigm (on its own terms) will continue to require thoughtful reflection as its pluralistic parameters come into clearer focus. Yet the proliferation of methods in the DBR paradigm should be seen as a positive. There are far too many theories of learning and ideological perspectives that have meaningful contributions to make to our knowledge of the world, communities, and learning to limit ourselves to a unitary approach to DBR, or set of methods. The paradigm has shown itself to have some core methodological principles, but there is no reason not to expect these to grow, expand, and evolve over time.

In an increasingly globalized, culturally diverse, and dynamic world, there is tremendous potential for innovation couched in this proliferation of DBR. Particularly in historically marginalized communities and across the Global South, we will need to know how learning theories can be lived out in productive ways in communities that have been understudied, and under-engaged. The DBR paradigm generally, and critical and CHAT iterations particularly, can fill an important need for participatory, theory-developing research in these contexts that simultaneously creates lived impacts. Participatory design research (PDR), social design experiments (SDE), and Change Laboratory models of DBR should be of particular interest and attention moving forward, as current trends toward culturally sustaining pedagogies and learning will need to be explored in depth and in close collaboration with communities, as participatory design partners, in the press toward liberatory educational innovations.

Bibliography

The following special issues of journals are encouraged starting points for engaging more deeply with current and past trends in design-based research.

• Bang, M. , & Vossoughi, S. (Eds.). (2016). Participatory design research and educational justice: Studying learning and relations within social change making [Special issue]. Cognition and Instruction , 34 (3).
• Barab, S. (Ed.). (2004). Design-based research [Special issue]. Journal of the Learning Sciences , 13 (1).
• Cole, M. , & The Distributed Literacy Consortium. (2006). The Fifth Dimension: An after-school program built on diversity . New York, NY: Russell Sage Foundation.
• Kelly, A. E. (Ed.). (2003). Special issue on the role of design in educational research [Special issue]. Educational Researcher , 32 (1).
• Arzubiaga, A. , Artiles, A. , King, K. , & Harris-Murri, N. (2008). Beyond research on cultural minorities: Challenges and implications of research as situated cultural practice. Exceptional Children , 74 (3), 309–327.
• Bang, M. , Faber, L. , Gurneau, J. , Marin, A. , & Soto, C. (2016). Community-based design research: Learning across generations and strategic transformations of institutional relations toward axiological innovations. Mind, Culture, and Activity , 23 (1), 28–41.
• Bang, M. , & Vossoughi, S. (2016). Participatory design research and educational justice: Studying learning and relations within social change making. Cognition and Instruction , 34 (3), 173–193.
• Barab, S. , Kinster, J. G. , Moore, J. , Cunningham, D. , & The ILF Design Team. (2001). Designing and building an online community: The struggle to support sociability in the Inquiry Learning Forum. Educational Technology Research and Development , 49 (4), 71–96.
• Barab, S. , & Squire, K. (2004). Design-based research: Putting a stake in the ground. Journal of the Learning Sciences , 13 (1), 1–14.
• Barab, S. A. , & Kirshner, D. (2001). Methodologies for capturing learner practices occurring as part of dynamic learning environments. Journal of the Learning Sciences , 10 (1–2), 5–15.
• Bell, P. (2004). On the theoretical breadth of design-based research in education. Educational Psychologist , 39 (4), 243–253.
• Bereiter, C. , & Scardamalia, M. (1989). Intentional learning as a goal of instruction. In L. B. Resnick (Ed.), Knowing, learning, and instruction: Essays in honor of Robert Glaser (pp. 361–392). Hillsdale, NJ: Lawrence Erlbaum.
• Booker, A. , & Goldman, S. (2016). Participatory design research as a practice for systemic repair: Doing hand-in-hand math research with families. Cognition and Instruction , 34 (3), 222–235.
• Brown, A. L. (1992). Design experiments: Theoretical and methodological challenges in creating complex interventions in classroom settings. Journal of the Learning Sciences , 2 (2), 141–178.
• Brown, A. , & Campione, J. C. (1996). Psychological theory and the design of innovative learning environments: On procedures, principles, and systems. In L. Schauble & R. Glaser (Eds.), Innovations in learning: New environments for education (pp. 289–325). Mahwah, NJ: Lawrence Erlbaum.
• Brown, A. L. , & Campione, J. C. (1998). Designing a community of young learners: Theoretical and practical lessons. In N. M. Lambert & B. L. McCombs (Eds.), How students learn: Reforming schools through learner-centered education (pp. 153–186). Washington, DC: American Psychological Association.
• Brown, A. , Campione, J. , Webber, L. , & McGilley, K. (1992). Interactive learning environments—A new look at learning and assessment. In B. R. Gifford & M. C. O’Connor (Eds.), Future assessment: Changing views of aptitude, achievement, and instruction (pp. 121–211). Boston, MA: Academic Press.
• Carnoy, M. , Jacobsen, R. , Mishel, L. , & Rothstein, R. (2005). The charter school dust-up: Examining the evidence on enrollment and achievement . Washington, DC: Economic Policy Institute.
• Carspecken, P. (1996). Critical ethnography in educational research . New York, NY: Routledge.
• Cobb, P. , Confrey, J. , diSessa, A. , Lehrer, R. , & Schauble, L. (2003). Design experiments in educational research. Educational Researcher , 32 (1), 9–13.
• Cobb, P. , & Steffe, L. P. (1983). The constructivist researcher as teacher and model builder. Journal for Research in Mathematics Education , 14 , 83–94.
• Coburn, C. , & Penuel, W. (2016). Research-practice partnerships in education: Outcomes, dynamics, and open questions. Educational Researcher , 45 (1), 48–54.
• Cole, M. (1995). From Moscow to the Fifth Dimension: An exploration in romantic science. In M. Cole & J. Wertsch (Eds.), Contemporary implications of Vygotsky and Luria (pp. 1–38). Worcester, MA: Clark University Press.
• Cole, M. (1996). Cultural psychology: A once and future discipline . Cambridge, MA: Harvard University Press.
• Cole, M. (2007). Sustaining model systems of educational activity: Designing for the long haul. In J. Campione , K. Metz , & A. S. Palinscar (Eds.), Children’s learning in and out of school: Essays in honor of Ann Brown (pp. 71–89). New York, NY: Routledge.
• Cole, M. , & Engeström, Y. (1993). A cultural historical approach to distributed cognition. In G. Saloman (Ed.), Distributed cognitions: Psychological and educational considerations (pp. 1–46). Cambridge, U.K.: Cambridge University Press.
• Cole, M. , & Engeström, Y. (2007). Cultural-historical approaches to designing for development. In J. Valsiner & A. Rosa (Eds.), The Cambridge handbook of sociocultural psychology , Cambridge, U.K.: Cambridge University Press.
• Cole, M. , & Underwood, C. (2013). The evolution of the 5th Dimension. In The Story of the Laboratory of Comparative Human Cognition: A polyphonic autobiography . https://lchcautobio.ucsd.edu/polyphonic-autobiography/section-5/chapter-12-the-later-life-of-the-5th-dimension-and-its-direct-progeny/ .
• Collins, A. (1992). Toward a design science of education. In E. Scanlon & T. O’Shea (Eds.), New directions in educational technology (pp. 15–22). New York, NY: Springer-Verlag.
• Collins, A. , Joseph, D. , & Bielaczyc, K. (2004). Design research: Theoretical and methodological issues. Journal of the Learning Sciences , 13 (1), 15–42.
• Dede, C. (2004). If design-based research is the answer, what is the question? A commentary on Collins, Joseph, and Bielaczyc; DiSessa and Cobb; and Fishman, Marx, Blumenthal, Krajcik, and Soloway in the JLS special issue on design-based research. Journal of the Learning Sciences , 13 (1), 105–114.
• Design-Based Research Collective . (2003). Design-based research: An emerging paradigm for educational inquiry. Educational Researcher , 32 (1), 5–8.
• DiGiacomo, D. , & Gutiérrez, K. D. (2015). Relational equity as a design tool within making and tinkering activities. Mind, Culture, and Activity , 22 (3), 1–15.
• diSessa, A. A. (1991). Local sciences: Viewing the design of human-computer systems as cognitive science. In J. M. Carroll (Ed.), Designing interaction: Psychology at the human-computer interface (pp. 162–202). Cambridge, U.K.: Cambridge University Press.
• diSessa, A. A. , & Cobb, P. (2004). Ontological innovation and the role of theory in design experiments. Journal of the Learning Sciences , 13 (1), 77–103.
• diSessa, A. A. , & Minstrell, J. (1998). Cultivating conceptual change with benchmark lessons. In J. G. Greeno & S. Goldman (Eds.), Thinking practices (pp. 155–187). Mahwah, NJ: Lawrence Erlbaum.
• Dominguez, M. (2015). Decolonizing teacher education: Explorations of expansive learning and culturally sustaining pedagogy in a social design experiment (Doctoral dissertation). University of Colorado, Boulder.
• Edelson, D. (2002). Design research: What we learn when we engage in design. Journal of the Learning Sciences , 11 (1), 105–121.
• Edwards, A. (2007). Relational agency in professional practice: A CHAT analysis. Actio: An International Journal of Human Activity Theory , 1 , 1–17.
• Edwards, A. (2009). Agency and activity theory: From the systemic to the relational. In A. Sannino , H. Daniels , & K. Gutiérrez (Eds.), Learning and expanding with activity theory (pp. 197–211). Cambridge, U.K.: Cambridge University Press.
• Engeström, Y. (1987). Learning by expanding . Helsinki, Finland: University of Helsinki, Department of Education.
• Engeström, Y. (2000). Can people learn to master their future? Journal of the Learning Sciences , 9 , 525–534.
• Engeström, Y. (2001). Expansive learning at work: Toward an activity theoretical reconceptualization. Journal of Education and Work , 14 (1), 133–156.
• Engeström, Y. (2007). Enriching the theory of expansive learning: Lessons from journeys toward co-configuration. Mind, Culture, and Activity , 14 (1–2), 23–39.
• Engeström, Y. (2008). Putting Vygotksy to work: The Change Laboratory as an application of double stimulation. In H. Daniels , M. Cole , & J. Wertsch (Eds.), Cambridge companion to Vygotsky (pp. 363–382). New York, NY: Cambridge University Press.
• Engeström, Y. (2011). From design experiments to formative interventions. Theory & Psychology , 21 (5), 598–628.
• Engeström, Y. , Engeström, R. , & Kärkkäinen, M. (1995). Polycontextuality and boundary crossing in expert cognition: Learning and problem solving in complex work activities. Learning and Instruction , 5 (4), 319–336.
• Engeström, Y. , & Sannino, A. (2010). Studies of expansive learning: Foundations, findings and future challenges. Educational Research Review , 5 (1), 1–24.
• Engeström, Y. , & Sannino, A. (2011). Discursive manifestations of contradictions in organizational change efforts: A methodological framework. Journal of Organizational Change Management , 24 (3), 368–387.
• Engeström, Y. , Sannino, A. , & Virkkunen, J. (2014). On the methodological demands of formative interventions. Mind, Culture, and Activity , 2 (2), 118–128.
• Erickson, F. , & Gutiérrez, K. (2002). Culture, rigor, and science in educational research. Educational Researcher , 31 (8), 21–24.
• Espinoza, M. (2009). A case study of the production of educational sanctuary in one migrant classroom. Pedagogies: An International Journal , 4 (1), 44–62.
• Espinoza, M. L. , & Vossoughi, S. (2014). Perceiving learning anew: Social interaction, dignity, and educational rights. Harvard Educational Review , 84 (3), 285–313.
• Fine, M. (1994). Dis-tance and other stances: Negotiations of power inside feminist research. In A. Gitlin (Ed.), Power and method (pp. 13–25). New York, NY: Routledge.
• Fishman, B. , Penuel, W. , Allen, A. , Cheng, B. , & Sabelli, N. (2013). Design-based implementation research: An emerging model for transforming the relationship of research and practice. National Society for the Study of Education , 112 (2), 136–156.
• Gravemeijer, K. (1994). Educational development and developmental research in mathematics education. Journal for Research in Mathematics Education , 25 (5), 443–471.
• Gutiérrez, K. (2005). Intersubjectivity and grammar in the third space . Scribner Award Lecture.
• Gutiérrez, K. (2008). Developing a sociocritical literacy in the third space. Reading Research Quarterly , 43 (2), 148–164.
• Gutiérrez, K. (2016). Designing resilient ecologies: Social design experiments and a new social imagination. Educational Researcher , 45 (3), 187–196.
• Gutiérrez, K. , Bien, A. , Selland, M. , & Pierce, D. M. (2011). Polylingual and polycultural learning ecologies: Mediating emergent academic literacies for dual language learners. Journal of Early Childhood Literacy , 11 (2), 232–261.
• Gutiérrez, K. , Engeström, Y. , & Sannino, A. (2016). Expanding educational research and interventionist methodologies. Cognition and Instruction , 34 (2), 275–284.
• Gutiérrez, K. , & Jurow, A. S. (2016). Social design experiments: Toward equity by design. Journal of Learning Sciences , 25 (4), 565–598.
• Gutiérrez, K. , & Penuel, W. R. (2014). Relevance to practice as a criterion for rigor. Educational Researcher , 43 (1), 19–23.
• Gutiérrez, K. , & Rogoff, B. (2003). Cultural ways of learning: Individual traits or repertoires of practice. Educational Researcher , 32 (5), 19–25.
• Gutierréz, K. , & Vossoughi, S. (2010). Lifting off the ground to return anew: Mediated praxis, transformative learning, and social design experiments. Journal of Teacher Education , 61 (1–2), 100–117.
• Hall, R. , & Jurow, A. S. (2015). Changing concepts in activity: Descriptive and design studies of consequential learning in conceptual practices. Educational Psychologist , 50 (3), 173–189.
• Harding, S. (1993). Rethinking standpoint epistemology: What is “strong objectivity”? In L. Alcoff & E. Potter (Eds.), Feminist epistemologies (pp. 49–82). New York, NY: Routledge.
• Hoadley, C. (2002). Creating context: Design-based research in creating and understanding CSCL. In G. Stahl (Ed.), Computer support for collaborative learning 2002 (pp. 453–462). Mahwah, NJ: Lawrence Erlbaum.
• Hoadley, C. (2004). Methodological alignment in design-based research. Educational Psychologist , 39 (4), 203–212.
• Joseph, D. (2004). The practice of design-based research: Uncovering the interplay between design, research, and the real-world context. Educational Psychologist , 39 (4), 235–242.
• Jurow, A. S. , & Shea, M. V. (2015). Learning in equity-oriented scale-making projects. Journal of the Learning Sciences , 24 (2), 286–307.
• Jurow, S. , Tracy, R. , Hotchkiss, J. , & Kirshner, B. (2012). Designing for the future: How the learning sciences can inform the trajectories of preservice teachers. Journal of Teacher Education , 63 (2), 147–60.
• Kärkkäinen, M. (1999). Teams as breakers of traditional work practices: A longitudinal study of planning and implementing curriculum units in elementary school teacher teams . Helsinki, Finland: University of Helsinki, Department of Education.
• Kelly, A. (2004). Design research in education: Yes, but is it methodological? Journal of the Learning Sciences , 13 (1), 115–128.
• Kelly, A. E. , & Sloane, F. C. (2003). Educational research and the problems of practice. Irish Educational Studies , 22 , 29–40.
• Kirshner, B. (2015). Youth activism in an era of education inequality . New York: New York University Press.
• Kirshner, B. , & Polman, J. L. (2013). Adaptation by design: A context-sensitive, dialogic approach to interventions. National Society for the Study of Education Yearbook , 112 (2), 215–236.
• Leander, K. M. , Phillips, N. C. , & Taylor, K. H. (2010). The changing social spaces of learning: Mapping new mobilities. Review of Research in Education , 34 , 329–394.
• Lesh, R. A. , & Kelly, A. E. (2000). Multi-tiered teaching experiments. In A. E. Kelly & R. A. Lesh (Eds.), Handbook of research design in mathematics and science education (pp. 197–230). Mahwah, NJ: Lawrence Erlbaum.
• Matusov, E. (1996). Intersubjectivty without agreement. Mind, Culture, and Activity , 3 (1), 29–45.
• Messick, S. (1992). The interplay of evidence and consequences in the validation of performance assessments. Educational Researcher , 23 (2), 13–23.
• Mosteller, F. , & Boruch, R. F. (Eds.). (2002). Evidence matters: Randomized trials in education research . Washington, DC: Brookings Institution Press.
• Newman, D. , Griffin, P. , & Cole, M. (1989). The construction zone: Working for cognitive change in school . London, U.K.: Cambridge University Press.
• Penuel, W. R. , Fishman, B. J. , Cheng, B. H. , & Sabelli, N. (2011). Organizing research and development at the intersection of learning, implementation, and design. Educational Researcher , 40 (7), 331–337.
• Polman, J. L. (2000). Designing project-based science: Connecting learners through guided inquiry . New York, NY: Teachers College Press.
• Ravitch, D. (2010). The death and life of the great American school system: How testing and choice are undermining education . New York, NY: Basic Books.
• Rogoff, B. (1990). Apprenticeship in thinking: Cognitive development in social context . New York, NY: Oxford University Press.
• Rogoff, B. (1995). Observing sociocultural activity on three planes: Participatory appropriation, guided participation, and apprenticeship. In J. V. Wertsch , P. D. Rio , & A. Alvarez (Eds.), Sociocultural studies of mind (pp. 139–164). Cambridge U.K.: Cambridge University Press.
• Saltman, K. J. (2007). Capitalizing on disaster: Taking and breaking public schools . Boulder, CO: Paradigm.
• Salvador, T. , Bell, G. , & Anderson, K. (1999). Design ethnography. Design Management Journal , 10 (4), 35–41.
• Sannino, A. (2011). Activity theory as an activist and interventionist theory. Theory & Psychology , 21 (5), 571–597.
• Sannino, A. , & Engeström, Y. (2016). Relational agency, double stimulation and the object of activity: An intervention study in a primary school. In A. Edwards (Ed.), Working relationally in and across practices: Cultural-historical approaches to collaboration (pp. 58–77). Cambridge, U.K.: Cambridge University Press.
• Scardamalia, M. , & Bereiter, C. (1991). Higher levels of agency for children in knowledge building: A challenge for the design of new knowledge media. Journal of the Learning Sciences , 1 , 37–68.
• Schoenfeld, A. H. (1982). Measures of problem solving performance and of problem solving instruction. Journal for Research in Mathematics Education , 13 , 31–49.
• Schoenfeld, A. H. (1985). Mathematical problem solving . Orlando, FL: Academic Press.
• Schoenfeld, A. H. (1992). On paradigms and methods: What do you do when the ones you know don’t do what you want them to? Issues in the analysis of data in the form of videotapes. Journal of the Learning Sciences , 2 (2), 179–214.
• Scribner, S. , & Cole, M. (1978). Literacy without schooling: Testing for intellectual effects. Harvard Educational Review , 48 (4), 448–461.
• Shavelson, R. J. , Phillips, D. C. , Towne, L. , & Feuer, M. J. (2003). On the science of education design studies. Educational Researcher , 32 (1), 25–28.
• Steffe, L. P. , & Thompson, P. W. (2000). Teaching experiment methodology: Underlying principles and essential elements. In A. Kelly & R. Lesh (Eds.), Handbook of research design in mathematics and science education (pp. 267–307). Mahwah, NJ: Erlbaum.
• Stevens, R. (2000). Divisions of labor in school and in the workplace: Comparing computer and paper-supported activities across settings. Journal of the Learning Sciences , 9 (4), 373–401.
• Suchman, L. (1995). Making work visible. Communications of the ACM , 38 (9), 57–64.
• Vakil, S. , de Royston, M. M. , Nasir, N. , & Kirshner, B. (2016). Rethinking race and power in design-based research: Reflections from the field. Cognition and Instruction , 34 (3), 194–209.
• van den Akker, J. (1999). Principles and methods of development research. In J. van den Akker , R. M. Branch , K. Gustafson , N. Nieveen , & T. Plomp (Eds.), Design approaches and tools in education and training (pp. 1–14). Boston, MA: Kluwer Academic.
• Virkkunen, J. , & Newnham, D. (2013). The Change Laboratory: A tool for collaborative development of work and education . Rotterdam, The Netherlands: Sense.
• White, B. Y. , & Frederiksen, J. R. (1998). Inquiry, modeling, and metacognition: Making science accessible to all students. Cognition and Instruction , 16 , 3–118.
• Zavala, M. (2016). Design, participation, and social change: What design in grassroots spaces can teach learning scientists. Cognition and Instruction , 34 (3), 236–249.

1. The reader should note the emergence of critical ethnography (e.g., Carspecken, 1996 ; Fine, 1994 ), and other more participatory models of ethnography that deviated from this traditional paradigm during this same time period. These new forms of ethnography comprised part of the genealogy of the more critical approaches to DBR, described later in this article.

2. The reader will also note that the adjective “qualitative” largely drops away from the acronym “DBR.” This is largely because, as described, DBR, as an exploration of naturalistic ecologies with multitudes of variables, and social and learning dynamics, necessarily demands a move beyond what can be captured by quantitative measurement alone. The qualitative nature of the research is thus implied and embedded as part of what makes DBR a unique and distinct methodology.

Related Articles

• Qualitative Data Analysis
• The Entanglements of Ethnography and Participatory Action Research (PAR) in Educational Research in North America
• Writing Educational Ethnography
• Qualitative Data Analysis and the Use of Theory
• Comparative Case Study Research
• Use of Qualitative Methods in Evaluation Studies
• Writing Qualitative Dissertations
• Ethnography in Early Childhood Education
• A History of Qualitative Research in Education in China
• Qualitative Research in the Field of Popular Education
• Qualitative Methodological Considerations for Studying Undocumented Students in the United States
• Culturally Responsive Evaluation as a Form of Critical Qualitative Inquiry
• Participatory Action Research in Education
• Complexity Theory as a Guide to Qualitative Methodology in Teacher Education
• Observing Schools and Classrooms

Printed from Oxford Research Encyclopedias, Education. Under the terms of the licence agreement, an individual user may print out a single article for personal use (for details see Privacy Policy and Legal Notice).

date: 24 April 2024

• Cookie Policy
• Privacy Policy
• Legal Notice
• Accessibility
• [66.249.64.20|185.126.86.119]
• 185.126.86.119

Character limit 500 /500

• Privacy Policy

Buy Me a Coffee

Home » Qualitative Research – Methods, Analysis Types and Guide

Qualitative Research – Methods, Analysis Types and Guide

Table of Contents

Qualitative Research

Qualitative research is a type of research methodology that focuses on exploring and understanding people’s beliefs, attitudes, behaviors, and experiences through the collection and analysis of non-numerical data. It seeks to answer research questions through the examination of subjective data, such as interviews, focus groups, observations, and textual analysis.

Qualitative research aims to uncover the meaning and significance of social phenomena, and it typically involves a more flexible and iterative approach to data collection and analysis compared to quantitative research. Qualitative research is often used in fields such as sociology, anthropology, psychology, and education.

Qualitative Research Methods

Qualitative Research Methods are as follows:

One-to-One Interview

This method involves conducting an interview with a single participant to gain a detailed understanding of their experiences, attitudes, and beliefs. One-to-one interviews can be conducted in-person, over the phone, or through video conferencing. The interviewer typically uses open-ended questions to encourage the participant to share their thoughts and feelings. One-to-one interviews are useful for gaining detailed insights into individual experiences.

Focus Groups

This method involves bringing together a group of people to discuss a specific topic in a structured setting. The focus group is led by a moderator who guides the discussion and encourages participants to share their thoughts and opinions. Focus groups are useful for generating ideas and insights, exploring social norms and attitudes, and understanding group dynamics.

Ethnographic Studies

This method involves immersing oneself in a culture or community to gain a deep understanding of its norms, beliefs, and practices. Ethnographic studies typically involve long-term fieldwork and observation, as well as interviews and document analysis. Ethnographic studies are useful for understanding the cultural context of social phenomena and for gaining a holistic understanding of complex social processes.

Text Analysis

This method involves analyzing written or spoken language to identify patterns and themes. Text analysis can be quantitative or qualitative. Qualitative text analysis involves close reading and interpretation of texts to identify recurring themes, concepts, and patterns. Text analysis is useful for understanding media messages, public discourse, and cultural trends.

This method involves an in-depth examination of a single person, group, or event to gain an understanding of complex phenomena. Case studies typically involve a combination of data collection methods, such as interviews, observations, and document analysis, to provide a comprehensive understanding of the case. Case studies are useful for exploring unique or rare cases, and for generating hypotheses for further research.

Process of Observation

This method involves systematically observing and recording behaviors and interactions in natural settings. The observer may take notes, use audio or video recordings, or use other methods to document what they see. Process of observation is useful for understanding social interactions, cultural practices, and the context in which behaviors occur.

Record Keeping

This method involves keeping detailed records of observations, interviews, and other data collected during the research process. Record keeping is essential for ensuring the accuracy and reliability of the data, and for providing a basis for analysis and interpretation.

This method involves collecting data from a large sample of participants through a structured questionnaire. Surveys can be conducted in person, over the phone, through mail, or online. Surveys are useful for collecting data on attitudes, beliefs, and behaviors, and for identifying patterns and trends in a population.

Qualitative data analysis is a process of turning unstructured data into meaningful insights. It involves extracting and organizing information from sources like interviews, focus groups, and surveys. The goal is to understand people’s attitudes, behaviors, and motivations

Qualitative Research Analysis Methods

Qualitative Research analysis methods involve a systematic approach to interpreting and making sense of the data collected in qualitative research. Here are some common qualitative data analysis methods:

Thematic Analysis

This method involves identifying patterns or themes in the data that are relevant to the research question. The researcher reviews the data, identifies keywords or phrases, and groups them into categories or themes. Thematic analysis is useful for identifying patterns across multiple data sources and for generating new insights into the research topic.

Content Analysis

This method involves analyzing the content of written or spoken language to identify key themes or concepts. Content analysis can be quantitative or qualitative. Qualitative content analysis involves close reading and interpretation of texts to identify recurring themes, concepts, and patterns. Content analysis is useful for identifying patterns in media messages, public discourse, and cultural trends.

Discourse Analysis

This method involves analyzing language to understand how it constructs meaning and shapes social interactions. Discourse analysis can involve a variety of methods, such as conversation analysis, critical discourse analysis, and narrative analysis. Discourse analysis is useful for understanding how language shapes social interactions, cultural norms, and power relationships.

Grounded Theory Analysis

This method involves developing a theory or explanation based on the data collected. Grounded theory analysis starts with the data and uses an iterative process of coding and analysis to identify patterns and themes in the data. The theory or explanation that emerges is grounded in the data, rather than preconceived hypotheses. Grounded theory analysis is useful for understanding complex social phenomena and for generating new theoretical insights.

Narrative Analysis

This method involves analyzing the stories or narratives that participants share to gain insights into their experiences, attitudes, and beliefs. Narrative analysis can involve a variety of methods, such as structural analysis, thematic analysis, and discourse analysis. Narrative analysis is useful for understanding how individuals construct their identities, make sense of their experiences, and communicate their values and beliefs.

Phenomenological Analysis

This method involves analyzing how individuals make sense of their experiences and the meanings they attach to them. Phenomenological analysis typically involves in-depth interviews with participants to explore their experiences in detail. Phenomenological analysis is useful for understanding subjective experiences and for developing a rich understanding of human consciousness.

Comparative Analysis

This method involves comparing and contrasting data across different cases or groups to identify similarities and differences. Comparative analysis can be used to identify patterns or themes that are common across multiple cases, as well as to identify unique or distinctive features of individual cases. Comparative analysis is useful for understanding how social phenomena vary across different contexts and groups.

Applications of Qualitative Research

Qualitative research has many applications across different fields and industries. Here are some examples of how qualitative research is used:

• Market Research: Qualitative research is often used in market research to understand consumer attitudes, behaviors, and preferences. Researchers conduct focus groups and one-on-one interviews with consumers to gather insights into their experiences and perceptions of products and services.
• Health Care: Qualitative research is used in health care to explore patient experiences and perspectives on health and illness. Researchers conduct in-depth interviews with patients and their families to gather information on their experiences with different health care providers and treatments.
• Education: Qualitative research is used in education to understand student experiences and to develop effective teaching strategies. Researchers conduct classroom observations and interviews with students and teachers to gather insights into classroom dynamics and instructional practices.
• Social Work : Qualitative research is used in social work to explore social problems and to develop interventions to address them. Researchers conduct in-depth interviews with individuals and families to understand their experiences with poverty, discrimination, and other social problems.
• Anthropology : Qualitative research is used in anthropology to understand different cultures and societies. Researchers conduct ethnographic studies and observe and interview members of different cultural groups to gain insights into their beliefs, practices, and social structures.
• Psychology : Qualitative research is used in psychology to understand human behavior and mental processes. Researchers conduct in-depth interviews with individuals to explore their thoughts, feelings, and experiences.
• Public Policy : Qualitative research is used in public policy to explore public attitudes and to inform policy decisions. Researchers conduct focus groups and one-on-one interviews with members of the public to gather insights into their perspectives on different policy issues.

How to Conduct Qualitative Research

Here are some general steps for conducting qualitative research:

• Identify your research question: Qualitative research starts with a research question or set of questions that you want to explore. This question should be focused and specific, but also broad enough to allow for exploration and discovery.
• Select your research design: There are different types of qualitative research designs, including ethnography, case study, grounded theory, and phenomenology. You should select a design that aligns with your research question and that will allow you to gather the data you need to answer your research question.
• Recruit participants: Once you have your research question and design, you need to recruit participants. The number of participants you need will depend on your research design and the scope of your research. You can recruit participants through advertisements, social media, or through personal networks.
• Collect data: There are different methods for collecting qualitative data, including interviews, focus groups, observation, and document analysis. You should select the method or methods that align with your research design and that will allow you to gather the data you need to answer your research question.
• Analyze data: Once you have collected your data, you need to analyze it. This involves reviewing your data, identifying patterns and themes, and developing codes to organize your data. You can use different software programs to help you analyze your data, or you can do it manually.
• Interpret data: Once you have analyzed your data, you need to interpret it. This involves making sense of the patterns and themes you have identified, and developing insights and conclusions that answer your research question. You should be guided by your research question and use your data to support your conclusions.
• Communicate results: Once you have interpreted your data, you need to communicate your results. This can be done through academic papers, presentations, or reports. You should be clear and concise in your communication, and use examples and quotes from your data to support your findings.

Examples of Qualitative Research

Here are some real-time examples of qualitative research:

• Customer Feedback: A company may conduct qualitative research to understand the feedback and experiences of its customers. This may involve conducting focus groups or one-on-one interviews with customers to gather insights into their attitudes, behaviors, and preferences.
• Healthcare : A healthcare provider may conduct qualitative research to explore patient experiences and perspectives on health and illness. This may involve conducting in-depth interviews with patients and their families to gather information on their experiences with different health care providers and treatments.
• Education : An educational institution may conduct qualitative research to understand student experiences and to develop effective teaching strategies. This may involve conducting classroom observations and interviews with students and teachers to gather insights into classroom dynamics and instructional practices.
• Social Work: A social worker may conduct qualitative research to explore social problems and to develop interventions to address them. This may involve conducting in-depth interviews with individuals and families to understand their experiences with poverty, discrimination, and other social problems.
• Anthropology : An anthropologist may conduct qualitative research to understand different cultures and societies. This may involve conducting ethnographic studies and observing and interviewing members of different cultural groups to gain insights into their beliefs, practices, and social structures.
• Psychology : A psychologist may conduct qualitative research to understand human behavior and mental processes. This may involve conducting in-depth interviews with individuals to explore their thoughts, feelings, and experiences.
• Public Policy: A government agency or non-profit organization may conduct qualitative research to explore public attitudes and to inform policy decisions. This may involve conducting focus groups and one-on-one interviews with members of the public to gather insights into their perspectives on different policy issues.

Purpose of Qualitative Research

The purpose of qualitative research is to explore and understand the subjective experiences, behaviors, and perspectives of individuals or groups in a particular context. Unlike quantitative research, which focuses on numerical data and statistical analysis, qualitative research aims to provide in-depth, descriptive information that can help researchers develop insights and theories about complex social phenomena.

Qualitative research can serve multiple purposes, including:

• Exploring new or emerging phenomena : Qualitative research can be useful for exploring new or emerging phenomena, such as new technologies or social trends. This type of research can help researchers develop a deeper understanding of these phenomena and identify potential areas for further study.
• Understanding complex social phenomena : Qualitative research can be useful for exploring complex social phenomena, such as cultural beliefs, social norms, or political processes. This type of research can help researchers develop a more nuanced understanding of these phenomena and identify factors that may influence them.
• Generating new theories or hypotheses: Qualitative research can be useful for generating new theories or hypotheses about social phenomena. By gathering rich, detailed data about individuals’ experiences and perspectives, researchers can develop insights that may challenge existing theories or lead to new lines of inquiry.
• Providing context for quantitative data: Qualitative research can be useful for providing context for quantitative data. By gathering qualitative data alongside quantitative data, researchers can develop a more complete understanding of complex social phenomena and identify potential explanations for quantitative findings.

When to use Qualitative Research

Here are some situations where qualitative research may be appropriate:

• Exploring a new area: If little is known about a particular topic, qualitative research can help to identify key issues, generate hypotheses, and develop new theories.
• Understanding complex phenomena: Qualitative research can be used to investigate complex social, cultural, or organizational phenomena that are difficult to measure quantitatively.
• Investigating subjective experiences: Qualitative research is particularly useful for investigating the subjective experiences of individuals or groups, such as their attitudes, beliefs, values, or emotions.
• Conducting formative research: Qualitative research can be used in the early stages of a research project to develop research questions, identify potential research participants, and refine research methods.
• Evaluating interventions or programs: Qualitative research can be used to evaluate the effectiveness of interventions or programs by collecting data on participants’ experiences, attitudes, and behaviors.

Characteristics of Qualitative Research

Qualitative research is characterized by several key features, including:

• Focus on subjective experience: Qualitative research is concerned with understanding the subjective experiences, beliefs, and perspectives of individuals or groups in a particular context. Researchers aim to explore the meanings that people attach to their experiences and to understand the social and cultural factors that shape these meanings.
• Use of open-ended questions: Qualitative research relies on open-ended questions that allow participants to provide detailed, in-depth responses. Researchers seek to elicit rich, descriptive data that can provide insights into participants’ experiences and perspectives.
• Sampling-based on purpose and diversity: Qualitative research often involves purposive sampling, in which participants are selected based on specific criteria related to the research question. Researchers may also seek to include participants with diverse experiences and perspectives to capture a range of viewpoints.
• Data collection through multiple methods: Qualitative research typically involves the use of multiple data collection methods, such as in-depth interviews, focus groups, and observation. This allows researchers to gather rich, detailed data from multiple sources, which can provide a more complete picture of participants’ experiences and perspectives.
• Inductive data analysis: Qualitative research relies on inductive data analysis, in which researchers develop theories and insights based on the data rather than testing pre-existing hypotheses. Researchers use coding and thematic analysis to identify patterns and themes in the data and to develop theories and explanations based on these patterns.
• Emphasis on researcher reflexivity: Qualitative research recognizes the importance of the researcher’s role in shaping the research process and outcomes. Researchers are encouraged to reflect on their own biases and assumptions and to be transparent about their role in the research process.

Advantages of Qualitative Research

Qualitative research offers several advantages over other research methods, including:

• Depth and detail: Qualitative research allows researchers to gather rich, detailed data that provides a deeper understanding of complex social phenomena. Through in-depth interviews, focus groups, and observation, researchers can gather detailed information about participants’ experiences and perspectives that may be missed by other research methods.
• Flexibility : Qualitative research is a flexible approach that allows researchers to adapt their methods to the research question and context. Researchers can adjust their research methods in real-time to gather more information or explore unexpected findings.
• Contextual understanding: Qualitative research is well-suited to exploring the social and cultural context in which individuals or groups are situated. Researchers can gather information about cultural norms, social structures, and historical events that may influence participants’ experiences and perspectives.
• Participant perspective : Qualitative research prioritizes the perspective of participants, allowing researchers to explore subjective experiences and understand the meanings that participants attach to their experiences.
• Theory development: Qualitative research can contribute to the development of new theories and insights about complex social phenomena. By gathering rich, detailed data and using inductive data analysis, researchers can develop new theories and explanations that may challenge existing understandings.
• Validity : Qualitative research can offer high validity by using multiple data collection methods, purposive and diverse sampling, and researcher reflexivity. This can help ensure that findings are credible and trustworthy.

Limitations of Qualitative Research

Qualitative research also has some limitations, including:

• Subjectivity : Qualitative research relies on the subjective interpretation of researchers, which can introduce bias into the research process. The researcher’s perspective, beliefs, and experiences can influence the way data is collected, analyzed, and interpreted.
• Limited generalizability: Qualitative research typically involves small, purposive samples that may not be representative of larger populations. This limits the generalizability of findings to other contexts or populations.
• Time-consuming: Qualitative research can be a time-consuming process, requiring significant resources for data collection, analysis, and interpretation.
• Resource-intensive: Qualitative research may require more resources than other research methods, including specialized training for researchers, specialized software for data analysis, and transcription services.
• Limited reliability: Qualitative research may be less reliable than quantitative research, as it relies on the subjective interpretation of researchers. This can make it difficult to replicate findings or compare results across different studies.
• Ethics and confidentiality: Qualitative research involves collecting sensitive information from participants, which raises ethical concerns about confidentiality and informed consent. Researchers must take care to protect the privacy and confidentiality of participants and obtain informed consent.

Also see Research Methods

About the author

Muhammad Hassan

Researcher, Academic Writer, Web developer

You may also like

Questionnaire – Definition, Types, and Examples

Case Study – Methods, Examples and Guide

Observational Research – Methods and Guide

Quantitative Research – Methods, Types and...

Qualitative Research Methods

Explanatory Research – Types, Methods, Guide

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

9.4 Types of qualitative research designs

Learning objectives.

• Define focus groups and outline how they differ from one-on-one interviews
• Describe how to determine the best size for focus groups
• Identify the important considerations in focus group composition
• Discuss how to moderate focus groups
• Identify the strengths and weaknesses of focus group methodology
• Describe case study research, ethnography, and phenomenology.

There are various types of approaches to qualitative research.  This chapter presents information about focus groups, which are often used in social work research.  It also introduces case studies, ethnography, and phenomenology.

Focus Groups

Focus groups resemble qualitative interviews in that a researcher may prepare a guide in advance and interact with participants by asking them questions. But anyone who has conducted both one-on-one interviews and focus groups knows that each is unique. In an interview, usually one member (the research participant) is most active while the other (the researcher) plays the role of listener, conversation guider, and question-asker. Focus groups , on the other hand, are planned discussions designed to elicit group interaction and “obtain perceptions on a defined area of interest in a permissive, nonthreatening environment” (Krueger & Casey, 2000, p. 5).  In focus groups, the researcher play a different role than in a one-on-one interview. The researcher’s aim is to get participants talking to each other,  to observe interactions among participants, and moderate the discussion.

There are numerous examples of focus group research. In their 2008 study, for example, Amy Slater and Marika Tiggemann (2010) conducted six focus groups with 49 adolescent girls between the ages of 13 and 15 to learn more about girls’ attitudes towards’ participation in sports. In order to get focus group participants to speak with one another rather than with the group facilitator, the focus group interview guide contained just two questions: “Can you tell me some of the reasons that girls stop playing sports or other physical activities?” and “Why do you think girls don’t play as much sport/physical activity as boys?” In another focus group study, Virpi Ylanne and Angie Williams (2009) held nine focus group sessions with adults of different ages to gauge their perceptions of how older characters are represented in television commercials. Among other considerations, the researchers were interested in discovering how focus group participants position themselves and others in terms of age stereotypes and identities during the group discussion. In both examples, the researchers’ core interest in group interaction could not have been assessed had interviews been conducted on a one-on-one basis, making the focus group method an ideal choice.

Who should be in your focus group?

In some ways, focus groups require more planning than other qualitative methods of data collection, such as one-on-one interviews in which a researcher may be better able to the dialogue. Researchers must take care to form focus groups with members who will want to interact with one another and to control the timing of the event so that participants are not asked nor expected to stay for a longer time than they’ve agreed to participate. The researcher should also be prepared to inform focus group participants of their responsibility to maintain the confidentiality of what is said in the group. But while the researcher can and should encourage all focus group members to maintain confidentiality, she should also clarify to participants that the unique nature of the group setting prevents her from being able to promise that confidentiality will be maintained by other participants. Once focus group members leave the research setting, researchers cannot control what they say to other people.

Group size should be determined in part by the topic of the interview and your sense of the likelihood that participants will have much to say without much prompting. If the topic is one about which you think participants feel passionately and will have much to say, a group of 3–5 could make sense. Groups larger than that, especially for heated topics, can easily become unmanageable. Some researchers say that a group of about 6–10 participants is the ideal size for focus group research (Morgan, 1997); others recommend that groups should include 3–12 participants (Adler & Clark, 2008).  The size of the focus group is ultimately the decision of the researcher. When forming groups and deciding how large or small to make them, take into consideration what you know about the topic and participants’ potential interest in, passion for, and feelings about the topic. Also consider your comfort level and experience in conducting focus groups. These factors will help you decide which size is right in your particular case.

It may seem counterintuitive, but in general, it is better to form focus groups consisting of participants who do not know one another than to create groups consisting of friends, relatives, or acquaintances (Agar & MacDonald, 1995).  The reason is that group members who know each other may not share some taken-for-granted knowledge or assumptions. In research, it is precisely the  taken-for-granted knowledge that is often of interest; thus, the focus group researcher should avoid setting up interactions where participants may be discouraged to question or raise issues that they take for granted. However, group members should not be so different from one another that participants will be unlikely to feel comfortable talking with one another.

Focus group researchers must carefully consider the composition of the groups they put together. In his text on conducting focus groups, Morgan (1997) suggests that “homogeneity in background and not homogeneity in attitudes” (p. 36) should be the goal, since participants must feel comfortable speaking up but must also have enough differences to facilitate a productive discussion.  Whatever composition a researcher designs for her focus groups, the important point to keep in mind is that focus group dynamics are shaped by multiple social contexts (Hollander, 2004). Participants’ silences as well as their speech may be shaped by gender, race, class, sexuality, age, or other background characteristics or social dynamics—all of which might be suppressed or exacerbated depending on the composition of the group. Hollander (2004) suggests that researchers must pay careful attention to group composition, must be attentive to group dynamics during the focus group discussion, and should use multiple methods of data collection in order to “untangle participants’ responses and their relationship to the social contexts of the focus group” (p. 632).

The role of the moderator

In addition to the importance of group composition, focus groups also require skillful moderation. A moderator is the researcher tasked with facilitating the conversation in the focus group. Participants may ask each other follow-up questions, agree or disagree with one another, display body language that tells us something about their feelings about the conversation, or even come up with questions not previously conceived of by the researcher. It is just these sorts of interactions and displays that are of interest to the researcher. A researcher conducting focus groups collects data on more than people’s direct responses to her question, as in interviews.

The moderator’s job is not to ask questions to each person individually, but to stimulate conversation between participants. It is important to set ground rules for focus groups at the outset of the discussion. Remind participants you’ve invited them to participate because you want to hear from all of them. Therefore, the group should aim to let just one person speak at a time and avoid letting just a couple of participants dominate the conversation. One way to do this is to begin the discussion by asking participants to briefly introduce themselves or to provide a brief response to an opening question. This will help set the tone of having all group members participate. Also, ask participants to avoid having side conversations; thoughts or reactions to what is said in the group are important and should be shared with everyone.

As the focus group gets rolling, the moderator will play a less active role as participants talk to one another. There may be times when the conversation stagnates or when you, as moderator, wish to guide the conversation in another direction. In these instances, it is important to demonstrate that you’ve been paying attention to what participants have said. Being prepared to interject statements or questions such as “I’d really like to hear more about what Sunil and Joe think about what Dominick and Jae have been saying” or “Several of you have mentioned X. What do others think about this?” will be important for keeping the conversation going. It can also help redirect the conversation, shift the focus to participants who have been less active in the group, and serve as a cue to those who may be dominating the conversation that it is time to allow others to speak. Researchers may choose to use multiple moderators to make managing these various tasks easier.

Moderators are often too busy working with participants to take diligent notes during a focus group. It is helpful to have a note-taker who can record participants’ responses (Liamputtong, 2011). The note-taker creates, in essence, the first draft of interpretation for the data in the study. They note themes in responses, nonverbal cues, and other information to be included in the analysis later on. Focus groups are analyzed in a similar way as interviews; however, the interactive dimension between participants adds another element to the analytical process. Researchers must attend to the group dynamics of each focus group, as “verbal and nonverbal expressions, the tactical use of humour, interruptions in interaction, and disagreement between participants” are all data that are vital to include in analysis (Liamputtong, 2011, p. 175). Note-takers record these elements in field notes, which allows moderators to focus on the conversation.

Strengths and weaknesses of focus groups

Focus groups share many of the strengths and weaknesses of one-on-one qualitative interviews. Both methods can yield very detailed, in-depth information; are excellent for studying social processes; and provide researchers with an opportunity not only to hear what participants say but also to observe what they do in terms of their body language. Focus groups offer the added benefit of giving researchers a chance to collect data on human interaction by observing how group participants respond and react to one another. Like one-on-one qualitative interviews, focus groups can also be quite expensive and time-consuming. However, there may be some savings with focus groups as it takes fewer group events than one-on-one interviews to gather data from the same number of people. Another potential drawback of focus groups, which is not a concern for one-on-one interviews, is that one or two participants might dominate the group, silencing other participants. Careful planning and skillful moderation on the part of the researcher are crucial for avoiding, or at least dealing with, such possibilities. The various strengths and weaknesses of focus group research are summarized in Table 91.

Grounded Theory

Grounded theory has been widely used since its development in the late 1960s (Glaser & Strauss, 1967). Largely derived from schools of sociology, grounded theory involves emersion of the researcher in the field and in the data. Researchers follow a systematic set of procedures and a simultaneous approach to data collection and analysis. Grounded theory is most often used to generate rich explanations of complex actions, processes, and transitions. The primary mode of data collection is one-on-one participant interviews. Sample sizes tend to range from 20 to 30 individuals, sampled purposively (Padgett, 2016). However, sample sizes can be larger or smaller, depending on data saturation. Data saturation is the point in the qualitative research data collection process when no new information is being discovered. Researchers use a constant comparative approach in which previously collected data are analyzed during the same time frame as new data are being collected.  This allows the researchers to determine when new information is no longer being gleaned from data collection and analysis — that data saturation has been reached — in order to conclude the data collection phase.

Rather than apply or test existing grand theories, or “Big T” theories, grounded theory focuses on “small t” theories (Padgett, 2016). Grand theories, or “Big T” theories, are systems of principles, ideas, and concepts used to predict phenomena. These theories are backed up by facts and tested hypotheses. “Small t” theories are speculative and contingent upon specific contexts. In grounded theory, these “small t” theories are grounded in events and experiences and emerge from the analysis of the data collected.

One notable application of grounded theory produced a “small t” theory of acceptance following cancer diagnoses (Jakobsson, Horvath, & Ahlberg, 2005). Using grounded theory, the researchers interviewed nine patients in western Sweden. Data collection and analysis stopped when saturation was reached. The researchers found that action and knowledge, given with respect and continuity led to confidence which led to acceptance. This “small t” theory continues to be applied and further explored in other contexts.

Case study research

Case study research is an intensive longitudinal study of a phenomenon at one or more research sites for the purpose of deriving detailed, contextualized inferences and understanding the dynamic process underlying a phenomenon of interest. Case research is a unique research design in that it can be used in an interpretive manner to build theories or in a positivist manner to test theories. The previous chapter on case research discusses both techniques in depth and provides illustrative exemplars. Furthermore, the case researcher is a neutral observer (direct observation) in the social setting rather than an active participant (participant observation). As with any other interpretive approach, drawing meaningful inferences from case research depends heavily on the observational skills and integrative abilities of the researcher.

Ethnography

The ethnographic research method, derived largely from the field of anthropology, emphasizes studying a phenomenon within the context of its culture. The researcher must be deeply immersed in the social culture over an extended period of time (usually 8 months to 2 years) and should engage, observe, and record the daily life of the studied culture and its social participants within their natural setting. The primary mode of data collection is participant observation, and data analysis involves a “sense-making” approach. In addition, the researcher must take extensive field notes, and narrate her experience in descriptive detail so that readers may experience the same culture as the researcher. In this method, the researcher has two roles: rely on her unique knowledge and engagement to generate insights (theory), and convince the scientific community of the trans-situational nature of the studied phenomenon.

The classic example of ethnographic research is Jane Goodall’s study of primate behaviors, where she lived with chimpanzees in their natural habitat at Gombe National Park in Tanzania, observed their behaviors, interacted with them, and shared their lives. During that process, she learnt and chronicled how chimpanzees seek food and shelter, how they socialize with each other, their communication patterns, their mating behaviors, and so forth. A more contemporary example of ethnographic research is Myra Bluebond-Langer’s (1996)14 study of decision making in families with children suffering from life-threatening illnesses, and the physical, psychological, environmental, ethical, legal, and cultural issues that influence such decision-making. The researcher followed the experiences of approximately 80 children with incurable illnesses and their families for a period of over two years. Data collection involved participant observation and formal/informal conversations with children, their parents and relatives, and health care providers to document their lived experience.

Phenomenology

Phenomenology is a research method that emphasizes the study of conscious experiences as a way of understanding the reality around us. Phenomenology is concerned with the systematic reflection and analysis of phenomena associated with conscious experiences, such as human judgment, perceptions, and actions, with the goal of (1) appreciating and describing social reality from the diverse subjective perspectives of the participants involved, and (2) understanding the symbolic meanings (“deep structure”) underlying these subjective experiences. Phenomenological inquiry requires that researchers eliminate any prior assumptions and personal biases, empathize with the participant’s situation, and tune into existential dimensions of that situation, so that they can fully understand the deep structures that drives the conscious thinking, feeling, and behavior of the studied participants.

Some researchers view phenomenology as a philosophy rather than as a research method. In response to this criticism, Giorgi and Giorgi (2003) developed an existential phenomenological research method to guide studies in this area. This method can be grouped into data collection and data analysis phases. In the data collection phase, participants embedded in a social phenomenon are interviewed to capture their subjective experiences and perspectives regarding the phenomenon under investigation. Examples of questions that may be asked include “can you describe a typical day” or “can you describe that particular incident in more detail?” These interviews are recorded and transcribed for further analysis. During data analysis, the researcher reads the transcripts to: (1) get a sense of the whole, and (2) establish “units of significance” that can faithfully represent participants’ subjective experiences. Examples of such units of significance are concepts such as “felt space” and “felt time,” which are then used to document participants’ psychological experiences. For instance, did participants feel safe, free, trapped, or joyous when experiencing a phenomenon (“felt-space”)? Did they feel that their experience was pressured, slow, or discontinuous (“felt-time”)? Phenomenological analysis should take into account the participants’ temporal landscape (i.e., their sense of past, present, and future), and the researcher must transpose herself in an imaginary sense in the participant’s situation (i.e., temporarily live the participant’s life). The participants’ lived experience is described in form of a narrative or using emergent themes. The analysis then delves into these themes to identify multiple layers of meaning while retaining the fragility and ambiguity of subjects’ lived experiences.

Key Takeaways

• In terms of focus group composition, homogeneity of background among participants is recommended while diverse attitudes within the group are ideal.
• The goal of a focus group is to get participants to talk with one another rather than the researcher.
• Like one-on-one qualitative interviews, focus groups can yield very detailed information, are excellent for studying social processes, and provide researchers with an opportunity to observe participants’ body language; they also allow researchers to observe social interaction.
• Focus groups can be expensive and time-consuming, as are one-on-one interviews; there is also the possibility that a few participants will dominate the group and silence others in the group.
• Other types of qualitative research include case studies, ethnography, and phenomenology.
• Data saturation – the point in the qualitative research data collection process when no new information is being discovered
• Focus groups- planned discussions designed to elicit group interaction and “obtain perceptions on a defined area of interest in a permissive, nonthreatening environment” (Krueger & Casey, 2000, p. 5)
• Moderator- the researcher tasked with facilitating the conversation in the focus group

Image attributions

target group by geralt CC-0

workplace team by Free-Photos CC-0

Foundations of Social Work Research Copyright © 2020 by Rebecca L. Mauldin is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License , except where otherwise noted.

Share This Book

Types Of Qualitative Research Designs And Methods

Qualitative research design comes in many forms. Understanding what qualitative research is and the various methods that fall under its…

Qualitative research design comes in many forms. Understanding what qualitative research is and the various methods that fall under its umbrella can help determine which method or design to use. Various techniques can achieve results, depending on the subject of study.

Types of qualitative research to explore social behavior or understand interactions within specific contexts include interviews, focus groups, observations and surveys. These identify concepts and relationships that aren’t easily observed through quantitative methods. Figuring out what to explore through qualitative research is the first step in picking the right study design.

Let’s look at the most common types of qualitative methods.

What Is Qualitative Research Design?

Types of qualitative research designs, how are qualitative answers analyzed, qualitative research design in business.

There are several types of qualitative research. The term refers to in-depth, exploratory studies that discover what people think, how they behave and the reasons behind their behavior. The qualitative researcher believes that to best understand human behavior, they need to know the context in which people are acting and making decisions.

Let’s define some basic terms.

Qualitative Method

A group of techniques that allow the researcher to gather information from participants to learn about their experiences, behaviors or beliefs. The types of qualitative research methods used in a specific study should be chosen as dictated by the data being gathered. For instance, to study how employers rate the skills of the engineering students they hired, qualitative research would be appropriate.

Quantitative Method

A group of techniques that allows the researcher to gather information from participants to measure variables. The data is numerical in nature. For instance, quantitative research can be used to study how many engineering students enroll in an MBA program.

Research Design

A plan or outline of how the researcher will proceed with the proposed research project. This defines the sample, the scope of work, the goals and objectives. It may also lay out a hypothesis to be tested. Research design could also combine qualitative and quantitative techniques.

Both qualitative and quantitative research are significant. Depending on the subject and the goals of the study, researchers choose one or the other or a combination of the two. This is all part of the qualitative research design process.

Before we look at some different types of qualitative research, it’s important to note that there’s no one correct approach to qualitative research design. No matter what the type of study, it’s important to carefully consider the design to ensure the method is suitable to the research question. Here are the types of qualitative research methods to choose from:

Cluster Sampling

This technique involves selecting participants from specific locations or teams (clusters). A researcher may set out to observe, interview, or create a focus group with participants linked by location, organization or some other commonality. For example, the researcher might select the top five teams that produce an organization’s finest work. The same can be done by looking at locations (stores in a geographic region). The benefit of this design is that it’s efficient in collecting opinions from specific working groups or areas. However, this limits the sample size to only those people who work within the cluster.

Random Sampling

This design involves randomly assigning participants into groups based on a set of variables (location, gender, race, occupation). In this design, each participant is assigned an equal chance of being selected into a particular group. For example, if the researcher wants to study how students from different colleges differ from one another in terms of workplace habits and friendships, a random sample could be chosen from the student population at these colleges. The purpose of this design is to create a more even distribution of participants across all groups. The researcher will need to choose which groups to include in the study.

Focus Groups

A focus group is a small group that meets to discuss specific issues. Participants are usually recruited randomly, although sometimes they might be recruited because of personal relationships with each other or because they represent part of a certain demographic (age, location). Focus groups are one of the most popular styles of qualitative research because they allow for individual views and opinions to be shared without introducing bias. Researchers gather data through face-to-face conversation or recorded observation.

Observation

This technique involves observing the interaction patterns in a particular situation. Researchers collect data by closely watching the behaviors of others. This method can only be used in certain settings, such as in the workplace or homes.

An interview is an open-ended conversation between a researcher and a participant in which the researcher asks predetermined questions. Successful interviews require careful preparation to ensure that participants are able to give accurate answers. This method allows researchers to collect specific information about their research topic, and participants are more likely to be honest when telling their stories. However, there’s no way to control the number of unique answers, and certain participants may feel uncomfortable sharing their personal details with a stranger.

A survey is a questionnaire used to gather information from a pool of people to get a large sample of responses. This study design allows researchers to collect more data than they would with individual interviews and observations. Depending on the nature of the survey, it may also not require participants to disclose sensitive information or details. On the flip side, it’s time-consuming and may not yield the answers researchers were looking for. It’s also difficult to collect and analyze answers from larger groups.

A large study can combine several of these methods. For instance, it can involve a survey to better understand which kind of organic produce consumers are looking for. It may also include questions on the frequency of such purchases—a numerical data point—alongside their views on the legitimacy of the organic tag, which is an open-ended qualitative question.

Knowledge of the types of qualitative research designs will help you achieve the results you desire.

With quantitative research, analysis of results is fairly straightforward. But, the nature of qualitative research design is such that turning the information collected into usable data can be a challenge. To do this, researchers have to code the non-numerical data for comparison and analysis.

The researcher goes through all their notes and recordings and codes them using a predetermined scheme. Codes are created by ‘stripping out’ words or phrases that seem to answer the questions posed. The researcher will need to decide which categories to code for. Sometimes this process can be time-consuming and difficult to do during the first few passes through the data. So, it’s a good idea to start off by coding a small amount of the data and conducting a thematic analysis to get a better understanding of how to proceed.

The data collected must be organized and analyzed to answer the research questions. There are three approaches to analyzing the data: exploratory, confirmatory and descriptive.

Explanatory Data Analysis

This approach involves looking for relationships within the data to make sense of it. This design can be useful if the research question is ambiguous or open-ended. Exploratory analysis is very flexible and can be used in a number of settings. But, it generally looks at the relationship between variables while the researcher is working with the data.

Confirmatory Data Analysis

This design is used when there’s a hypothesis or theory to be tested. Confirmatory research seeks to test how well past findings apply to new observations by comparing them to statistical tests that quantify relationships between variables. It can also use prior research findings to predict new results.

Descriptive Data Analysis

In this design, the researcher will describe patterns that can be observed from the data. The researcher will take raw data and interpret it with an eye for patterns to formulate a theory that can eventually be tested with quantitative data. The qualitative design is ideal for exploring events that can’t be observed (such as people’s thoughts) or when a process is being evaluated.

With careful planning and insightful analysis, qualitative research is a versatile and useful tool in business, public policy and social studies. In the workplace, managers can use it to understand markets and consumers better or to study the health of an organization.

Businesses conduct qualitative research for many reasons. Harappa’s Thinking Critically course prepares professionals to use such data to understand their work better. Driven by experienced faculty with real-world experience, the course equips employees on a growth trajectory with frameworks and skills to use their reasoning abilities to build better arguments. It’s possible to build more effective teams. Find out how with Harappa.

Explore Harappa Diaries to learn more about topics such as What is Qualitative Research , Quantitative Vs Qualitative Research , Examples of Phenomenological Research and Tips For Studying Online to upgrade your knowledge and skills.

Home » Education » What is Research Design in Qualitative Research

What is Research Design in Qualitative Research

The research design is the overall plan of your research study. It defines the type of study you are going to undertake, sub-type, research problem, hypothesis , variables (dependent and independent), experimental design,  data collection methods, etc. The design of your research may depend on the type of information the researcher wants to uncover as well. Thus, there is a difference between research designs used in quantitative research and qualitative research.

This article looks at, 

1. What is Qualitative Research       – Characteristics of Qualitative Research 2. What is Research Design in Qualitative Research

What is Qualitative Research

Qualitative research is investigative or exploratory in nature and is used by researchers to observe human habits and behaviour. A qualitative approach is used to analyse a problem in depth and develop theories or hypothesis. It is also used to understand underlying causes, opinions, motivations behind something and to discover trends in opinions and thoughts. This is considered to be a subjective approach since this contains observations and descriptions instead of purely statistical data.

Characteristics of Qualitative Research

• Subjective approach
• Used in soft science (social science)
• Complex and broad focus
• Based on observations, communication, and individual interpretation
• Dialectic, inductive reasoning
• Words as the basic element of analysis
• Data collection is done by interviews, observation and analysis of documents.

The design of any research study would depend on the type of information the researcher wants to uncover. There are different types of qualitative research designs; in this article, we are looking at six qualitative research designs:

Figure 1: Types of Qualitative Research Design

Grounded Theory

Grounded theory approach is a special approach whose main purpose is to develop theories. This type of study doesn’t commence with any pre-existing hypothesis or theories . Instead, the theory is developed after the analysis of the collected data. The researcher involved in this research design will first collect data through interviews, observations, review of records or a combination of these methods. He or she will then analyse the collected data and notice recurring ideas and concepts, which will be tagged with codes. As the data analysis progresses, codes will be grouped into concepts and concepts into categories. Categories will then serve as the basis for the theories.

Historical Studies

Historical studies examine past events to understand the present and expect potential future effects. The process involves selecting an appropriate topic after reading related literature, developing research questions, finding an inventory of sources such as achieves, papers, private libraries, etc., verifying their validity and reliability, and collecting data. The analysis of data in this approach will involve the synthesis of all information, and reconciliation of conflicting information.

Case Studies

A case study is a detailed investigation of the development of a single phenomenon or an individual over a period of time. Case studies are generally used to explore complex issues using both qualitative and quantitative data. They are holistic, systematic, context-sensitive and layered. They start with the identification of the research problem, which is then followed by the selection of the cases and the data collection and analysis method. The next step involves the collection of data in the field and analysis. Since case studies focus on a single phenomenon, it is somewhat difficult to use this data to form generalisations.

Action Research

Action research is a type of a research approach that aims to solve an immediate problem. This design can involve a combination of analytical, investigative and evaluative methods designed to diagnose and solve problems. The researcher, who will also act as a participant, will first identify a research problem, clarify theories regarding it and then identify research questions. Next, he will collect data on the problem, and organize them. Then the data will be analyzed and interpreted, and a plan will be created to address the problem. The last steps of this research design involve the implementation of this plan and evaluation of the results of this plan.

Phenomenology

Phenomenology is a study that aims to understand the subjective, lived experiences and perspectives of participants. This is based on the main idea that there are multiple interpretations of the same experience and these multiple interpretations or meanings make up of reality. This approach involves long and intensive interview sessions with various participants in order to gather information about unique individual experiences which offer a rich description of human experiences.

Ethnography

Ethnography is the systematic study of communities and cultures. Researchers engaged in ethnographic studies participate, either overtly or covertly, in the daily lives of the population that are being studied. They spend an extensive time period among this population, watching what happens, listening to what’s being said and asking questions. Interviews, observations and analysis of records are the main methods of data collection in an ethnographic study. The main aim of an ethnographic research is exploring and studying culture from an insider’s point of view.

Conclusion 

As explained above, the research design of a qualitative research study will depend on the type of information the researcher wants to uncover. Although this article only discusses six research approaches, a researcher can use more methods or create his or her own design to discover and evaluate information about a research problem. 

About the Author: Hasa

Hasanthi is a seasoned content writer and editor with over 8 years of experience. Armed with a BA degree in English and a knack for digital marketing, she explores her passions for literature, history, culture, and food through her engaging and informative writing.

​You May Also Like These

Leave a reply cancel reply.

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• My Account Login
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 22 April 2024

The design and evaluation of gamified online role-play as a telehealth training strategy in dental education: an explanatory sequential mixed-methods study

• Chayanid Teerawongpairoj 1 ,
• Chanita Tantipoj 1 &
• Kawin Sipiyaruk 2  

Scientific Reports volume  14 , Article number:  9216 ( 2024 ) Cite this article

Metrics details

• Health care
• Health services
• Public health

To evaluate user perceptions and educational impact of gamified online role-play in teledentistry as well as to construct a conceptual framework highlighting how to design this interactive learning strategy, this research employed an explanatory sequential mixed-methods design. Participants were requested to complete self-perceived assessments toward confidence and awareness in teledentistry before and after participating in a gamified online role-play. They were also asked to complete a satisfaction questionnaire and participate in an in-depth interview to investigate their learning experience. The data were analyzed using descriptive statistics, paired sample t-test, one-way analysis of variance, and framework analysis. There were 18 participants who completed self-perceived assessments and satisfaction questionnaire, in which 12 of them participated in a semi-structured interview. There were statistically significant increases in self-perceived confidence and awareness after participating in the gamified online role-play ( P  < 0.001). In addition, the participants were likely to be satisfied with this learning strategy, where usefulness was perceived as the most positive aspect with a score of 4.44 out of 5, followed by ease of use (4.40) and enjoyment (4.03). The conceptual framework constructed from the qualitative findings has revealed five key elements in designing a gamified online role-play, including learner profile, learning settings, pedagogical components, interactive functions, and educational impact. The gamified online role-play has demonstrated its potential in improving self-perceived confidence and awareness in teledentistry. The conceptual framework developed in this research could be considered to design and implement a gamified online role-play in dental education. This research provides valuable evidence on the educational impact of gamified online role-play in teledentistry and how it could be designed and implemented in dental education. This information would be supportive for dental instructors or educators who are considering to implement teledentistry training in their practice.

Similar content being viewed by others

Games in dental education: playing to learn or learning to play?

Andresa Costa Pereira & Anthony Damien Walmsley

A bibliometric analysis of the use of the Gamification Octalysis Framework in training: evidence from Web of Science

Sattwik Mohanty & Prabu Christopher B

Helping general dental practitioners use the Index of Orthodontic Treatment Need: an assessment of available educational apps

David G. Murray, Gautham Sivamurthy & Peter Mossey

Introduction

Telehealth has gained significant attention from various organization due to its potential to improve healthcare quality and accessibility 1 . It can be supportive in several aspects in healthcare, including medical and nursing services, to enhance continuous monitoring and follow-up 2 . Its adoption has increased substantially during the COVID-19 pandemic, aiming to provide convenient healthcare services 3 . Even though the COVID-19 outbreak has passed, many patients still perceive telehealth as an effective tool in reducing a number of visits and enhancing access to health care services 4 , 5 . This supports the use of telehealth in the post-COVID-19 era.

Teledentistry, a form of telehealth specific to dentistry, has been employed to improve access to dental services 6 . This system offers benefits ranging from online history taking, oral diagnosis, treatment monitoring, and interdisciplinary communication among dental professionals, enabling comprehensive and holistic treatment planning for patients 7 . Teledentistry can also reduce travel time and costs associated with dental appointments 8 , 9 , 10 . There is evidence that teledentistry serves as a valuable tool to enhance access to dental care for patients 11 . Additionally, in the context of long-term management in patients, telehealth has contributed to patient-centered care, by enhancing their surrounding environments 12 . Therefore, teledentistry should be emphasized as one of digital dentistry to enhance treatment quality.

Albeit the benefits of teledentistry, available evidence demonstrates challenges and concerns in the implementation of telehealth. Lack of awareness and knowledge in the use of telehealth can hinder the adoption of telehealth 13 . Legal issues and privacy concerns also emerge as significant challenges in telehealth use 14 . Moreover, online communication skills and technology literacy, including competency in using technological tools and applications, have been frequently reported as challenges in teledentistry 15 , 16 . Concerns regarding limitations stemming from the lack of physical examination are also significant 17 . These challenges and complexities may impact the accuracy of diagnosis and the security and confidentiality of patient information. Therefore, telehealth training for dental professionals emerges as essential prerequisites to effectively navigate the use of teledentistry, fostering confidence and competence in remote oral healthcare delivery.

The feasibility and practicality of telehealth in dental education present ongoing challenges and concerns. Given the limitations of teledentistry compared to face-to-face appointments, areas of training should encompass the telehealth system, online communication, technical issues, confidentiality concerns, and legal compliance 18 . However, there is currently no educational strategy that effectively demonstrates the importance and application of teledentistry 19 . A role-play can be considered as a teaching strategy where learners play a role that closely resembles real-life scenarios. A well-organized storytelling allows learner to manage problematic situations, leading to the development of problem-solving skill 20 , 21 . When compared to traditional lecture-based learning, learners can also enhance their communication skills through conversations with simulated patients 22 , 23 . In addition, they could express their thoughts and emotions during a role-play through experiential learning 20 , 24 , 25 . Role-play through video teleconference would be considered as a distance learning tool for training dental professionals to effectively use teledentistry.

While there have been studies supporting online role-play as an effective learning tool due to its impact of flexibility, engagement, and anonymity 26 , 27 , no evidence has been yet reported whether or not this learning strategy could have potential for training teledentistry. Given the complicated issues in telehealth, role-play for training teledentistry should incorporate different learning aspects compared to face-to-face communication with patients. In addition, game components have proved to be supportive in dental education 28 , 29 . Consequently, this research aimed to evaluate user perceptions and educational impact of gamified online role-play to enhance learner competence and awareness in using teledentistry as well as to construct a conceptual framework highlighting how to design and implement this interactive learning strategy. This research would introduce and promote the design and implementation of gamified online role-play as a learning tool for training teledentistry. To achieve the aim, specific objectives were established as follows:

1. To design a gamified online role-play for teledentistry training.

2. To investigate learner perceptions regarding their confidence and awareness in the use of teledentistry after completing the gamified online role-play.

3. To explore user satisfactions toward the use of gamified online role-play.

4. To develop a conceptual framework for designing and implementing a gamified online role-play for teledentistry training.

Materials and methods

Research design.

This research employed an explanatory sequential mixed-methods design, where a quantitative phase was firstly performed followed by a qualitative phase 30 , 31 . The quantitative phase was conducted based on pre-experimental research using one-group pretest–posttest design. Participants were requested to complete self-perceived assessments toward confidence and awareness in the use of teledentistry before and after participating in a gamified online role-play. They were also asked to complete a satisfaction questionnaire in using a gamified online role-play for training teledentistry. The qualitative phase was afterwards conducted to explore in-depth information through semi-structured interviews, in order to enhance an understanding of the quantitative phase, and to develop a conceptual framework for designing and implementing an online role-play for training teledentistry.

A gamified online role-play for training teledentistry

A gamified online role-play was designed and developed by the author team. To ensure its educational impact was significant, the expected learning outcomes were formulated based on insights gathered from a survey with experienced instructors from the Department of Advanced General Dentistry, Faculty of Dentistry, Mahidol University. These learning outcomes covered areas of online communication skill, technical issues, technology literacy of patients, limitations of physical examination, and privacy concerns of personal information. Learning scenario and instructional content were subsequently designed to support learners in achieving the expected learning outcomes, with their alignments validated by three experts in dental education. A professional actress underwent training to role-play a patient with a dental problem, requesting a virtual consultation or teledentistry. Before conducting data collection, the simulated patient was required to undergo a training and adjusting process with a pilot group under supervision of two experts in advanced general dentistry and dental education who had experience with teledentistry to ensure realism and completeness of learning content.

According to the role-play scenario, an actress was assigned to portray a 34-year-old female with chief complaints of pain around both ears, accompanied by difficulties in chewing food due to tooth loss. She was instructed to express her anxiety and nervousness about addressing these issues. Additionally, it was specified that she could not take a day off from work during this period. Despite this constraint, she required a dental consultation to receive advice for initial self-care, as her symptoms significantly impacted her daily life. Furthermore, she was designated to encounter difficulties with the technological use of the teledentistry platform.

The game components were implemented into the online role-play to enhance motivation and engagement. As challenge and randomness appear to be game elements 32 , 33 , five challenge cards were designed and embedded into the online role-play, where a participant was asked to randomly select one of them before interacting with the simulated patient. The challenging situations were potential technical concerns which could occur frequently during video conferencing, including network problems (e.g., internet disconnection and poor connection) and audiovisual quality issues. The participants were blinded to the selected card, while it was revealed to only the simulated patient. The challenging conditions were mimicked by the organizers and simulated patient, allowing learners to deal with difficulties. Therefore, both challenges and randomness were implemented into this learning intervention not only to create learning situations but also to enhance engagement.

A feedback system was carefully considered and implemented into the gamified online role-play. Immediate feedback appears to be a key feature of interactive learning environments 29 . Formative feedback was instantly delivered to learners through verbal and non-verbal communication, including words (content), tone of voice, facial expressions, and gestures of the simulated patient. This type of feedback allowed participants to reflect on whether or not their inputs were appropriate, enabling them to learn from their mistakes, or so-called the role of failure 34 . Summative feedback was also provided at the end of the role-play through a reflection from a simulated patient and suggestions from an instructor.

Learners were able to interact with the simulated patient using an online meeting room by Cisco WebEx. According to the research setting (Fig.  1 ), a learner was asked to participate in the role-play activity using a computer laptop in a soundproof room, while a simulated patient was arranged in a prepared location showing her residential environment. The researcher and instructor also joined the online meeting room and observed the interaction between the simulated patient and learners during the role-play activity whether or not all necessary information was accurately obtained. The role-play activity took around 30 minutes.

A diagram demonstrating the setting of gamified online role-play.

Research participants

Quantitative phase.

The participants in this research were postgraduate students from the Residency Training Program in Advanced General Dentistry at Mahidol University Faculty of Dentistry in academic year 2022, using a volunteer sampling. This program was selected because its objective was to develop graduates capable of integrating competencies from various dental disciplines to provide comprehensive dental care for both normal patients and those with special needs. Therefore, teledentistry should be a supportive component of their service. The recruitment procedure involved posting a recruiting text in the group chat of the residents. Those interested in participating in the research were informed to directly contact us to request more information, and they were subsequently allowed to decide whether they would like to participate. This approach ensured that participation was voluntary. Although there could be a non-response bias within this non-probability sampling technique 35 , it was considered as appropriate for this study, as participants were willing to have contribution in the learning activity, and therefore accurate and reliable research findings with no dropout could be achieved 36 .

The inclusion and exclusion criteria were established to determine the eligibility of prospective participants for this research. This study included postgraduate students from Years 1 to 3 in the Residency Training Program in Advanced General Dentistry at Mahidol University Faculty of Dentistry, enrolled during the academic year 2022. They were also required to at least complete the first semester to be eligible for this research to ensure familiarity with comprehensive dental care. However, they were excluded if they had previous involvement in the pilot testing of the gamified online role-play or if they were not fluent in the Thai language. The sample size was determined using a formula for two dependent samples (comparing means) 37 . To detect a difference in self-perceived confidence and awareness between pre- and post-assessments at a power of 90% and a level of statistical significance of 1%, five participants were required. With an assumed dropout rate of 20%, the number of residents per year (Year 1–3) was set to be 6. Therefore, 18 residents were required for this research.

Qualitative phase

The participants from the quantitative phase were selected for semi-structured interviews using a purposive sampling. This sampling method involved the selection of information-rich participants based on specific criteria deemed relevant to the research objective and to ensure a diverse representation of perspectives and experiences within the sample group 38 . In this research, the information considered for the purposive sampling included demographic data (e.g., sex and year of study), along with self-perceived assessment scores. By incorporating perceptions from a variety of participants, a broad spectrum of insights from different experiences in comprehensive dental practice and diverse improvement levels in self-perceived confidence and awareness could inform the design and implementation of the training program effectively. The sample size for this phase was determined based on data saturation, wherein interviews continued until no new information or emerging themes were retrieved. This method ensured thorough exploration of the research topic and maximized the richness of the qualitative data obtained.

Outcome assessments

To evaluate the gamified online role-play, a triangular design approach was employed, enabling the researchers to compare the research outcomes from different assessment methods. In this research, self-perceived assessments (confidence and awareness) in teledentistry, satisfactions toward gamified online role-play, and learner experience were assessed to assure the quality and feasibility of the gamified online role-play.

Self-perceived confidence and awareness toward teledentistry

All participants were requested to rate their perceptions of teledentistry before and after participating in the gamified online role-play (Supplementary material 1 ). The self-perceived assessment was developed based on previous literature 39 , 40 , 41 , 42 . The assessment scores would inform whether or not the participants could improve their self-perceived confidence and awareness through a learning activity. The assessment consisted of two parts, which were (1) self-perceived confidence and (2) self-perceived awareness. Each part contained six items, which were similar between the pre- and post-assessments. All items were designed using a 5-point Likert scale, where 1 being ‘strongly disagree’ and 5 being ‘strongly agree’.

Satisfactions toward the gamified online role-play

All participants were asked to complete the satisfaction questionnaire after participating in the gamified online role-play, to investigate whether or not they felt satisfied with their learning (Supplementary material 2 ). The questionnaire was developed based on previous literature regarding gamification and role-play 41 , 42 , 43 , 44 . Most of the items were designed using a 5-point Likert scale, where 1 being ‘very dissatisfied’ and 5 being ‘very satisfied’. They were grouped into three aspects, which were (1) Perceived usefulness, (2) Perceived ease of use, and (3) Perceived enjoyment.

Learner experiences within the gamified online role-play

Semi-structured interviews were conducted with the purposively selected participants to gather in-depth information regarding their learning experiences within the gamified online role-play. This technique allowed researchers to ask additional interesting topics raised from the responses of participants. A topic guide for interviews were constructed based on the findings of previous literature 45 , 46 , 47 . The interview was conducted in a private room by a researcher who was trained in conducting qualitative research including interviews. The interview sessions took approximately 45–60 minutes, where all responses from participants were recorded using a digital audio recorder with their permission. The recorded audios were transcribed using a verbatim technique by a transcription service under a confidential agreement.

Validity and reliability of data collection tools

To enhance the quality of self-perceived assessment and satisfaction questionnaire, they were piloted and revised to assure their validity and reliability. According to the content validity, three experts in advanced general dentistry were asked to evaluate the questionnaire, where problematic items were iteratively revised until they achieved the index of item-objective congruence (IOC) higher than 0.5. To perform a test–retest reliability, the validated versions of both self-perceived assessment and satisfaction questionnaire were afterwards piloted in residents from other programs, and the data were analyzed using an intraclass correlation coefficient (ICC), where the values of all items were 0.7 or greater. The data from the first pilot completion of both data collection tools were analyzed using Cronbach’s alpha to ensure the internal consistency of all constructs. The problematic items were deleted to achieve the coefficient alpha of 0.7 or greater for all constructs, which was considered as acceptable internal consistency.

Data analysis

The quantitative data retrieved from self-perceived assessment and satisfaction questionnaire were analyzed with the Statistical Package for Social Sciences software (SPSS, version 29, IBM Corp.). Descriptive statistics were performed to present an overview of the data. The scores from pre- and post-assessments were analyzed using a paired sample t-test to evaluate whether or not the participants would better self-perceive their confidence and awareness in teledentistry after participating in the gamified online role-play. One-way analysis of variance (ANOVA) was conducted to compare whether or not there were statistically significant differences in self-perceived assessment and satisfaction scores among the three academic years.

The qualitative data retrieved from semi-structured interviews were analyzed using a framework analysis, where its procedure involved transcription, familiarization with the interview data, coding, developing an analytical framework, indexing, charting, and data interpreting qualitative findings 48 . In this research, the initial codes had been pre-defined from previous literature and subsequently adjusted following the analysis of each transcript to develop an analytical framework (themes and subthemes), requiring several iterations until no additional codes emerged. Subsequently, the established categories and codes were applied consistently across all transcripts (indexing). The data from each transcript were then charted to develop a matrix, facilitating the management and summarization of qualitative findings. This method enabled the researchers to compare and contrast differences within the data and to identify connections between categories, thereby exploring their relationships and informing data interpretation.

The procedure of framework analysis necessitated a transparent process for data management and interpretation of emerging themes to ensure the robustness of research 49 . The transparency of this analytic approach enabled two researchers (C.Te. and K.S.) to independently analyze the qualitative data, and the emerging themes afterwards were discussed to obtain consensus among the researchers. This technique can be considered as a triangular approach to assure the intercoder reliability and internal validity of this research. The transparent process also allowed an external expert in dental education to verify the accuracy of the analysis. All emerging themes and the decision on data saturation were based on a discussion of all researchers until an agreement was made. NVivo (version 14, QSR International) was used to performed the qualitative data analysis. Subsequently, a conceptual framework was constructed to demonstrate emerging themes and subthemes together with their relationships.

Ethical consideration

The ethical approval for the study was approved by the Institutional Review Board of Faculty of Dentistry and Faculty of Pharmacy, Mahidol University on 29 th September 2022, the ethical approval number: MU-DT/PY-IRB 2022/049.2909. All methods were performed in accordance with the relevant guidelines and regulations. Although the data were not anonymous in nature as they contained identifiable data, they were coded prior to the analysis to assure confidentiality of participants.

Informed consent

Informed consent was obtained from all participants.

There were 18 residents from Year 1 to 3 of the Residency Training Program in Advanced General Dentistry who participated in this research (six from each year). Of these, there were 14 females and 4 males. There was no participant dropout, as all of them completed all required tasks, including the pre- and post-perceived assessments, gamified online role-play, and satisfaction questionnaire. According to the purposive sampling, the participants from the quantitative phase were selected for semi-structured interviews by considering sex, year of study, and self-perceived assessment scores. Twelve students (ten females and two males) participated in semi-structured interviews, where their characteristics are presented in Table 1 .

Internal consistency of all constructs

The data collected from the research participants, in addition to the pilot samples, were analyzed with Cronbach’s alpha to confirm the internal consistency. The coefficient alpha of all constructs demonstrated high internal consistency, as demonstrated in Table 2 .

Self-perceived assessments toward confidence and awareness of teledentistry

There were statistically significant increases in the assessment scores of self-perceived confidence and awareness after participating in the gamified online role-play ( P  < 0.001). According to Table 3 , there was an increase in self-perceived confidence from 3.38 (SD = 0.68) for the pre-assessment to 4.22 (SD = 0.59) for the post-assessment ( P  < 0.001). The findings of self-perceived awareness also showed score improvement from 4.16 (SD = 0.48) to 4.55 (SD = 0.38) after interacting with the simulated patient ( P  < 0.001).

According to Fig.  2 , participants demonstrated a higher level of self-perceived assessments for both self-confidence and awareness in all aspects after participating in the gamified online role-play for teledentistry training.

Self-perceived assessments toward confidence and awareness of teledentistry.

When comparing the self-perceived assessment scores toward confidence and awareness in the use of teledentistry among the three years of study (Year 1–3), there were no statistically significant differences in the pre-assessment, post-assessment score, and score difference (Table 4 ).

Satisfactions toward the use of gamified online role-play

According to Fig.  3 , participants exhibited high levels of satisfaction with the use of gamified online role-play across all three aspects. The aspect of usefulness received the highest satisfaction rating with a score of 4.44 (SD = 0.23) out of 5, followed by ease of use and enjoyment, scoring 4.40 (SD = 0.23) and 4.03 (SD = 0.21), respectively. Particularly, participants expressed the highest satisfaction levels regarding the usefulness of gamified online role-play for identifying their role (Mean = 4.72, SD = 0.46) and developing problem-solving skills associated with teledentistry (Mean = 4.61, SD = 0.50). Additionally, they reported satisfaction with the learning sequence presented in the gamified online role-play (Mean = 4.61, SD = 0.50). However, participants did not strongly perceive that the format of the gamified online role-play could engage them with the learning task for an extended period (Mean = 3.72, SD = 0.83).

Satisfactions toward the use of gamified online role-play.

When comparing the satisfaction levels perceived by participants from different academic years (Table 5 ), no statistically significant differences were observed among the three groups for all three aspects ( P  > 0.05).

Following the framework analysis of qualitative data, there were five emerging themes, including: (1) learner profile, (2) learning settings of the gamified online role-play, (3) pedagogical components, (4) interactive functions, and (5) educational impact.

Theme 1: Learner profile

Learner experience and preferences appeared to have impact on how the participants perceived the use of gamified online role-play for teledentistry training. When learners preferred role-play or realized benefits of teledentistry, they were likely to support this learning intervention. In addition, they could have seen an overall picture of the assigned tasks before participating in this research.

“I had experience with a role-play activity when I was dental undergraduates, and I like this kind of learning where someone role-plays a patient with specific personalities in various contexts. This could be a reason why I felt interested to participate in this task (the gamified online role-play). I also believed that it would be supportive for my clinical practice.” Participant 12, Year 1, Female “Actually, I' have seen in several videos (about teledentistry), where dentists were teaching patients to perform self-examinations, such as checking their own mouth and taking pictures for consultations. Therefore, I could have thought about what I would experience during the activity (within the gamified online role-play).” Participant 8, Year 2, Female

Theme 2: Learning settings of the gamified online role-play

Subtheme 2.1: location.

Participants had agreed that the location for conducting a gamified online role-play should be in a private room without any disturbances, enabling learners to focus on the simulated patient. This could allow them to effectively communicate and understand of the needs of patient, leading to a better grasp of lesson content. In addition, the environments of both learners and simulated patient should be authentic to the learning quality.

“The room should be a private space without any disturbances. This will make us feel confident and engage in conversations with the simulated patient.” Participant 10, Year 1, Female “… simulating a realistic environment can engage me to interact with the simulated patient more effectively ...” Participant 8, Year 2, Female

Subtheme 2.2: Time allocated for the gamified online role-play

The time allocated for the gamified online role-play in this research was considered as appropriate, as participants believed that a 30-minutes period should be suitable to take information and afterwards give some advice to their patient. In addition, a 10-minutes discussion on how they interact with the patient could be supportive for participants to enhance their competencies in the use of teledentistry.

“… it would probably take about 20 minutes because we would need to gather a lot of information … it might need some time to request and gather various information … maybe another 10-15 minutes to provide some advice.” Participant 7, Year 1, Female “I think during the class … we could allocate around 30 minutes for role-play, … we may have discussion of learner performance for 10-15 minutes ... I think it should not be longer than 45 minutes in total.” Participant 6, Year 2, Female

Subtheme 2.3: Learning consequence within a postgraduate curriculum

Most participants suggested that the gamified online role-play in teledentistry should be arranged in the first year of their postgraduate program. This could maximize the effectiveness of online role-play, as they would be able to implement teledentistry for their clinical practice since the beginning of their training. However, some participants suggested that this learning approach could be rearranged in either second or third year of the program. As they already had experience in clinical practice, the gamified online role-play would reinforce their competence in teledentistry.

"Actually, it would be great if this session could be scheduled in the first year … I would feel more comfortable when dealing with my patients through an online platform." Participant 11, Year 2, Male "I believe this approach should be implemented in the first year because it allows students to be trained in teledentistry before being exposed to real patients. However, if this approach is implemented in either the second or third year when they have already had experience in patient care, they would be able to better learn from conversations with simulated patients." Participant 4, Year 3, Male

Theme 3: Pedagogical components

Subtheme 3.1: learning content.

Learning content appeared to be an important component of pedagogical aspect, as it would inform what participants should learn from the gamified online role-play. Based on the interview data, participants reported they could learn how to use a video teleconference platform for teledentistry. The conditions of simulated patient embedded in an online role-play also allowed them to realize the advantages of teledentistry. In addition, dental problems assigned to the simulated patient could reveal the limitations of teledentistry for participants.

“The learning tasks (within the gamified online role-play) let me know how to manage patients through the teleconference.” Participant 5, Year 2, Female “… there seemed to be limitations (of teledentistry) … there could be a risk of misdiagnosis … the poor quality of video may lead to diagnostic errors … it is difficult for patients to capture their oral lesions.” Participant 3, Year 2, Female

Subtheme 3.2: Feedback

During the use of online role-play, the simulated patient can provide formative feedback to participants through facial expressions and tones of voice, enabling participants to observe and learn to adjust their inquiries more accurately. In addition, at the completion of the gamified online role-play, summative feedback provided by instructors could summarize the performance of participants leading to further improvements in the implementation of teledentistry.

“I knew (whether or not I interacted correctly) from the gestures and emotions of the simulated patient between the conversation. I could have learnt from feedback provided during the role-play, especially from the facial expressions of the patient.” Participant 11, Year 2, Male “The feedback provided at the end let me know how well I performed within the learning tasks.” Participant 2, Year 1, Female

Theme 4: Interactive functions

Subtheme 4.1: the authenticity of the simulated patient.

Most participants believed that a simulated patient with high acting performance could enhance the flow of role-play, allowing learners to experience real consequences. The appropriate level of authenticity could engage learners with the learning activity, as they would have less awareness of time passing in the state of flow. Therefore, they could learn better from the gamified online role-play.

"It was so realistic. ... This allowed me to talk with the simulated patient naturally ... At first, when we were talking, I was not sure how I should perform … but afterwards I no longer had any doubts and felt like I wanted to explain things to her even more." Participant 3, Year 2, Female "At first, I believed that if there was a factor that could influence learning, it would probably be a simulated patient. I was impressed by how this simulated patient could perform very well. It made the conversation flow smoothly and gradually." Participant 9, Year 3, Female

Subtheme 4.2: Entertaining features

Participants were likely to be satisfied with the entertaining features embedded in the gamified online role-play. They felt excited when they were being exposed to the unrevealed challenge which they had randomly selected. In addition, participants suggested to have more learning scenarios or simulated patients where they could randomly select to enhance randomness and excitement.

“It was a playful experience while communicating with the simulated patient. There are elements of surprise from the challenge cards that make the conversation more engaging, and I did not feel bored during the role-play.” Participant 4, Year 3, Male “I like the challenge card we randomly selected, as we had no idea what we would encounter … more scenarios like eight choices and we can randomly choose to be more excited. I think we do not need additional challenge cards, as some of them have already been embedded in patient conditions.” Participant 5, Year 2, Female

Subtheme 4.3: Level of difficulty

Participants suggested the gamified online role-play to have various levels of difficulty, so learners could have a chance to select a suitable level for their competence. The difficulties could be represented through patient conditions (e.g., systemic diseases or socioeconomic status), personal health literacy, and emotional tendencies. They also recommended to design the gamified online role-play to have different levels where learners could select an option that is suitable for them.

“The patient had hidden their information, and I needed to bring them out from the conversation.” Participant 12, Year 1, Female “Patients' emotions could be more sensitive to increase level of challenges. This can provide us with more opportunities to enhance our management skills in handling patient emotions.” Participant 11, Year 2, Male “… we can gradually increase the difficult level, similar to playing a game. These challenges could be related to the simulated patient, such as limited knowledge or difficulties in communication, which is likely to occur in our profession.” Participant 6, Year 2, Female

Theme 5: Educational impact

Subtheme 5.1: self-perceived confidence in teledentistry, communication skills.

Participants were likely to perceive that they could learn from the gamified online role-play and felt more confident in the use of teledentistry. This educational impact was mostly achieved from the online conversation within the role-play activity, where the participants could improve their communication skills through a video teleconference platform.

“I feel like the online role-play was a unique form of learning. I believe that I gained confidence from the online communication the simulated patient. I could develop skills to communicate effectively with real patients.” Participant 11, Year 2, Male “I believe it support us to train communication skills ... It allowed us to practice both listening and speaking skills more comprehensively.” Participant 4, Year 3, Male

Critical thinking and problem-solving skills

In addition to communication skills, participants reported that challenges embedded in the role-play allowed them to enhance critical thinking and problem-solving skills, which were a set of skills required to deal with potential problems in the use of teledentistry.

"It was a way of training before experiencing real situations … It allowed us to think critically whether or not what we performed with the simulated patients was appropriate." Participant 7, Year 1, Female “It allowed us to learn how to effectively solve the arranged problems in simulated situation. We needed to solve problems in order to gather required information from the patient and think about how to deliver dental advice through teledentistry.” Participant 11, Year 2, Male

Subtheme 5.2: Self perceived awareness in teledentistry

Participants believed that they could realize the necessity of teledentistry from the gamified online role-play. The storytelling or patient conditions allowed learners to understand how teledentistry could have both physical and psychological support for dental patients.

“From the activity, I would consider teledentistry as a convenient tool for communicating with patients, especially if a patient cannot go to a dental office”. Participant 5, Year 2, Female “I learned about the benefits of teledentistry, particularly in terms of follow-up. The video conference platform could support information sharing, such as drawing images or presenting treatment plans, to patients.” Participant 8, Year 2, Female

A conceptual framework of learning experience within a gamified online role-play

Based on the qualitative findings, a conceptual framework was developed in which a gamified online role-play was conceptualized as a learning strategy in supporting learners to be able to implement teledentistry in their clinical practice (Fig.  4 ).

The conceptual framework of key elements in designing a gamified online role-play.

The conceptual framework has revealed key elements to be considered in designing a gamified online role-play. Learner profile, learning settings, pedagogical components, and interactive functions are considered as influential factors toward user experience within the gamified online role-play. The well-designed learning activity will support learners to achieve expected learning outcomes, considered as educational impact of the gamified online role-play. The contributions of these five key elements to the design of gamified online role-play were interpreted, as follows:

Learner profile: This element tailors the design of gamified online role-plays for teledentistry training involves considering the background knowledge, skills, and experiences of target learners to ensure relevance and engagement.

Learning settings: The element focuses the planning for gamified online role-plays in teledentistry training involves selecting appropriate contexts, such as location and timing, to enhance accessibility and achieve learning outcomes effectively.

Pedagogical components: This element emphasizes the alignment between learning components and learning outcomes within gamified online role-plays, to ensure that the content together with effective feedback design can support learners in improving their competencies from their mistakes.

Interactive functions: This element highlights interactivity features integrated into gamified online role-plays, such as the authenticity and entertaining components to enhance immersion and engagement, together with game difficulty for optimal flow. All these features should engage learners with the learning activities until the achievement of learner outcomes.

Educational impact: This element represents the expected learning outcomes, which will inform the design of learning content and activities within gamified online role-plays. In addition, this element could be considered to evaluate the efficacy of gamified online role-plays, reflecting how well learning designs align with the learning outcomes.

A gamified online role-play can be considered as a learning strategy for teledentistry according to its educational impact. This pedagogical approach could mimic real-life practice, where dental learners could gain experience in the use of teledentistry in simulated situations before interacting with actual patients. Role-play could provide learners opportunities to develop their required competencies, especially communication and real-time decision-making skills, in a predictable and safe learning environment 20 , 23 , 46 . Potential obstacles could also be arranged for learners to deal with, leading to the enhancement of problem-solving skill 50 . In addition, the recognition of teledentistry benefits can enhance awareness and encourage its adoption and implementation, which could be explained by the technology acceptance model 51 . Therefore, a gamified online role-play with a robust design and implementation appeared to have potential in enhancing self-perceived confidence and awareness in the use of teledentistry.

The pedagogical components comprised learning content, which was complemented by assessment and feedback. Learners could develop their competence with engagement through the learning content, gamified by storytelling of the online role-play 52 , 53 . Immediate feedback provided through facial expression and voice tone of simulated patients allowed participants to learn from their failure, considered as a key feature of game-based learning 29 , 45 . The discussion of summative feedback provided from an instructor at the end of role-play activity could support a debriefing process enabling participants to reflect their learning experience, considered as important of simulation-based game 54 . These key considerations should be initially considered in the design of gamified online role-play.

The interactive functions can be considered as another key component for designing and evaluating the gamified online role-play 45 . Several participants enjoyed with a learning process within the gamified online role-play and suggested it to have more learning scenarios. In other words, this tool could engage learners with an instructional process, leading to the achievement of learning outcomes 29 , 45 . As challenge and randomness appear to be game elements 32 , 33 , this learning intervention assigned a set of cards with obstacle tasks for learners to randomly pick up before interacting with simulated patients, which was perceived by participants as a feature to make the role-play more challenging and engaging. This is consistent with previous research, where challenging content for simulated patients could make learners more engaged with a learning process 55 . However, the balance between task challenges and learner competencies is certainly required for the design of learning activities 56 , 57 . The authenticity of simulated patient and immediate feedback could also affect the game flow, leading to the enhancement of learner engagement 45 . These elements could engage participants with a learning process, leading to the enhancement of educational impact.

The educational settings for implementing gamified online role-play into dental curriculum should be another concern. This aspect has been recognized as significant in existing evidence 45 . As this research found no significant differences in all aspects among the three groups of learners, this learning intervention demonstrated the potential for its implementation at any time of postgraduate dental curriculum. This argument can be supported by previous evidence where a role-play could be adaptable for learning at any time, as it requires a short learning period but provides learners with valuable experience prior to being exposed in real-life scenarios 58 . This strategy also provides opportunities for learners who have any question or concern to seek advice or guidance from their instructors 59 . Although the gamified online role-play can be arranged in the program at any time, the first academic year should be considered, as dental learners would be confidence in implementing teledentistry for their clinical practice.

While a gamified online role-play demonstrated its strengths as an interactive learning strategy specifically for teledentistry, there are a couple of potential drawbacks that need to be addressed. The requirement for synchronous participation could limit the flexibility of access time for learners (synchronous interactivity limitation). With only one learner able to engage with a simulated patient at a time (limited participants), more simulated patients would be required if there are a number of learners, otherwise they would need to wait for their turn. Time and resources are significantly required for preparing simulated patients 60 . Despite the use of trained and calibrated professional actors/actresses, inauthenticity may be perceived during role-plays, requiring a significant amount of effort to achieve both interactional and clinical authenticities 46 . Future research could investigate asynchronous learning approaches utilizing non-player character (NPC) controlled by an artificial intelligence system as a simulated patient. This setup would enable multiple learners to have the flexibility to engage with the material at their own pace and at times convenient to them 29 . While there are potential concerns about using gamified online role-plays, this interactive learning intervention offers opportunities for dental professionals to enhance their teledentistry competency in a safe and engaging environment.

Albeit the robust design and data collection tools to assure reliability and validity as well as transparency of this study, a few limitations were raised leading to a potential of further research. While this research recruited only postgraduate students to evaluate the feasibility of gamified online role-play in teledentistry training, further research should include not only experienced dental practitioners but also undergraduate students to confirm its potential use in participants with different learner profiles. More learning scenarios in other dental specialties should also be included to validate its effectiveness, as different specialties could have different limitations and variations. Additional learning scenarios from various dental disciplines should be considered to validate the effectiveness of gamified online role-plays, as different specialties may present unique limitations and variations. A randomized controlled trial with robust design should be required to compare the effectiveness of gamified online role-play with different approaches in training the use of teledentistry.

Conclusions

This research supports the design and implementation of a gamified online role-play in dental education, as dental learners could develop self-perceived confidence and awareness with satisfaction. A well-designed gamified online role-play is necessary to support learners to achieve expected learning outcomes, and the conceptual framework developed in this research can serve as a guidance to design and implement this interactive learning strategy in dental education. However, further research with robust design should be required to validate and ensure the educational impact of gamified online role-play in dental education. Additionally, efforts should be made to develop gamified online role-play in asynchronous learning approaches to enhance the flexibility of learning activities.

Data availability

The data that support the findings of this study are available from the corresponding author, up-on reasonable request. The data are not publicly available due to information that could compromise the privacy of research participants.

Van Dyk, L. A review of telehealth service implementation frameworks. Int. J. Environ. Res. Public Health 11 (2), 1279–1298 (2014).

Article   PubMed   PubMed Central   Google Scholar  

Bartz, C. C. Nursing care in telemedicine and telehealth across the world. Soins. 61 (810), 57–59 (2016).

Article   PubMed   Google Scholar  

Lin, G.S.S., Koh, S.H., Ter, K.Z., Lim, C.W., Sultana, S., Tan, W.W. Awareness, knowledge, attitude, and practice of teledentistry among dental practitioners during COVID-19: A systematic review and meta-analysis. Medicina (Kaunas). 58 (1), 130 (2022).

Wolf, T.G., Schulze, R.K.W., Ramos-Gomez, F., Campus, G. Effectiveness of telemedicine and teledentistry after the COVID-19 pandemic. Int. J. Environ. Res. Public Health. 19 (21), 13857 (2022).

Gajarawala, S. N. & Pelkowski, J. N. Telehealth benefits and barriers. J. Nurse Pract. 17 (2), 218–221 (2021).

Jampani, N. D., Nutalapati, R., Dontula, B. S. & Boyapati, R. Applications of teledentistry: A literature review and update. J. Int. Soc. Prev. Community Dent. 1 (2), 37–44 (2011).

Article   CAS   PubMed   PubMed Central   Google Scholar  

Khan, S. A. & Omar, H. Teledentistry in practice: literature review. Telemed. J. E. Health. 19 (7), 565–567 (2013).

Baheti, M. J. B. S., Toshniwal, N. G. & Misal, A. Teledentistry: A need of the era. Int. J. Dent. Med. Res. 1 (2), 80–91 (2014).

Google Scholar  

Datta, N., Derenne, J., Sanders, M. & Lock, J. D. Telehealth transition in a comprehensive care unit for eating disorders: Challenges and long-term benefits. Int. J. Eat. Disord. 53 (11), 1774–1779 (2020).

Bursell, S. E., Brazionis, L. & Jenkins, A. Telemedicine and ocular health in diabetes mellitus. Clin. Exp. Optom. 95 (3), 311–327 (2012).

da Costa, C. B., Peralta, F. D. S. & Ferreira de Mello, A. L. S. How has teledentistry been applied in public dental health services? An integrative review. Telemed. J. E. Health. 26 (7), 945–954 (2020).

Heckemann, B., Wolf, A., Ali, L., Sonntag, S. M. & Ekman, I. Discovering untapped relationship potential with patients in telehealth: A qualitative interview study. BMJ Open. 6 (3), e009750 (2016).

Pérez-Noboa, B., Soledispa-Carrasco, A., Padilla, V. S. & Velasquez, W. Teleconsultation apps in the COVID-19 pandemic: The case of Guayaquil City, Ecuador. IEEE Eng. Manag. Rev. 49 (1), 27–37 (2021).

Article   Google Scholar  

Wamsley, C. E., Kramer, A., Kenkel, J. M. & Amirlak, B. Trends and challenges of telehealth in an academic institution: The unforeseen benefits of the COVID-19 global pandemic. Aesthetic Surg. J. 41 (1), 109–118 (2020).

Jonasdottir, S. K., Thordardottir, I. & Jonsdottir, T. Health professionals’ perspective towards challenges and opportunities of telehealth service provision: A scoping review. Int. J. Med. Inform. 167 , 104862 (2022).

Tan, S. H. X., Lee, C. K. J., Yong, C. W. & Ding, Y. Y. Scoping review: Facilitators and barriers in the adoption of teledentistry among older adults. Gerodontology. 38 (4), 351–365 (2021).

Minervini, G. et al. Teledentistry in the management of patients with dental and temporomandibular disorders. BioMed. Res. Int. 2022 , 7091153 (2022).

Edirippulige, S. & Armfield, N. Education and training to support the use of clinical telehealth: A review of the literature. J. Telemed. Telecare. 23 (2), 273–282 (2017).

Article   CAS   PubMed   Google Scholar  

Mariño, R. & Ghanim, A. Teledentistry: A systematic review of the literature. J. Telemed. Telecare. 19 (4), 179–183 (2013).

Article   ADS   PubMed   Google Scholar  

Armitage-Chan, E. & Whiting, M. Teaching professionalism: Using role-play simulations to generate professionalism learning outcomes. J. Vet. Med. Educ. 43 (4), 359–363 (2016).

Spyropoulos, F., Trichakis, I. & Vozinaki, A.-E. A narrative-driven role-playing game for raising flood awareness. Sustainability. 14 (1), 554 (2022).

Jiang, W. K. et al. Role-play in endodontic teaching: A case study. Chin. J. Dent. Res. 23 (4), 281–288 (2020).

PubMed   Google Scholar  

Vizeshfar, F., Zare, M. & Keshtkaran, Z. Role-play versus lecture methods in community health volunteers. Nurse Educ. Today. 79 , 175–179 (2019).

Nestel, D. & Tierney, T. Role-play for medical students learning about communication: Guidelines for maximising benefits. BMC Med. Educ. 7 , 3 (2007).

Gelis, A. et al. Peer role-play for training communication skills in medical students: A systematic review. Simulat. Health. 15 (2), 106–111 (2020).

Cornelius, S., Gordon, C. & Harris, M. Role engagement and anonymity in synchronous online role play. Int. Rev. Res. Open Distrib. Learn. 12 (5), 57–73 (2011).

Bell, M. Online role-play: Anonymity, engagement and risk. Educ. Med. Int. 38 (4), 251–260 (2001).

Sipiyaruk, K., Gallagher, J. E., Hatzipanagos, S. & Reynolds, P. A. A rapid review of serious games: From healthcare education to dental education. Eur. J. Dent. Educ. 22 (4), 243–257 (2018).

Sipiyaruk, K., Hatzipanagos, S., Reynolds, P. A. & Gallagher, J. E. Serious games and the COVID-19 pandemic in dental education: An integrative review of the literature. Computers. 10 (4), 42 (2021).

Morse, J.M., Niehaus, L. Mixed Method Design: Principles and Procedures. (2016).

Creswell, J. W. Research Design: Qualitative, Quantitative, and Mixed Methods Approaches 3rd edn. (SAGE Publications, 2009).

Cheng, V. W. S., Davenport, T., Johnson, D., Vella, K. & Hickie, I. B. Gamification in apps and technologies for improving mental health and well-being: Systematic review. JMIR Ment. Health. 6 (6), e13717 (2019).

Gallego-Durán, F. J. et al. A guide for game-design-based gamification. Informatics. 6 (4), 49 (2019).

Gee, J. P. Learning and games. In The Ecology of Games: Connecting Youth, Games, and Learning (ed. Salen, K.) 21–40 (MIT Press, 2008).

Cheung, K. L., ten Klooster, P. M., Smit, C., de Vries, H. & Pieterse, M. E. The impact of non-response bias due to sampling in public health studies: A comparison of voluntary versus mandatory recruitment in a Dutch national survey on adolescent health. BMC Public Health. 17 (1), 276 (2017).

Murairwa, S. Voluntary sampling design. Int. J. Adv. Res. Manag. Social Sci. 4 (2), 185–200 (2015).

Chow, S.-C., Shao, J., Wang, H. & Lokhnygina, Y. Sample Size Calculations in Clinical Research (CRC Press, 2017).

Book   Google Scholar  

Palinkas, L. A. et al. Purposeful sampling for qualitative data collection and analysis in mixed method implementation research. Administration Policy Mental Health Mental Health Services Res. 42 (5), 533–544 (2015).

McIlvried, D. E., Prucka, S. K., Herbst, M., Barger, C. & Robin, N. H. The use of role-play to enhance medical student understanding of genetic counseling. Genet. Med. 10 (10), 739–744 (2008).

Schlegel, C., Woermann, U., Shaha, M., Rethans, J.-J. & van der Vleuten, C. Effects of communication training on real practice performance: A role-play module versus a standardized patient module. J. Nursing Educ. 51 (1), 16–22 (2012).

Manzoor, I. M. F. & Hashmi, N. R. Medical students’ perspective about role-plays as a teaching strategy in community medicine. J. Coll. Physicians Surg. Pak. 22 (4), 222–225 (2012).

Cornes, S., Gelfand, J. M. & Calton, B. Foundational telemedicine workshop for first-year medical students developed during a pandemic. MedEdPORTAL. 17 , 11171 (2021).

King, J., Hill, K. & Gleason, A. All the world’sa stage: Evaluating psychiatry role-play based learning for medical students. Austral. Psychiatry. 23 (1), 76–79 (2015).

Arayapisit, T. et al. An educational board game for learning orofacial spaces: An experimental study comparing collaborative and competitive approaches. Anatomical Sci. Educ. 16 (4), 666–676 (2023).

Sipiyaruk, K., Hatzipanagos, S., Vichayanrat, T., Reynolds, P.A., Gallagher, J.E. Evaluating a dental public health game across two learning contexts. Educ. Sci. 12 (8), 517 (2022).

Pilnick, A. et al. Using conversation analysis to inform role play and simulated interaction in communications skills training for healthcare professionals: Identifying avenues for further development through a scoping review. BMC Med. Educ. 18 (1), 267 (2018).

Lane, C. & Rollnick, S. The use of simulated patients and role-play in communication skills training: A review of the literature to August 2005. Patient Educ. Counseling. 67 (1), 13–20 (2007).

Gale, N. K., Heath, G., Cameron, E., Rashid, S. & Redwood, S. Using the framework method for the analysis of qualitative data in multi-disciplinary health research. BMC Med. Res. Methodol. 13 (1), 117 (2013).

Ritchie, J., Lewis, J., Nicholls, C. M. & Ormston, R. Qualitative Research Practice: A Guide for Social Science Students and Researchers (Sage, 2014).

Chen, J. C. & Martin, A. R. Role-play simulations as a transformative methodology in environmental education. J. Transform. Educ. 13 (1), 85–102 (2015).

Davis, F. D. Perceived usefulness, perceived ease of use, and user acceptance of information technology. Manag. Inform. Syst. Quart. 13 (3), 319–340 (1989).

Novak, E., Johnson, T. E., Tenenbaum, G. & Shute, V. J. Effects of an instructional gaming characteristic on learning effectiveness, efficiency, and engagement: Using a storyline for teaching basic statistical skills. Interact. Learn. Environ. 24 (3), 523–538 (2016).

Marchiori, E. J. et al. A narrative metaphor to facilitate educational game authoring. Comput. Educ. 58 (1), 590–599 (2012).

Luctkar-Flude, M. et al. Effectiveness of debriefing methods for virtual simulation: A systematic review. Clin. Simulat. Nursing. 57 , 18–30 (2021).

Joyner, B. & Young, L. Teaching medical students using role play: Twelve tips for successful role plays. Med. Teach. 28 (3), 225–229 (2006).

Csikszentmihalyi, M. Flow: The Psychology of Optimal Performance (HarperCollins Publishers, 1990).

Buajeeb, W., Chokpipatkun, J., Achalanan, N., Kriwattanawong, N. & Sipiyaruk, K. The development of an online serious game for oral diagnosis and treatment planning: Evaluation of knowledge acquisition and retention. BMC Med. Educ. 23 (1), 830 (2023).

Littlefield, J. H., Hahn, H. B. & Meyer, A. S. Evaluation of a role-play learning exercise in an ambulatory clinic setting. Adv. Health. Sci. Educ. Theory Pract. 4 (2), 167–173 (1999).

Alkin, M. C. & Christie, C. A. The use of role-play in teaching evaluation. Am. J. Evaluat. 23 (2), 209–218 (2002).

Lovell, K. L., Mavis, B. E., Turner, J. L., Ogle, K. S. & Griffith, M. Medical students as standardized patients in a second-year performance-based assessment experience. Med. Educ. Online. 3 (1), 4301 (1998).

Download references

Acknowledgements

The authors would like to express our sincere gratitude to participants for their contributions in this research. We would also like to thank the experts who provided their helpful suggestions in the validation process of the data collection tools.

This research project was funded by the Faculty of Dentistry, Mahidol University. The APC was funded by Mahidol University.

Author information

Authors and affiliations.

Department of Advanced General Dentistry, Faculty of Dentistry, Mahidol University, Bangkok, Thailand

Chayanid Teerawongpairoj & Chanita Tantipoj

Department of Orthodontics, Faculty of Dentistry, Mahidol University, Bangkok, Thailand

Kawin Sipiyaruk

You can also search for this author in PubMed   Google Scholar

Contributions

Conceptualization, C.Te., C.Ta., and K.S.; methodology, C.Te., C.Ta., and K.S.; validation, C.Te., C.Ta., and K.S.; investigation, C.Te. and K.S.; formal analysis, C.Te., C.Ta., and K.S.; resources, C.Te., C.Ta., and K.S.; data curation, C.Ta. and K.S.; writing-original draft preparation, C.Te., C.Ta., and K.S.; writing-review and editing, C.Te., C.Ta., and K.S. All authors have read and agreed to the published version of the manuscript.

Corresponding author

Correspondence to Kawin Sipiyaruk .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary information 1., supplementary information 2., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Teerawongpairoj, C., Tantipoj, C. & Sipiyaruk, K. The design and evaluation of gamified online role-play as a telehealth training strategy in dental education: an explanatory sequential mixed-methods study. Sci Rep 14 , 9216 (2024). https://doi.org/10.1038/s41598-024-58425-9

Download citation

Received : 30 September 2023

Accepted : 28 March 2024

Published : 22 April 2024

DOI : https://doi.org/10.1038/s41598-024-58425-9

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Dental education
• Distance learning
• Game-based learning
• Gamification

By submitting a comment you agree to abide by our Terms and Community Guidelines . If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Numbers, Facts and Trends Shaping Your World

Read our research on:

Full Topic List

Regions & Countries

• Publications
• Our Methods
• Short Reads
• Tools & Resources

Read Our Research On:

Writing Survey Questions

Perhaps the most important part of the survey process is the creation of questions that accurately measure the opinions, experiences and behaviors of the public. Accurate random sampling will be wasted if the information gathered is built on a shaky foundation of ambiguous or biased questions. Creating good measures involves both writing good questions and organizing them to form the questionnaire.

Questionnaire design is a multistage process that requires attention to many details at once. Designing the questionnaire is complicated because surveys can ask about topics in varying degrees of detail, questions can be asked in different ways, and questions asked earlier in a survey may influence how people respond to later questions. Researchers are also often interested in measuring change over time and therefore must be attentive to how opinions or behaviors have been measured in prior surveys.

Surveyors may conduct pilot tests or focus groups in the early stages of questionnaire development in order to better understand how people think about an issue or comprehend a question. Pretesting a survey is an essential step in the questionnaire design process to evaluate how people respond to the overall questionnaire and specific questions, especially when questions are being introduced for the first time.

For many years, surveyors approached questionnaire design as an art, but substantial research over the past forty years has demonstrated that there is a lot of science involved in crafting a good survey questionnaire. Here, we discuss the pitfalls and best practices of designing questionnaires.

Question development

There are several steps involved in developing a survey questionnaire. The first is identifying what topics will be covered in the survey. For Pew Research Center surveys, this involves thinking about what is happening in our nation and the world and what will be relevant to the public, policymakers and the media. We also track opinion on a variety of issues over time so we often ensure that we update these trends on a regular basis to better understand whether people’s opinions are changing.

At Pew Research Center, questionnaire development is a collaborative and iterative process where staff meet to discuss drafts of the questionnaire several times over the course of its development. We frequently test new survey questions ahead of time through qualitative research methods such as  focus groups , cognitive interviews, pretesting (often using an  online, opt-in sample ), or a combination of these approaches. Researchers use insights from this testing to refine questions before they are asked in a production survey, such as on the ATP.

Measuring change over time

Many surveyors want to track changes over time in people’s attitudes, opinions and behaviors. To measure change, questions are asked at two or more points in time. A cross-sectional design surveys different people in the same population at multiple points in time. A panel, such as the ATP, surveys the same people over time. However, it is common for the set of people in survey panels to change over time as new panelists are added and some prior panelists drop out. Many of the questions in Pew Research Center surveys have been asked in prior polls. Asking the same questions at different points in time allows us to report on changes in the overall views of the general public (or a subset of the public, such as registered voters, men or Black Americans), or what we call “trending the data”.

When measuring change over time, it is important to use the same question wording and to be sensitive to where the question is asked in the questionnaire to maintain a similar context as when the question was asked previously (see  question wording  and  question order  for further information). All of our survey reports include a topline questionnaire that provides the exact question wording and sequencing, along with results from the current survey and previous surveys in which we asked the question.

The Center’s transition from conducting U.S. surveys by live telephone interviewing to an online panel (around 2014 to 2020) complicated some opinion trends, but not others. Opinion trends that ask about sensitive topics (e.g., personal finances or attending religious services ) or that elicited volunteered answers (e.g., “neither” or “don’t know”) over the phone tended to show larger differences than other trends when shifting from phone polls to the online ATP. The Center adopted several strategies for coping with changes to data trends that may be related to this change in methodology. If there is evidence suggesting that a change in a trend stems from switching from phone to online measurement, Center reports flag that possibility for readers to try to head off confusion or erroneous conclusions.

Open- and closed-ended questions

One of the most significant decisions that can affect how people answer questions is whether the question is posed as an open-ended question, where respondents provide a response in their own words, or a closed-ended question, where they are asked to choose from a list of answer choices.

For example, in a poll conducted after the 2008 presidential election, people responded very differently to two versions of the question: “What one issue mattered most to you in deciding how you voted for president?” One was closed-ended and the other open-ended. In the closed-ended version, respondents were provided five options and could volunteer an option not on the list.

When explicitly offered the economy as a response, more than half of respondents (58%) chose this answer; only 35% of those who responded to the open-ended version volunteered the economy. Moreover, among those asked the closed-ended version, fewer than one-in-ten (8%) provided a response other than the five they were read. By contrast, fully 43% of those asked the open-ended version provided a response not listed in the closed-ended version of the question. All of the other issues were chosen at least slightly more often when explicitly offered in the closed-ended version than in the open-ended version. (Also see  “High Marks for the Campaign, a High Bar for Obama”  for more information.)

Researchers will sometimes conduct a pilot study using open-ended questions to discover which answers are most common. They will then develop closed-ended questions based off that pilot study that include the most common responses as answer choices. In this way, the questions may better reflect what the public is thinking, how they view a particular issue, or bring certain issues to light that the researchers may not have been aware of.

When asking closed-ended questions, the choice of options provided, how each option is described, the number of response options offered, and the order in which options are read can all influence how people respond. One example of the impact of how categories are defined can be found in a Pew Research Center poll conducted in January 2002. When half of the sample was asked whether it was “more important for President Bush to focus on domestic policy or foreign policy,” 52% chose domestic policy while only 34% said foreign policy. When the category “foreign policy” was narrowed to a specific aspect – “the war on terrorism” – far more people chose it; only 33% chose domestic policy while 52% chose the war on terrorism.

In most circumstances, the number of answer choices should be kept to a relatively small number – just four or perhaps five at most – especially in telephone surveys. Psychological research indicates that people have a hard time keeping more than this number of choices in mind at one time. When the question is asking about an objective fact and/or demographics, such as the religious affiliation of the respondent, more categories can be used. In fact, they are encouraged to ensure inclusivity. For example, Pew Research Center’s standard religion questions include more than 12 different categories, beginning with the most common affiliations (Protestant and Catholic). Most respondents have no trouble with this question because they can expect to see their religious group within that list in a self-administered survey.

In addition to the number and choice of response options offered, the order of answer categories can influence how people respond to closed-ended questions. Research suggests that in telephone surveys respondents more frequently choose items heard later in a list (a “recency effect”), and in self-administered surveys, they tend to choose items at the top of the list (a “primacy” effect).

Because of concerns about the effects of category order on responses to closed-ended questions, many sets of response options in Pew Research Center’s surveys are programmed to be randomized to ensure that the options are not asked in the same order for each respondent. Rotating or randomizing means that questions or items in a list are not asked in the same order to each respondent. Answers to questions are sometimes affected by questions that precede them. By presenting questions in a different order to each respondent, we ensure that each question gets asked in the same context as every other question the same number of times (e.g., first, last or any position in between). This does not eliminate the potential impact of previous questions on the current question, but it does ensure that this bias is spread randomly across all of the questions or items in the list. For instance, in the example discussed above about what issue mattered most in people’s vote, the order of the five issues in the closed-ended version of the question was randomized so that no one issue appeared early or late in the list for all respondents. Randomization of response items does not eliminate order effects, but it does ensure that this type of bias is spread randomly.

Questions with ordinal response categories – those with an underlying order (e.g., excellent, good, only fair, poor OR very favorable, mostly favorable, mostly unfavorable, very unfavorable) – are generally not randomized because the order of the categories conveys important information to help respondents answer the question. Generally, these types of scales should be presented in order so respondents can easily place their responses along the continuum, but the order can be reversed for some respondents. For example, in one of Pew Research Center’s questions about abortion, half of the sample is asked whether abortion should be “legal in all cases, legal in most cases, illegal in most cases, illegal in all cases,” while the other half of the sample is asked the same question with the response categories read in reverse order, starting with “illegal in all cases.” Again, reversing the order does not eliminate the recency effect but distributes it randomly across the population.

Question wording

The choice of words and phrases in a question is critical in expressing the meaning and intent of the question to the respondent and ensuring that all respondents interpret the question the same way. Even small wording differences can substantially affect the answers people provide.

[View more Methods 101 Videos ]

An example of a wording difference that had a significant impact on responses comes from a January 2003 Pew Research Center survey. When people were asked whether they would “favor or oppose taking military action in Iraq to end Saddam Hussein’s rule,” 68% said they favored military action while 25% said they opposed military action. However, when asked whether they would “favor or oppose taking military action in Iraq to end Saddam Hussein’s rule  even if it meant that U.S. forces might suffer thousands of casualties, ” responses were dramatically different; only 43% said they favored military action, while 48% said they opposed it. The introduction of U.S. casualties altered the context of the question and influenced whether people favored or opposed military action in Iraq.

There has been a substantial amount of research to gauge the impact of different ways of asking questions and how to minimize differences in the way respondents interpret what is being asked. The issues related to question wording are more numerous than can be treated adequately in this short space, but below are a few of the important things to consider:

First, it is important to ask questions that are clear and specific and that each respondent will be able to answer. If a question is open-ended, it should be evident to respondents that they can answer in their own words and what type of response they should provide (an issue or problem, a month, number of days, etc.). Closed-ended questions should include all reasonable responses (i.e., the list of options is exhaustive) and the response categories should not overlap (i.e., response options should be mutually exclusive). Further, it is important to discern when it is best to use forced-choice close-ended questions (often denoted with a radio button in online surveys) versus “select-all-that-apply” lists (or check-all boxes). A 2019 Center study found that forced-choice questions tend to yield more accurate responses, especially for sensitive questions.  Based on that research, the Center generally avoids using select-all-that-apply questions.

It is also important to ask only one question at a time. Questions that ask respondents to evaluate more than one concept (known as double-barreled questions) – such as “How much confidence do you have in President Obama to handle domestic and foreign policy?” – are difficult for respondents to answer and often lead to responses that are difficult to interpret. In this example, it would be more effective to ask two separate questions, one about domestic policy and another about foreign policy.

In general, questions that use simple and concrete language are more easily understood by respondents. It is especially important to consider the education level of the survey population when thinking about how easy it will be for respondents to interpret and answer a question. Double negatives (e.g., do you favor or oppose  not  allowing gays and lesbians to legally marry) or unfamiliar abbreviations or jargon (e.g., ANWR instead of Arctic National Wildlife Refuge) can result in respondent confusion and should be avoided.

Similarly, it is important to consider whether certain words may be viewed as biased or potentially offensive to some respondents, as well as the emotional reaction that some words may provoke. For example, in a 2005 Pew Research Center survey, 51% of respondents said they favored “making it legal for doctors to give terminally ill patients the means to end their lives,” but only 44% said they favored “making it legal for doctors to assist terminally ill patients in committing suicide.” Although both versions of the question are asking about the same thing, the reaction of respondents was different. In another example, respondents have reacted differently to questions using the word “welfare” as opposed to the more generic “assistance to the poor.” Several experiments have shown that there is much greater public support for expanding “assistance to the poor” than for expanding “welfare.”

We often write two versions of a question and ask half of the survey sample one version of the question and the other half the second version. Thus, we say we have two  forms  of the questionnaire. Respondents are assigned randomly to receive either form, so we can assume that the two groups of respondents are essentially identical. On questions where two versions are used, significant differences in the answers between the two forms tell us that the difference is a result of the way we worded the two versions.

One of the most common formats used in survey questions is the “agree-disagree” format. In this type of question, respondents are asked whether they agree or disagree with a particular statement. Research has shown that, compared with the better educated and better informed, less educated and less informed respondents have a greater tendency to agree with such statements. This is sometimes called an “acquiescence bias” (since some kinds of respondents are more likely to acquiesce to the assertion than are others). This behavior is even more pronounced when there’s an interviewer present, rather than when the survey is self-administered. A better practice is to offer respondents a choice between alternative statements. A Pew Research Center experiment with one of its routinely asked values questions illustrates the difference that question format can make. Not only does the forced choice format yield a very different result overall from the agree-disagree format, but the pattern of answers between respondents with more or less formal education also tends to be very different.

One other challenge in developing questionnaires is what is called “social desirability bias.” People have a natural tendency to want to be accepted and liked, and this may lead people to provide inaccurate answers to questions that deal with sensitive subjects. Research has shown that respondents understate alcohol and drug use, tax evasion and racial bias. They also may overstate church attendance, charitable contributions and the likelihood that they will vote in an election. Researchers attempt to account for this potential bias in crafting questions about these topics. For instance, when Pew Research Center surveys ask about past voting behavior, it is important to note that circumstances may have prevented the respondent from voting: “In the 2012 presidential election between Barack Obama and Mitt Romney, did things come up that kept you from voting, or did you happen to vote?” The choice of response options can also make it easier for people to be honest. For example, a question about church attendance might include three of six response options that indicate infrequent attendance. Research has also shown that social desirability bias can be greater when an interviewer is present (e.g., telephone and face-to-face surveys) than when respondents complete the survey themselves (e.g., paper and web surveys).

Lastly, because slight modifications in question wording can affect responses, identical question wording should be used when the intention is to compare results to those from earlier surveys. Similarly, because question wording and responses can vary based on the mode used to survey respondents, researchers should carefully evaluate the likely effects on trend measurements if a different survey mode will be used to assess change in opinion over time.

Question order

Once the survey questions are developed, particular attention should be paid to how they are ordered in the questionnaire. Surveyors must be attentive to how questions early in a questionnaire may have unintended effects on how respondents answer subsequent questions. Researchers have demonstrated that the order in which questions are asked can influence how people respond; earlier questions can unintentionally provide context for the questions that follow (these effects are called “order effects”).

One kind of order effect can be seen in responses to open-ended questions. Pew Research Center surveys generally ask open-ended questions about national problems, opinions about leaders and similar topics near the beginning of the questionnaire. If closed-ended questions that relate to the topic are placed before the open-ended question, respondents are much more likely to mention concepts or considerations raised in those earlier questions when responding to the open-ended question.

For closed-ended opinion questions, there are two main types of order effects: contrast effects ( where the order results in greater differences in responses), and assimilation effects (where responses are more similar as a result of their order).

An example of a contrast effect can be seen in a Pew Research Center poll conducted in October 2003, a dozen years before same-sex marriage was legalized in the U.S. That poll found that people were more likely to favor allowing gays and lesbians to enter into legal agreements that give them the same rights as married couples when this question was asked after one about whether they favored or opposed allowing gays and lesbians to marry (45% favored legal agreements when asked after the marriage question, but 37% favored legal agreements without the immediate preceding context of a question about same-sex marriage). Responses to the question about same-sex marriage, meanwhile, were not significantly affected by its placement before or after the legal agreements question.

Another experiment embedded in a December 2008 Pew Research Center poll also resulted in a contrast effect. When people were asked “All in all, are you satisfied or dissatisfied with the way things are going in this country today?” immediately after having been asked “Do you approve or disapprove of the way George W. Bush is handling his job as president?”; 88% said they were dissatisfied, compared with only 78% without the context of the prior question.

Responses to presidential approval remained relatively unchanged whether national satisfaction was asked before or after it. A similar finding occurred in December 2004 when both satisfaction and presidential approval were much higher (57% were dissatisfied when Bush approval was asked first vs. 51% when general satisfaction was asked first).

Several studies also have shown that asking a more specific question before a more general question (e.g., asking about happiness with one’s marriage before asking about one’s overall happiness) can result in a contrast effect. Although some exceptions have been found, people tend to avoid redundancy by excluding the more specific question from the general rating.

Assimilation effects occur when responses to two questions are more consistent or closer together because of their placement in the questionnaire. We found an example of an assimilation effect in a Pew Research Center poll conducted in November 2008 when we asked whether Republican leaders should work with Obama or stand up to him on important issues and whether Democratic leaders should work with Republican leaders or stand up to them on important issues. People were more likely to say that Republican leaders should work with Obama when the question was preceded by the one asking what Democratic leaders should do in working with Republican leaders (81% vs. 66%). However, when people were first asked about Republican leaders working with Obama, fewer said that Democratic leaders should work with Republican leaders (71% vs. 82%).

The order questions are asked is of particular importance when tracking trends over time. As a result, care should be taken to ensure that the context is similar each time a question is asked. Modifying the context of the question could call into question any observed changes over time (see  measuring change over time  for more information).

A questionnaire, like a conversation, should be grouped by topic and unfold in a logical order. It is often helpful to begin the survey with simple questions that respondents will find interesting and engaging. Throughout the survey, an effort should be made to keep the survey interesting and not overburden respondents with several difficult questions right after one another. Demographic questions such as income, education or age should not be asked near the beginning of a survey unless they are needed to determine eligibility for the survey or for routing respondents through particular sections of the questionnaire. Even then, it is best to precede such items with more interesting and engaging questions. One virtue of survey panels like the ATP is that demographic questions usually only need to be asked once a year, not in each survey.

U.S. Surveys

Other research methods, sign up for our weekly newsletter.

Fresh data delivered Saturday mornings

1615 L St. NW, Suite 800 Washington, DC 20036 USA (+1) 202-419-4300 | Main (+1) 202-857-8562 | Fax (+1) 202-419-4372 |  Media Inquiries

Research Topics

• Age & Generations
• Coronavirus (COVID-19)
• Economy & Work
• Family & Relationships
• Gender & LGBTQ
• Immigration & Migration
• International Affairs
• Internet & Technology
• Methodological Research
• News Habits & Media
• Non-U.S. Governments
• Other Topics
• Politics & Policy
• Race & Ethnicity
• Email Newsletters

ABOUT PEW RESEARCH CENTER  Pew Research Center is a nonpartisan fact tank that informs the public about the issues, attitudes and trends shaping the world. It conducts public opinion polling, demographic research, media content analysis and other empirical social science research. Pew Research Center does not take policy positions. It is a subsidiary of  The Pew Charitable Trusts .

Copyright 2024 Pew Research Center

Terms & Conditions

Privacy Policy

Cookie Settings

Reprints, Permissions & Use Policy