qualitative research and evaluation methods 2015

• Request new password
• Create a new account

Qualitative Research & Evaluation Methods: Integrating Theory and Practice

Student resources.

This site is intended to enhance your use of  Qualitative Research & Evaluation Methods, Fourth Edition ,  by Michael Quinn Patton. Please note that all the materials on this site are especially geared toward maximizing your understanding of the material. 

Drawing on more than 40 years of experience conducting applied social science research and program evaluation, author Michael Quinn Patton has crafted the most comprehensive and systematic book on qualitative research and evaluation methods, inquiry frameworks, and analysis options available today. Now offering more balance between applied research and evaluation, this  Fourth Edition  illuminates all aspects of qualitative inquiry through new examples, stories, and cartoons; more than a hundred new summarizing and synthesizing exhibits; and a wide range of new highlight sections/sidebars that elaborate on important and emergent issues. For the first time, full case studies are included to illustrate extended research and evaluation examples. In addition, each chapter features an extended "rumination," written in a voice and style more emphatic and engaging than traditional textbook style, about a core issue of persistent debate and controversy.

Acknowledgments

We gratefully acknowledge Michael Quinn Patton for writing an excellent text and creating the materials on this site.

Qualitative Research & Evaluation Methods: Integrating Theory and Practice, Edition 4

About this ebook, ratings and reviews.

• Flag inappropriate

About the author

Michael Quinn Patton is author of more than a dozen books on evaluation including Qualitative Research & Evaluation Methods, 4th ed (2015), Blue Marble Evaluation (2020), Principles-Focused Evaluation (2018), Facilitating Evaluation (2018) and Developmental Evaluation (2011). Based in Minnesota, he was on the faculty of the University of Minnesota for 18 years and is a former president of the American Evaluation Association (AEA). Michael is a recipient of the Alva and Gunnar Myrdal Evaluation Practice Award, the Paul F. Lazarsfeld Evaluation Theory Award, and the Research on Evaluation Award, all from AEA He has also received the Lester F. Ward Distinguished Contribution to Applied and Clinical Sociology Award from the Association for Applied and Clinical Sociology. In 2021 he received the first Transformative Evaluator Award from EvalYouth. He is an active speaker, trainer, and workshop presenter who has conducted applied research and evaluation on a broad range of issues and has worked with organizations and programs at the international, national, state, provincial, and local levels. Michael has three children—a musician, an engineer, and an evaluator—and four grandchildren. When not evaluating, he enjoys exploring the woods and rivers of Minnesota, where he lives.

Rate this ebook

Reading information, more by michael quinn patton.

Similar ebooks

This website may not work correctly because your browser is out of date. Please update your browser .

• Qualitative research & evaluation methods: integrating theory and practice

Resource link

The fourth edition of Michael Quinn Patton's  Qualitative Research & Evaluation Methods Integrating Theory and Practice,  published by Sage Publications, analyses and provides clear guidance and advice for using a range of different qualitative methods for evaluation.

• Module 1. How qualitative inquiry contributes to our understanding of the world
• Module 2. What makes qualitative data qualitative
• Module 3. Making methods decisions
• Module 4. The fruit of qualitative methods: Chapter summary and conclusion
• Module 5. Strategic design principles for qualitative inquiry
• Module 6. Strategic principles guiding data collection and fieldwork
• Module 7. Strategic principles for qualitative analysis and reporting findings
• Module 8: Integrating the 12 strategic qualitative principles in practice
• Module 9. Understanding the Paradigms Debate: Quants versus Quals
• Module 10. Introduction to Qualitative Inquiry Frameworks
• Module 11. Ethnography and Autoethnography
• Module 12. Positivism, Postpositivism, Empiricism and Foundationalist Epistemologies
• Module 13. Grounded Theory and Realism
• Module 14 Phenomenology and Heuristic Inquiry
• Module 15 Social Constructionism, Constructivism, Postmodernism, and Narrative Inquiry
• Module 16. Ethnomethodology, Semiotics, and Symbolic Interaction, Hermeneutics and Ecological Psychology
• Module 17 Systems Theory and Complexity Theory
• Module 18. Pragmatism, Generic Qualitative Inquiry, and Utilization-Focused Evaluation
• Module 19 Patterns and themes across inquiry frameworks: Chapter summary and conclusions
• Module 20. Practical purposes, concrete questions, and actionable answers: Illuminating and enhancing quality
• Module 21. Program evaluation applications: Focus on outcomes
• Module 22 Specialized qualitative evaluation applications
• Module 23 Evaluating program models and theories of change, and evaluation models especially aligned with qualitative methods
• Module 24 Interactive and participatory qualitative applications
• Module 25 Democratic evaluation, indigenous research and evaluation, capacity building, and cultural competence
• Module 26 Special methodological applications
• Module 27 A vision of the utility of qualitative methods: Chapter summary and conclusion
• Module 28 Design thinking: Questions derive from purpose, design answers questions
• Module 29 Date Collection Decisions
• Module 30 Purposeful sampling and case selection: Overview of strategies and options
• Module 31 Single-significant-case sampling as a design strategy
• Module 32 Comparison-focused sampling options
• Module 33 Group characteristics sampling strategies and options
• Module 34 Concept and theoretical sampling strategies and options
• Module 35. Instrumental-use multiple-case sampling
• Module 36 Sequential and emergence-driven sampling strategies and options
• Module 37 Analytically focused sampling
• Module 38 Mixed, stratified, and nested purposeful sampling strategies
• Module 39 Information-rich cases
• Module 40 Sample size for qualitative designs
• Module 41 Mixed methods designs
• Module 42 Qualitative design chapter summary and conclusion: Methods choices and decisions
• Module 43 The Power of direct observation
• Module 44. Variations in observational methods
• Module 45. Variations in duration of observations and site visits: From rapid reconnaissance to longitudinal studies over years
• Module 46. Variations in observational focus and summary of dimensions along which fieldwork varies
• Module 47. What to observe: Sensitizing concepts
• Module 48. Integrating what to observe with how to observe
• Module 49. Unobtrusive observations and indicators, and documents and archival fieldwork
• Module 50. Observing oneself: Reflexivity and Creativity, and Review of Fieldwork Dimensions
• Module 51. Doing Fieldwork: The Data Gathering Process
• Module 52. Stages of fieldwork: Entry into the field
• Module 53. Routinization of fieldwork: The dynamics of the second stage
• Module 54. Bringing fieldwork to a close
• Module 55. The observer and what is observed: Unity, separation, and reactivity
• Module 56. Chapter summary and conclusion: Guidelines for fieldwork
• Module 57 The Interview Society: Diversity of applications
• Module 58 Distinguishing interview approaches and types of interviews
• Module 59 Question options and skilled question formulation
• Module 60 Rapport, neutrality, and the interview relationship
• Module 61 Interviewing groups and cross-cultural interviewing
• Module 62. Creative modes of qualitative inquiry
• Module 63. Ethical issues and challenges in qualitative interviewing
• Module 64. Personal reflections on interviewing, and chapter summary and conclusion
• Module 65. Setting the Context for Qualitative Analysis: Challenge, Purpose, and Focus
• Module 66. Thick description and case studies: The bedrock of qualitative analysis
• Module 67. Qualitative Analysis Approaches: Identifying Patterns and Themes
• Module 68. The intellectual and operational work of analysis
• Module 69. Logical and matrix analyses, and synthesizing qualitative studies
• Module 70. Interpreting findings, determining substantive significance, phenomenological essence, and hermeneutic interpretation
• Module 71. Causal explanation thorough qualitative analysis
• Module 72. New analysis directions: Contribution analysis, participatory analysis, and qualitative counterfactuals
• Module 73. Writing up and reporting findings, including using visuals
• Module 74. Special analysis and reporting issues: Mixed methods, focused communications, and principles-focused report exemplar.
• Module 75 Chapter summary and conclusion, plus case study exhibits
• Module 76. Analytical processes for enhancing credibility: systematically engaging and questioning the data
• Module 77. Four triangulation processes for enhancing credibility
• Part 1, universal criteria, and traditional scientific research versus constructivist criteria
• Part 2: artistic, participatory, critical change, systems, pragmatic, and mixed criteria
• Module 80 Credibility of the inquirer
• Module 81 Generalizations, Extrapolations, Transferability, Principles, and Lessons learned
• Module 82 Enhancing the credibility and utility of qualitative inquiry by addressing philosophy of science issues

Patton, M. Q. (2014).  Qualitative Research & Evaluation Methods: Integrative Theory and Practice . SAGE Publications.

'Qualitative research & evaluation methods: integrating theory and practice' is referenced in:

• Week 47: Rumination #3: Fools' gold: the widely touted methodological "gold standard" is neither golden nor a standard

Framework/Guide

• Rainbow Framework :  Sample

Back to top

© 2022 BetterEvaluation. All right reserved.

• Writing, Research & Publishing Guides

Buy new: $33.99 $33.99 $3.99 delivery May 6 - 9 Ships from: familybookvalues Sold by: familybookvalues

Buy used: $9.68.

Fulfillment by Amazon (FBA) is a service we offer sellers that lets them store their products in Amazon's fulfillment centers, and we directly pack, ship, and provide customer service for these products. Something we hope you'll especially enjoy: FBA items qualify for FREE Shipping and Amazon Prime.

If you're a seller, Fulfillment by Amazon can help you grow your business. Learn more about the program.

Download the free Kindle app and start reading Kindle books instantly on your smartphone, tablet, or computer - no Kindle device required .

Read instantly on your browser with Kindle for Web.

Using your mobile phone camera - scan the code below and download the Kindle app.

Image Unavailable

• To view this video download Flash Player

Follow the author

Qualitative Evaluation and Research Methods 2nd Edition

There is a newer edition of this item:.

Purchase options and add-ons

Once again setting the standard for the field, the Second Edition of Qualitative Evaluation Methods reflects the tremendous explosion of interest in qualitative methods over the past decade. Thoroughly revised and updated, this new edition includes three new chapters on Theoretic Foundations of Qualitative Inquiry, Particularly Appropriate Qualitative Applications and Quality, and Credibility of Qualitative Analysis. Patton has also a completely updated literature review and citations section to reflect the mass of new research in qualitative methods in the last ten years. It will be of interest to anyone involved in evaluation of any kind.

• ISBN-10 0803937792
• ISBN-13 978-0803937796
• Edition 2nd
• Publisher SAGE Publications, Inc
• Publication date February 1, 1990
• Language English
• Dimensions 1.25 x 5.75 x 9 inches
• Print length 536 pages
• See all details

Customers who viewed this item also viewed

Editorial Reviews

About the author.

Michael Quinn Patton  is author of more than a dozen books on evaluation including Qualitative Research & Evaluation Methods, 4th ed (2015), Blue Marble Evaluation (2020), Principles-Focused Evaluation (2018), Facilitating Evaluation (2018) and Developmental Evaluation (2011). Based in Minnesota, he was on the faculty of the University of Minnesota for 18 years and is a former president of the American Evaluation Association (AEA). Michael is a recipient of the Alva and Gunnar Myrdal Evaluation Practice Award, the Paul F. Lazarsfeld Evaluation Theory Award, and the Research on Evaluation Award, all from AEA He has also received the Lester F. Ward Distinguished Contribution to Applied and Clinical Sociology Award from the Association for Applied and Clinical Sociology. In 2021 he received the first Transformative Evaluator Award from EvalYouth. He is an active speaker, trainer, and workshop presenter who has conducted applied research and evaluation on a broad range of issues and has worked with organizations and programs at the international, national, state, provincial, and local levels. Michael has three children―a musician, an engineer, and an evaluator―and four grandchildren. When not evaluating, he enjoys exploring the woods and rivers of Minnesota, where he lives.

Product details

• Publisher ‏ : ‎ SAGE Publications, Inc; 2nd edition (February 1, 1990)
• Language ‏ : ‎ English
• Hardcover ‏ : ‎ 536 pages
• ISBN-10 ‏ : ‎ 0803937792
• ISBN-13 ‏ : ‎ 978-0803937796
• Item Weight ‏ : ‎ 1.65 pounds
• Dimensions ‏ : ‎ 1.25 x 5.75 x 9 inches
• #633 in Social Sciences Methodology
• #1,101 in Medical Psychology of Sexuality
• #1,263 in Social Sciences Research

About the author

Michael quinn patton.

Michael Quinn Patton lives in Minnesota where, according to the state's poet laureate, Garrison Keillor, "all the women are strong, all the men are good looking, and all the children are above average." It was this lack of interesting statistical variation in Minnesota that led him to qualitative inquiry despite the strong quantitative orientation of his doctoral studies in sociology at the University of Wisconsin. He serves on the graduate faculty of The Union Institute, a nontraditional, interdisciplinary, nonresidential and individually designed doctoral program.

He was on the faculty of the University of Minnesota for 18 years, including five years as Director of the Minnesota Center for Social Research, where he was awarded the Morse-Amoco Award for innovative teaching. He won the University of Minnesota storytelling competition and has authored several other books which include Utilization-Focused Evaluation, Creative Evaluation, Practical Evaluation, How to Use Qualitative Methods in Evaluation, and Family Sexual Abuse: Frontline Research and Evaluation.

He edited Culture and Evaluation for the journal New Direction in Program Evaluation. His creative nonfiction book, Grand Canyon Celebration: A Father-Son Journey of Discovery, was a finalist for 1999 Minnesota Book of the Year.He is former President of the American Evaluation Association and the only recipient of both the Alva and Gunner Myrdal Award for Outstanding Contributions to Useful and Practical Evaluation from the Evaluation Research Society and the Paul F. Lazarsfeld Award for Lifelong Contributions to Evaluation Theory from the American Evaluation Association. The Society for Applied Sociology awarded him the 2001 Lester F. Ward Award for Outstanding Contributions to Applied Sociology.

Customer reviews

Customer Reviews, including Product Star Ratings help customers to learn more about the product and decide whether it is the right product for them.

To calculate the overall star rating and percentage breakdown by star, we don’t use a simple average. Instead, our system considers things like how recent a review is and if the reviewer bought the item on Amazon. It also analyzed reviews to verify trustworthiness.

• Sort reviews by Top reviews Most recent Top reviews

Top reviews from the United States

There was a problem filtering reviews right now. please try again later..

Top reviews from other countries

• Amazon Newsletter
• About Amazon
• Accessibility
• Sustainability
• Press Center
• Investor Relations
• Amazon Devices
• Amazon Science
• Sell on Amazon
• Sell apps on Amazon
• Supply to Amazon
• Protect & Build Your Brand
• Become an Affiliate
• Become a Delivery Driver
• Start a Package Delivery Business
• Advertise Your Products
• Self-Publish with Us
• Become an Amazon Hub Partner
• › See More Ways to Make Money
• Amazon Visa
• Amazon Store Card
• Amazon Secured Card
• Amazon Business Card
• Shop with Points
• Credit Card Marketplace
• Reload Your Balance
• Amazon Currency Converter
• Your Account
• Your Orders
• Shipping Rates & Policies
• Amazon Prime
• Returns & Replacements
• Manage Your Content and Devices
• Recalls and Product Safety Alerts
• Conditions of Use
• Privacy Notice
• Consumer Health Data Privacy Disclosure
• Your Ads Privacy Choices

• Technical Support
• Find My Rep

You are here

Research and Evaluation in Education and Psychology Integrating Diversity With Quantitative, Qualitative, and Mixed Methods

• Donna M. Mertens - Gallaudet University, USA
• Description

Updated to align with the American Psychological Association and the National Council of Accreditation of Teacher Education accreditation requirements.   Focused on increasing the credibility of research and evaluation, the Fifth Edition of Research and Evaluation in Education and Psychology: Integrating Diversity with Quantitative, Qualitative, and Mixed Methods incorporates the viewpoints of various research paradigms into its descriptions of these methods. Students will learn to identify, evaluate, and practice good research, with special emphasis on conducting research in culturally complex communities, based on the perspectives of women, LGBTQ communities, ethnic/racial minorities, and people with disabilities. In each chapter, Dr. Donna M. Mertens carefully explains a step of the research process—from the literature review to analysis and reporting—and includes a sample study and abstract to illustrate the concepts discussed.   The new edition includes over 30 new research studies and contemporary examples to demonstrate research methods including:

• Black girls and school discipline: The complexities of being overrepresented and understudied (Annamma, S.A., Anyon, Y., Joseph, N.M., Farrar, J., Greer, E., Downing, B., & Simmons, J.)
• Learning Cooperatively under Challenging Circumstances: Cooperation among Students in High-Risk Contexts in El Salvador (Christine Schmalenbach)
• Replicated Evidence of Racial and Ethnic Disparities in Disability Identification in U.S. Schools (Morgan, et. al.)
• Relation of white-matter microstructure to reading ability and disability in beginning readers (Christodoulu, et. al.)
• Arts and mixed methods research: an innovative methodological merger (Archibald, M.M. & Gerber, N.)   

See what’s new to this edition by selecting the Features tab on this page. Should you need additional information or have questions regarding the HEOA information provided for this title, including what is new to this edition, please email [email protected] . Please include your name, contact information, and the name of the title for which you would like more information. For information on the HEOA, please go to http://ed.gov/policy/highered/leg/hea08/index.html .

For assistance with your order: Please email us at [email protected] or connect with your SAGE representative.

SAGE 2455 Teller Road Thousand Oaks, CA 91320 www.sagepub.com

Supplements

The open-access  Student Study Site  includes the following:

• EXCLUSIVE! Access to certain full-text  SAGE journal articles   have been carefully selected for each chapter. Each article supports and expands on the concepts presented in the chapter. 

study.sagepub.com/mertens5e

Password-protected  Instructor Resources  include the following:

• Editable, chapter-specific Microsoft®  PowerPoint® slides  offer you complete flexibility in easily creating a multimedia presentation for your course. Highlight essential content, features, and artwork from the book.
• Sample course syllabi  for semester and quarter courses provide suggested models for use when creating the syllabi for your courses.
• EXCLUSIVE! Access to certain full-text  SAGE journal articles  that have been carefully selected for each chapter. Each article supports and expands on the concepts presented in the chapter. Combine cutting-edge academic journal scholarship with the topics in your course for a robust classroom experience.
• Chapter-specific exercises and activities offer practical application of the concepts and can be used in-class or as assignments.
• Tables and figures from the book available for download.

This text aligns well with the various Research Methods courses offered in the Department of Educational Policy Studies (EPS). This text is particularly relevant to/for those whose concentration is in Research, Methods, and Statistics (RMS).

NEW TO THIS EDITION: 

• Includes increased coverage of digital research throughout , with examples of research studies that use Twitter, Facebook, and digital data collection strategies.
• Increased coverage of mixed methods studies as examples is used throughout so readers can see how mixed methods can be used with a wide range of research perspectives and methods.
• Intersectionality is added as an essential part of effectively researching with marginalized communities.
• The book aligns with the latest APA and NCATE accreditation standards by increasing the focus on issues related to cultural competency.
• Ethics in research is addressed throughout , emphasizing its importance throughout the process of planning, conducting, and using research.
• Critical issues that have arisen in society are addressed , including violence in schools, sexual abuse, disproportionately severe discipline practices with students of color, and strategies to support inclusion of people with disabilities.
• More about the use of digital resources to disseminate and support the use of research findings is included.
• Reporting and publication of research is aligned with the APA recommendations for quantitative, qualitative, and mixed methods research

KEY FEATURES:  

• Postpositivist, constructivist, transformative, and pragmatic paradigms  discussed, showing how researchers’ views of the world underlie their choice of research approaches.
• Conducting research in culturally complex communities  emphasized throughout.
• A  step-by-step overview of the entire research process.
• Detailed explanation of  how to write a literature review and plan a dissertation.
• Pedagogy that includes  Extending Your Thinking  throughout, providing opportunity for critical application of concepts.

Sample Materials & Chapters

1: An Introduction to Research and Ethical Practice

3: Literature Review and Focusing the Research

For instructors

Select a purchasing option.

• Find My Rep

You are here

Qualitative Research & Evaluation Methods Integrating Theory and Practice

• Michael Quinn Patton - Utilization-Focused Evaluation, Saint Paul, MN
• Description
• Author(s) / Editor(s)

Drawing on more than 40 years of experience conducting applied social science research and program evaluation, author Michael Quinn Patton has crafted the most comprehensive and systematic book on qualitative research and evaluation methods, inquiry frameworks, and analysis options available today. Now offering more balance between applied research and evaluation, t his Fourth Edition illuminates all aspects of qualitative inquiry through new examples, stories, and cartoons; more than a hundred new summarizing and synthesizing exhibits; and a wide range of new highlight sections/sidebars that elaborate on important and emergent issues . For the first time, full case studies are included to illustrate extended research and evaluation examples. In addition, each chapter features an extended "rumination," written in a voice and style more emphatic and engaging than traditional textbook style, about a core issue of persistent debate and controversy.

Supplements

"Very thoughtful and thorough coverage of qualitative design and study."

  “The content itself, based in years of thinking, reading, doing, conversing, is a huge strength. Reading the chapters is like sitting at the feet of one of the masters.”  

“I can’t emphasize enough the quality, detail, and depth of the presentation of research design and methods… Students and experienced researchers will appreciate the depth of presentation of potential qualitative paradigms, theoretical orientations and frameworks as well as special methodological applications that are often not covered in other qualitative texts.”

“It is refreshing to see a text that engages the multiple philosophical and historical trajectories within a qualitative research tradition while integrating this discussion so well with the practice of research design, fieldwork strategies, and data analysis.”

I have used Patton for this course historically; I will continue to use him for this course. I have used previous editions as well.

Great book - not currently teaching a course in evaluation - will definitely consider this text when I do next teach such a course.

Mae’r llyfr yma yn wych, yn enwedig y bennod ar fframweithiau damcaniaethol ac athroniaeth. Rwyf wedi ei argymell i nifer o fyfyrwyr ôl-radd sydd wrthi’n cynllunio traethodau hir - trwy gwrs ‘Yr ymchwilydd ansoddol’ y Coleg Cymraeg Cenedlaethol (CCC) a hefyd myfyrwyr Bangor (Cymraeg a Saesneg ei hiaith). Rwyf hefyd wedi gofyn i’r llyfrgellydd gwyddorau cymdeithasol archebu un neu ddau o gopïau i’r llyfrgell.

Adopted Creswell 3rd ed (Sage)

Good resource for students looking to develop their qualitative research skills.

Will use as a secondary text in the Qualitative Research Methods course I teach

Preview this book

Sample materials & chapters.

Table of Contents

Michael Quinn Patton

Michael Quinn Patton is author of more than a dozen books on evaluation including Qualitative Research & Evaluation Methods, 4th ed (2015), Blue Marble Evaluation (2020), Principles-Focused Evaluation (2018), Facilitating Evaluation (2018) and Developmental Evaluation (2011). Based in Minnesota, he was on the faculty of the University of Minnesota for 18 years and is a former president of the American Evaluation Association (AEA). Michael is a recipient of the Alva and Gunnar Myrdal Evaluation Practice Award, the Paul F. Lazarsfeld Evaluation Theory Award, and the Research on Evaluation Award, all from AEA He has also received the... More About Author

Purchasing options

Please select a format:

Order from:

• Buy from SAGE

Related Products

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Neurol Res Pract

How to use and assess qualitative research methods

Loraine busetto.

1 Department of Neurology, Heidelberg University Hospital, Im Neuenheimer Feld 400, 69120 Heidelberg, Germany

Wolfgang Wick

2 Clinical Cooperation Unit Neuro-Oncology, German Cancer Research Center, Heidelberg, Germany

Christoph Gumbinger

Associated data.

Not applicable.

This paper aims to provide an overview of the use and assessment of qualitative research methods in the health sciences. Qualitative research can be defined as the study of the nature of phenomena and is especially appropriate for answering questions of why something is (not) observed, assessing complex multi-component interventions, and focussing on intervention improvement. The most common methods of data collection are document study, (non-) participant observations, semi-structured interviews and focus groups. For data analysis, field-notes and audio-recordings are transcribed into protocols and transcripts, and coded using qualitative data management software. Criteria such as checklists, reflexivity, sampling strategies, piloting, co-coding, member-checking and stakeholder involvement can be used to enhance and assess the quality of the research conducted. Using qualitative in addition to quantitative designs will equip us with better tools to address a greater range of research problems, and to fill in blind spots in current neurological research and practice.

The aim of this paper is to provide an overview of qualitative research methods, including hands-on information on how they can be used, reported and assessed. This article is intended for beginning qualitative researchers in the health sciences as well as experienced quantitative researchers who wish to broaden their understanding of qualitative research.

What is qualitative research?

Qualitative research is defined as “the study of the nature of phenomena”, including “their quality, different manifestations, the context in which they appear or the perspectives from which they can be perceived” , but excluding “their range, frequency and place in an objectively determined chain of cause and effect” [ 1 ]. This formal definition can be complemented with a more pragmatic rule of thumb: qualitative research generally includes data in form of words rather than numbers [ 2 ].

Why conduct qualitative research?

Because some research questions cannot be answered using (only) quantitative methods. For example, one Australian study addressed the issue of why patients from Aboriginal communities often present late or not at all to specialist services offered by tertiary care hospitals. Using qualitative interviews with patients and staff, it found one of the most significant access barriers to be transportation problems, including some towns and communities simply not having a bus service to the hospital [ 3 ]. A quantitative study could have measured the number of patients over time or even looked at possible explanatory factors – but only those previously known or suspected to be of relevance. To discover reasons for observed patterns, especially the invisible or surprising ones, qualitative designs are needed.

While qualitative research is common in other fields, it is still relatively underrepresented in health services research. The latter field is more traditionally rooted in the evidence-based-medicine paradigm, as seen in " research that involves testing the effectiveness of various strategies to achieve changes in clinical practice, preferably applying randomised controlled trial study designs (...) " [ 4 ]. This focus on quantitative research and specifically randomised controlled trials (RCT) is visible in the idea of a hierarchy of research evidence which assumes that some research designs are objectively better than others, and that choosing a "lesser" design is only acceptable when the better ones are not practically or ethically feasible [ 5 , 6 ]. Others, however, argue that an objective hierarchy does not exist, and that, instead, the research design and methods should be chosen to fit the specific research question at hand – "questions before methods" [ 2 , 7 – 9 ]. This means that even when an RCT is possible, some research problems require a different design that is better suited to addressing them. Arguing in JAMA, Berwick uses the example of rapid response teams in hospitals, which he describes as " a complex, multicomponent intervention – essentially a process of social change" susceptible to a range of different context factors including leadership or organisation history. According to him, "[in] such complex terrain, the RCT is an impoverished way to learn. Critics who use it as a truth standard in this context are incorrect" [ 8 ] . Instead of limiting oneself to RCTs, Berwick recommends embracing a wider range of methods , including qualitative ones, which for "these specific applications, (...) are not compromises in learning how to improve; they are superior" [ 8 ].

Research problems that can be approached particularly well using qualitative methods include assessing complex multi-component interventions or systems (of change), addressing questions beyond “what works”, towards “what works for whom when, how and why”, and focussing on intervention improvement rather than accreditation [ 7 , 9 – 12 ]. Using qualitative methods can also help shed light on the “softer” side of medical treatment. For example, while quantitative trials can measure the costs and benefits of neuro-oncological treatment in terms of survival rates or adverse effects, qualitative research can help provide a better understanding of patient or caregiver stress, visibility of illness or out-of-pocket expenses.

How to conduct qualitative research?

Given that qualitative research is characterised by flexibility, openness and responsivity to context, the steps of data collection and analysis are not as separate and consecutive as they tend to be in quantitative research [ 13 , 14 ]. As Fossey puts it : “sampling, data collection, analysis and interpretation are related to each other in a cyclical (iterative) manner, rather than following one after another in a stepwise approach” [ 15 ]. The researcher can make educated decisions with regard to the choice of method, how they are implemented, and to which and how many units they are applied [ 13 ]. As shown in Fig.  1 , this can involve several back-and-forth steps between data collection and analysis where new insights and experiences can lead to adaption and expansion of the original plan. Some insights may also necessitate a revision of the research question and/or the research design as a whole. The process ends when saturation is achieved, i.e. when no relevant new information can be found (see also below: sampling and saturation). For reasons of transparency, it is essential for all decisions as well as the underlying reasoning to be well-documented.

Iterative research process

While it is not always explicitly addressed, qualitative methods reflect a different underlying research paradigm than quantitative research (e.g. constructivism or interpretivism as opposed to positivism). The choice of methods can be based on the respective underlying substantive theory or theoretical framework used by the researcher [ 2 ].

Data collection

The methods of qualitative data collection most commonly used in health research are document study, observations, semi-structured interviews and focus groups [ 1 , 14 , 16 , 17 ].

Document study

Document study (also called document analysis) refers to the review by the researcher of written materials [ 14 ]. These can include personal and non-personal documents such as archives, annual reports, guidelines, policy documents, diaries or letters.

Observations

Observations are particularly useful to gain insights into a certain setting and actual behaviour – as opposed to reported behaviour or opinions [ 13 ]. Qualitative observations can be either participant or non-participant in nature. In participant observations, the observer is part of the observed setting, for example a nurse working in an intensive care unit [ 18 ]. In non-participant observations, the observer is “on the outside looking in”, i.e. present in but not part of the situation, trying not to influence the setting by their presence. Observations can be planned (e.g. for 3 h during the day or night shift) or ad hoc (e.g. as soon as a stroke patient arrives at the emergency room). During the observation, the observer takes notes on everything or certain pre-determined parts of what is happening around them, for example focusing on physician-patient interactions or communication between different professional groups. Written notes can be taken during or after the observations, depending on feasibility (which is usually lower during participant observations) and acceptability (e.g. when the observer is perceived to be judging the observed). Afterwards, these field notes are transcribed into observation protocols. If more than one observer was involved, field notes are taken independently, but notes can be consolidated into one protocol after discussions. Advantages of conducting observations include minimising the distance between the researcher and the researched, the potential discovery of topics that the researcher did not realise were relevant and gaining deeper insights into the real-world dimensions of the research problem at hand [ 18 ].

Semi-structured interviews

Hijmans & Kuyper describe qualitative interviews as “an exchange with an informal character, a conversation with a goal” [ 19 ]. Interviews are used to gain insights into a person’s subjective experiences, opinions and motivations – as opposed to facts or behaviours [ 13 ]. Interviews can be distinguished by the degree to which they are structured (i.e. a questionnaire), open (e.g. free conversation or autobiographical interviews) or semi-structured [ 2 , 13 ]. Semi-structured interviews are characterized by open-ended questions and the use of an interview guide (or topic guide/list) in which the broad areas of interest, sometimes including sub-questions, are defined [ 19 ]. The pre-defined topics in the interview guide can be derived from the literature, previous research or a preliminary method of data collection, e.g. document study or observations. The topic list is usually adapted and improved at the start of the data collection process as the interviewer learns more about the field [ 20 ]. Across interviews the focus on the different (blocks of) questions may differ and some questions may be skipped altogether (e.g. if the interviewee is not able or willing to answer the questions or for concerns about the total length of the interview) [ 20 ]. Qualitative interviews are usually not conducted in written format as it impedes on the interactive component of the method [ 20 ]. In comparison to written surveys, qualitative interviews have the advantage of being interactive and allowing for unexpected topics to emerge and to be taken up by the researcher. This can also help overcome a provider or researcher-centred bias often found in written surveys, which by nature, can only measure what is already known or expected to be of relevance to the researcher. Interviews can be audio- or video-taped; but sometimes it is only feasible or acceptable for the interviewer to take written notes [ 14 , 16 , 20 ].

Focus groups

Focus groups are group interviews to explore participants’ expertise and experiences, including explorations of how and why people behave in certain ways [ 1 ]. Focus groups usually consist of 6–8 people and are led by an experienced moderator following a topic guide or “script” [ 21 ]. They can involve an observer who takes note of the non-verbal aspects of the situation, possibly using an observation guide [ 21 ]. Depending on researchers’ and participants’ preferences, the discussions can be audio- or video-taped and transcribed afterwards [ 21 ]. Focus groups are useful for bringing together homogeneous (to a lesser extent heterogeneous) groups of participants with relevant expertise and experience on a given topic on which they can share detailed information [ 21 ]. Focus groups are a relatively easy, fast and inexpensive method to gain access to information on interactions in a given group, i.e. “the sharing and comparing” among participants [ 21 ]. Disadvantages include less control over the process and a lesser extent to which each individual may participate. Moreover, focus group moderators need experience, as do those tasked with the analysis of the resulting data. Focus groups can be less appropriate for discussing sensitive topics that participants might be reluctant to disclose in a group setting [ 13 ]. Moreover, attention must be paid to the emergence of “groupthink” as well as possible power dynamics within the group, e.g. when patients are awed or intimidated by health professionals.

Choosing the “right” method

As explained above, the school of thought underlying qualitative research assumes no objective hierarchy of evidence and methods. This means that each choice of single or combined methods has to be based on the research question that needs to be answered and a critical assessment with regard to whether or to what extent the chosen method can accomplish this – i.e. the “fit” between question and method [ 14 ]. It is necessary for these decisions to be documented when they are being made, and to be critically discussed when reporting methods and results.

Let us assume that our research aim is to examine the (clinical) processes around acute endovascular treatment (EVT), from the patient’s arrival at the emergency room to recanalization, with the aim to identify possible causes for delay and/or other causes for sub-optimal treatment outcome. As a first step, we could conduct a document study of the relevant standard operating procedures (SOPs) for this phase of care – are they up-to-date and in line with current guidelines? Do they contain any mistakes, irregularities or uncertainties that could cause delays or other problems? Regardless of the answers to these questions, the results have to be interpreted based on what they are: a written outline of what care processes in this hospital should look like. If we want to know what they actually look like in practice, we can conduct observations of the processes described in the SOPs. These results can (and should) be analysed in themselves, but also in comparison to the results of the document analysis, especially as regards relevant discrepancies. Do the SOPs outline specific tests for which no equipment can be observed or tasks to be performed by specialized nurses who are not present during the observation? It might also be possible that the written SOP is outdated, but the actual care provided is in line with current best practice. In order to find out why these discrepancies exist, it can be useful to conduct interviews. Are the physicians simply not aware of the SOPs (because their existence is limited to the hospital’s intranet) or do they actively disagree with them or does the infrastructure make it impossible to provide the care as described? Another rationale for adding interviews is that some situations (or all of their possible variations for different patient groups or the day, night or weekend shift) cannot practically or ethically be observed. In this case, it is possible to ask those involved to report on their actions – being aware that this is not the same as the actual observation. A senior physician’s or hospital manager’s description of certain situations might differ from a nurse’s or junior physician’s one, maybe because they intentionally misrepresent facts or maybe because different aspects of the process are visible or important to them. In some cases, it can also be relevant to consider to whom the interviewee is disclosing this information – someone they trust, someone they are otherwise not connected to, or someone they suspect or are aware of being in a potentially “dangerous” power relationship to them. Lastly, a focus group could be conducted with representatives of the relevant professional groups to explore how and why exactly they provide care around EVT. The discussion might reveal discrepancies (between SOPs and actual care or between different physicians) and motivations to the researchers as well as to the focus group members that they might not have been aware of themselves. For the focus group to deliver relevant information, attention has to be paid to its composition and conduct, for example, to make sure that all participants feel safe to disclose sensitive or potentially problematic information or that the discussion is not dominated by (senior) physicians only. The resulting combination of data collection methods is shown in Fig.  2 .

Possible combination of data collection methods

Attributions for icons: “Book” by Serhii Smirnov, “Interview” by Adrien Coquet, FR, “Magnifying Glass” by anggun, ID, “Business communication” by Vectors Market; all from the Noun Project

The combination of multiple data source as described for this example can be referred to as “triangulation”, in which multiple measurements are carried out from different angles to achieve a more comprehensive understanding of the phenomenon under study [ 22 , 23 ].

Data analysis

To analyse the data collected through observations, interviews and focus groups these need to be transcribed into protocols and transcripts (see Fig.  3 ). Interviews and focus groups can be transcribed verbatim , with or without annotations for behaviour (e.g. laughing, crying, pausing) and with or without phonetic transcription of dialects and filler words, depending on what is expected or known to be relevant for the analysis. In the next step, the protocols and transcripts are coded , that is, marked (or tagged, labelled) with one or more short descriptors of the content of a sentence or paragraph [ 2 , 15 , 23 ]. Jansen describes coding as “connecting the raw data with “theoretical” terms” [ 20 ]. In a more practical sense, coding makes raw data sortable. This makes it possible to extract and examine all segments describing, say, a tele-neurology consultation from multiple data sources (e.g. SOPs, emergency room observations, staff and patient interview). In a process of synthesis and abstraction, the codes are then grouped, summarised and/or categorised [ 15 , 20 ]. The end product of the coding or analysis process is a descriptive theory of the behavioural pattern under investigation [ 20 ]. The coding process is performed using qualitative data management software, the most common ones being InVivo, MaxQDA and Atlas.ti. It should be noted that these are data management tools which support the analysis performed by the researcher(s) [ 14 ].

From data collection to data analysis

Attributions for icons: see Fig. ​ Fig.2, 2 , also “Speech to text” by Trevor Dsouza, “Field Notes” by Mike O’Brien, US, “Voice Record” by ProSymbols, US, “Inspection” by Made, AU, and “Cloud” by Graphic Tigers; all from the Noun Project

How to report qualitative research?

Protocols of qualitative research can be published separately and in advance of the study results. However, the aim is not the same as in RCT protocols, i.e. to pre-define and set in stone the research questions and primary or secondary endpoints. Rather, it is a way to describe the research methods in detail, which might not be possible in the results paper given journals’ word limits. Qualitative research papers are usually longer than their quantitative counterparts to allow for deep understanding and so-called “thick description”. In the methods section, the focus is on transparency of the methods used, including why, how and by whom they were implemented in the specific study setting, so as to enable a discussion of whether and how this may have influenced data collection, analysis and interpretation. The results section usually starts with a paragraph outlining the main findings, followed by more detailed descriptions of, for example, the commonalities, discrepancies or exceptions per category [ 20 ]. Here it is important to support main findings by relevant quotations, which may add information, context, emphasis or real-life examples [ 20 , 23 ]. It is subject to debate in the field whether it is relevant to state the exact number or percentage of respondents supporting a certain statement (e.g. “Five interviewees expressed negative feelings towards XYZ”) [ 21 ].

How to combine qualitative with quantitative research?

Qualitative methods can be combined with other methods in multi- or mixed methods designs, which “[employ] two or more different methods [ …] within the same study or research program rather than confining the research to one single method” [ 24 ]. Reasons for combining methods can be diverse, including triangulation for corroboration of findings, complementarity for illustration and clarification of results, expansion to extend the breadth and range of the study, explanation of (unexpected) results generated with one method with the help of another, or offsetting the weakness of one method with the strength of another [ 1 , 17 , 24 – 26 ]. The resulting designs can be classified according to when, why and how the different quantitative and/or qualitative data strands are combined. The three most common types of mixed method designs are the convergent parallel design , the explanatory sequential design and the exploratory sequential design. The designs with examples are shown in Fig.  4 .

Three common mixed methods designs

In the convergent parallel design, a qualitative study is conducted in parallel to and independently of a quantitative study, and the results of both studies are compared and combined at the stage of interpretation of results. Using the above example of EVT provision, this could entail setting up a quantitative EVT registry to measure process times and patient outcomes in parallel to conducting the qualitative research outlined above, and then comparing results. Amongst other things, this would make it possible to assess whether interview respondents’ subjective impressions of patients receiving good care match modified Rankin Scores at follow-up, or whether observed delays in care provision are exceptions or the rule when compared to door-to-needle times as documented in the registry. In the explanatory sequential design, a quantitative study is carried out first, followed by a qualitative study to help explain the results from the quantitative study. This would be an appropriate design if the registry alone had revealed relevant delays in door-to-needle times and the qualitative study would be used to understand where and why these occurred, and how they could be improved. In the exploratory design, the qualitative study is carried out first and its results help informing and building the quantitative study in the next step [ 26 ]. If the qualitative study around EVT provision had shown a high level of dissatisfaction among the staff members involved, a quantitative questionnaire investigating staff satisfaction could be set up in the next step, informed by the qualitative study on which topics dissatisfaction had been expressed. Amongst other things, the questionnaire design would make it possible to widen the reach of the research to more respondents from different (types of) hospitals, regions, countries or settings, and to conduct sub-group analyses for different professional groups.

How to assess qualitative research?

A variety of assessment criteria and lists have been developed for qualitative research, ranging in their focus and comprehensiveness [ 14 , 17 , 27 ]. However, none of these has been elevated to the “gold standard” in the field. In the following, we therefore focus on a set of commonly used assessment criteria that, from a practical standpoint, a researcher can look for when assessing a qualitative research report or paper.

Assessors should check the authors’ use of and adherence to the relevant reporting checklists (e.g. Standards for Reporting Qualitative Research (SRQR)) to make sure all items that are relevant for this type of research are addressed [ 23 , 28 ]. Discussions of quantitative measures in addition to or instead of these qualitative measures can be a sign of lower quality of the research (paper). Providing and adhering to a checklist for qualitative research contributes to an important quality criterion for qualitative research, namely transparency [ 15 , 17 , 23 ].

Reflexivity

While methodological transparency and complete reporting is relevant for all types of research, some additional criteria must be taken into account for qualitative research. This includes what is called reflexivity, i.e. sensitivity to the relationship between the researcher and the researched, including how contact was established and maintained, or the background and experience of the researcher(s) involved in data collection and analysis. Depending on the research question and population to be researched this can be limited to professional experience, but it may also include gender, age or ethnicity [ 17 , 27 ]. These details are relevant because in qualitative research, as opposed to quantitative research, the researcher as a person cannot be isolated from the research process [ 23 ]. It may influence the conversation when an interviewed patient speaks to an interviewer who is a physician, or when an interviewee is asked to discuss a gynaecological procedure with a male interviewer, and therefore the reader must be made aware of these details [ 19 ].

Sampling and saturation

The aim of qualitative sampling is for all variants of the objects of observation that are deemed relevant for the study to be present in the sample “ to see the issue and its meanings from as many angles as possible” [ 1 , 16 , 19 , 20 , 27 ] , and to ensure “information-richness [ 15 ]. An iterative sampling approach is advised, in which data collection (e.g. five interviews) is followed by data analysis, followed by more data collection to find variants that are lacking in the current sample. This process continues until no new (relevant) information can be found and further sampling becomes redundant – which is called saturation [ 1 , 15 ] . In other words: qualitative data collection finds its end point not a priori , but when the research team determines that saturation has been reached [ 29 , 30 ].

This is also the reason why most qualitative studies use deliberate instead of random sampling strategies. This is generally referred to as “ purposive sampling” , in which researchers pre-define which types of participants or cases they need to include so as to cover all variations that are expected to be of relevance, based on the literature, previous experience or theory (i.e. theoretical sampling) [ 14 , 20 ]. Other types of purposive sampling include (but are not limited to) maximum variation sampling, critical case sampling or extreme or deviant case sampling [ 2 ]. In the above EVT example, a purposive sample could include all relevant professional groups and/or all relevant stakeholders (patients, relatives) and/or all relevant times of observation (day, night and weekend shift).

Assessors of qualitative research should check whether the considerations underlying the sampling strategy were sound and whether or how researchers tried to adapt and improve their strategies in stepwise or cyclical approaches between data collection and analysis to achieve saturation [ 14 ].

Good qualitative research is iterative in nature, i.e. it goes back and forth between data collection and analysis, revising and improving the approach where necessary. One example of this are pilot interviews, where different aspects of the interview (especially the interview guide, but also, for example, the site of the interview or whether the interview can be audio-recorded) are tested with a small number of respondents, evaluated and revised [ 19 ]. In doing so, the interviewer learns which wording or types of questions work best, or which is the best length of an interview with patients who have trouble concentrating for an extended time. Of course, the same reasoning applies to observations or focus groups which can also be piloted.

Ideally, coding should be performed by at least two researchers, especially at the beginning of the coding process when a common approach must be defined, including the establishment of a useful coding list (or tree), and when a common meaning of individual codes must be established [ 23 ]. An initial sub-set or all transcripts can be coded independently by the coders and then compared and consolidated after regular discussions in the research team. This is to make sure that codes are applied consistently to the research data.

Member checking

Member checking, also called respondent validation , refers to the practice of checking back with study respondents to see if the research is in line with their views [ 14 , 27 ]. This can happen after data collection or analysis or when first results are available [ 23 ]. For example, interviewees can be provided with (summaries of) their transcripts and asked whether they believe this to be a complete representation of their views or whether they would like to clarify or elaborate on their responses [ 17 ]. Respondents’ feedback on these issues then becomes part of the data collection and analysis [ 27 ].

Stakeholder involvement

In those niches where qualitative approaches have been able to evolve and grow, a new trend has seen the inclusion of patients and their representatives not only as study participants (i.e. “members”, see above) but as consultants to and active participants in the broader research process [ 31 – 33 ]. The underlying assumption is that patients and other stakeholders hold unique perspectives and experiences that add value beyond their own single story, making the research more relevant and beneficial to researchers, study participants and (future) patients alike [ 34 , 35 ]. Using the example of patients on or nearing dialysis, a recent scoping review found that 80% of clinical research did not address the top 10 research priorities identified by patients and caregivers [ 32 , 36 ]. In this sense, the involvement of the relevant stakeholders, especially patients and relatives, is increasingly being seen as a quality indicator in and of itself.

How not to assess qualitative research

The above overview does not include certain items that are routine in assessments of quantitative research. What follows is a non-exhaustive, non-representative, experience-based list of the quantitative criteria often applied to the assessment of qualitative research, as well as an explanation of the limited usefulness of these endeavours.

Protocol adherence

Given the openness and flexibility of qualitative research, it should not be assessed by how well it adheres to pre-determined and fixed strategies – in other words: its rigidity. Instead, the assessor should look for signs of adaptation and refinement based on lessons learned from earlier steps in the research process.

Sample size

For the reasons explained above, qualitative research does not require specific sample sizes, nor does it require that the sample size be determined a priori [ 1 , 14 , 27 , 37 – 39 ]. Sample size can only be a useful quality indicator when related to the research purpose, the chosen methodology and the composition of the sample, i.e. who was included and why.

Randomisation

While some authors argue that randomisation can be used in qualitative research, this is not commonly the case, as neither its feasibility nor its necessity or usefulness has been convincingly established for qualitative research [ 13 , 27 ]. Relevant disadvantages include the negative impact of a too large sample size as well as the possibility (or probability) of selecting “ quiet, uncooperative or inarticulate individuals ” [ 17 ]. Qualitative studies do not use control groups, either.

Interrater reliability, variability and other “objectivity checks”

The concept of “interrater reliability” is sometimes used in qualitative research to assess to which extent the coding approach overlaps between the two co-coders. However, it is not clear what this measure tells us about the quality of the analysis [ 23 ]. This means that these scores can be included in qualitative research reports, preferably with some additional information on what the score means for the analysis, but it is not a requirement. Relatedly, it is not relevant for the quality or “objectivity” of qualitative research to separate those who recruited the study participants and collected and analysed the data. Experiences even show that it might be better to have the same person or team perform all of these tasks [ 20 ]. First, when researchers introduce themselves during recruitment this can enhance trust when the interview takes place days or weeks later with the same researcher. Second, when the audio-recording is transcribed for analysis, the researcher conducting the interviews will usually remember the interviewee and the specific interview situation during data analysis. This might be helpful in providing additional context information for interpretation of data, e.g. on whether something might have been meant as a joke [ 18 ].

Not being quantitative research

Being qualitative research instead of quantitative research should not be used as an assessment criterion if it is used irrespectively of the research problem at hand. Similarly, qualitative research should not be required to be combined with quantitative research per se – unless mixed methods research is judged as inherently better than single-method research. In this case, the same criterion should be applied for quantitative studies without a qualitative component.

The main take-away points of this paper are summarised in Table ​ Table1. 1 . We aimed to show that, if conducted well, qualitative research can answer specific research questions that cannot to be adequately answered using (only) quantitative designs. Seeing qualitative and quantitative methods as equal will help us become more aware and critical of the “fit” between the research problem and our chosen methods: I can conduct an RCT to determine the reasons for transportation delays of acute stroke patients – but should I? It also provides us with a greater range of tools to tackle a greater range of research problems more appropriately and successfully, filling in the blind spots on one half of the methodological spectrum to better address the whole complexity of neurological research and practice.

Take-away-points

Acknowledgements

Abbreviations, authors’ contributions.

LB drafted the manuscript; WW and CG revised the manuscript; all authors approved the final versions.

no external funding.

Availability of data and materials

Ethics approval and consent to participate, consent for publication, competing interests.

The authors declare no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

We will keep fighting for all libraries - stand with us!

Internet Archive Audio

• This Just In
• Grateful Dead
• Old Time Radio
• 78 RPMs and Cylinder Recordings
• Audio Books & Poetry
• Computers, Technology and Science
• Music, Arts & Culture
• News & Public Affairs
• Spirituality & Religion
• Radio News Archive

• Flickr Commons
• Occupy Wall Street Flickr
• NASA Images
• Solar System Collection
• Ames Research Center

• All Software
• Old School Emulation
• MS-DOS Games
• Historical Software
• Classic PC Games
• Software Library
• Kodi Archive and Support File
• Vintage Software
• CD-ROM Software
• CD-ROM Software Library
• Software Sites
• Tucows Software Library
• Shareware CD-ROMs
• Software Capsules Compilation
• CD-ROM Images
• ZX Spectrum
• DOOM Level CD

• Smithsonian Libraries
• FEDLINK (US)
• Lincoln Collection
• American Libraries
• Canadian Libraries
• Universal Library
• Project Gutenberg
• Children's Library
• Biodiversity Heritage Library
• Books by Language
• Additional Collections

• Prelinger Archives
• Democracy Now!
• Occupy Wall Street
• TV NSA Clip Library
• Animation & Cartoons
• Arts & Music
• Computers & Technology
• Cultural & Academic Films
• Ephemeral Films
• Sports Videos
• Videogame Videos
• Youth Media

Search the history of over 866 billion web pages on the Internet.

Mobile Apps

• Wayback Machine (iOS)
• Wayback Machine (Android)

Browser Extensions

Archive-it subscription.

• Explore the Collections
• Build Collections

Save Page Now

Capture a web page as it appears now for use as a trusted citation in the future.

Please enter a valid web address

• Donate Donate icon An illustration of a heart shape

Qualitative research and evaluation methods

Bookreader item preview, share or embed this item, flag this item for.

• Graphic Violence
• Explicit Sexual Content
• Hate Speech
• Misinformation/Disinformation
• Marketing/Phishing/Advertising
• Misleading/Inaccurate/Missing Metadata

plus-circle Add Review comment Reviews

3 Favorites

Better World Books

DOWNLOAD OPTIONS

No suitable files to display here.

IN COLLECTIONS

Uploaded by station26.cebu on March 2, 2021

SIMILAR ITEMS (based on metadata)

An Introduction to Qualitative Evaluation Approaches

• First Online: 10 February 2022

Cite this chapter

• Charles P. Friedman 4 ,
• Jeremy C. Wyatt 5 &
• Joan S. Ash 6  

Part of the book series: Health Informatics ((HI))

1431 Accesses

This chapter is the first of three devoted to describing qualitative methods for evaluating informatics interventions. Its major goals are to describe when qualitative methods are appropriate and to offer a general framework for understanding how studies using these methods are conducted. Chapters 15 and 16 provide much more detailed tours through the methods of qualitative evaluation.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

• Available as PDF
• Read on any device
• Instant download
• Own it forever
• Available as EPUB and PDF
• Compact, lightweight edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info
• Durable hardcover edition

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Berg BL, Lune H. Qualitative research methods for social scientists. 8th ed. Pearson: Boston, MA; 2012.

Google Scholar  

Crabtree BF, Miller WL. Doing qualitative research. 2nd ed. Thousand Oaks, CA: Sage; 1999.

Cresswell JW. Research design: qualitative, quantitative, and mixed methods approaches. Thousand Oaks, CA: Sage; 2019.

Davis WS. Business systems design and analysis. Belmont, CA: Wadsworth; 1994.

Forsythe DE. Using ethnography to build a working system: rethinking basic design assumptions. Symp Comput Applications Med Care. 1992;16:505–9.

Forsythe DE. New bottles, old wine: hidden cultural assumptions in a computerized explanation system for migraine sufferers. Med Anthropol Q. 1996;10:551–74.

Article   CAS   Google Scholar  

Forsythe DE. Using ethnography to investigate life scientists’ information needs. Bull Med Libr Assoc. 1998;86(3):402–9.

Goedhart NS, Zuiderent-Jerak T, Woudstra J, Broerse JEW, Betten AW, Dedding C. Persistent inequitable design and implementation of patient portals for users at the margins. J Am Med Inform Assoc. 2021;28(2):276–83.

Article   Google Scholar  

House ER. Evaluating with validity. Beverly Hills: Sage; 1980.

J Am Med Inform Assoc 2021;28(2):197–423. (special issue).

Kaplan B. Initial impact of a clinical laboratory computer system. J Med Syst. 1987;11:137–47.

Kaplan B, Duchon D. Combining qualitative and quantitative methods in information systems research: a case study. MIS Q. 1988;12:571–86.

Keikhosrokiani P. Perspectives in the development of mobile medical information systems: life cycle, management, methodological approach and application. Amsterdam: Elsevier; 2019.

Norman DA. The design of everyday things. New York: Basic Books; 1996.

Sittig DF, Singh H. A new sociotechnical model for studying health information technology in complex adaptive healthcare systems. Qual Saf Heath Care. 2010;19(suppl 3):i68–74.

Strauss A, Corbin J. Basics of qualitative research: grounded theory procedures and techniques. Newbury Park, CA: Sage; 1990.

Toye F. Not everything that can be counted counts, and not everything that counts can be counted. Br J Pain. 2015;9:7.

University of Technology Sydney. Adapting research methods in the COVID-19 pandemic: resources for researchers, 2nd ed. December, 2020.

Winograd T, Flores F. Understanding computers and cognition: a new foundation for design. Reading, MA: Addison-Wesley; 1987.

Download references

Author information

Authors and affiliations.

Department of Learning Health Sciences, University of Michigan Medical School, Ann Arbor, MI, USA

Charles P. Friedman

Department of Primary Care, Population Sciences and Medical Education, School of Medicine, University of Southampton, Southampton, UK

Jeremy C. Wyatt

Department of Medical Informatics and Clinical Epidemiology, School of Medicine, Oregon Health & Science University, Portland, OR, USA

Joan S. Ash

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Charles P. Friedman .

Answers to Self-Tests

Self-test 14.1.

Qualitative methods can be especially useful at the beginning of a project because they can yield information in the form of a needs assessment. Later, after implementation, they can be extremely valuable if used to provide feedback for improving the information resource.

Systems analysis processes attempt to be sensitive to the context into which the system will be placed and they are also iterative like qualitative processes.

These methods are inductive and interpretive because they require the study team to be open minded, to learn from participants, and to offer explanations and describe the meaning of cultural attributes.

Rights and permissions

Reprints and permissions

Copyright information

© 2022 Springer Nature Switzerland AG

About this chapter

Friedman, C.P., Wyatt, J.C., Ash, J.S. (2022). An Introduction to Qualitative Evaluation Approaches. In: Evaluation Methods in Biomedical and Health Informatics. Health Informatics. Springer, Cham. https://doi.org/10.1007/978-3-030-86453-8_14

Download citation

DOI : https://doi.org/10.1007/978-3-030-86453-8_14

Published : 10 February 2022

Publisher Name : Springer, Cham

Print ISBN : 978-3-030-86452-1

Online ISBN : 978-3-030-86453-8

eBook Packages : Medicine Medicine (R0)

The design and evaluation of gamified online role-play as a telehealth training strategy in dental education: an explanatory sequential mixed-methods study

• Chayanid Teerawongpairoj 1 ,
• Chanita Tantipoj 1 &
• Kawin Sipiyaruk 2  

Scientific Reports volume  14 , Article number:  9216 ( 2024 ) Cite this article

103 Accesses

Metrics details

• Health care
• Health services
• Public health

To evaluate user perceptions and educational impact of gamified online role-play in teledentistry as well as to construct a conceptual framework highlighting how to design this interactive learning strategy, this research employed an explanatory sequential mixed-methods design. Participants were requested to complete self-perceived assessments toward confidence and awareness in teledentistry before and after participating in a gamified online role-play. They were also asked to complete a satisfaction questionnaire and participate in an in-depth interview to investigate their learning experience. The data were analyzed using descriptive statistics, paired sample t-test, one-way analysis of variance, and framework analysis. There were 18 participants who completed self-perceived assessments and satisfaction questionnaire, in which 12 of them participated in a semi-structured interview. There were statistically significant increases in self-perceived confidence and awareness after participating in the gamified online role-play ( P  < 0.001). In addition, the participants were likely to be satisfied with this learning strategy, where usefulness was perceived as the most positive aspect with a score of 4.44 out of 5, followed by ease of use (4.40) and enjoyment (4.03). The conceptual framework constructed from the qualitative findings has revealed five key elements in designing a gamified online role-play, including learner profile, learning settings, pedagogical components, interactive functions, and educational impact. The gamified online role-play has demonstrated its potential in improving self-perceived confidence and awareness in teledentistry. The conceptual framework developed in this research could be considered to design and implement a gamified online role-play in dental education. This research provides valuable evidence on the educational impact of gamified online role-play in teledentistry and how it could be designed and implemented in dental education. This information would be supportive for dental instructors or educators who are considering to implement teledentistry training in their practice.

Similar content being viewed by others

Games in dental education: playing to learn or learning to play?

A bibliometric analysis of the use of the Gamification Octalysis Framework in training: evidence from Web of Science

Helping general dental practitioners use the Index of Orthodontic Treatment Need: an assessment of available educational apps

Introduction.

Telehealth has gained significant attention from various organization due to its potential to improve healthcare quality and accessibility 1 . It can be supportive in several aspects in healthcare, including medical and nursing services, to enhance continuous monitoring and follow-up 2 . Its adoption has increased substantially during the COVID-19 pandemic, aiming to provide convenient healthcare services 3 . Even though the COVID-19 outbreak has passed, many patients still perceive telehealth as an effective tool in reducing a number of visits and enhancing access to health care services 4 , 5 . This supports the use of telehealth in the post-COVID-19 era.

Teledentistry, a form of telehealth specific to dentistry, has been employed to improve access to dental services 6 . This system offers benefits ranging from online history taking, oral diagnosis, treatment monitoring, and interdisciplinary communication among dental professionals, enabling comprehensive and holistic treatment planning for patients 7 . Teledentistry can also reduce travel time and costs associated with dental appointments 8 , 9 , 10 . There is evidence that teledentistry serves as a valuable tool to enhance access to dental care for patients 11 . Additionally, in the context of long-term management in patients, telehealth has contributed to patient-centered care, by enhancing their surrounding environments 12 . Therefore, teledentistry should be emphasized as one of digital dentistry to enhance treatment quality.

Albeit the benefits of teledentistry, available evidence demonstrates challenges and concerns in the implementation of telehealth. Lack of awareness and knowledge in the use of telehealth can hinder the adoption of telehealth 13 . Legal issues and privacy concerns also emerge as significant challenges in telehealth use 14 . Moreover, online communication skills and technology literacy, including competency in using technological tools and applications, have been frequently reported as challenges in teledentistry 15 , 16 . Concerns regarding limitations stemming from the lack of physical examination are also significant 17 . These challenges and complexities may impact the accuracy of diagnosis and the security and confidentiality of patient information. Therefore, telehealth training for dental professionals emerges as essential prerequisites to effectively navigate the use of teledentistry, fostering confidence and competence in remote oral healthcare delivery.

The feasibility and practicality of telehealth in dental education present ongoing challenges and concerns. Given the limitations of teledentistry compared to face-to-face appointments, areas of training should encompass the telehealth system, online communication, technical issues, confidentiality concerns, and legal compliance 18 . However, there is currently no educational strategy that effectively demonstrates the importance and application of teledentistry 19 . A role-play can be considered as a teaching strategy where learners play a role that closely resembles real-life scenarios. A well-organized storytelling allows learner to manage problematic situations, leading to the development of problem-solving skill 20 , 21 . When compared to traditional lecture-based learning, learners can also enhance their communication skills through conversations with simulated patients 22 , 23 . In addition, they could express their thoughts and emotions during a role-play through experiential learning 20 , 24 , 25 . Role-play through video teleconference would be considered as a distance learning tool for training dental professionals to effectively use teledentistry.

While there have been studies supporting online role-play as an effective learning tool due to its impact of flexibility, engagement, and anonymity 26 , 27 , no evidence has been yet reported whether or not this learning strategy could have potential for training teledentistry. Given the complicated issues in telehealth, role-play for training teledentistry should incorporate different learning aspects compared to face-to-face communication with patients. In addition, game components have proved to be supportive in dental education 28 , 29 . Consequently, this research aimed to evaluate user perceptions and educational impact of gamified online role-play to enhance learner competence and awareness in using teledentistry as well as to construct a conceptual framework highlighting how to design and implement this interactive learning strategy. This research would introduce and promote the design and implementation of gamified online role-play as a learning tool for training teledentistry. To achieve the aim, specific objectives were established as follows:

1. To design a gamified online role-play for teledentistry training.

2. To investigate learner perceptions regarding their confidence and awareness in the use of teledentistry after completing the gamified online role-play.

3. To explore user satisfactions toward the use of gamified online role-play.

4. To develop a conceptual framework for designing and implementing a gamified online role-play for teledentistry training.

Materials and methods

Research design.

This research employed an explanatory sequential mixed-methods design, where a quantitative phase was firstly performed followed by a qualitative phase 30 , 31 . The quantitative phase was conducted based on pre-experimental research using one-group pretest–posttest design. Participants were requested to complete self-perceived assessments toward confidence and awareness in the use of teledentistry before and after participating in a gamified online role-play. They were also asked to complete a satisfaction questionnaire in using a gamified online role-play for training teledentistry. The qualitative phase was afterwards conducted to explore in-depth information through semi-structured interviews, in order to enhance an understanding of the quantitative phase, and to develop a conceptual framework for designing and implementing an online role-play for training teledentistry.

A gamified online role-play for training teledentistry

A gamified online role-play was designed and developed by the author team. To ensure its educational impact was significant, the expected learning outcomes were formulated based on insights gathered from a survey with experienced instructors from the Department of Advanced General Dentistry, Faculty of Dentistry, Mahidol University. These learning outcomes covered areas of online communication skill, technical issues, technology literacy of patients, limitations of physical examination, and privacy concerns of personal information. Learning scenario and instructional content were subsequently designed to support learners in achieving the expected learning outcomes, with their alignments validated by three experts in dental education. A professional actress underwent training to role-play a patient with a dental problem, requesting a virtual consultation or teledentistry. Before conducting data collection, the simulated patient was required to undergo a training and adjusting process with a pilot group under supervision of two experts in advanced general dentistry and dental education who had experience with teledentistry to ensure realism and completeness of learning content.

According to the role-play scenario, an actress was assigned to portray a 34-year-old female with chief complaints of pain around both ears, accompanied by difficulties in chewing food due to tooth loss. She was instructed to express her anxiety and nervousness about addressing these issues. Additionally, it was specified that she could not take a day off from work during this period. Despite this constraint, she required a dental consultation to receive advice for initial self-care, as her symptoms significantly impacted her daily life. Furthermore, she was designated to encounter difficulties with the technological use of the teledentistry platform.

The game components were implemented into the online role-play to enhance motivation and engagement. As challenge and randomness appear to be game elements 32 , 33 , five challenge cards were designed and embedded into the online role-play, where a participant was asked to randomly select one of them before interacting with the simulated patient. The challenging situations were potential technical concerns which could occur frequently during video conferencing, including network problems (e.g., internet disconnection and poor connection) and audiovisual quality issues. The participants were blinded to the selected card, while it was revealed to only the simulated patient. The challenging conditions were mimicked by the organizers and simulated patient, allowing learners to deal with difficulties. Therefore, both challenges and randomness were implemented into this learning intervention not only to create learning situations but also to enhance engagement.

A feedback system was carefully considered and implemented into the gamified online role-play. Immediate feedback appears to be a key feature of interactive learning environments 29 . Formative feedback was instantly delivered to learners through verbal and non-verbal communication, including words (content), tone of voice, facial expressions, and gestures of the simulated patient. This type of feedback allowed participants to reflect on whether or not their inputs were appropriate, enabling them to learn from their mistakes, or so-called the role of failure 34 . Summative feedback was also provided at the end of the role-play through a reflection from a simulated patient and suggestions from an instructor.

Learners were able to interact with the simulated patient using an online meeting room by Cisco WebEx. According to the research setting (Fig.  1 ), a learner was asked to participate in the role-play activity using a computer laptop in a soundproof room, while a simulated patient was arranged in a prepared location showing her residential environment. The researcher and instructor also joined the online meeting room and observed the interaction between the simulated patient and learners during the role-play activity whether or not all necessary information was accurately obtained. The role-play activity took around 30 minutes.

A diagram demonstrating the setting of gamified online role-play.

Research participants

Quantitative phase.

The participants in this research were postgraduate students from the Residency Training Program in Advanced General Dentistry at Mahidol University Faculty of Dentistry in academic year 2022, using a volunteer sampling. This program was selected because its objective was to develop graduates capable of integrating competencies from various dental disciplines to provide comprehensive dental care for both normal patients and those with special needs. Therefore, teledentistry should be a supportive component of their service. The recruitment procedure involved posting a recruiting text in the group chat of the residents. Those interested in participating in the research were informed to directly contact us to request more information, and they were subsequently allowed to decide whether they would like to participate. This approach ensured that participation was voluntary. Although there could be a non-response bias within this non-probability sampling technique 35 , it was considered as appropriate for this study, as participants were willing to have contribution in the learning activity, and therefore accurate and reliable research findings with no dropout could be achieved 36 .

The inclusion and exclusion criteria were established to determine the eligibility of prospective participants for this research. This study included postgraduate students from Years 1 to 3 in the Residency Training Program in Advanced General Dentistry at Mahidol University Faculty of Dentistry, enrolled during the academic year 2022. They were also required to at least complete the first semester to be eligible for this research to ensure familiarity with comprehensive dental care. However, they were excluded if they had previous involvement in the pilot testing of the gamified online role-play or if they were not fluent in the Thai language. The sample size was determined using a formula for two dependent samples (comparing means) 37 . To detect a difference in self-perceived confidence and awareness between pre- and post-assessments at a power of 90% and a level of statistical significance of 1%, five participants were required. With an assumed dropout rate of 20%, the number of residents per year (Year 1–3) was set to be 6. Therefore, 18 residents were required for this research.

Qualitative phase

The participants from the quantitative phase were selected for semi-structured interviews using a purposive sampling. This sampling method involved the selection of information-rich participants based on specific criteria deemed relevant to the research objective and to ensure a diverse representation of perspectives and experiences within the sample group 38 . In this research, the information considered for the purposive sampling included demographic data (e.g., sex and year of study), along with self-perceived assessment scores. By incorporating perceptions from a variety of participants, a broad spectrum of insights from different experiences in comprehensive dental practice and diverse improvement levels in self-perceived confidence and awareness could inform the design and implementation of the training program effectively. The sample size for this phase was determined based on data saturation, wherein interviews continued until no new information or emerging themes were retrieved. This method ensured thorough exploration of the research topic and maximized the richness of the qualitative data obtained.

Outcome assessments

To evaluate the gamified online role-play, a triangular design approach was employed, enabling the researchers to compare the research outcomes from different assessment methods. In this research, self-perceived assessments (confidence and awareness) in teledentistry, satisfactions toward gamified online role-play, and learner experience were assessed to assure the quality and feasibility of the gamified online role-play.

Self-perceived confidence and awareness toward teledentistry

All participants were requested to rate their perceptions of teledentistry before and after participating in the gamified online role-play (Supplementary material 1 ). The self-perceived assessment was developed based on previous literature 39 , 40 , 41 , 42 . The assessment scores would inform whether or not the participants could improve their self-perceived confidence and awareness through a learning activity. The assessment consisted of two parts, which were (1) self-perceived confidence and (2) self-perceived awareness. Each part contained six items, which were similar between the pre- and post-assessments. All items were designed using a 5-point Likert scale, where 1 being ‘strongly disagree’ and 5 being ‘strongly agree’.

Satisfactions toward the gamified online role-play

All participants were asked to complete the satisfaction questionnaire after participating in the gamified online role-play, to investigate whether or not they felt satisfied with their learning (Supplementary material 2 ). The questionnaire was developed based on previous literature regarding gamification and role-play 41 , 42 , 43 , 44 . Most of the items were designed using a 5-point Likert scale, where 1 being ‘very dissatisfied’ and 5 being ‘very satisfied’. They were grouped into three aspects, which were (1) Perceived usefulness, (2) Perceived ease of use, and (3) Perceived enjoyment.

Learner experiences within the gamified online role-play

Semi-structured interviews were conducted with the purposively selected participants to gather in-depth information regarding their learning experiences within the gamified online role-play. This technique allowed researchers to ask additional interesting topics raised from the responses of participants. A topic guide for interviews were constructed based on the findings of previous literature 45 , 46 , 47 . The interview was conducted in a private room by a researcher who was trained in conducting qualitative research including interviews. The interview sessions took approximately 45–60 minutes, where all responses from participants were recorded using a digital audio recorder with their permission. The recorded audios were transcribed using a verbatim technique by a transcription service under a confidential agreement.

Validity and reliability of data collection tools

To enhance the quality of self-perceived assessment and satisfaction questionnaire, they were piloted and revised to assure their validity and reliability. According to the content validity, three experts in advanced general dentistry were asked to evaluate the questionnaire, where problematic items were iteratively revised until they achieved the index of item-objective congruence (IOC) higher than 0.5. To perform a test–retest reliability, the validated versions of both self-perceived assessment and satisfaction questionnaire were afterwards piloted in residents from other programs, and the data were analyzed using an intraclass correlation coefficient (ICC), where the values of all items were 0.7 or greater. The data from the first pilot completion of both data collection tools were analyzed using Cronbach’s alpha to ensure the internal consistency of all constructs. The problematic items were deleted to achieve the coefficient alpha of 0.7 or greater for all constructs, which was considered as acceptable internal consistency.

Data analysis

The quantitative data retrieved from self-perceived assessment and satisfaction questionnaire were analyzed with the Statistical Package for Social Sciences software (SPSS, version 29, IBM Corp.). Descriptive statistics were performed to present an overview of the data. The scores from pre- and post-assessments were analyzed using a paired sample t-test to evaluate whether or not the participants would better self-perceive their confidence and awareness in teledentistry after participating in the gamified online role-play. One-way analysis of variance (ANOVA) was conducted to compare whether or not there were statistically significant differences in self-perceived assessment and satisfaction scores among the three academic years.

The qualitative data retrieved from semi-structured interviews were analyzed using a framework analysis, where its procedure involved transcription, familiarization with the interview data, coding, developing an analytical framework, indexing, charting, and data interpreting qualitative findings 48 . In this research, the initial codes had been pre-defined from previous literature and subsequently adjusted following the analysis of each transcript to develop an analytical framework (themes and subthemes), requiring several iterations until no additional codes emerged. Subsequently, the established categories and codes were applied consistently across all transcripts (indexing). The data from each transcript were then charted to develop a matrix, facilitating the management and summarization of qualitative findings. This method enabled the researchers to compare and contrast differences within the data and to identify connections between categories, thereby exploring their relationships and informing data interpretation.

The procedure of framework analysis necessitated a transparent process for data management and interpretation of emerging themes to ensure the robustness of research 49 . The transparency of this analytic approach enabled two researchers (C.Te. and K.S.) to independently analyze the qualitative data, and the emerging themes afterwards were discussed to obtain consensus among the researchers. This technique can be considered as a triangular approach to assure the intercoder reliability and internal validity of this research. The transparent process also allowed an external expert in dental education to verify the accuracy of the analysis. All emerging themes and the decision on data saturation were based on a discussion of all researchers until an agreement was made. NVivo (version 14, QSR International) was used to performed the qualitative data analysis. Subsequently, a conceptual framework was constructed to demonstrate emerging themes and subthemes together with their relationships.

Ethical consideration

The ethical approval for the study was approved by the Institutional Review Board of Faculty of Dentistry and Faculty of Pharmacy, Mahidol University on 29 th September 2022, the ethical approval number: MU-DT/PY-IRB 2022/049.2909. All methods were performed in accordance with the relevant guidelines and regulations. Although the data were not anonymous in nature as they contained identifiable data, they were coded prior to the analysis to assure confidentiality of participants.

Informed consent

Informed consent was obtained from all participants.

There were 18 residents from Year 1 to 3 of the Residency Training Program in Advanced General Dentistry who participated in this research (six from each year). Of these, there were 14 females and 4 males. There was no participant dropout, as all of them completed all required tasks, including the pre- and post-perceived assessments, gamified online role-play, and satisfaction questionnaire. According to the purposive sampling, the participants from the quantitative phase were selected for semi-structured interviews by considering sex, year of study, and self-perceived assessment scores. Twelve students (ten females and two males) participated in semi-structured interviews, where their characteristics are presented in Table 1 .

Internal consistency of all constructs

The data collected from the research participants, in addition to the pilot samples, were analyzed with Cronbach’s alpha to confirm the internal consistency. The coefficient alpha of all constructs demonstrated high internal consistency, as demonstrated in Table 2 .

Self-perceived assessments toward confidence and awareness of teledentistry

There were statistically significant increases in the assessment scores of self-perceived confidence and awareness after participating in the gamified online role-play ( P  < 0.001). According to Table 3 , there was an increase in self-perceived confidence from 3.38 (SD = 0.68) for the pre-assessment to 4.22 (SD = 0.59) for the post-assessment ( P  < 0.001). The findings of self-perceived awareness also showed score improvement from 4.16 (SD = 0.48) to 4.55 (SD = 0.38) after interacting with the simulated patient ( P  < 0.001).

According to Fig.  2 , participants demonstrated a higher level of self-perceived assessments for both self-confidence and awareness in all aspects after participating in the gamified online role-play for teledentistry training.

Self-perceived assessments toward confidence and awareness of teledentistry.

When comparing the self-perceived assessment scores toward confidence and awareness in the use of teledentistry among the three years of study (Year 1–3), there were no statistically significant differences in the pre-assessment, post-assessment score, and score difference (Table 4 ).

Satisfactions toward the use of gamified online role-play

According to Fig.  3 , participants exhibited high levels of satisfaction with the use of gamified online role-play across all three aspects. The aspect of usefulness received the highest satisfaction rating with a score of 4.44 (SD = 0.23) out of 5, followed by ease of use and enjoyment, scoring 4.40 (SD = 0.23) and 4.03 (SD = 0.21), respectively. Particularly, participants expressed the highest satisfaction levels regarding the usefulness of gamified online role-play for identifying their role (Mean = 4.72, SD = 0.46) and developing problem-solving skills associated with teledentistry (Mean = 4.61, SD = 0.50). Additionally, they reported satisfaction with the learning sequence presented in the gamified online role-play (Mean = 4.61, SD = 0.50). However, participants did not strongly perceive that the format of the gamified online role-play could engage them with the learning task for an extended period (Mean = 3.72, SD = 0.83).

Satisfactions toward the use of gamified online role-play.

When comparing the satisfaction levels perceived by participants from different academic years (Table 5 ), no statistically significant differences were observed among the three groups for all three aspects ( P  > 0.05).

Following the framework analysis of qualitative data, there were five emerging themes, including: (1) learner profile, (2) learning settings of the gamified online role-play, (3) pedagogical components, (4) interactive functions, and (5) educational impact.

Theme 1: Learner profile

Learner experience and preferences appeared to have impact on how the participants perceived the use of gamified online role-play for teledentistry training. When learners preferred role-play or realized benefits of teledentistry, they were likely to support this learning intervention. In addition, they could have seen an overall picture of the assigned tasks before participating in this research.

“I had experience with a role-play activity when I was dental undergraduates, and I like this kind of learning where someone role-plays a patient with specific personalities in various contexts. This could be a reason why I felt interested to participate in this task (the gamified online role-play). I also believed that it would be supportive for my clinical practice.” Participant 12, Year 1, Female “Actually, I' have seen in several videos (about teledentistry), where dentists were teaching patients to perform self-examinations, such as checking their own mouth and taking pictures for consultations. Therefore, I could have thought about what I would experience during the activity (within the gamified online role-play).” Participant 8, Year 2, Female

Theme 2: Learning settings of the gamified online role-play

Subtheme 2.1: location.

Participants had agreed that the location for conducting a gamified online role-play should be in a private room without any disturbances, enabling learners to focus on the simulated patient. This could allow them to effectively communicate and understand of the needs of patient, leading to a better grasp of lesson content. In addition, the environments of both learners and simulated patient should be authentic to the learning quality.

“The room should be a private space without any disturbances. This will make us feel confident and engage in conversations with the simulated patient.” Participant 10, Year 1, Female “… simulating a realistic environment can engage me to interact with the simulated patient more effectively ...” Participant 8, Year 2, Female

Subtheme 2.2: Time allocated for the gamified online role-play

The time allocated for the gamified online role-play in this research was considered as appropriate, as participants believed that a 30-minutes period should be suitable to take information and afterwards give some advice to their patient. In addition, a 10-minutes discussion on how they interact with the patient could be supportive for participants to enhance their competencies in the use of teledentistry.

“… it would probably take about 20 minutes because we would need to gather a lot of information … it might need some time to request and gather various information … maybe another 10-15 minutes to provide some advice.” Participant 7, Year 1, Female “I think during the class … we could allocate around 30 minutes for role-play, … we may have discussion of learner performance for 10-15 minutes ... I think it should not be longer than 45 minutes in total.” Participant 6, Year 2, Female

Subtheme 2.3: Learning consequence within a postgraduate curriculum

Most participants suggested that the gamified online role-play in teledentistry should be arranged in the first year of their postgraduate program. This could maximize the effectiveness of online role-play, as they would be able to implement teledentistry for their clinical practice since the beginning of their training. However, some participants suggested that this learning approach could be rearranged in either second or third year of the program. As they already had experience in clinical practice, the gamified online role-play would reinforce their competence in teledentistry.

"Actually, it would be great if this session could be scheduled in the first year … I would feel more comfortable when dealing with my patients through an online platform." Participant 11, Year 2, Male "I believe this approach should be implemented in the first year because it allows students to be trained in teledentistry before being exposed to real patients. However, if this approach is implemented in either the second or third year when they have already had experience in patient care, they would be able to better learn from conversations with simulated patients." Participant 4, Year 3, Male

Theme 3: Pedagogical components

Subtheme 3.1: learning content.

Learning content appeared to be an important component of pedagogical aspect, as it would inform what participants should learn from the gamified online role-play. Based on the interview data, participants reported they could learn how to use a video teleconference platform for teledentistry. The conditions of simulated patient embedded in an online role-play also allowed them to realize the advantages of teledentistry. In addition, dental problems assigned to the simulated patient could reveal the limitations of teledentistry for participants.

“The learning tasks (within the gamified online role-play) let me know how to manage patients through the teleconference.” Participant 5, Year 2, Female “… there seemed to be limitations (of teledentistry) … there could be a risk of misdiagnosis … the poor quality of video may lead to diagnostic errors … it is difficult for patients to capture their oral lesions.” Participant 3, Year 2, Female

Subtheme 3.2: Feedback

During the use of online role-play, the simulated patient can provide formative feedback to participants through facial expressions and tones of voice, enabling participants to observe and learn to adjust their inquiries more accurately. In addition, at the completion of the gamified online role-play, summative feedback provided by instructors could summarize the performance of participants leading to further improvements in the implementation of teledentistry.

“I knew (whether or not I interacted correctly) from the gestures and emotions of the simulated patient between the conversation. I could have learnt from feedback provided during the role-play, especially from the facial expressions of the patient.” Participant 11, Year 2, Male “The feedback provided at the end let me know how well I performed within the learning tasks.” Participant 2, Year 1, Female

Theme 4: Interactive functions

Subtheme 4.1: the authenticity of the simulated patient.

Most participants believed that a simulated patient with high acting performance could enhance the flow of role-play, allowing learners to experience real consequences. The appropriate level of authenticity could engage learners with the learning activity, as they would have less awareness of time passing in the state of flow. Therefore, they could learn better from the gamified online role-play.

"It was so realistic. ... This allowed me to talk with the simulated patient naturally ... At first, when we were talking, I was not sure how I should perform … but afterwards I no longer had any doubts and felt like I wanted to explain things to her even more." Participant 3, Year 2, Female "At first, I believed that if there was a factor that could influence learning, it would probably be a simulated patient. I was impressed by how this simulated patient could perform very well. It made the conversation flow smoothly and gradually." Participant 9, Year 3, Female

Subtheme 4.2: Entertaining features

Participants were likely to be satisfied with the entertaining features embedded in the gamified online role-play. They felt excited when they were being exposed to the unrevealed challenge which they had randomly selected. In addition, participants suggested to have more learning scenarios or simulated patients where they could randomly select to enhance randomness and excitement.

“It was a playful experience while communicating with the simulated patient. There are elements of surprise from the challenge cards that make the conversation more engaging, and I did not feel bored during the role-play.” Participant 4, Year 3, Male “I like the challenge card we randomly selected, as we had no idea what we would encounter … more scenarios like eight choices and we can randomly choose to be more excited. I think we do not need additional challenge cards, as some of them have already been embedded in patient conditions.” Participant 5, Year 2, Female

Subtheme 4.3: Level of difficulty

Participants suggested the gamified online role-play to have various levels of difficulty, so learners could have a chance to select a suitable level for their competence. The difficulties could be represented through patient conditions (e.g., systemic diseases or socioeconomic status), personal health literacy, and emotional tendencies. They also recommended to design the gamified online role-play to have different levels where learners could select an option that is suitable for them.

“The patient had hidden their information, and I needed to bring them out from the conversation.” Participant 12, Year 1, Female “Patients' emotions could be more sensitive to increase level of challenges. This can provide us with more opportunities to enhance our management skills in handling patient emotions.” Participant 11, Year 2, Male “… we can gradually increase the difficult level, similar to playing a game. These challenges could be related to the simulated patient, such as limited knowledge or difficulties in communication, which is likely to occur in our profession.” Participant 6, Year 2, Female

Theme 5: Educational impact

Subtheme 5.1: self-perceived confidence in teledentistry, communication skills.

Participants were likely to perceive that they could learn from the gamified online role-play and felt more confident in the use of teledentistry. This educational impact was mostly achieved from the online conversation within the role-play activity, where the participants could improve their communication skills through a video teleconference platform.

“I feel like the online role-play was a unique form of learning. I believe that I gained confidence from the online communication the simulated patient. I could develop skills to communicate effectively with real patients.” Participant 11, Year 2, Male “I believe it support us to train communication skills ... It allowed us to practice both listening and speaking skills more comprehensively.” Participant 4, Year 3, Male

Critical thinking and problem-solving skills

In addition to communication skills, participants reported that challenges embedded in the role-play allowed them to enhance critical thinking and problem-solving skills, which were a set of skills required to deal with potential problems in the use of teledentistry.

"It was a way of training before experiencing real situations … It allowed us to think critically whether or not what we performed with the simulated patients was appropriate." Participant 7, Year 1, Female “It allowed us to learn how to effectively solve the arranged problems in simulated situation. We needed to solve problems in order to gather required information from the patient and think about how to deliver dental advice through teledentistry.” Participant 11, Year 2, Male

Subtheme 5.2: Self perceived awareness in teledentistry

Participants believed that they could realize the necessity of teledentistry from the gamified online role-play. The storytelling or patient conditions allowed learners to understand how teledentistry could have both physical and psychological support for dental patients.

“From the activity, I would consider teledentistry as a convenient tool for communicating with patients, especially if a patient cannot go to a dental office”. Participant 5, Year 2, Female “I learned about the benefits of teledentistry, particularly in terms of follow-up. The video conference platform could support information sharing, such as drawing images or presenting treatment plans, to patients.” Participant 8, Year 2, Female

A conceptual framework of learning experience within a gamified online role-play

Based on the qualitative findings, a conceptual framework was developed in which a gamified online role-play was conceptualized as a learning strategy in supporting learners to be able to implement teledentistry in their clinical practice (Fig.  4 ).

The conceptual framework of key elements in designing a gamified online role-play.

The conceptual framework has revealed key elements to be considered in designing a gamified online role-play. Learner profile, learning settings, pedagogical components, and interactive functions are considered as influential factors toward user experience within the gamified online role-play. The well-designed learning activity will support learners to achieve expected learning outcomes, considered as educational impact of the gamified online role-play. The contributions of these five key elements to the design of gamified online role-play were interpreted, as follows:

Learner profile: This element tailors the design of gamified online role-plays for teledentistry training involves considering the background knowledge, skills, and experiences of target learners to ensure relevance and engagement.

Learning settings: The element focuses the planning for gamified online role-plays in teledentistry training involves selecting appropriate contexts, such as location and timing, to enhance accessibility and achieve learning outcomes effectively.

Pedagogical components: This element emphasizes the alignment between learning components and learning outcomes within gamified online role-plays, to ensure that the content together with effective feedback design can support learners in improving their competencies from their mistakes.

Interactive functions: This element highlights interactivity features integrated into gamified online role-plays, such as the authenticity and entertaining components to enhance immersion and engagement, together with game difficulty for optimal flow. All these features should engage learners with the learning activities until the achievement of learner outcomes.

Educational impact: This element represents the expected learning outcomes, which will inform the design of learning content and activities within gamified online role-plays. In addition, this element could be considered to evaluate the efficacy of gamified online role-plays, reflecting how well learning designs align with the learning outcomes.

A gamified online role-play can be considered as a learning strategy for teledentistry according to its educational impact. This pedagogical approach could mimic real-life practice, where dental learners could gain experience in the use of teledentistry in simulated situations before interacting with actual patients. Role-play could provide learners opportunities to develop their required competencies, especially communication and real-time decision-making skills, in a predictable and safe learning environment 20 , 23 , 46 . Potential obstacles could also be arranged for learners to deal with, leading to the enhancement of problem-solving skill 50 . In addition, the recognition of teledentistry benefits can enhance awareness and encourage its adoption and implementation, which could be explained by the technology acceptance model 51 . Therefore, a gamified online role-play with a robust design and implementation appeared to have potential in enhancing self-perceived confidence and awareness in the use of teledentistry.

The pedagogical components comprised learning content, which was complemented by assessment and feedback. Learners could develop their competence with engagement through the learning content, gamified by storytelling of the online role-play 52 , 53 . Immediate feedback provided through facial expression and voice tone of simulated patients allowed participants to learn from their failure, considered as a key feature of game-based learning 29 , 45 . The discussion of summative feedback provided from an instructor at the end of role-play activity could support a debriefing process enabling participants to reflect their learning experience, considered as important of simulation-based game 54 . These key considerations should be initially considered in the design of gamified online role-play.

The interactive functions can be considered as another key component for designing and evaluating the gamified online role-play 45 . Several participants enjoyed with a learning process within the gamified online role-play and suggested it to have more learning scenarios. In other words, this tool could engage learners with an instructional process, leading to the achievement of learning outcomes 29 , 45 . As challenge and randomness appear to be game elements 32 , 33 , this learning intervention assigned a set of cards with obstacle tasks for learners to randomly pick up before interacting with simulated patients, which was perceived by participants as a feature to make the role-play more challenging and engaging. This is consistent with previous research, where challenging content for simulated patients could make learners more engaged with a learning process 55 . However, the balance between task challenges and learner competencies is certainly required for the design of learning activities 56 , 57 . The authenticity of simulated patient and immediate feedback could also affect the game flow, leading to the enhancement of learner engagement 45 . These elements could engage participants with a learning process, leading to the enhancement of educational impact.

The educational settings for implementing gamified online role-play into dental curriculum should be another concern. This aspect has been recognized as significant in existing evidence 45 . As this research found no significant differences in all aspects among the three groups of learners, this learning intervention demonstrated the potential for its implementation at any time of postgraduate dental curriculum. This argument can be supported by previous evidence where a role-play could be adaptable for learning at any time, as it requires a short learning period but provides learners with valuable experience prior to being exposed in real-life scenarios 58 . This strategy also provides opportunities for learners who have any question or concern to seek advice or guidance from their instructors 59 . Although the gamified online role-play can be arranged in the program at any time, the first academic year should be considered, as dental learners would be confidence in implementing teledentistry for their clinical practice.

While a gamified online role-play demonstrated its strengths as an interactive learning strategy specifically for teledentistry, there are a couple of potential drawbacks that need to be addressed. The requirement for synchronous participation could limit the flexibility of access time for learners (synchronous interactivity limitation). With only one learner able to engage with a simulated patient at a time (limited participants), more simulated patients would be required if there are a number of learners, otherwise they would need to wait for their turn. Time and resources are significantly required for preparing simulated patients 60 . Despite the use of trained and calibrated professional actors/actresses, inauthenticity may be perceived during role-plays, requiring a significant amount of effort to achieve both interactional and clinical authenticities 46 . Future research could investigate asynchronous learning approaches utilizing non-player character (NPC) controlled by an artificial intelligence system as a simulated patient. This setup would enable multiple learners to have the flexibility to engage with the material at their own pace and at times convenient to them 29 . While there are potential concerns about using gamified online role-plays, this interactive learning intervention offers opportunities for dental professionals to enhance their teledentistry competency in a safe and engaging environment.

Albeit the robust design and data collection tools to assure reliability and validity as well as transparency of this study, a few limitations were raised leading to a potential of further research. While this research recruited only postgraduate students to evaluate the feasibility of gamified online role-play in teledentistry training, further research should include not only experienced dental practitioners but also undergraduate students to confirm its potential use in participants with different learner profiles. More learning scenarios in other dental specialties should also be included to validate its effectiveness, as different specialties could have different limitations and variations. Additional learning scenarios from various dental disciplines should be considered to validate the effectiveness of gamified online role-plays, as different specialties may present unique limitations and variations. A randomized controlled trial with robust design should be required to compare the effectiveness of gamified online role-play with different approaches in training the use of teledentistry.

Conclusions

This research supports the design and implementation of a gamified online role-play in dental education, as dental learners could develop self-perceived confidence and awareness with satisfaction. A well-designed gamified online role-play is necessary to support learners to achieve expected learning outcomes, and the conceptual framework developed in this research can serve as a guidance to design and implement this interactive learning strategy in dental education. However, further research with robust design should be required to validate and ensure the educational impact of gamified online role-play in dental education. Additionally, efforts should be made to develop gamified online role-play in asynchronous learning approaches to enhance the flexibility of learning activities.

Data availability

The data that support the findings of this study are available from the corresponding author, up-on reasonable request. The data are not publicly available due to information that could compromise the privacy of research participants.

Van Dyk, L. A review of telehealth service implementation frameworks. Int. J. Environ. Res. Public Health 11 (2), 1279–1298 (2014).

Article   PubMed   PubMed Central   Google Scholar  

Bartz, C. C. Nursing care in telemedicine and telehealth across the world. Soins. 61 (810), 57–59 (2016).

Article   PubMed   Google Scholar  

Lin, G.S.S., Koh, S.H., Ter, K.Z., Lim, C.W., Sultana, S., Tan, W.W. Awareness, knowledge, attitude, and practice of teledentistry among dental practitioners during COVID-19: A systematic review and meta-analysis. Medicina (Kaunas). 58 (1), 130 (2022).

Wolf, T.G., Schulze, R.K.W., Ramos-Gomez, F., Campus, G. Effectiveness of telemedicine and teledentistry after the COVID-19 pandemic. Int. J. Environ. Res. Public Health. 19 (21), 13857 (2022).

Gajarawala, S. N. & Pelkowski, J. N. Telehealth benefits and barriers. J. Nurse Pract. 17 (2), 218–221 (2021).

Jampani, N. D., Nutalapati, R., Dontula, B. S. & Boyapati, R. Applications of teledentistry: A literature review and update. J. Int. Soc. Prev. Community Dent. 1 (2), 37–44 (2011).

Article   CAS   PubMed   PubMed Central   Google Scholar  

Khan, S. A. & Omar, H. Teledentistry in practice: literature review. Telemed. J. E. Health. 19 (7), 565–567 (2013).

Baheti, M. J. B. S., Toshniwal, N. G. & Misal, A. Teledentistry: A need of the era. Int. J. Dent. Med. Res. 1 (2), 80–91 (2014).

Google Scholar  

Datta, N., Derenne, J., Sanders, M. & Lock, J. D. Telehealth transition in a comprehensive care unit for eating disorders: Challenges and long-term benefits. Int. J. Eat. Disord. 53 (11), 1774–1779 (2020).

Bursell, S. E., Brazionis, L. & Jenkins, A. Telemedicine and ocular health in diabetes mellitus. Clin. Exp. Optom. 95 (3), 311–327 (2012).

da Costa, C. B., Peralta, F. D. S. & Ferreira de Mello, A. L. S. How has teledentistry been applied in public dental health services? An integrative review. Telemed. J. E. Health. 26 (7), 945–954 (2020).

Heckemann, B., Wolf, A., Ali, L., Sonntag, S. M. & Ekman, I. Discovering untapped relationship potential with patients in telehealth: A qualitative interview study. BMJ Open. 6 (3), e009750 (2016).

Pérez-Noboa, B., Soledispa-Carrasco, A., Padilla, V. S. & Velasquez, W. Teleconsultation apps in the COVID-19 pandemic: The case of Guayaquil City, Ecuador. IEEE Eng. Manag. Rev. 49 (1), 27–37 (2021).

Article   Google Scholar  

Wamsley, C. E., Kramer, A., Kenkel, J. M. & Amirlak, B. Trends and challenges of telehealth in an academic institution: The unforeseen benefits of the COVID-19 global pandemic. Aesthetic Surg. J. 41 (1), 109–118 (2020).

Jonasdottir, S. K., Thordardottir, I. & Jonsdottir, T. Health professionals’ perspective towards challenges and opportunities of telehealth service provision: A scoping review. Int. J. Med. Inform. 167 , 104862 (2022).

Tan, S. H. X., Lee, C. K. J., Yong, C. W. & Ding, Y. Y. Scoping review: Facilitators and barriers in the adoption of teledentistry among older adults. Gerodontology. 38 (4), 351–365 (2021).

Minervini, G. et al. Teledentistry in the management of patients with dental and temporomandibular disorders. BioMed. Res. Int. 2022 , 7091153 (2022).

Edirippulige, S. & Armfield, N. Education and training to support the use of clinical telehealth: A review of the literature. J. Telemed. Telecare. 23 (2), 273–282 (2017).

Article   CAS   PubMed   Google Scholar  

Mariño, R. & Ghanim, A. Teledentistry: A systematic review of the literature. J. Telemed. Telecare. 19 (4), 179–183 (2013).

Article   ADS   PubMed   Google Scholar  

Armitage-Chan, E. & Whiting, M. Teaching professionalism: Using role-play simulations to generate professionalism learning outcomes. J. Vet. Med. Educ. 43 (4), 359–363 (2016).

Spyropoulos, F., Trichakis, I. & Vozinaki, A.-E. A narrative-driven role-playing game for raising flood awareness. Sustainability. 14 (1), 554 (2022).

Jiang, W. K. et al. Role-play in endodontic teaching: A case study. Chin. J. Dent. Res. 23 (4), 281–288 (2020).

PubMed   Google Scholar  

Vizeshfar, F., Zare, M. & Keshtkaran, Z. Role-play versus lecture methods in community health volunteers. Nurse Educ. Today. 79 , 175–179 (2019).

Nestel, D. & Tierney, T. Role-play for medical students learning about communication: Guidelines for maximising benefits. BMC Med. Educ. 7 , 3 (2007).

Gelis, A. et al. Peer role-play for training communication skills in medical students: A systematic review. Simulat. Health. 15 (2), 106–111 (2020).

Cornelius, S., Gordon, C. & Harris, M. Role engagement and anonymity in synchronous online role play. Int. Rev. Res. Open Distrib. Learn. 12 (5), 57–73 (2011).

Bell, M. Online role-play: Anonymity, engagement and risk. Educ. Med. Int. 38 (4), 251–260 (2001).

Sipiyaruk, K., Gallagher, J. E., Hatzipanagos, S. & Reynolds, P. A. A rapid review of serious games: From healthcare education to dental education. Eur. J. Dent. Educ. 22 (4), 243–257 (2018).

Sipiyaruk, K., Hatzipanagos, S., Reynolds, P. A. & Gallagher, J. E. Serious games and the COVID-19 pandemic in dental education: An integrative review of the literature. Computers. 10 (4), 42 (2021).

Morse, J.M., Niehaus, L. Mixed Method Design: Principles and Procedures. (2016).

Creswell, J. W. Research Design: Qualitative, Quantitative, and Mixed Methods Approaches 3rd edn. (SAGE Publications, 2009).

Cheng, V. W. S., Davenport, T., Johnson, D., Vella, K. & Hickie, I. B. Gamification in apps and technologies for improving mental health and well-being: Systematic review. JMIR Ment. Health. 6 (6), e13717 (2019).

Gallego-Durán, F. J. et al. A guide for game-design-based gamification. Informatics. 6 (4), 49 (2019).

Gee, J. P. Learning and games. In The Ecology of Games: Connecting Youth, Games, and Learning (ed. Salen, K.) 21–40 (MIT Press, 2008).

Cheung, K. L., ten Klooster, P. M., Smit, C., de Vries, H. & Pieterse, M. E. The impact of non-response bias due to sampling in public health studies: A comparison of voluntary versus mandatory recruitment in a Dutch national survey on adolescent health. BMC Public Health. 17 (1), 276 (2017).

Murairwa, S. Voluntary sampling design. Int. J. Adv. Res. Manag. Social Sci. 4 (2), 185–200 (2015).

Chow, S.-C., Shao, J., Wang, H. & Lokhnygina, Y. Sample Size Calculations in Clinical Research (CRC Press, 2017).

Book   Google Scholar  

Palinkas, L. A. et al. Purposeful sampling for qualitative data collection and analysis in mixed method implementation research. Administration Policy Mental Health Mental Health Services Res. 42 (5), 533–544 (2015).

McIlvried, D. E., Prucka, S. K., Herbst, M., Barger, C. & Robin, N. H. The use of role-play to enhance medical student understanding of genetic counseling. Genet. Med. 10 (10), 739–744 (2008).

Schlegel, C., Woermann, U., Shaha, M., Rethans, J.-J. & van der Vleuten, C. Effects of communication training on real practice performance: A role-play module versus a standardized patient module. J. Nursing Educ. 51 (1), 16–22 (2012).

Manzoor, I. M. F. & Hashmi, N. R. Medical students’ perspective about role-plays as a teaching strategy in community medicine. J. Coll. Physicians Surg. Pak. 22 (4), 222–225 (2012).

Cornes, S., Gelfand, J. M. & Calton, B. Foundational telemedicine workshop for first-year medical students developed during a pandemic. MedEdPORTAL. 17 , 11171 (2021).

King, J., Hill, K. & Gleason, A. All the world’sa stage: Evaluating psychiatry role-play based learning for medical students. Austral. Psychiatry. 23 (1), 76–79 (2015).

Arayapisit, T. et al. An educational board game for learning orofacial spaces: An experimental study comparing collaborative and competitive approaches. Anatomical Sci. Educ. 16 (4), 666–676 (2023).

Sipiyaruk, K., Hatzipanagos, S., Vichayanrat, T., Reynolds, P.A., Gallagher, J.E. Evaluating a dental public health game across two learning contexts. Educ. Sci. 12 (8), 517 (2022).

Pilnick, A. et al. Using conversation analysis to inform role play and simulated interaction in communications skills training for healthcare professionals: Identifying avenues for further development through a scoping review. BMC Med. Educ. 18 (1), 267 (2018).

Lane, C. & Rollnick, S. The use of simulated patients and role-play in communication skills training: A review of the literature to August 2005. Patient Educ. Counseling. 67 (1), 13–20 (2007).

Gale, N. K., Heath, G., Cameron, E., Rashid, S. & Redwood, S. Using the framework method for the analysis of qualitative data in multi-disciplinary health research. BMC Med. Res. Methodol. 13 (1), 117 (2013).

Ritchie, J., Lewis, J., Nicholls, C. M. & Ormston, R. Qualitative Research Practice: A Guide for Social Science Students and Researchers (Sage, 2014).

Chen, J. C. & Martin, A. R. Role-play simulations as a transformative methodology in environmental education. J. Transform. Educ. 13 (1), 85–102 (2015).

Davis, F. D. Perceived usefulness, perceived ease of use, and user acceptance of information technology. Manag. Inform. Syst. Quart. 13 (3), 319–340 (1989).

Novak, E., Johnson, T. E., Tenenbaum, G. & Shute, V. J. Effects of an instructional gaming characteristic on learning effectiveness, efficiency, and engagement: Using a storyline for teaching basic statistical skills. Interact. Learn. Environ. 24 (3), 523–538 (2016).

Marchiori, E. J. et al. A narrative metaphor to facilitate educational game authoring. Comput. Educ. 58 (1), 590–599 (2012).

Luctkar-Flude, M. et al. Effectiveness of debriefing methods for virtual simulation: A systematic review. Clin. Simulat. Nursing. 57 , 18–30 (2021).

Joyner, B. & Young, L. Teaching medical students using role play: Twelve tips for successful role plays. Med. Teach. 28 (3), 225–229 (2006).

Csikszentmihalyi, M. Flow: The Psychology of Optimal Performance (HarperCollins Publishers, 1990).

Buajeeb, W., Chokpipatkun, J., Achalanan, N., Kriwattanawong, N. & Sipiyaruk, K. The development of an online serious game for oral diagnosis and treatment planning: Evaluation of knowledge acquisition and retention. BMC Med. Educ. 23 (1), 830 (2023).

Littlefield, J. H., Hahn, H. B. & Meyer, A. S. Evaluation of a role-play learning exercise in an ambulatory clinic setting. Adv. Health. Sci. Educ. Theory Pract. 4 (2), 167–173 (1999).

Alkin, M. C. & Christie, C. A. The use of role-play in teaching evaluation. Am. J. Evaluat. 23 (2), 209–218 (2002).

Lovell, K. L., Mavis, B. E., Turner, J. L., Ogle, K. S. & Griffith, M. Medical students as standardized patients in a second-year performance-based assessment experience. Med. Educ. Online. 3 (1), 4301 (1998).

Download references

Acknowledgements

The authors would like to express our sincere gratitude to participants for their contributions in this research. We would also like to thank the experts who provided their helpful suggestions in the validation process of the data collection tools.

This research project was funded by the Faculty of Dentistry, Mahidol University. The APC was funded by Mahidol University.

Author information

Authors and affiliations.

Department of Advanced General Dentistry, Faculty of Dentistry, Mahidol University, Bangkok, Thailand

Chayanid Teerawongpairoj & Chanita Tantipoj

Department of Orthodontics, Faculty of Dentistry, Mahidol University, Bangkok, Thailand

Kawin Sipiyaruk

You can also search for this author in PubMed   Google Scholar

Contributions

Conceptualization, C.Te., C.Ta., and K.S.; methodology, C.Te., C.Ta., and K.S.; validation, C.Te., C.Ta., and K.S.; investigation, C.Te. and K.S.; formal analysis, C.Te., C.Ta., and K.S.; resources, C.Te., C.Ta., and K.S.; data curation, C.Ta. and K.S.; writing-original draft preparation, C.Te., C.Ta., and K.S.; writing-review and editing, C.Te., C.Ta., and K.S. All authors have read and agreed to the published version of the manuscript.

Corresponding author

Correspondence to Kawin Sipiyaruk .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary information 1., supplementary information 2., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Teerawongpairoj, C., Tantipoj, C. & Sipiyaruk, K. The design and evaluation of gamified online role-play as a telehealth training strategy in dental education: an explanatory sequential mixed-methods study. Sci Rep 14 , 9216 (2024). https://doi.org/10.1038/s41598-024-58425-9

Download citation

Received : 30 September 2023

Accepted : 28 March 2024

Published : 22 April 2024

DOI : https://doi.org/10.1038/s41598-024-58425-9

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Dental education
• Distance learning
• Game-based learning
• Gamification

By submitting a comment you agree to abide by our Terms and Community Guidelines . If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.