case study learning method

• Columbia University in the City of New York
• Office of Teaching, Learning, and Innovation
• University Policies
• Columbia Online
• Academic Calendar
• Resources and Technology
• Resources and Guides

Case Method Teaching and Learning

What is the case method? How can the case method be used to engage learners? What are some strategies for getting started? This guide helps instructors answer these questions by providing an overview of the case method while highlighting learner-centered and digitally-enhanced approaches to teaching with the case method. The guide also offers tips to instructors as they get started with the case method and additional references and resources.

On this page:

What is case method teaching.

• Case Method at Columbia

Why use the Case Method?

Case method teaching approaches, how do i get started.

• Additional Resources

The CTL is here to help!

For support with implementing a case method approach in your course, email [email protected] to schedule your 1-1 consultation .

Cite this resource: Columbia Center for Teaching and Learning (2019). Case Method Teaching and Learning. Columbia University. Retrieved from [today’s date] from https://ctl.columbia.edu/resources-and-technology/resources/case-method/  

Case method 1 teaching is an active form of instruction that focuses on a case and involves students learning by doing 2 3 . Cases are real or invented stories 4  that include “an educational message” or recount events, problems, dilemmas, theoretical or conceptual issue that requires analysis and/or decision-making.

Case-based teaching simulates real world situations and asks students to actively grapple with complex problems 5 6 This method of instruction is used across disciplines to promote learning, and is common in law, business, medicine, among other fields. See Table 1 below for a few types of cases and the learning they promote.

Table 1: Types of cases and the learning they promote.

For a more complete list, see Case Types & Teaching Methods: A Classification Scheme from the National Center for Case Study Teaching in Science.

Back to Top

Case Method Teaching and Learning at Columbia

The case method is actively used in classrooms across Columbia, at the Morningside campus in the School of International and Public Affairs (SIPA), the School of Business, Arts and Sciences, among others, and at Columbia University Irving Medical campus.

Faculty Spotlight:

Professor Mary Ann Price on Using Case Study Method to Place Pre-Med Students in Real-Life Scenarios

Read more  

Professor De Pinho on Using the Case Method in the Mailman Core

Case method teaching has been found to improve student learning, to increase students’ perception of learning gains, and to meet learning objectives 8 9 . Faculty have noted the instructional benefits of cases including greater student engagement in their learning 10 , deeper student understanding of concepts, stronger critical thinking skills, and an ability to make connections across content areas and view an issue from multiple perspectives 11 . 

Through case-based learning, students are the ones asking questions about the case, doing the problem-solving, interacting with and learning from their peers, “unpacking” the case, analyzing the case, and summarizing the case. They learn how to work with limited information and ambiguity, think in professional or disciplinary ways, and ask themselves “what would I do if I were in this specific situation?”

The case method bridges theory to practice, and promotes the development of skills including: communication, active listening, critical thinking, decision-making, and metacognitive skills 12 , as students apply course content knowledge, reflect on what they know and their approach to analyzing, and make sense of a case. 

Though the case method has historical roots as an instructor-centered approach that uses the Socratic dialogue and cold-calling, it is possible to take a more learner-centered approach in which students take on roles and tasks traditionally left to the instructor. 

Cases are often used as “vehicles for classroom discussion” 13 . Students should be encouraged to take ownership of their learning from a case. Discussion-based approaches engage students in thinking and communicating about a case. Instructors can set up a case activity in which students are the ones doing the work of “asking questions, summarizing content, generating hypotheses, proposing theories, or offering critical analyses” 14 . 

The role of the instructor is to share a case or ask students to share or create a case to use in class, set expectations, provide instructions, and assign students roles in the discussion. Student roles in a case discussion can include: 

• discussion “starters” get the conversation started with a question or posing the questions that their peers came up with; 
• facilitators listen actively, validate the contributions of peers, ask follow-up questions, draw connections, refocus the conversation as needed; 
• recorders take-notes of the main points of the discussion, record on the board, upload to CourseWorks, or type and project on the screen; and 
• discussion “wrappers” lead a summary of the main points of the discussion. 

Prior to the case discussion, instructors can model case analysis and the types of questions students should ask, co-create discussion guidelines with students, and ask for students to submit discussion questions. During the discussion, the instructor can keep time, intervene as necessary (however the students should be doing the talking), and pause the discussion for a debrief and to ask students to reflect on what and how they learned from the case activity. 

Note: case discussions can be enhanced using technology. Live discussions can occur via video-conferencing (e.g., using Zoom ) or asynchronous discussions can occur using the Discussions tool in CourseWorks (Canvas) .

Table 2 includes a few interactive case method approaches. Regardless of the approach selected, it is important to create a learning environment in which students feel comfortable participating in a case activity and learning from one another. See below for tips on supporting student in how to learn from a case in the “getting started” section and how to create a supportive learning environment in the Guide for Inclusive Teaching at Columbia . 

Table 2. Strategies for Engaging Students in Case-Based Learning

Approaches to case teaching should be informed by course learning objectives, and can be adapted for small, large, hybrid, and online classes. Instructional technology can be used in various ways to deliver, facilitate, and assess the case method. For instance, an online module can be created in CourseWorks (Canvas) to structure the delivery of the case, allow students to work at their own pace, engage all learners, even those reluctant to speak up in class, and assess understanding of a case and student learning. Modules can include text, embedded media (e.g., using Panopto or Mediathread ) curated by the instructor, online discussion, and assessments. Students can be asked to read a case and/or watch a short video, respond to quiz questions and receive immediate feedback, post questions to a discussion, and share resources. 

For more information about options for incorporating educational technology to your course, please contact your Learning Designer .

To ensure that students are learning from the case approach, ask them to pause and reflect on what and how they learned from the case. Time to reflect  builds your students’ metacognition, and when these reflections are collected they provides you with insights about the effectiveness of your approach in promoting student learning.

Well designed case-based learning experiences: 1) motivate student involvement, 2) have students doing the work, 3) help students develop knowledge and skills, and 4) have students learning from each other.  

Designing a case-based learning experience should center around the learning objectives for a course. The following points focus on intentional design. 

Identify learning objectives, determine scope, and anticipate challenges. 

• Why use the case method in your course? How will it promote student learning differently than other approaches? 
• What are the learning objectives that need to be met by the case method? What knowledge should students apply and skills should they practice? 
• What is the scope of the case? (a brief activity in a single class session to a semester-long case-based course; if new to case method, start small with a single case). 
• What challenges do you anticipate (e.g., student preparation and prior experiences with case learning, discomfort with discussion, peer-to-peer learning, managing discussion) and how will you plan for these in your design? 
• If you are asking students to use transferable skills for the case method (e.g., teamwork, digital literacy) make them explicit. 

Determine how you will know if the learning objectives were met and develop a plan for evaluating the effectiveness of the case method to inform future case teaching. 

• What assessments and criteria will you use to evaluate student work or participation in case discussion? 
• How will you evaluate the effectiveness of the case method? What feedback will you collect from students? 
• How might you leverage technology for assessment purposes? For example, could you quiz students about the case online before class, accept assignment submissions online, use audience response systems (e.g., PollEverywhere) for formative assessment during class? 

Select an existing case, create your own, or encourage students to bring course-relevant cases, and prepare for its delivery

• Where will the case method fit into the course learning sequence? 
• Is the case at the appropriate level of complexity? Is it inclusive, culturally relevant, and relatable to students? 
• What materials and preparation will be needed to present the case to students? (e.g., readings, audiovisual materials, set up a module in CourseWorks). 

Plan for the case discussion and an active role for students

• What will your role be in facilitating case-based learning? How will you model case analysis for your students? (e.g., present a short case and demo your approach and the process of case learning) (Davis, 2009). 
• What discussion guidelines will you use that include your students’ input? 
• How will you encourage students to ask and answer questions, summarize their work, take notes, and debrief the case? 
• If students will be working in groups, how will groups form? What size will the groups be? What instructions will they be given? How will you ensure that everyone participates? What will they need to submit? Can technology be leveraged for any of these areas? 
• Have you considered students of varied cognitive and physical abilities and how they might participate in the activities/discussions, including those that involve technology? 

Student preparation and expectations

• How will you communicate about the case method approach to your students? When will you articulate the purpose of case-based learning and expectations of student engagement? What information about case-based learning and expectations will be included in the syllabus?
• What preparation and/or assignment(s) will students complete in order to learn from the case? (e.g., read the case prior to class, watch a case video prior to class, post to a CourseWorks discussion, submit a brief memo, complete a short writing assignment to check students’ understanding of a case, take on a specific role, prepare to present a critique during in-class discussion).

Andersen, E. and Schiano, B. (2014). Teaching with Cases: A Practical Guide . Harvard Business Press. 

Bonney, K. M. (2015). Case Study Teaching Method Improves Student Performance and Perceptions of Learning Gains†. Journal of Microbiology & Biology Education , 16 (1), 21–28. https://doi.org/10.1128/jmbe.v16i1.846

Davis, B.G. (2009). Chapter 24: Case Studies. In Tools for Teaching. Second Edition. Jossey-Bass. 

Garvin, D.A. (2003). Making the Case: Professional Education for the world of practice. Harvard Magazine. September-October 2003, Volume 106, Number 1, 56-107.

Golich, V.L. (2000). The ABCs of Case Teaching. International Studies Perspectives. 1, 11-29. 

Golich, V.L.; Boyer, M; Franko, P.; and Lamy, S. (2000). The ABCs of Case Teaching. Pew Case Studies in International Affairs. Institute for the Study of Diplomacy. 

Heath, J. (2015). Teaching & Writing Cases: A Practical Guide. The Case Center, UK. 

Herreid, C.F. (2011). Case Study Teaching. New Directions for Teaching and Learning. No. 128, Winder 2011, 31 – 40. 

Herreid, C.F. (2007). Start with a Story: The Case Study Method of Teaching College Science . National Science Teachers Association. Available as an ebook through Columbia Libraries. 

Herreid, C.F. (2006). “Clicker” Cases: Introducing Case Study Teaching Into Large Classrooms. Journal of College Science Teaching. Oct 2006, 36(2). https://search.proquest.com/docview/200323718?pq-origsite=gscholar  

Krain, M. (2016). Putting the Learning in Case Learning? The Effects of Case-Based Approaches on Student Knowledge, Attitudes, and Engagement. Journal on Excellence in College Teaching. 27(2), 131-153. 

Lundberg, K.O. (Ed.). (2011). Our Digital Future: Boardrooms and Newsrooms. Knight Case Studies Initiative. 

Popil, I. (2011). Promotion of critical thinking by using case studies as teaching method. Nurse Education Today, 31(2), 204–207. https://doi.org/10.1016/j.nedt.2010.06.002

Schiano, B. and Andersen, E. (2017). Teaching with Cases Online . Harvard Business Publishing. 

Thistlethwaite, JE; Davies, D.; Ekeocha, S.; Kidd, J.M.; MacDougall, C.; Matthews, P.; Purkis, J.; Clay D. (2012). The effectiveness of case-based learning in health professional education: A BEME systematic review . Medical Teacher. 2012; 34(6): e421-44. 

Yadav, A.; Lundeberg, M.; DeSchryver, M.; Dirkin, K.; Schiller, N.A.; Maier, K. and Herreid, C.F. (2007). Teaching Science with Case Studies: A National Survey of Faculty Perceptions of the Benefits and Challenges of Using Cases. Journal of College Science Teaching; Sept/Oct 2007; 37(1). 

Weimer, M. (2013). Learner-Centered Teaching: Five Key Changes to Practice. Second Edition. Jossey-Bass.

Additional resources 

Teaching with Cases , Harvard Kennedy School of Government. 

Features “what is a teaching case?” video that defines a teaching case, and provides documents to help students prepare for case learning, Common case teaching challenges and solutions, tips for teaching with cases. 

Promoting excellence and innovation in case method teaching: Teaching by the Case Method , Christensen Center for Teaching & Learning. Harvard Business School. 

National Center for Case Study Teaching in Science . University of Buffalo. 

A collection of peer-reviewed STEM cases to teach scientific concepts and content, promote process skills and critical thinking. The Center welcomes case submissions. Case classification scheme of case types and teaching methods:

• Different types of cases: analysis case, dilemma/decision case, directed case, interrupted case, clicker case, a flipped case, a laboratory case. 
• Different types of teaching methods: problem-based learning, discussion, debate, intimate debate, public hearing, trial, jigsaw, role-play. 

Columbia Resources

Resources available to support your use of case method: The University hosts a number of case collections including: the Case Consortium (a collection of free cases in the fields of journalism, public policy, public health, and other disciplines that include teaching and learning resources; SIPA’s Picker Case Collection (audiovisual case studies on public sector innovation, filmed around the world and involving SIPA student teams in producing the cases); and Columbia Business School CaseWorks , which develops teaching cases and materials for use in Columbia Business School classrooms.

Center for Teaching and Learning

The Center for Teaching and Learning (CTL) offers a variety of programs and services for instructors at Columbia. The CTL can provide customized support as you plan to use the case method approach through implementation. Schedule a one-on-one consultation. 

Office of the Provost

The Hybrid Learning Course Redesign grant program from the Office of the Provost provides support for faculty who are developing innovative and technology-enhanced pedagogy and learning strategies in the classroom. In addition to funding, faculty awardees receive support from CTL staff as they redesign, deliver, and evaluate their hybrid courses.

The Start Small! Mini-Grant provides support to faculty who are interested in experimenting with one new pedagogical strategy or tool. Faculty awardees receive funds and CTL support for a one-semester period.

Explore our teaching resources.

• Blended Learning
• Contemplative Pedagogy
• Inclusive Teaching Guide
• FAQ for Teaching Assistants
• Metacognition

CTL resources and technology for you.

• Overview of all CTL Resources and Technology
• The origins of this method can be traced to Harvard University where in 1870 the Law School began using cases to teach students how to think like lawyers using real court decisions. This was followed by the Business School in 1920 (Garvin, 2003). These professional schools recognized that lecture mode of instruction was insufficient to teach critical professional skills, and that active learning would better prepare learners for their professional lives. ↩
• Golich, V.L. (2000). The ABCs of Case Teaching. International Studies Perspectives. 1, 11-29. ↩
• Herreid, C.F. (2007). Start with a Story: The Case Study Method of Teaching College Science . National Science Teachers Association. Available as an ebook through Columbia Libraries. ↩
• Davis, B.G. (2009). Chapter 24: Case Studies. In Tools for Teaching. Second Edition. Jossey-Bass. ↩
• Andersen, E. and Schiano, B. (2014). Teaching with Cases: A Practical Guide . Harvard Business Press. ↩
• Lundberg, K.O. (Ed.). (2011). Our Digital Future: Boardrooms and Newsrooms. Knight Case Studies Initiative. ↩
• Heath, J. (2015). Teaching & Writing Cases: A Practical Guide. The Case Center, UK. ↩
• Bonney, K. M. (2015). Case Study Teaching Method Improves Student Performance and Perceptions of Learning Gains†. Journal of Microbiology & Biology Education , 16 (1), 21–28. https://doi.org/10.1128/jmbe.v16i1.846 ↩
• Krain, M. (2016). Putting the Learning in Case Learning? The Effects of Case-Based Approaches on Student Knowledge, Attitudes, and Engagement. Journal on Excellence in College Teaching. 27(2), 131-153. ↩
• Thistlethwaite, JE; Davies, D.; Ekeocha, S.; Kidd, J.M.; MacDougall, C.; Matthews, P.; Purkis, J.; Clay D. (2012). The effectiveness of case-based learning in health professional education: A BEME systematic review . Medical Teacher. 2012; 34(6): e421-44. ↩
• Yadav, A.; Lundeberg, M.; DeSchryver, M.; Dirkin, K.; Schiller, N.A.; Maier, K. and Herreid, C.F. (2007). Teaching Science with Case Studies: A National Survey of Faculty Perceptions of the Benefits and Challenges of Using Cases. Journal of College Science Teaching; Sept/Oct 2007; 37(1). ↩
• Popil, I. (2011). Promotion of critical thinking by using case studies as teaching method. Nurse Education Today, 31(2), 204–207. https://doi.org/10.1016/j.nedt.2010.06.002 ↩
• Weimer, M. (2013). Learner-Centered Teaching: Five Key Changes to Practice. Second Edition. Jossey-Bass. ↩
• Herreid, C.F. (2006). “Clicker” Cases: Introducing Case Study Teaching Into Large Classrooms. Journal of College Science Teaching. Oct 2006, 36(2). https://search.proquest.com/docview/200323718?pq-origsite=gscholar ↩

This website uses cookies to identify users, improve the user experience and requires cookies to work. By continuing to use this website, you consent to Columbia University's use of cookies and similar technologies, in accordance with the Columbia University Website Cookie Notice .

How it works

For Business

Join Mind Tools

Article • 10 min read

Case Study-Based Learning

Enhancing learning through immediate application.

By the Mind Tools Content Team

If you've ever tried to learn a new concept, you probably appreciate that "knowing" is different from "doing." When you have an opportunity to apply your knowledge, the lesson typically becomes much more real.

Adults often learn differently from children, and we have different motivations for learning. Typically, we learn new skills because we want to. We recognize the need to learn and grow, and we usually need – or want – to apply our newfound knowledge soon after we've learned it.

A popular theory of adult learning is andragogy (the art and science of leading man, or adults), as opposed to the better-known pedagogy (the art and science of leading children). Malcolm Knowles , a professor of adult education, was considered the father of andragogy, which is based on four key observations of adult learners:

• Adults learn best if they know why they're learning something.
• Adults often learn best through experience.
• Adults tend to view learning as an opportunity to solve problems.
• Adults learn best when the topic is relevant to them and immediately applicable.

This means that you'll get the best results with adults when they're fully involved in the learning experience. Give an adult an opportunity to practice and work with a new skill, and you have a solid foundation for high-quality learning that the person will likely retain over time.

So, how can you best use these adult learning principles in your training and development efforts? Case studies provide an excellent way of practicing and applying new concepts. As such, they're very useful tools in adult learning, and it's important to understand how to get the maximum value from them.

What Is a Case Study?

Case studies are a form of problem-based learning, where you present a situation that needs a resolution. A typical business case study is a detailed account, or story, of what happened in a particular company, industry, or project over a set period of time.

The learner is given details about the situation, often in a historical context. The key players are introduced. Objectives and challenges are outlined. This is followed by specific examples and data, which the learner then uses to analyze the situation, determine what happened, and make recommendations.

The depth of a case depends on the lesson being taught. A case study can be two pages, 20 pages, or more. A good case study makes the reader think critically about the information presented, and then develop a thorough assessment of the situation, leading to a well-thought-out solution or recommendation.

Why Use a Case Study?

Case studies are a great way to improve a learning experience, because they get the learner involved, and encourage immediate use of newly acquired skills.

They differ from lectures or assigned readings because they require participation and deliberate application of a broad range of skills. For example, if you study financial analysis through straightforward learning methods, you may have to calculate and understand a long list of financial ratios (don't worry if you don't know what these are). Likewise, you may be given a set of financial statements to complete a ratio analysis. But until you put the exercise into context, you may not really know why you're doing the analysis.

With a case study, however, you might explore whether a bank should provide financing to a borrower, or whether a company is about to make a good acquisition. Suddenly, the act of calculating ratios becomes secondary – it's more important to understand what the ratios tell you. This is how case studies can make the difference between knowing what to do, and knowing how, when, and why to do it.

Then, what really separates case studies from other practical forms of learning – like scenarios and simulations – is the ability to compare the learner's recommendations with what actually happened. When you know what really happened, it's much easier to evaluate the "correctness" of the answers given.

When to Use a Case Study

As you can see, case studies are powerful and effective training tools. They also work best with practical, applied training, so make sure you use them appropriately.

Remember these tips:

• Case studies tend to focus on why and how to apply a skill or concept, not on remembering facts and details. Use case studies when understanding the concept is more important than memorizing correct responses.
• Case studies are great team-building opportunities. When a team gets together to solve a case, they'll have to work through different opinions, methods, and perspectives.
• Use case studies to build problem-solving skills, particularly those that are valuable when applied, but are likely to be used infrequently. This helps people get practice with these skills that they might not otherwise get.
• Case studies can be used to evaluate past problem solving. People can be asked what they'd do in that situation, and think about what could have been done differently.

Ensuring Maximum Value From Case Studies

The first thing to remember is that you already need to have enough theoretical knowledge to handle the questions and challenges in the case study. Otherwise, it can be like trying to solve a puzzle with some of the pieces missing.

Here are some additional tips for how to approach a case study. Depending on the exact nature of the case, some tips will be more relevant than others.

• Read the case at least three times before you start any analysis. Case studies usually have lots of details, and it's easy to miss something in your first, or even second, reading.
• Once you're thoroughly familiar with the case, note the facts. Identify which are relevant to the tasks you've been assigned. In a good case study, there are often many more facts than you need for your analysis.
• If the case contains large amounts of data, analyze this data for relevant trends. For example, have sales dropped steadily, or was there an unexpected high or low point?
• If the case involves a description of a company's history, find the key events, and consider how they may have impacted the current situation.
• Consider using techniques like SWOT analysis and Porter's Five Forces Analysis to understand the organization's strategic position.
• Stay with the facts when you draw conclusions. These include facts given in the case as well as established facts about the environmental context. Don't rely on personal opinions when you put together your answers.

Writing a Case Study

You may have to write a case study yourself. These are complex documents that take a while to research and compile. The quality of the case study influences the quality of the analysis. Here are some tips if you want to write your own:

• Write your case study as a structured story. The goal is to capture an interesting situation or challenge and then bring it to life with words and information. You want the reader to feel a part of what's happening.
• Present information so that a "right" answer isn't obvious. The goal is to develop the learner's ability to analyze and assess, not necessarily to make the same decision as the people in the actual case.
• Do background research to fully understand what happened and why. You may need to talk to key stakeholders to get their perspectives as well.
• Determine the key challenge. What needs to be resolved? The case study should focus on one main question or issue.
• Define the context. Talk about significant events leading up to the situation. What organizational factors are important for understanding the problem and assessing what should be done? Include cultural factors where possible.
• Identify key decision makers and stakeholders. Describe their roles and perspectives, as well as their motivations and interests.
• Make sure that you provide the right data to allow people to reach appropriate conclusions.
• Make sure that you have permission to use any information you include.

A typical case study structure includes these elements:

• Executive summary. Define the objective, and state the key challenge.
• Opening paragraph. Capture the reader's interest.
• Scope. Describe the background, context, approach, and issues involved.
• Presentation of facts. Develop an objective picture of what's happening.
• Description of key issues. Present viewpoints, decisions, and interests of key parties.

Because case studies have proved to be such effective teaching tools, many are already written. Some excellent sources of free cases are The Times 100 , CasePlace.org , and Schroeder & Schroeder Inc . You can often search for cases by topic or industry. These cases are expertly prepared, based mostly on real situations, and used extensively in business schools to teach management concepts.

Case studies are a great way to improve learning and training. They provide learners with an opportunity to solve a problem by applying what they know.

There are no unpleasant consequences for getting it "wrong," and cases give learners a much better understanding of what they really know and what they need to practice.

Case studies can be used in many ways, as team-building tools, and for skill development. You can write your own case study, but a large number are already prepared. Given the enormous benefits of practical learning applications like this, case studies are definitely something to consider adding to your next training session.

Knowles, M. (1973). 'The Adult Learner: A Neglected Species [online].' Available here .

You've accessed 1 of your 2 free resources.

Get unlimited access

Discover more content

What is leadership video.

An Introduction to Leadership

An Introduction to Planning

An Overview of Planning and How the Different Types of Plans Relate to Each Other

Add comment

Comments (0)

Be the first to comment!

Try Mind Tools for FREE

Get unlimited access to all our career-boosting content and member benefits with our 7-day free trial.

Sign-up to our newsletter

Subscribing to the Mind Tools newsletter will keep you up-to-date with our latest updates and newest resources.

Subscribe now

Business Skills

Personal Development

Leadership and Management

Member Extras

Most Popular

Newest Releases

Team Briefings

Onboarding With STEPS

Mind Tools Store

About Mind Tools Content

Discover something new today

New pain points podcast - perfectionism.

Why Am I Such a Perfectionist?

Pain Points Podcast - Building Trust

Developing and Strengthening Trust at Work

How Emotionally Intelligent Are You?

Boosting Your People Skills

Self-Assessment

What's Your Leadership Style?

Learn About the Strengths and Weaknesses of the Way You Like to Lead

Recommended for you

Seeing the big picture with mind maps.

Exploring ideas from all angles

Animated Video

Business Operations and Process Management

Strategy Tools

Customer Service

Business Ethics and Values

Handling Information and Data

Project Management

Knowledge Management

Self-Development and Goal Setting

Time Management

Presentation Skills

Learning Skills

Career Skills

Communication Skills

Negotiation, Persuasion and Influence

Working With Others

Difficult Conversations

Creativity Tools

Self-Management

Work-Life Balance

Stress Management and Wellbeing

Coaching and Mentoring

Change Management

Team Management

Managing Conflict

Delegation and Empowerment

Performance Management

Leadership Skills

Developing Your Team

Talent Management

Problem Solving

Decision Making

Member Podcast

• SUGGESTED TOPICS
• The Magazine
• Newsletters
• Managing Yourself
• Managing Teams
• Work-life Balance
• The Big Idea
• Data & Visuals
• Reading Lists
• Case Selections
• HBR Learning
• Topic Feeds
• Account Settings
• Email Preferences

What the Case Study Method Really Teaches

• Nitin Nohria

Seven meta-skills that stick even if the cases fade from memory.

It’s been 100 years since Harvard Business School began using the case study method. Beyond teaching specific subject matter, the case study method excels in instilling meta-skills in students. This article explains the importance of seven such skills: preparation, discernment, bias recognition, judgement, collaboration, curiosity, and self-confidence.

During my decade as dean of Harvard Business School, I spent hundreds of hours talking with our alumni. To enliven these conversations, I relied on a favorite question: “What was the most important thing you learned from your time in our MBA program?”

• Nitin Nohria is the George F. Baker Professor of Business Administration, Distinguished University Service Professor, and former dean of Harvard Business School.

Partner Center

Our websites may use cookies to personalize and enhance your experience. By continuing without changing your cookie settings, you agree to this collection. For more information, please see our University Websites Privacy Notice .

Center for Excellence in Teaching and Learning

• Case Based Learning

What is the case method?

In case-based learning, students learn to interact with and manipulate basic foundational knowledge by working with situations resembling specific real-world scenarios.

How does it work?

Case studies encourage students to use critical thinking skills to identify and narrow an issue, develop and evaluate alternatives, and offer a solution.  In fact, Nkhoma (2016), who studied the value of developing case-based learning activities based on Bloom’s Taxonomy of thinking skills, suggests that this approach encourages deep learning through critical thinking:

Sherfield (2004) confirms this, asserting that working through case studies can begin to build and expand these six critical thinking strategies:

• Emotional restraint
• Questioning
• Distinguishing fact from fiction
• Searching for ambiguity

What makes a good case?

Case-based learning can focus on anything from a one-sentence physics word problem to a textbook-sized nursing case or a semester-long case in a law course.  Though we often assume that a case is a “problem,” Ellet (2007) suggests that most cases entail one of four types of situations:

• Evaluations
• What are the facts you know about the case?
• What are some logical assumptions you can make about the case?
• What are the problems involved in the case as you see it?
• What is the root problem (the main issue)?
• What do you estimate is the cause of the root problem?
• What are the reasons that the root problem exists?
• What is the solution to the problem?
• Are there any moral or ethical considerations to your solution?
• What are the real-world implications for this case?
• How might the lives of the people in the case study be changed because of your proposed solution?
• Where in your world (campus/town/country) might a problem like this occur?
• Where could someone get help with this problem?
• What personal advice would you give to the person or people concerned?

Adapted from Sherfield’s Case Studies for the First Year (2004)

Some faculty buy prepared cases from publishers, but many create their own based on their unique course needs.  When introducing case-based learning to students, be sure to offer a series of guidelines or questions to prompt deep thinking.  One option is to provide a scenario followed by questions; for example, questions designed for a first year experience problem might include these:

Before you begin, take a look at what others are doing with cases in your field.  Pre-made case studies are available from various publishers, and you can find case-study templates online.

• Choose scenarios carefully
• Tell a story from beginning to end, including many details
• Create real-life characters and use quotes when possible
• Write clearly and concisely and format the writing simply
• Ask students to reflect on their learning—perhaps identifying connections between the lesson and specific course learning outcomes—after working a case

Additional Resources

• Barnes, Louis B. et al. Teaching and the Case Method , 3 rd (1994). Harvard, 1994.
• Campoy, Renee. Case Study Analysis in the Classroom: Becoming a Reflective Teacher . Sage Publications, 2005.
• Ellet, William. The Case Study Handbook . Harvard, 2007.
• Herreid, Clyde Freeman, ed. Start with a Story: The Case Study Method of Teaching College Science . NSTA, 2007.
• Herreid, Clyde Freeman, et al. Science Stories: Using Case Studies to Teach Critical Thinking . NSTA, 2012.
• Nkhoma, M., Lam, et al. Developing case-based learning activities based on the revised Bloom’s Taxonomy . Proceedings of Informing Science & IT Education Conference (In SITE) 2016, 85-93. 2016.
• Rolls, Geoff. Classic Case Studies in Psychology , 3 rd Hodder Education, Bookpoint, 2014.
• Sherfield, Robert M., et al. Case Studies for the First Year . Pearson, 2004.
• Shulman, Judith H., ed. Case Methods in Teacher Education . Teacher’s College, 1992.

Quick Links

• Developing Learning Objectives
• Creating Your Syllabus
• Active Learning
• Service Learning
• Critical Thinking and other Higher-Order Thinking Skills
• Group and Team Based Learning
• Integrating Technology in the Classroom
• Effective PowerPoint Design
• Hybrid and Hybrid Limited Course Design
• Online Course Design

Consult with our CETL Professionals

Consultation services are available to all UConn faculty at all campuses at no charge.

Search form

• About Faculty Development and Support
• Programs and Funding Opportunities

Consultations, Observations, and Services

• Strategic Resources & Digital Publications
• Canvas @ Yale Support
• Learning Environments @ Yale
• Teaching Workshops
• Teaching Consultations and Classroom Observations
• Teaching Programs
• Spring Teaching Forum
• Written and Oral Communication Workshops and Panels
• Writing Resources & Tutorials
• About the Graduate Writing Laboratory
• Writing and Public Speaking Consultations
• Writing Workshops and Panels
• Writing Peer-Review Groups
• Writing Retreats and All Writes
• Online Writing Resources for Graduate Students
• About Teaching Development for Graduate and Professional School Students
• Teaching Programs and Grants
• Teaching Forums
• Resources for Graduate Student Teachers
• About Undergraduate Writing and Tutoring
• Academic Strategies Program
• The Writing Center
• STEM Tutoring & Programs
• Humanities & Social Sciences
• Center for Language Study
• Online Course Catalog
• Antiracist Pedagogy
• NECQL 2019: NorthEast Consortium for Quantitative Literacy XXII Meeting
• STEMinar Series
• Teaching in Context: Troubling Times
• Helmsley Postdoctoral Teaching Scholars
• Pedagogical Partners
• Instructional Materials
• Evaluation & Research
• STEM Education Job Opportunities
• Yale Connect
• Online Education Legal Statements

You are here

Case-based learning.

Case-based learning (CBL) is an established approach used across disciplines where students apply their knowledge to real-world scenarios, promoting higher levels of cognition (see Bloom’s Taxonomy ). In CBL classrooms, students typically work in groups on case studies, stories involving one or more characters and/or scenarios.  The cases present a disciplinary problem or problems for which students devise solutions under the guidance of the instructor. CBL has a strong history of successful implementation in medical, law, and business schools, and is increasingly used within undergraduate education, particularly within pre-professional majors and the sciences (Herreid, 1994). This method involves guided inquiry and is grounded in constructivism whereby students form new meanings by interacting with their knowledge and the environment (Lee, 2012).

There are a number of benefits to using CBL in the classroom. In a review of the literature, Williams (2005) describes how CBL: utilizes collaborative learning, facilitates the integration of learning, develops students’ intrinsic and extrinsic motivation to learn, encourages learner self-reflection and critical reflection, allows for scientific inquiry, integrates knowledge and practice, and supports the development of a variety of learning skills.

CBL has several defining characteristics, including versatility, storytelling power, and efficient self-guided learning.  In a systematic analysis of 104 articles in health professions education, CBL was found to be utilized in courses with less than 50 to over 1000 students (Thistlethwaite et al., 2012). In these classrooms, group sizes ranged from 1 to 30, with most consisting of 2 to 15 students.  Instructors varied in the proportion of time they implemented CBL in the classroom, ranging from one case spanning two hours of classroom time, to year-long case-based courses. These findings demonstrate that instructors use CBL in a variety of ways in their classrooms.

The stories that comprise the framework of case studies are also a key component to CBL’s effectiveness. Jonassen and Hernandez-Serrano (2002, p.66) describe how storytelling:

Is a method of negotiating and renegotiating meanings that allows us to enter into other’s realms of meaning through messages they utter in their stories,

Helps us find our place in a culture,

Allows us to explicate and to interpret, and

Facilitates the attainment of vicarious experience by helping us to distinguish the positive models to emulate from the negative model.

Neurochemically, listening to stories can activate oxytocin, a hormone that increases one’s sensitivity to social cues, resulting in more empathy, generosity, compassion and trustworthiness (Zak, 2013; Kosfeld et al., 2005). The stories within case studies serve as a means by which learners form new understandings through characters and/or scenarios.

CBL is often described in conjunction or in comparison with problem-based learning (PBL). While the lines are often confusingly blurred within the literature, in the most conservative of definitions, the features distinguishing the two approaches include that PBL involves open rather than guided inquiry, is less structured, and the instructor plays a more passive role. In PBL multiple solutions to the problem may exit, but the problem is often initially not well-defined. PBL also has a stronger emphasis on developing self-directed learning. The choice between implementing CBL versus PBL is highly dependent on the goals and context of the instruction.  For example, in a comparison of PBL and CBL approaches during a curricular shift at two medical schools, students and faculty preferred CBL to PBL (Srinivasan et al., 2007). Students perceived CBL to be a more efficient process and more clinically applicable. However, in another context, PBL might be the favored approach.

In a review of the effectiveness of CBL in health profession education, Thistlethwaite et al. (2012), found several benefits:

Students enjoyed the method and thought it enhanced their learning,

Instructors liked how CBL engaged students in learning,

CBL seemed to facilitate small group learning, but the authors could not distinguish between whether it was the case itself or the small group learning that occurred as facilitated by the case.

Other studies have also reported on the effectiveness of CBL in achieving learning outcomes (Bonney, 2015; Breslin, 2008; Herreid, 2013; Krain, 2016). These findings suggest that CBL is a vehicle of engagement for instruction, and facilitates an environment whereby students can construct knowledge.

Science – Students are given a scenario to which they apply their basic science knowledge and problem-solving skills to help them solve the case. One example within the biological sciences is two brothers who have a family history of a genetic illness. They each have mutations within a particular sequence in their DNA. Students work through the case and draw conclusions about the biological impacts of these mutations using basic science. Sample cases: You are Not the Mother of Your Children ; Organic Chemisty and Your Cellphone: Organic Light-Emitting Diodes ;   A Light on Physics: F-Number and Exposure Time

Medicine – Medical or pre-health students read about a patient presenting with specific symptoms. Students decide which questions are important to ask the patient in their medical history, how long they have experienced such symptoms, etc. The case unfolds and students use clinical reasoning, propose relevant tests, develop a differential diagnoses and a plan of treatment. Sample cases: The Case of the Crying Baby: Surgical vs. Medical Management ; The Plan: Ethics and Physician Assisted Suicide ; The Haemophilus Vaccine: A Victory for Immunologic Engineering

Public Health – A case study describes a pandemic of a deadly infectious disease. Students work through the case to identify Patient Zero, the person who was the first to spread the disease, and how that individual became infected.  Sample cases: The Protective Parent ; The Elusive Tuberculosis Case: The CDC and Andrew Speaker ; Credible Voice: WHO-Beijing and the SARS Crisis

Law – A case study presents a legal dilemma for which students use problem solving to decide the best way to advise and defend a client. Students are presented information that changes during the case.  Sample cases: Mortgage Crisis Call (abstract) ; The Case of the Unpaid Interns (abstract) ; Police-Community Dialogue (abstract)

Business – Students work on a case study that presents the history of a business success or failure. They apply business principles learned in the classroom and assess why the venture was successful or not. Sample cases: SELCO-Determining a path forward ; Project Masiluleke: Texting and Testing to Fight HIV/AIDS in South Africa ; Mayo Clinic: Design Thinking in Healthcare

Humanities - Students consider a case that presents a theater facing financial and management difficulties. They apply business and theater principles learned in the classroom to the case, working together to create solutions for the theater. Sample cases: David Geffen School of Drama

Recommendations

Finding and Writing Cases

Consider utilizing or adapting open access cases - The availability of open resources and databases containing cases that instructors can download makes this approach even more accessible in the classroom. Two examples of open databases are the Case Center on Public Leadership and Harvard Kennedy School (HKS) Case Program , which focus on government, leadership and public policy case studies.

• Consider writing original cases - In the event that an instructor is unable to find open access cases relevant to their course learning objectives, they may choose to write their own. See the following resources on case writing: Cooking with Betty Crocker: A Recipe for Case Writing ; The Way of Flesch: The Art of Writing Readable Cases ;   Twixt Fact and Fiction: A Case Writer’s Dilemma ; And All That Jazz: An Essay Extolling the Virtues of Writing Case Teaching Notes .

Implementing Cases

Take baby steps if new to CBL - While entire courses and curricula may involve case-based learning, instructors who desire to implement on a smaller-scale can integrate a single case into their class, and increase the number of cases utilized over time as desired.

Use cases in classes that are small, medium or large - Cases can be scaled to any course size. In large classes with stadium seating, students can work with peers nearby, while in small classes with more flexible seating arrangements, teams can move their chairs closer together. CBL can introduce more noise (and energy) in the classroom to which an instructor often quickly becomes accustomed. Further, students can be asked to work on cases outside of class, and wrap up discussion during the next class meeting.

Encourage collaborative work - Cases present an opportunity for students to work together to solve cases which the historical literature supports as beneficial to student learning (Bruffee, 1993). Allow students to work in groups to answer case questions.

Form diverse teams as feasible - When students work within diverse teams they can be exposed to a variety of perspectives that can help them solve the case. Depending on the context of the course, priorities, and the background information gathered about the students enrolled in the class, instructors may choose to organize student groups to allow for diversity in factors such as current course grades, gender, race/ethnicity, personality, among other items.  

Use stable teams as appropriate - If CBL is a large component of the course, a research-supported practice is to keep teams together long enough to go through the stages of group development: forming, storming, norming, performing and adjourning (Tuckman, 1965).

Walk around to guide groups - In CBL instructors serve as facilitators of student learning. Walking around allows the instructor to monitor student progress as well as identify and support any groups that may be struggling. Teaching assistants can also play a valuable role in supporting groups.

Interrupt strategically - Only every so often, for conversation in large group discussion of the case, especially when students appear confused on key concepts. An effective practice to help students meet case learning goals is to guide them as a whole group when the class is ready. This may include selecting a few student groups to present answers to discussion questions to the entire class, asking the class a question relevant to the case using polling software, and/or performing a mini-lesson on an area that appears to be confusing among students.  

Assess student learning in multiple ways - Students can be assessed informally by asking groups to report back answers to various case questions. This practice also helps students stay on task, and keeps them accountable. Cases can also be included on exams using related scenarios where students are asked to apply their knowledge.

Barrows HS. (1996). Problem-based learning in medicine and beyond: a brief overview. New Directions for Teaching and Learning, 68, 3-12.  

Bonney KM. (2015). Case Study Teaching Method Improves Student Performance and Perceptions of Learning Gains. Journal of Microbiology and Biology Education, 16(1): 21-28.

Breslin M, Buchanan, R. (2008) On the Case Study Method of Research and Teaching in Design.  Design Issues, 24(1), 36-40.

Bruffee KS. (1993). Collaborative learning: Higher education, interdependence, and authority of knowledge. Johns Hopkins University Press, Baltimore, MD.

Herreid CF. (2013). Start with a Story: The Case Study Method of Teaching College Science, edited by Clyde Freeman Herreid. Originally published in 2006 by the National Science Teachers Association (NSTA); reprinted by the National Center for Case Study Teaching in Science (NCCSTS) in 2013.

Herreid CH. (1994). Case studies in science: A novel method of science education. Journal of Research in Science Teaching, 23(4), 221–229.

Jonassen DH and Hernandez-Serrano J. (2002). Case-based reasoning and instructional design: Using stories to support problem solving. Educational Technology, Research and Development, 50(2), 65-77.  

Kosfeld M, Heinrichs M, Zak PJ, Fischbacher U, Fehr E. (2005). Oxytocin increases trust in humans. Nature, 435, 673-676.

Krain M. (2016) Putting the learning in case learning? The effects of case-based approaches on student knowledge, attitudes, and engagement. Journal on Excellence in College Teaching, 27(2), 131-153.

Lee V. (2012). What is Inquiry-Guided Learning?  New Directions for Learning, 129:5-14.

Nkhoma M, Sriratanaviriyakul N. (2017). Using case method to enrich students’ learning outcomes. Active Learning in Higher Education, 18(1):37-50.

Srinivasan et al. (2007). Comparing problem-based learning with case-based learning: Effects of a major curricular shift at two institutions. Academic Medicine, 82(1): 74-82.

Thistlethwaite JE et al. (2012). The effectiveness of case-based learning in health professional education. A BEME systematic review: BEME Guide No. 23.  Medical Teacher, 34, e421-e444.

Tuckman B. (1965). Development sequence in small groups. Psychological Bulletin, 63(6), 384-99.

Williams B. (2005). Case-based learning - a review of the literature: is there scope for this educational paradigm in prehospital education? Emerg Med, 22, 577-581.

Zak, PJ (2013). How Stories Change the Brain. Retrieved from: https://greatergood.berkeley.edu/article/item/how_stories_change_brain

YOU MAY BE INTERESTED IN

The Poorvu Center for Teaching and Learning routinely supports members of the Yale community with individual instructional consultations and classroom observations.

Instructional Enhancement Fund

The Instructional Enhancement Fund (IEF) awards grants of up to $500 to support the timely integration of new learning activities into an existing undergraduate or graduate course. All Yale instructors of record, including tenured and tenure-track faculty, clinical instructional faculty, lecturers, lectors, and part-time acting instructors (PTAIs), are eligible to apply. Award decisions are typically provided within two weeks to help instructors implement ideas for the current semester.

Reserve a Room

The Poorvu Center for Teaching and Learning partners with departments and groups on-campus throughout the year to share its space. Please review the reservation form and submit a request.

Case Study Teaching and Learning

• First Online: 09 July 2020

Cite this chapter

• Riann Singh 2 &
• Shalini Ramdeo 2  

2179 Accesses

This chapter is the first of three chapters that explore case studies in OD. This chapter presents the fundamentals of case method teaching and learning. It is important to understand such fundamentals before presenting OD cases to readers. In OD and other related fields, the case approach is widely used as a pedagogy for learning by making decisions on information about an issue or problem. Case method teaching and learning strategies attempt to bridge the gap between theoretical and practical applications in any field of study. The chapter also presents the reader with the basics of case method approaches, provides an explanation of its importance in OD, describes how students should approach case method learning, and outlines how they can approach case analysis and discussion.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

• Available as PDF
• Read on any device
• Instant download
• Own it forever
• Available as EPUB and PDF
• Compact, lightweight edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Burgoyne, J., and A. Mumford. 2001. Learning from the case method: A report to the European case. In Clearing house . Cranfield: ECCH.

Google Scholar  

McDade, S.A. 1995. Case study pedagogy to advance critical thinking. Teaching of Psychology 22 (1): 9–10.

Article   Google Scholar  

Geering, J. 2004. What is a case study and what is it good for? American Political Science Review 98 (2): 341–354.

———. 2001. Learning from the case method: A report to the European case. In Clearing house . Cranfield: ECCH.

Golich, V.L., M. Boyer, P. Franko, and S. Lamy. 2000. The ABCs of case teaching. Pew Case Studies in International Affairs .

Bloom, B.S. 1956. Taxonomy of educational objectives: The classification of educational goals . New York, NY: Longmans, Green.

Pun, Hubert. Ivey publishing case teaching and writing workshop 2018. The University of the West Indies-Arthur Lok Jack Global School of Business.

Further Readings

Download references

Author information

Authors and affiliations.

The University of the West Indies, St. Augustine Campus, Trinidad, Saint Kitts and Nevis

Riann Singh & Shalini Ramdeo

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Riann Singh .

Rights and permissions

Reprints and permissions

Copyright information

© 2020 The Author(s)

About this chapter

Singh, R., Ramdeo, S. (2020). Case Study Teaching and Learning. In: Leading Organizational Development and Change. Palgrave Macmillan, Cham. https://doi.org/10.1007/978-3-030-39123-2_21

Download citation

DOI : https://doi.org/10.1007/978-3-030-39123-2_21

Published : 09 July 2020

Publisher Name : Palgrave Macmillan, Cham

Print ISBN : 978-3-030-39122-5

Online ISBN : 978-3-030-39123-2

eBook Packages : Business and Management Business and Management (R0)

Share this chapter

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

• Illinois Online
• Illinois Remote

• TA Resources
• Teaching Consultation
• Teaching Portfolio Program
• Grad Academy for College Teaching
• Faculty Events
• The Art of Teaching
• 2022 Illinois Summer Teaching Institute
• Large Classes
• Leading Discussions
• Laboratory Classes
• Lecture-Based Classes
• Planning a Class Session
• Questioning Strategies
• Classroom Assessment Techniques (CATs)
• Problem-Based Learning (PBL)
• The Case Method
• Community-Based Learning: Service Learning
• Group Learning
• Just-in-Time Teaching
• Creating a Syllabus
• Motivating Students
• Dealing With Cheating
• Discouraging & Detecting Plagiarism
• Diversity & Creating an Inclusive Classroom
• Harassment & Discrimination
• Professional Conduct
• Foundations of Good Teaching
• Student Engagement
• Assessment Strategies
• Course Design
• Student Resources
• Teaching Tips
• Graduate Teacher Certificate
• Certificate in Foundations of Teaching
• Teacher Scholar Certificate
• Certificate in Technology-Enhanced Teaching
• Master Course in Online Teaching (MCOT)
• 2022 Celebration of College Teaching
• 2023 Celebration of College Teaching
• Hybrid Teaching and Learning Certificate
• Classroom Observation Etiquette
• Teaching Philosophy Statement
• Pedagogical Literature Review
• Scholarship of Teaching and Learning
• Instructor Stories
• Podcast: Teach Talk Listen Learn
• Universal Design for Learning

Sign-Up to receive Teaching and Learning news and events

Cases are narratives, situations, select data samplings, or statements that present unresolved and provocative issues, situations, or questions (Indiana University Teaching Handbook, 2005). The case method is a participatory, discussion-based way of learning where students gain skills in critical thinking, communication, and group dynamics. It is a type of problem-based learning . Often seen in the professional schools of medicine, law, and business, the case method is now used successfully in disciplines such as engineering, chemistry, education, and journalism. Students can work through a case during class as a whole or in small groups.

In addition to the definition above, the case method of teaching (or learning):

• Is a partnership between students and teacher as well as among students.
• Promotes more effective contextual learning and long-term retention.
• Involves trust that students will find the answers.
• Answers questions not only of “how” but “why.”
• Provides students the opportunity to “walk around the problem” and to see varied perspectives.

(Bruner, 2002, and Christensen, Garvin, and Sweet, 1991)

What is the value of the case method?

Bruner (1991) states that the case method:

• Is effective: It employs active learning, involves self-discovery where the teacher serves as facilitator.
• Builds the capacity for critical thinking: It uses questioning skills as modeled by the teacher and employs discussion and debates.
• Exercises an administrative point of view: Students must develop a framework for making decisions.
• Models a learning environment: It offers an exchange and flow of ideas from one person to another and achieves trust, respect, and risk-taking.
• Models the process of inductive learning-from-experience: It is valuable in promoting life-long learning. It also promotes more effective contextual learning and long-term retention.
• Mimics the real world: Decisions are sometimes based not on absolute values of right and wrong, but on relative values and uncertainty.

What are some ways to use the case method appropriately?

Choose an appropriate case

Cases can be any of the following (Indiana University Teaching Handbook, 2005):

• Finished cases based on facts; these are useful for purposes of analysis.
• Unfinished open-ended cases; where the results are not clear yet, so the student must predict, make suggestions, and conclusions.
• Fictional cases that the teacher writes; the difficulty is in writing these cases so they reflect a real-world situation.
• Original documents, such as the use of news articles, reports, data sets, ethnographies; an interesting case would be to provide two sides of a scenario.

Develop effective questions

Think about ways to start the discussion such as using a hypothetical example or employing the background knowledge of your students.

Get students prepared

To prepare for the next class ask students to think about the following questions:

• What is the problem or decision?
• Who is the key decision-maker?
• Who are the other people involved?
• What caused the problem?
• What are some underlying assumptions or objectives?
• What decision needs to be made?
• Are there alternative responses?

Set ground rules with your students

For effective class discussion suggest the following to your students:

• Carefully listen to the discussion, but do not wait too long to participate.
• Collaboration and respect should always be present.
• Provide value-added comments, suggestions, or questions. Strive to think of the class objective by keeping the discussion going toward constructive inquiry and solutions.

Other suggestions

• Try to refrain from being the “sage on the stage” or a monopolizer. If you are, students are merely absorbing and not engaging with the material in the way that the case method allows.
• Make sure the students have finished presenting their perspective before interjecting. Wait and check their body language before adding or changing the discussion.
• Take note of the progress and the content in the discussion. One way is by using the board or computer to structure the comments. Another way, particularly useful where there is a conflict or multiple alternatives, is the two-column method. In this method, the teacher makes two columns: “For and Against” or “Alternative A and Alternative B.” All arguments/comments are listed in the respective column before discussions or evaluations occur. Don't forget to note supportive evidence.
• In addition to the discussion method, you can also try debates, role-plays, and simulations as ways to uncover the lesson from the case.
• If you decide to grade participation, make sure that your grading system is an accurate and defensible portrayal of the contributions.

In conclusion, cases are a valuable way for learning to occur. It takes a fair amount of preparation by both the teacher and the students, but don't forget these benefits (Bruner, 2002):

• The teacher is learning as well as the students. Because of the interactive nature of this method, the teacher constantly “encounters fresh perspective on old problems or tests classic solutions to new problems.”
• The students are having fun, are motivated and engaged. If done well, the students are working collaboratively to support each other.

Where can I learn more?

• Case Studies, Center for Teaching, Vanderbilt University
• Case-based Teaching, Center for Research on Teaching and Learning, University of Michigan
• Barnes, L. B., Christensen, C. R., & Hansen, A. J. (1994). Teaching and the case method (3rd ed.). Boston: Harvard Business School Press.
• Boehrer, J., & Linsky, M. (1990). Teaching with cases: Learning to question. In M. D. Svinicki (Ed.), New Directions for Teaching and Learning: No. 42, The changing face of college teaching . San Francisco: Jossey-Bass.
• Bruner, R. (2002). Socrates' muse: Reflections on effective case discussion leadership . New York: McGraw-Hill.
• Christensen, C. R., Garvin, D. A., & Sweet, A. (Eds.). (1991). Education for judgment: The artistry of discussion leadership . Boston: Harvard Business School Press.
• Indiana University, Bloomington, Campus Instructional Consulting. (n.d.). Teaching with the case method. In Indiana University Teaching Handbook . Retrieved June 23, 2010, from http://www.teaching.iub.edu/wrapper_big.php?section_id=case
• Mitchell, T., & Rosenstiel, T. (2003). Background and tips for case study teaching . Retrieved June 23, 2010, from http://www.journalism.org/node/1757

Center for Innovation in Teaching & Learning

249 Armory Building 505 East Armory Avenue Champaign, IL 61820

217 333-1462

Email: [email protected]

Office of the Provost

• Harvard Business School →
• Case Method Project →

Case Method Teaching

• Teacher Workshops
• Student Experience
• Case Method Teaching →

What is the Case Method?

The core pedagogy of Harvard Business School since the early 20th century, the case method boasts a unique ability to make complex concepts accessible and develop students’ leadership skills, all while creating an engaging intellectual atmosphere.

A “case” is a short narrative document – a story – that presents a particular challenge facing an individual or organization. Each case reflects the information available to decision-makers at the time, and builds to a particular decision point, but without revealing what decision was actually made. For each class, students are asked to read the case and to put themselves in the shoes of the actual decision-makers to consider what they themselves would have done given the information available at the time.

To help situate the decision, each case also provides background history on the most relevant issues and events leading up to the decision point. For example, a case that focuses on a decision facing Martin Luther King Jr. during the campaign for black voting rights in 1965 also traces the broader civil rights movement, as well as the history of segregation and black disenfranchisement in the U.S. since the Civil War.

In the classroom, the instructor poses questions to guide student discussion. These carefully designed questions are the key to any successful case-method course. An experienced case-method teacher is often thinking several steps ahead, anticipating what points and questions might be raised and standing ready with follow-up questions to guide the group. Active participation in class is essential to the case method’s success, and the grading metrics reflect its importance. Students quickly learn to speak up, to challenge each other, and to build on each other’s ideas.

Any professor or teacher can teach by the case method. Content expertise beyond the case itself is helpful but not required. To assist both experienced and new case-method teachers, each case has its own teaching plan: a flexible road-map for the instructor that suggests specific questions, strategies for eliciting key insights, and ideas for organizing student responses visually on a blackboard. For some cases, more extensive supporting documents known as “teaching notes” are available to fully brief the instructor on the relevant history and the multiple levels of questions he or she might consider.

“ I’ve never had this experience as a teacher before, and it's explicitly due to the case method — it's a game changer. ”

• Our Mission

Making Learning Relevant With Case Studies

The open-ended problems presented in case studies give students work that feels connected to their lives.

To prepare students for jobs that haven’t been created yet, we need to teach them how to be great problem solvers so that they’ll be ready for anything. One way to do this is by teaching content and skills using real-world case studies, a learning model that’s focused on reflection during the problem-solving process. It’s similar to project-based learning, but PBL is more focused on students creating a product.

Case studies have been used for years by businesses, law and medical schools, physicians on rounds, and artists critiquing work. Like other forms of problem-based learning, case studies can be accessible for every age group, both in one subject and in interdisciplinary work.

You can get started with case studies by tackling relatable questions like these with your students:

• How can we limit food waste in the cafeteria?
• How can we get our school to recycle and compost waste? (Or, if you want to be more complex, how can our school reduce its carbon footprint?)
• How can we improve school attendance?
• How can we reduce the number of people who get sick at school during cold and flu season?

Addressing questions like these leads students to identify topics they need to learn more about. In researching the first question, for example, students may see that they need to research food chains and nutrition. Students often ask, reasonably, why they need to learn something, or when they’ll use their knowledge in the future. Learning is most successful for students when the content and skills they’re studying are relevant, and case studies offer one way to create that sense of relevance.

Teaching With Case Studies

Ultimately, a case study is simply an interesting problem with many correct answers. What does case study work look like in classrooms? Teachers generally start by having students read the case or watch a video that summarizes the case. Students then work in small groups or individually to solve the case study. Teachers set milestones defining what students should accomplish to help them manage their time.

During the case study learning process, student assessment of learning should be focused on reflection. Arthur L. Costa and Bena Kallick’s Learning and Leading With Habits of Mind gives several examples of what this reflection can look like in a classroom: 

Journaling: At the end of each work period, have students write an entry summarizing what they worked on, what worked well, what didn’t, and why. Sentence starters and clear rubrics or guidelines will help students be successful. At the end of a case study project, as Costa and Kallick write, it’s helpful to have students “select significant learnings, envision how they could apply these learnings to future situations, and commit to an action plan to consciously modify their behaviors.”

Interviews: While working on a case study, students can interview each other about their progress and learning. Teachers can interview students individually or in small groups to assess their learning process and their progress.

Student discussion: Discussions can be unstructured—students can talk about what they worked on that day in a think-pair-share or as a full class—or structured, using Socratic seminars or fishbowl discussions. If your class is tackling a case study in small groups, create a second set of small groups with a representative from each of the case study groups so that the groups can share their learning.

4 Tips for Setting Up a Case Study

1. Identify a problem to investigate: This should be something accessible and relevant to students’ lives. The problem should also be challenging and complex enough to yield multiple solutions with many layers.

2. Give context: Think of this step as a movie preview or book summary. Hook the learners to help them understand just enough about the problem to want to learn more.

3. Have a clear rubric: Giving structure to your definition of quality group work and products will lead to stronger end products. You may be able to have your learners help build these definitions.

4. Provide structures for presenting solutions: The amount of scaffolding you build in depends on your students’ skill level and development. A case study product can be something like several pieces of evidence of students collaborating to solve the case study, and ultimately presenting their solution with a detailed slide deck or an essay—you can scaffold this by providing specified headings for the sections of the essay.

Problem-Based Teaching Resources

There are many high-quality, peer-reviewed resources that are open source and easily accessible online.

• The National Center for Case Study Teaching in Science at the University at Buffalo built an online collection of more than 800 cases that cover topics ranging from biochemistry to economics. There are resources for middle and high school students.
• Models of Excellence , a project maintained by EL Education and the Harvard Graduate School of Education, has examples of great problem- and project-based tasks—and corresponding exemplary student work—for grades pre-K to 12.
• The Interdisciplinary Journal of Problem-Based Learning at Purdue University is an open-source journal that publishes examples of problem-based learning in K–12 and post-secondary classrooms.
• The Tech Edvocate has a list of websites and tools related to problem-based learning.

In their book Problems as Possibilities , Linda Torp and Sara Sage write that at the elementary school level, students particularly appreciate how they feel that they are taken seriously when solving case studies. At the middle school level, “researchers stress the importance of relating middle school curriculum to issues of student concern and interest.” And high schoolers, they write, find the case study method “beneficial in preparing them for their future.”

What is the Case Study Method?

Overview Dropdown up

Overview dropdown down, celebrating 100 years of the case method at hbs.

The 2021-2022 academic year marks the 100-year anniversary of the introduction of the case method at Harvard Business School. Today, the HBS case method is employed in the HBS MBA program, in Executive Education programs, and in dozens of other business schools around the world. As Dean Srikant Datar's says, the case method has withstood the test of time.

Case Discussion Preparation Details Expand All Collapse All

In self-reflection in self-reflection dropdown down, in a small group setting in a small group setting dropdown down, in the classroom in the classroom dropdown down, beyond the classroom beyond the classroom dropdown down, how the case method creates value dropdown up, how the case method creates value dropdown down, in self-reflection, in a small group setting, in the classroom, beyond the classroom.

How Cases Unfold In the Classroom

How cases unfold in the classroom dropdown up, how cases unfold in the classroom dropdown down, preparation guidelines expand all collapse all, read the professor's assignment or discussion questions read the professor's assignment or discussion questions dropdown down, read the first few paragraphs and then skim the case read the first few paragraphs and then skim the case dropdown down, reread the case, underline text, and make margin notes reread the case, underline text, and make margin notes dropdown down, note the key problems on a pad of paper and go through the case again note the key problems on a pad of paper and go through the case again dropdown down, how to prepare for case discussions dropdown up, how to prepare for case discussions dropdown down, read the professor's assignment or discussion questions, read the first few paragraphs and then skim the case, reread the case, underline text, and make margin notes, note the key problems on a pad of paper and go through the case again, case study best practices expand all collapse all, prepare prepare dropdown down, discuss discuss dropdown down, participate participate dropdown down, relate relate dropdown down, apply apply dropdown down, note note dropdown down, understand understand dropdown down, case study best practices dropdown up, case study best practices dropdown down, participate, what can i expect on the first day dropdown down.

Most programs begin with registration, followed by an opening session and a dinner. If your travel plans necessitate late arrival, please be sure to notify us so that alternate registration arrangements can be made for you. Please note the following about registration:

HBS campus programs – Registration takes place in the Chao Center.

India programs – Registration takes place outside the classroom.

Other off-campus programs – Registration takes place in the designated facility.

What happens in class if nobody talks? Dropdown down

Professors are here to push everyone to learn, but not to embarrass anyone. If the class is quiet, they'll often ask a participant with experience in the industry in which the case is set to speak first. This is done well in advance so that person can come to class prepared to share. Trust the process. The more open you are, the more willing you’ll be to engage, and the more alive the classroom will become.

Does everyone take part in "role-playing"? Dropdown down

Professors often encourage participants to take opposing sides and then debate the issues, often taking the perspective of the case protagonists or key decision makers in the case.

View Frequently Asked Questions

Subscribe to Our Emails

• Utility Menu

harvardchan_logo.png

Harvard T.H. Chan School of Public Health Case-Based Teaching & Learning Initiative

Teaching cases & active learning resources for public health education, teaching & learning with the case method.

2023. Case Compendium, University of California Berkeley Haas School of Business Center for Equity, Gender & Leadership . Visit website This resource, compiled by the Berkeley Haas Center for Equity, Gender & Leadership, is "a case compendium that includes: (a) case studies with diverse protagonists, and (b) case studies that build “equity fluency” by focusing on DEI-related issues and opportunities. The goal of the compendium is to support professors at Haas, and business schools globally, to identify cases they can use in their own classrooms, and ultimately contribute to advancing DEI in education and business."

Kane, N.M. , 2014. Benefits of Case-Based Teaching . Watch video Watch a demonstration of Prof. Nancy Kane teaching public health with the case method. (Part 3 of 3, 3 minutes)

Kane, N.M. , 2014. Case teaching demonstration: Should a health plan cover medical tourism? . Watch video Watch a demonstration of Prof. Nancy Kane teaching public health with the case method. (Part 2 of 3, 17 minutes)

Kane, N.M. , 2014. Case-based teaching at the Harvard T.H. Chan School of Public Health . Watch video Watch a demonstration of Prof. Nancy Kane teaching public health with the case method. (Part 1 of 3, 10 minutes)

2019. The Case Centre . Visit website A non-profit clearing house for materials on the case method, the Case Centre holds a large and diverse collection of cases, articles, book chapters and teaching materials, including the collections of leading business schools across the globe.

Austin, S.B. & Sonneville, K.R. , 2013. Closing the "know-do" gap: training public health professionals in eating disorders prevention via case-method teaching. International Journal of Eating Disorders , 46 (5) , pp. 533-537. Read online Abstract Expansion of our societies' capacity to prevent eating disorders will require strategic integration of the topic into the curricula of professional training programs. An ideal way to integrate new content into educational programs is through the case-method approach, a teaching method that is more effective than traditional teaching techniques. The Strategic Training Initiative for the Prevention of Eating Disorders has begun developing cases designed to be used in classroom settings to engage students in topical, high-impact issues in public health approaches to eating disorders prevention and screening. Dissemination of these cases will provide an opportunity for students in public health training programs to learn material in a meaningful context by actively applying skills as they are learning them, helping to bridge the "know-do" gap. The new curriculum is an important step toward realizing the goal that public health practitioners be fully equipped to address the challenge of eating disorders prevention. "Expansion of our societies' capacity to prevent eating disorders will require strategic integration of the topic into the curricula of professional training programs. An ideal way to integrate new content into educational programs is through the case-method approach, a teaching method that is more effective than traditional teaching techniques." Access full article with HarvardKey . 

Ellet, W. , 2018. The Case Study Handbook, Revised Edition: A Student's Guide , Harvard Business School Publishing. Publisher's Version "If you're like many people, you may find interpreting and writing about cases mystifying and time-consuming. In The Case Study Handbook, Revised Edition , William Ellet presents a potent new approach for efficiently analyzing, discussing, and writing about cases."

Andersen, E. & Schiano, B. , 2014. Teaching with Cases: A Practical Guide , Harvard Business School Publishing. Publisher's Version "The class discussion inherent in case teaching is well known for stimulating the development of students' critical thinking skills, yet instructors often need guidance on managing that class discussion to maximize learning. Teaching with Cases focuses on practical advice for instructors that can be easily implemented. It covers how to plan a course, how to teach it, and how to evaluate it." 

Honan, J. & Sternman Rule, C. , 2002. Case Method Instruction Versus Lecture-Based Instruction R. Reis, ed. Tomorrow's Professor . Read online "Faculty and discussion leaders who incorporate the case study method into their teaching offer various reasons for their enthusiasm for this type of pedagogy over more traditional, such as lecture-based, instructional methods and routes to learning." Exerpt from the book Using Cases in Higher Education: A Guide for Faculty and Administrators , by James P. Honan and Cheryl Sternman Rule.

Austin, J. , 1993. Teaching Notes: Communicating the Teacher's Wisdom , Harvard Business School Publishing. Publisher's Version "Provides guidance for the preparation of teaching notes. Sets forth the rationale for teaching notes, what they should contain and why, and how they can be prepared. Based on the experiences of Harvard Business School faculty."

Abell, D. , 1997. What makes a good case? . ECCHO–The Newsletter of the European Case Clearing House , 17 (1) , pp. 4-7. Read online "Case writing is both art and science. There are few, if any, specific prescriptions or recipes, but there are key ingredients that appear to distinguish excellent cases from the run-of-the-mill. This technical note lists ten ingredients to look for if you are teaching somebody else''s case - and to look out for if you are writing it yourself."

Herreid, C.F. , 2001. Don't! What not to do when teaching cases. Journal of College Science Teaching , 30 (5) , pp. 292. Read online "Be warned, I am about to unleash a baker’s dozen of 'don’ts' for aspiring case teachers willing to try running a classroom discussion armed with only a couple of pages of a story and a lot of chutzpah."

Garvin, D.A. , 2003. Making the case: Professional education for the world of practice . Harvard Magazine , 106 (1) , pp. 56-65. Read online A history and overview of the case-method in professional schools, which all “face the same difficult challenge: how to prepare students for the world of practice. Time in the classroom must somehow translate directly into real-world activity: how to diagnose, decide, and act."

• Writing a case (8)
• Writing a teaching note (4)
• Active learning (12)
• Active listening (1)
• Asking effective questions (5)
• Assessing learning (1)
• Engaging students (5)
• Leading discussion (10)
• Managing the classroom (4)
• Planning a course (1)
• Problem-based learning (1)
• Teaching & learning with the case method (14)
• Teaching inclusively (3)

Using Case Studies to Teach

Why Use Cases?

Many students are more inductive than deductive reasoners, which means that they learn better from examples than from logical development starting with basic principles. The use of case studies can therefore be a very effective classroom technique.

Case studies are have long been used in business schools, law schools, medical schools and the social sciences, but they can be used in any discipline when instructors want students to explore how what they have learned applies to real world situations. Cases come in many formats, from a simple “What would you do in this situation?” question to a detailed description of a situation with accompanying data to analyze. Whether to use a simple scenario-type case or a complex detailed one depends on your course objectives.

Most case assignments require students to answer an open-ended question or develop a solution to an open-ended problem with multiple potential solutions. Requirements can range from a one-paragraph answer to a fully developed group action plan, proposal or decision.

Common Case Elements

Most “full-blown” cases have these common elements:

• A decision-maker who is grappling with some question or problem that needs to be solved.
• A description of the problem’s context (a law, an industry, a family).
• Supporting data, which can range from data tables to links to URLs, quoted statements or testimony, supporting documents, images, video, or audio.

Case assignments can be done individually or in teams so that the students can brainstorm solutions and share the work load.

The following discussion of this topic incorporates material presented by Robb Dixon of the School of Management and Rob Schadt of the School of Public Health at CEIT workshops. Professor Dixon also provided some written comments that the discussion incorporates.

Advantages to the use of case studies in class

A major advantage of teaching with case studies is that the students are actively engaged in figuring out the principles by abstracting from the examples. This develops their skills in:

• Problem solving
• Analytical tools, quantitative and/or qualitative, depending on the case
• Decision making in complex situations
• Coping with ambiguities

Guidelines for using case studies in class

In the most straightforward application, the presentation of the case study establishes a framework for analysis. It is helpful if the statement of the case provides enough information for the students to figure out solutions and then to identify how to apply those solutions in other similar situations. Instructors may choose to use several cases so that students can identify both the similarities and differences among the cases.

Depending on the course objectives, the instructor may encourage students to follow a systematic approach to their analysis.  For example:

• What is the issue?
• What is the goal of the analysis?
• What is the context of the problem?
• What key facts should be considered?
• What alternatives are available to the decision-maker?
• What would you recommend — and why?

An innovative approach to case analysis might be to have students  role-play the part of the people involved in the case. This not only actively engages students, but forces them to really understand the perspectives of the case characters. Videos or even field trips showing the venue in which the case is situated can help students to visualize the situation that they need to analyze.

Accompanying Readings

Case studies can be especially effective if they are paired with a reading assignment that introduces or explains a concept or analytical method that applies to the case. The amount of emphasis placed on the use of the reading during the case discussion depends on the complexity of the concept or method. If it is straightforward, the focus of the discussion can be placed on the use of the analytical results. If the method is more complex, the instructor may need to walk students through its application and the interpretation of the results.

Leading the Case Discussion and Evaluating Performance

Decision cases are more interesting than descriptive ones. In order to start the discussion in class, the instructor can start with an easy, noncontroversial question that all the students should be able to answer readily. However, some of the best case discussions start by forcing the students to take a stand. Some instructors will ask a student to do a formal “open” of the case, outlining his or her entire analysis.  Others may choose to guide discussion with questions that move students from problem identification to solutions.  A skilled instructor steers questions and discussion to keep the class on track and moving at a reasonable pace.

In order to motivate the students to complete the assignment before class as well as to stimulate attentiveness during the class, the instructor should grade the participation—quantity and especially quality—during the discussion of the case. This might be a simple check, check-plus, check-minus or zero. The instructor should involve as many students as possible. In order to engage all the students, the instructor can divide them into groups, give each group several minutes to discuss how to answer a question related to the case, and then ask a randomly selected person in each group to present the group’s answer and reasoning. Random selection can be accomplished through rolling of dice, shuffled index cards, each with one student’s name, a spinning wheel, etc.

Tips on the Penn State U. website: http://tlt.its.psu.edu/suggestions/cases/

If you are interested in using this technique in a science course, there is a good website on use of case studies in the sciences at the University of Buffalo.

Dunne, D. and Brooks, K. (2004) Teaching with Cases (Halifax, NS: Society for Teaching and Learning in Higher Education), ISBN 0-7703-8924-4 (Can be ordered at http://www.bookstore.uwo.ca/ at a cost of $15.00)

• --> Login or Sign Up

The Case Study Teaching Method

It is easy to get confused between the case study method and the case method , particularly as it applies to legal education. The case method in legal education was invented by Christopher Columbus Langdell, Dean of Harvard Law School from 1870 to 1895. Langdell conceived of a way to systematize and simplify legal education by focusing on previous case law that furthered principles or doctrines. To that end, Langdell wrote the first casebook, entitled A Selection of Cases on the Law of Contracts , a collection of settled cases that would illuminate the current state of contract law. Students read the cases and came prepared to analyze them during Socratic question-and-answer sessions in class.

The Harvard Business School case study approach grew out of the Langdellian method. But instead of using established case law, business professors chose real-life examples from the business world to highlight and analyze business principles. HBS-style case studies typically consist of a short narrative (less than 25 pages), told from the point of view of a manager or business leader embroiled in a dilemma. Case studies provide readers with an overview of the main issue; background on the institution, industry, and individuals involved; and the events that led to the problem or decision at hand. Cases are based on interviews or public sources; sometimes, case studies are disguised versions of actual events or composites based on the faculty authors’ experience and knowledge of the subject. Cases are used to illustrate a particular set of learning objectives; as in real life, rarely are there precise answers to the dilemma at hand.

Our suite of free materials offers a great introduction to the case study method. We also offer review copies of our products free of charge to educators and staff at degree-granting institutions.

For more information on the case study teaching method, see:

• Martha Minow and Todd Rakoff: A Case for Another Case Method
• HLS Case Studies Blog: Legal Education’s 9 Big Ideas
• Teaching Units: Problem Solving , Advanced Problem Solving , Skills , Decision Making and Leadership , Professional Development for Law Firms , Professional Development for In-House Counsel
• Educator Community: Tips for Teachers

Watch this informative video about the Problem-Solving Workshop:

<< Previous: About Harvard Law School Case Studies | Next: Downloading Case Studies >>

• Business Essentials
• Leadership & Management
• Credential of Leadership, Impact, and Management in Business (CLIMB)
• Entrepreneurship & Innovation
• Digital Transformation
• Finance & Accounting
• Business in Society
• For Organizations
• Support Portal
• Media Coverage
• Founding Donors
• Leadership Team

• Harvard Business School →
• HBS Online →
• Business Insights →

Business Insights

Harvard Business School Online's Business Insights Blog provides the career insights you need to achieve your goals and gain confidence in your business skills.

• Career Development
• Communication
• Decision-Making
• Earning Your MBA
• Negotiation
• News & Events
• Productivity
• Staff Spotlight
• Student Profiles
• Work-Life Balance
• AI Essentials for Business
• Alternative Investments
• Business Analytics
• Business Strategy
• Business and Climate Change
• Design Thinking and Innovation
• Digital Marketing Strategy
• Disruptive Strategy
• Economics for Managers
• Entrepreneurship Essentials
• Financial Accounting
• Global Business
• Launching Tech Ventures
• Leadership Principles
• Leadership, Ethics, and Corporate Accountability
• Leading with Finance
• Management Essentials
• Negotiation Mastery
• Organizational Leadership
• Power and Influence for Positive Impact
• Strategy Execution
• Sustainable Business Strategy
• Sustainable Investing
• Winning with Digital Platforms

5 Benefits of Learning Through the Case Study Method

• 28 Nov 2023

While several factors make HBS Online unique —including a global Community and real-world outcomes —active learning through the case study method rises to the top.

In a 2023 City Square Associates survey, 74 percent of HBS Online learners who also took a course from another provider said HBS Online’s case method and real-world examples were better by comparison.

Here’s a primer on the case method, five benefits you could gain, and how to experience it for yourself.

Access your free e-book today.

What Is the Harvard Business School Case Study Method?

The case study method , or case method , is a learning technique in which you’re presented with a real-world business challenge and asked how you’d solve it. After working through it yourself and with peers, you’re told how the scenario played out.

HBS pioneered the case method in 1922. Shortly before, in 1921, the first case was written.

“How do you go into an ambiguous situation and get to the bottom of it?” says HBS Professor Jan Rivkin, former senior associate dean and chair of HBS's master of business administration (MBA) program, in a video about the case method . “That skill—the skill of figuring out a course of inquiry to choose a course of action—that skill is as relevant today as it was in 1921.”

Originally developed for the in-person MBA classroom, HBS Online adapted the case method into an engaging, interactive online learning experience in 2014.

In HBS Online courses , you learn about each case from the business professional who experienced it. After reviewing their videos, you’re prompted to take their perspective and explain how you’d handle their situation.

You then get to read peers’ responses, “star” them, and comment to further the discussion. Afterward, you learn how the professional handled it and their key takeaways.

HBS Online’s adaptation of the case method incorporates the famed HBS “cold call,” in which you’re called on at random to make a decision without time to prepare.

“Learning came to life!” said Sheneka Balogun , chief administration officer and chief of staff at LeMoyne-Owen College, of her experience taking the Credential of Readiness (CORe) program . “The videos from the professors, the interactive cold calls where you were randomly selected to participate, and the case studies that enhanced and often captured the essence of objectives and learning goals were all embedded in each module. This made learning fun, engaging, and student-friendly.”

If you’re considering taking a course that leverages the case study method, here are five benefits you could experience.

5 Benefits of Learning Through Case Studies

1. take new perspectives.

The case method prompts you to consider a scenario from another person’s perspective. To work through the situation and come up with a solution, you must consider their circumstances, limitations, risk tolerance, stakeholders, resources, and potential consequences to assess how to respond.

Taking on new perspectives not only can help you navigate your own challenges but also others’. Putting yourself in someone else’s situation to understand their motivations and needs can go a long way when collaborating with stakeholders.

2. Hone Your Decision-Making Skills

Another skill you can build is the ability to make decisions effectively . The case study method forces you to use limited information to decide how to handle a problem—just like in the real world.

Throughout your career, you’ll need to make difficult decisions with incomplete or imperfect information—and sometimes, you won’t feel qualified to do so. Learning through the case method allows you to practice this skill in a low-stakes environment. When facing a real challenge, you’ll be better prepared to think quickly, collaborate with others, and present and defend your solution.

3. Become More Open-Minded

As you collaborate with peers on responses, it becomes clear that not everyone solves problems the same way. Exposing yourself to various approaches and perspectives can help you become a more open-minded professional.

When you’re part of a diverse group of learners from around the world, your experiences, cultures, and backgrounds contribute to a range of opinions on each case.

On the HBS Online course platform, you’re prompted to view and comment on others’ responses, and discussion is encouraged. This practice of considering others’ perspectives can make you more receptive in your career.

“You’d be surprised at how much you can learn from your peers,” said Ratnaditya Jonnalagadda , a software engineer who took CORe.

In addition to interacting with peers in the course platform, Jonnalagadda was part of the HBS Online Community , where he networked with other professionals and continued discussions sparked by course content.

“You get to understand your peers better, and students share examples of businesses implementing a concept from a module you just learned,” Jonnalagadda said. “It’s a very good way to cement the concepts in one's mind.”

4. Enhance Your Curiosity

One byproduct of taking on different perspectives is that it enables you to picture yourself in various roles, industries, and business functions.

“Each case offers an opportunity for students to see what resonates with them, what excites them, what bores them, which role they could imagine inhabiting in their careers,” says former HBS Dean Nitin Nohria in the Harvard Business Review . “Cases stimulate curiosity about the range of opportunities in the world and the many ways that students can make a difference as leaders.”

Through the case method, you can “try on” roles you may not have considered and feel more prepared to change or advance your career .

5. Build Your Self-Confidence

Finally, learning through the case study method can build your confidence. Each time you assume a business leader’s perspective, aim to solve a new challenge, and express and defend your opinions and decisions to peers, you prepare to do the same in your career.

According to a 2022 City Square Associates survey , 84 percent of HBS Online learners report feeling more confident making business decisions after taking a course.

“Self-confidence is difficult to teach or coach, but the case study method seems to instill it in people,” Nohria says in the Harvard Business Review . “There may well be other ways of learning these meta-skills, such as the repeated experience gained through practice or guidance from a gifted coach. However, under the direction of a masterful teacher, the case method can engage students and help them develop powerful meta-skills like no other form of teaching.”

How to Experience the Case Study Method

If the case method seems like a good fit for your learning style, experience it for yourself by taking an HBS Online course. Offerings span seven subject areas, including:

• Business essentials
• Leadership and management
• Entrepreneurship and innovation
• Finance and accounting
• Business in society

No matter which course or credential program you choose, you’ll examine case studies from real business professionals, work through their challenges alongside peers, and gain valuable insights to apply to your career.

Are you interested in discovering how HBS Online can help advance your career? Explore our course catalog and download our free guide —complete with interactive workbook sections—to determine if online learning is right for you and which course to take.

About the Author

Brought to you by:

Learning by the Case Method

By: John S. Hammond

Helps students fully capitalize on the case method. Succinctly introduces the benefits, format, and process while offering lots of "how-to" advice. Widely used as an introductory handout for…

• Length: 4 page(s)
• Publication Date: Apr 1, 1976
• Discipline: Teaching & the Case Method
• Product #: 376241-PDF-ENG

What's included:

• Educator Copy

$4.95 per student

degree granting course

$8.95 per student

non-degree granting course

Get access to this material, plus much more with a free Educator Account:

• Access to world-famous HBS cases
• Up to 60% off materials for your students
• Resources for teaching online
• Tips and reviews from other Educators

Already registered? Sign in

• Student Registration
• Non-Academic Registration
• Included Materials

Helps students fully capitalize on the case method. Succinctly introduces the benefits, format, and process while offering lots of "how-to" advice. Widely used as an introductory handout for executive, MBA, and undergraduate courses. Describes how case discussion enables the exchange of managerial experience and knowledge. Emphasizes the need to identify the real issues and do rigorous analysis in the course of reaching a management decision and that there is typically more than one "right" answer.

Apr 1, 1976 (Revised: Apr 16, 2002)

Discipline:

Teaching & the Case Method

Harvard Business School

376241-PDF-ENG

We use cookies to understand how you use our site and to improve your experience, including personalizing content. Learn More . By continuing to use our site, you accept our use of cookies and revised Privacy Policy .

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• SAGE Open Nurs
• v.7; Jan-Dec 2021

Case Study Analysis as an Effective Teaching Strategy: Perceptions of Undergraduate Nursing Students From a Middle Eastern Country

Vidya seshan.

1 Maternal and Child Health Department, College of Nursing, Sultan Qaboos University, P.O. Box 66 Al-Khoudh, Postal Code 123, Muscat, Oman

Gerald Amandu Matua

2 Fundamentals and Administration Department, College of Nursing, Sultan Qaboos University, P.O. Box 66 Al-Khoudh, Postal Code 123, Muscat, Oman

Divya Raghavan

Judie arulappan, iman al hashmi, erna judith roach, sheeba elizebath sunderraj, emi john prince.

3 Griffith University, Nathan Campus, Queensland 4111

Background: Case study analysis is an active, problem-based, student-centered, teacher-facilitated teaching strategy preferred in undergraduate programs as they help the students in developing critical thinking skills. Objective: It determined the effectiveness of case study analysis as an effective teacher-facilitated strategy in an undergraduate nursing program. Methodology: A descriptive qualitative research design using focus group discussion method guided the study. The sample included undergraduate nursing students enrolled in the Maternal Health Nursing Course during the Academic Years 2017 and 2018. The researcher used a purposive sampling technique and a total of 22 students participated in the study, through five (5) focus groups, with each focus group comprising between four to six nursing students. Results: In total, nine subthemes emerged from the three themes. The themes were “Knowledge development”, “Critical thinking and Problem solving”, and “Communication and Collaboration”. Regarding “Knowledge development”, the students perceived case study analysis method as contributing toward deeper understanding of the course content thereby helping to reduce the gap between theory and practice especially during clinical placement. The “Enhanced critical thinking ability” on the other hand implies that case study analysis increased student's ability to think critically and aroused problem-solving interest in the learners. The “Communication and Collaboration” theme implies that case study analysis allowed students to share their views, opinions, and experiences with others and this enabled them to communicate better with others and to respect other's ideas which further enhanced their team building capacities. Conclusion: This method is effective for imparting professional knowledge and skills in undergraduate nursing education and it results in deeper level of learning and helps in the application of theoretical knowledge into clinical practice. It also broadened students’ perspectives, improved their cooperation capacity and their communication with each other. Finally, it enhanced student's judgment and critical thinking skills which is key for their success.

Introduction/Background

Recently, educators started to advocate for teaching modalities that not only transfer knowledge ( Shirani Bidabadi et al., 2016 ), but also foster critical and higher-order thinking and student-centered learning ( Wang & Farmer, 2008 ; Onweh & Akpan, 2014). Therefore, educators need to utilize proven teaching strategies to produce positive outcomes for learners (Onweh & Akpan, 2014). Informed by this view point, a teaching strategy is considered effective if it results in purposeful learning ( Centra, 1993 ; Sajjad, 2010 ) and allows the teacher to create situations that promote appropriate learning (Braskamp & Ory, 1994) to achieve the desired outcome ( Hodges et al., 2020 ). Since teaching methods impact student learning significantly, educators need to continuously test the effectives of their teaching strategies to ensure desired learning outcomes for their students given today's dynamic learning environments ( Farashahi & Tajeddin, 2018 ).

In this study, the researchers sought to study the effectiveness of case study analysis as an active, problem-based, student-centered, teacher-facilitated strategy in a baccalaureate-nursing program. This choice of teaching method is supported by the fact that nowadays, active teaching-learning is preferred in undergraduate programs because, they not only make students more powerful actors in professional life ( Bean, 2011 ; Yang et al., 2013 ), but they actually help learners to develop critical thinking skills ( Clarke, 2010 ). In fact, students who undergo such teaching approaches usually become more resourceful in integrating theory with practice, especially as they solve their case scenarios ( Chen et al., 2019 ; Farashahi & Tajeddin, 2018 ; Savery, 2019 ).

Review of Literature

As a pedagogical strategy, case studies allow the learner to integrate theory with real-life situations as they devise solutions to the carefully designed scenarios ( Farashahi & Tajeddin, 2018 ; Hermens & Clarke, 2009). Another important known observation is that case-study-based teaching exposes students to different cases, decision contexts and the environment to experience teamwork and interpersonal relations as “they learn by doing” thus benefiting from possibilities that traditional lectures hardly create ( Farashahi & Tajeddin, 2018 ; Garrison & Kanuka, 2004 ).

Another merit associated with case study method of teaching is the fact that students can apply and test their perspectives and knowledge in line with the tenets of Kolb et al.'s (2014) “experiential learning model”. This model advocates for the use of practical experience as the source of one's learning and development. Proponents of case study-based teaching note that unlike passive lectures where student input is limited, case studies allow them to draw from their own experience leading to the development of higher-order thinking and retention of knowledge.

Case scenario-based teaching also encourages learners to engage in reflective practice as they cooperate with others to solve the cases and share views during case scenario analysis and presentation ( MsDade, 1995 ).

This method results in “idea marriage” as learners articulate their views about the case scenario. This “idea marriage” phenomenon occurs through knowledge transfer from one situation to another as learners analyze scenarios, compare notes with each other, and develop multiple perspectives of the case scenario. In fact, recent evidence shows that authentic case-scenarios help learners to acquire problem solving and collaborative capabilities, including the ability to express their own views firmly and respectfully, which is vital for future success in both professional and personal lives ( Eronen et al., 2019 ; Yajima & Takahashi, 2017 ). In recognition of this higher education trend toward student-focused learning, educators are now increasingly expected to incorporate different strategies in their teaching.

This study demonstrated that when well implemented, educators can use active learning strategies like case study analysis to aid critical thinking, problem-solving, and collaborative capabilities in undergraduate students. This study is significant because the findings will help educators in the country and in the region to incorporate active learning strategies such as case study analysis to aid critical thinking, problem-solving, and collaborative capabilities in undergraduate students. Besides, most studies on the case study method in nursing literature mostly employ quantitative methods. The shortage of published research on the case study method in the Arabian Gulf region and the scanty use of qualitative methods further justify why we adopted the focus group method for inquiry.

A descriptive qualitative research design using focus group discussion method guided the study. The authors chose this method because it is not only inexpensive, flexible, stimulating but it is also known to help with information recall and results in rich data ( Matua et al., 2014 ; Streubert & Carpenter, 2011 ). Furthermore, as evidenced in the literature, the focus group discussion method is often used when there is a need to gain an in-depth understanding of poorly understood phenomena as the case in our study. The choice of this method is further supported by the scarcity of published research related to the use of case study analysis as a teaching strategy in the Middle Eastern region, thereby further justifying the need for an exploratory research approach for our study.

As a recommended strategy, the researchers generated data from information-rich purposively selected group of baccalaureate nursing students who had experienced both traditional lectures and cased-based teaching approaches. The focus group interviews allowed the study participants to express their experiences and perspectives in their own words. In addition, the investigators integrated participants’ self-reported experiences with their own observations and this enhanced the study findings ( Morgan & Bottorff, 2010 ; Nyumba et al., 2018 ; Parker & Tritter, 2006 ).

Eligibility Criteria

In order to be eligible to participate in the study, the participants had to:

• be a baccalaureate nursing student in College of Nursing, Sultan Qaboos University
• register for Maternity Nursing Course in 2017 and 2018.
• attend all the Case Study Analysis sessions in the courses before the study.
• show a willingness to participate in the study voluntarily and share their views freely.

The population included the undergraduate nursing students enrolled in the Maternal Health Nursing Course during the Academic Years 2017 and 2018.

The researcher used a purposive sampling technique to choose participants who were capable of actively participating and discussing their views in the focus group interviews. This technique enabled the researchers to select participants who could provide rich information and insights about case study analysis method as an effective teaching strategy. The final study sample included baccalaureate nursing students who agreed to participate in the study by signing a written informed consent. In total, twenty-two (22) students participated in the study, through five focus groups, with each focus group comprising between four and six students. The number of participants was determined by the stage at which data saturation was reached. The point of data saturation is when no new information emerges from additional participants interviewed ( Saunders et al., 2018 ).Focus group interviews were stopped once data saturation was achieved. Qualitative research design with focus group discussion allowed the researchers to generate data from information-rich purposively selected group of baccalaureate nursing students who had experienced both traditional lectures and case-based teaching approaches. The focus group interviews allowed the study participants to express their perspectives in their own words. In addition, the investigators enhanced the study findings by integrating participants’ self-reported experiences with the researchers’ own observations and notes during the study.

The study took place at College of Nursing; Sultan Qaboos University, Oman's premier public university, in Muscat. This is the only setting chosen for the study. The participants are the students who were enrolled in Maternal Health Nursing course during 2017 and 2018. The interviews occurred in the teaching rooms after official class hours. Students who did not participate in the study learnt the course content using the traditional lecture based method.

Ethical Considerations

Permission to conduct the study was granted by the College Research and Ethics Committee (XXXX). Prior to the interviews, each participant was informed about the purpose, benefits as well as the risks associated with participating in the study and clarifications were made by the principal researcher. After completing this ethical requirement, each student who accepted to participate in the study proceeded to sign an informed consent form signifying that their participation in the focus group interview was entirely voluntary and based on free will.

The anonymity of study participants and confidentiality of their data was upheld throughout the focus group interviews and during data analysis. To enhance confidentiality and anonymity of the data, each participant was assigned a unique code number which was used throughout data analysis and reporting phases. To further assure the confidentiality of the research data and anonymity of the participants, all research-related data were kept safe, under lock and key and through digital password protection, with unhindered access only available to the research team.

Research Intervention

In Fall 2017 and Spring 2018 semesters, as a method of teaching Maternal Health Nursing course, all students participated in two group-based case study analysis exercises which were implemented in the 7 th and 13 th weeks. This was done after the students were introduced to the case study method using a sample case study prior to the study. The instructor explained to the students how to solve the sample problem, including how to accomplish the role-specific competencies in the courses through case study analysis. In both weeks, each group consisting of six to seven students was assigned to different case scenarios to analyze and work on, after which they presented their collective solution to the case scenarios to the larger class of 40 students. The case scenarios used in both weeks were peer-reviewed by the researchers prior to the study.

Pilot Study

A group of three students participated as a pilot group for the study. However, the students who participated in the pilot study were not included in the final study as is general the principle with qualitative inquiry because of possible prior exposure “contamination”. The purpose of piloting was to gather data to provide guidance for a substantive study focusing on testing the data collection procedure, the interview process including the sequence and number of questions and probes and recording equipment efficacy. After the pilot phase, the lessons learned from the pilot were incorporated to ensure smooth operations during the actual focus group interview ( Malmqvist et al., 2019 .

Data Collection

The focus group interviews took place after the target population was exposed to case study analysis method in Maternal Health Nursing course during the Fall 2017 and Spring 2018 semesters. Before data collection began, the research team pilot tested the focus group interview guide to ensure that all the guide questions were clear and well understood by study participants.

In total, five (5) focus groups participated in the study, with each group comprising between four and six students. The focus group interviews lasted between 60 and 90 min. In addition to the interview guide questions, participants’ responses to unanswered questions were elicited using prompts to facilitate information flow whenever required. As a best practice, all the interviews were audio-recorded in addition to extensive field notes taken by one of the researchers. The focus group interviews continued until data saturation occurred in all the five (5) focus groups.

Credibility

In this study, participant's descriptions were digitally audio recorded to ensure that no information was lost. In order to ensure that the results are accurate, verbatim transcriptions of the audio recordings were done supported by interview notes. Furthermore, interpretations of the researcher were verified and supported with existing literature with oversight from the research team.

Transferability

The researcher provided a detailed description about the study settings, participants, sampling technique, and the process of data collection and analyses. The researcher used verbatim quotes from various participants to aid the transferability of the results.

Dependability

The researcher ensured that the research process is clearly documented, traceable, and logical to achieve dependability of the research findings. Furthermore, the researcher transparently described the research steps, procedures and process from the start of the research project to the reporting of the findings.

Confirmability

In this study, confirmability of the study findings was achieved through the researcher's efforts to make the findings credible, dependable, and transferable.

Data Analysis

Data were analyzed manually after the lead researcher integrated the verbatim transcriptions with the extensive field notes to form the final data set. Data were analyzed thematically under three thematic areas of a) knowledge development; b) critical thinking and problem solving; and (c) communication and collaboration, which are linked to the study objectives. The researchers used the Six (6) steps approach to conduct a trustworthy thematic analysis: (1) familiarization with the research data, (2) generating initial codes, (3) searching for themes, (4) reviewing the themes, (5) defining and naming themes, (6) writing the report ( Nowell et al., 2017 ).

The analysis process started with each team member individually reading and re-reading the transcripts several times and then identifying meaning units linked to the three thematic areas. The co-authors then discussed in-depth the various meaning units linked to the thematic statements until consensus was reached and final themes emerged based on the study objectives.

A total of 22 undergraduate third-year baccalaureate nursing students who were enrolled in the Maternal Health Nursing Course during the Academic Years 2017 and 2018 participated in the study, through five focus groups, with each group comprising four to six students. Of these, 59% were females and 41% were males. In total, nine subthemes emerged from the three themes. Under knowledge development, emerged the subthemes, “ deepened understanding of content ; “ reduced gap between theory and practice” and “ improved test-taking ability ”. While under Critical thinking and problem solving, emerged the subthemes, “ enhanced critical thinking ability ” and “ heightened curiosity”. The third thematic area of communication and collaboration yielded, “ improved communication ability ”; “ enhanced team-building capacity ”; “ effective collaboration” and “ improved presentation skills ”, details of which are summarized in Table 1 .

Table 1.

Objective Linked Themes and Student Perceptions of Outcome Case Study Analysis.

Theme 1: Knowledge Development

In terms of knowledge development, students expressed delight at the inclusion of case study analysis as a method during their regular theory class. The first subtheme related to knowledge development that supports the adoption of the case study approach is its perceived benefit of ‘ deepened understanding of content ’ by the students as vividly described by this participant:

“ I was able to perform well in the in-course exams as this teaching method enhanced my understanding of the content rather than memorizing ” (FGD#3).

The second subtheme related to knowledge development was informed by participants’ observation that teaching them using case study analysis method ‘ reduced the gap between theory and practice’. This participant's claim stem from the realization that, a case study scenario his group analyzed in the previous week helped him and his colleagues to competently deal with a similar situation during clinical placement the following week, as articulated below:

“ You see when I was caring for mothers in antenatal unit, I could understand the condition better and could plan her care well because me and my group already analyzed a similar situation in class last week which the teacher gave us, this made our work easier in the ward”. (FGD#7).

Another student added that:

“ It was useful as what is taught in the theory class could be applied to the clinical cases.”

This ‘theory-practice’ connection was particularly useful in helping students to better understand how to manage patients with different health conditions. Interestingly, the students reported that they were more likely to link a correct nursing care plan to patients whose conditions were close to the case study scenarios they had already studied in class as herein affirmed:

“ …when in the hospital I felt I could perceive the treatment modality and plan for [a particular] nursing care well when I [had] discussed with my team members and referred the textbook resource while performing case study discussion”. (FGD#17).

In a similar way, another student added:

“…I could relate with the condition I have seen in the clinical area. So this has given me a chance to recall the condition and relate the theory to practice”. (FGD#2) .

The other subtheme closely related to case study scenarios as helping to deepen participant's understanding of the course content, is the notion that this teaching strategy also resulted in ‘ improved test taking-ability’ as this participant's verbatim statement confirms:

“ I could answer the questions related to the cases discussed [much] better during in-course exams. Also [the case scenarios] helped me a great deal to critically think and answer my exam papers” (FGD#11).

Theme 2: Critical Thinking and Problem Solving

In this subtheme, students found the case study analysis as an excellent method to learn disease conditions in the two courses. This perceived success with the case study approach is associated with the method's ability to ‘ enhance students’ critical thinking ability’ as this student declares:

“ This method of teaching increased my ability to think critically as the cases are the situations, where we need to think to solve the situation”. (FGD#5)

This enhanced critical thinking ability attributed to case study scenario analysis was also manifested during patient care where students felt it allowed them to experience a “ flow of patient care” leading to better patient management planning as would typically occur during case scenario analysis. In support of this finding, a participant mentioned that:

“ …I could easily connect the flow of patient care provided and hence was able to plan for [his] management as often required during case study discussion” (FGD#12)

Another subtheme linked with this theme is the “ heightened curiosity” associated with the case scenario discussions. It was clear from the findings that the cases aroused curiosity in the mind of the students. This heightened interest meant that during class discussion, baccalaureate nursing students became active learners, eager to discover the next set of action as herein affirmed:

“… from the beginning of discussion with the group, I was eager to find the answer to questions presented and wanted to learn the best way for patient management” (FGD#14)

Theme 3: Communication and Collaboration

In terms of its impact on student communication, the subtheme revealed that case study analysis resulted in “ improved communication ability” among the nursing students . This enhanced ability of students to exchange ideas with each other may be attributed to the close interaction required to discuss and solve their assigned case scenarios as described by the participant below:

“ as [case study analysis] was done in the way of group discussion, I felt me and my friends communicated more within the group as we discussed our condition. We also learnt from each other, and we became better with time.” (FGD#21).

The next subtheme further augments the notion that case study analysis activities helped to “ enhance team-building capacity” of students as this participant affirmatively narrates:

“ students have the opportunity to meet face to face to share their views, opinion, and their experience, as this build on the way they can communicate with each other and respect each other's opinions and enhance team-building”. (FGD#19).

Another subtheme revealed from the findings show that the small groups in which the case analysis occurs allowed the learners to have deeper and more focused conversations with one another, resulting in “ an effective collaboration between students” as herein declared:

“ We could collaborate effectively as we further went into a deep conversation on the case to solve”. (FGD#16).

Similarly, another student noted that:

“ …discussion of case scenarios helped us to prepare better for clinical postings and simulation lab experience” (FGD#5) .

A fourth subtheme related to communication found that students also identified that case study analysis resulted in “ improved presentation skills”. This is attributed in part to the preparation students have to go through as part of their routine case study discussion activities, which include organizing their presentations and justifying and integrating their ideas. Besides readying themselves for case presentations, the advice, motivation, and encouragement such students receive from their faculty members and colleagues makes them better presenters as confirmed below:

“ …teachers gave us enough time to prepare, hence I was able to present in front of the class regarding the finding from our group.” (FGD#16).

In this study, the researches explored learner's perspectives on how one of the active teaching strategies, case study analysis method impacted their knowledge development, critical thinking, and problem solving as well as communication and collaboration ability.

Knowledge Development

In terms of knowledge development, the nursing students perceived case study analysis as contributing toward: (a) deeper understanding of content, (b) reducing gap between theory and practice, and (c) improving test-taking ability. Deeper learning” implies better grasping and retention of course content. It may also imply a deeper understanding of course content combined with learner's ability to apply that understanding to new problems including grasping core competencies expected in future practice situations (Rickles et al., 2019; Rittle-Johnson et al., 2020 ). Deeper learning therefore occurs due to the disequilibrium created by the case scenario, which is usually different from what the learner already knows ( Hattie, 2017 ). Hence, by “forcing” students to compare and discuss various options in the quest to solve the “imbalance” embedded in case scenarios, students dig deeper in their current understanding of a given content including its application to the broader context ( Manalo, 2019 ). This movement to a deeper level of understanding arises from carefully crafted case scenarios that instructors use to stimulate learning in the desired area (Nottingham, 2017; Rittle-Johnson et al., 2020 ). The present study demonstrated that indeed such carefully crafted case study scenarios did encourage students to engage more deeply with course content. This finding supports the call by educators to adopt case study as an effective strategy.

Another finding that case study analysis method helps in “ reducing the gap between theory and practice ” implies that the method helps students to maintain a proper balance between theory and practice, where they can see how theoretical knowledge has direct practical application in the clinical area. Ajani and Moez (2011) argue that to enable students to link theory and practice effectively, nurse educators should introduce them to different aspects of knowledge and practice as with case study analysis. This dual exposure ensures that students are proficient in theory and clinical skills. This finding further amplifies the call for educators to adequately prepare students to match the demands and realities of modern clinical environments ( Hickey, 2010 ). This expectation can be met by ensuring that student's knowledge and skills that are congruent with hospital requirements ( Factor et al., 2017 ) through adoption of case study analysis method which allows integration of clinical knowledge in classroom discussion on regular basis.

The third finding, related to “improved test taking ability”, implies that case study analysis helped them to perform better in their examination, noting that their experience of going through case scenario analysis helped them to answer similar cases discussed in class much better during examinations. Martinez-Rodrigo et al. (2017) report similar findings in a study conducted among Spanish electrical engineering students who were introduced to problem-based cooperative learning strategies, which is similar to case study analysis method. Analysis of student's results showed that their grades and pass rates increased considerably compared to previous years where traditional lecture-based method was used. Similar results were reported by Bonney (2015) in an even earlier study conducted among biology students in Kings Borough community college students, in New York, United States. When student's performance in examination questions covered by case studies was compared with class-room discussions, and text-book reading, case study analysis approach was significantly more effective compared to traditional methods in aiding students’ performance in their examinations. This finding therefore further demonstrates that case study analysis method indeed improves student's test taking ability.

Critical Thinking and Problem Solving

In terms of critical thinking and problem-solving ability, the use of case study analysis resulted in two subthemes: (a) enhanced critical thinking ability and (b) heightened learner curiosity. The “ enhanced critical thinking ability” implies that case analysis increased student's ability to think critically as they navigated through the case scenarios. This observation agrees with the findings of an earlier questionnaire-based study conducted among 145 undergraduate business administration students at Chittagong University, Bangladesh, that showed 81% of respondents agree that case study analysis develops critical thinking ability and enables students to do better problem analysis ( Muhiuddin & Jahan, 2006 ). This observation agrees with the findings of an earlier study conducted among 145 undergraduate business administration students at Chittagong University, Bangladesh. The study showed that 81% of respondents agreed that case study analysis facilitated the development of critical thinking ability in the learners and enabled the students to perform better with problem analysis ( Muhiuddin & Jahan, 2006 ).

More recently, Suwono et al. (2017) found similar results in a quasi-experimental research conducted at a Malaysian university. The research findings showed that there was a significant difference in biological literacy and critical thinking skills between the students taught using socio-biological case-based learning and those taught using traditional lecture-based learning. The researchers concluded that case-based learning enhanced the biological literacy and critical thinking skills of the students. The current study adds to the existing pedagogical knowledge base that case study methodology can indeed help to deepen learner's critical thinking and problem solving ability.

The second subtheme related to “ heightened learner curiosity” seems to suggest that the case studies aroused problem-solving interest in learners. This observation agrees with two earlier studies by Tiwari et al. (2006) and Flanagan and McCausland (2007) who both reported that most students enjoyed case-based teaching. The authors add that the case study method also improved student's clinical reasoning, diagnostic interpretation of patient information as well as their ability to think logically when presented a challenge in the classroom and in the clinical area. Jackson and Ward (2012) similarly reported that first year engineering undergraduates experienced enhanced student motivation. The findings also revealed that the students venturing self-efficacy increased much like their awareness of the importance of key aspects of the course for their future careers. The authors conclude that the case-based method appears to motivate students to autonomously gather, analyze and present data to solve a given case. The researchers observed enhanced personal and collaborative efforts among the learners, including improved communication ability. Further still, learners were more willing to challenge conventional wisdom, and showed higher “softer” skills after exposure to case analysis based teaching method. These findings like that of the current study indicate that teaching using case based analysis approach indeed motivates students to engage more in their learning, there by resulting in deeper learning.

Communication and Collaboration

Case study analysis is also perceived to result in: (a) improved communication ability; (b) enhanced team -building capacity, (c) effective collaboration ability, and (d) enhanced presentation skills. The “ improved communication ability ” manifested in learners being better able to exchange ideas with peers, communicating their views more clearly and collaborating more effectively with their colleagues to address any challenges that arise. Fini et al. (2018) report comparable results in a study involving engineering students who were subjected to case scenario brainstorming activities about sustainability concepts and their implications in transportation engineering in selected courses. The results show that this intervention significantly improved student's communication skills besides their higher-order cognitive, self-efficacy and teamwork skills. The researchers concluded that involving students in brainstorming activities related to problem identification including their practical implications, is an effective teaching strategy. Similarly, a Korean study by Park and Choi (2018) that sought to analyze the effects of case-based communication training involving 112 sophomore nursing students concluded that case-based training program improved the students’ critical thinking ability and communication competence. This finding seems to support further the use of case based teaching as an effective teaching-learning strategy.

The “ enhanced team-building capacity” arose from the opportunity students had in sharing their views, opinions, and experiences where they learned to communicate with each other and respect each other's ideas which further enhance team building. Fini et al. (2018) similarly noted that increased teamwork levels were seen among their study respondents when the researchers subjected engineering students to case scenario based-brainstorming activities as occurs with case study analysis teaching. Likewise, Lairamore et al. (2013) report similar results in their study that showed that case study analysis method increased team work ability and readiness among students from five health disciplines in a US-based study.

The finding that case study analysis teaching method resulted in “ effective collaboration ability” among students manifested as students entered into deep conversation as they solved the case scenarios. Rezaee and Mosalanejad (2015) assert that such innovative learning strategies result in noticeable educational outcomes, such as greater satisfaction with and enjoyment of the learning process ( Wellmon et al., 2012 ). Further, positive attitudes toward learning and collaboration have been noted leading to deeper learning as students prepare for case discussions ( Rezaee & Mosalanejad, 2015 ). This results show that case study analysis can be utilized by educators to foster professional collaboration among their learners, which is one of the key expectations of new graduates today.

The finding associated with “improved presentation skills” is consistent with the results of a descriptive study in Saudi Arabia that compared case study and traditional lectures in the teaching of physiology course to undergraduate nursing students. The researchers found that case-based teaching improved student’ overall knowledge and performance in the course including facilitating the acquisition of skills compared to traditional lectures ( Majeed, 2014 ). Noblitt et al. (2010) report similar findings in their study that compares traditional presentation approach with the case study method for developing and improving student's oral communication skills. This finding extends our understanding that case study method improves learners’ presentation skills.

The study was limited to level third year nursing students belonging to only one college and the sample size, which might limit the transferability of the study findings to other settings.

Implications for Practice

These study findings add to the existing body of knowledge that places case study based teaching as a tested method that promotes perception learning where students’ senses are engaged as a result of the real-life and authentic clinical scenarios ( Malesela, 2009 ), resulting in deeper learning and achievement of long-lasting knowledge ( Fiscus, 2018 ). The students reported that case scenario discussions broadened their perspectives, improved their cooperation capacity and communication with each other. This teaching method, in turn, offers students an opportunity to enhance their judgment and critical thinking skills by applying theory into practice.

These skills are critically important because nurses need to have the necessary knowledge and skills to plan high quality care for their patients to achieve a speedy recovery. In order to attain this educational goal, nurse educators have to prepare students through different student- centered strategies. The findings of our study appear to show that when appropriately used, case-based teaching results in acquisition of disciplinary knowledge manifested by deepened understanding of course content, as well as reducing the gap between theory and practice and enhancing learner's test-taking-ability. The study also showed that cased based teaching enhanced learner's critical thinking ability and curiosity to seek and acquire a deeper knowledge. Finally, the study results indicate that case study analysis results in improved communication and enhanced team-building capacity, collaborative ability and improved oral communication and presentation skills. The study findings and related evidence from literature show that case study analysis is well- suited approach for imparting knowledge and skills in baccalaureate nursing education.

This study evaluated the usefulness of Case Study Analysis as a teaching strategy. We found that this method of teaching helps encourages deeper learning among students. For instructors, it provides the opportunity to tailor learning experiences for students to undertake in depth study in order to stimulate deeper understanding of the desired content. The researchers conclude that if the cases are carefully selected according to the level of the students, and are written realistically and creatively and the group discussions keep students well engaged, case study analysis method is more effective than other traditional lecture methods in facilitating deeper and transferable learning/skills acquisition in undergraduate courses.

Conflict of Interest: The authors declare no conflict of interest.

ORCID iD: Judie Arulappan https://orcid.org/0000-0003-2788-2755

• Ajani K., Moez S. (2011). Gap between knowledge and practice in nursing . Procedia-Social and Behavioral Sciences , 15 , 3927–3931. 10.1016/j.sbspro.2011.04.396 [ CrossRef ] [ Google Scholar ]
• Bean J. C. (2011). Engaging ideas: The professor’s guide to integrating writing critical thinking and active-learning in the classroom (2nd ed.). Jossey-Bass. [ Google Scholar ]
• Bonney K. M. (2015). Case study teaching method improves student performance and perceptions of learning gains . Journal of Microbiology & Biology Education , 16 ( 1 ), 21–28. 10.1128/jmbe.v16i1.846 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Braskamp L. A., Ory J. C. (1994). Assessing faculty work: Enhancing individual and institutional performance . Jossey-Bass Higher and Adult Education Series. Jossey-Bass Inc. [ Google Scholar ]
• Centra J. A. (1993). Reflective faculty evaluation: Enhancing teaching and determining faculty effectiveness . Jossey-Bass. [ Google Scholar ]
• Chen W., Shah U. V., Brechtelsbauer C. (2019). A framework for hands-on learning in chemical engineering education—training students with the end goal in mind . Education for Chemical Engineers , 28 , 25–29. 10.1016/j.ece.2019.03.002 [ CrossRef ] [ Google Scholar ]
• Clarke J. (2010). Student centered teaching methods in a Chinese setting . Nurse Education Today , 30 ( 1 ), 15–19. 10.1016/j.nedt.2009.05.009 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Eronen L., Kokko S., Sormunen K. (2019). Escaping the subject-based class: A Finnish case study of developing transversal competencies in a transdisciplinary course . The Curriculum Journal , 30 ( 3 ), 264–278. 10.1080/09585176.2019.1568271 [ CrossRef ] [ Google Scholar ]
• Factor E. M. R., Matienzo E. T., de Guzman A. B. (2017). A square peg in a round hole: Theory-practice gap from the lens of Filipino student nurses . Nurse Education Today , 57 , 82–87. 10.1016/j.nedt.2017.07.004 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Farashahi M., Tajeddin M. (2018). Effectiveness of teaching methods in business education: A comparison study on the learning outcomes of lectures, case studies and simulations . The International Journal of Management Education , 16 ( 1 ), 131–142. 10.1016/j.ijme.2018.01.003 [ CrossRef ] [ Google Scholar ]
• Fini E. H., Awadallah F., Parast M. M., Abu-Lebdeh T. (2018). The impact of project-based learning on improving student learning outcomes of sustainability concepts in transportation engineering courses . European Journal of Engineering Education , 43 ( 3 ), 473–488. 10.1080/03043797.2017.1393045 [ CrossRef ] [ Google Scholar ]
• Fiscus J. (2018). Reflection in Motion: A Case Study of Reflective Practice in the Composition Classroom [ Doctoral dissertation ]. Source: http://hdl.handle.net/1773/42299 [ Google Scholar ]
• Flanagan N. A., McCausland L. (2007). Teaching around the cycle: Strategies for teaching theory to undergraduate nursing students . Nursing Education Perspectives , 28 ( 6 ), 310–314. [ PubMed ] [ Google Scholar ]
• Garrison D. R., Kanuka H. (2004). Blended learning: Uncovering its transformative potential in higher education . The internet and higher education , 7 ( 2 ), 95–105. 10.1016/j.iheduc.2004.02.001 [ CrossRef ] [ Google Scholar ]
• Hattie J. (2017). Foreword . In Nottingham J. (Ed.), The learning challenge: How to guide your students through the learning pit to achieve deeper understanding . Corwin Press, p. xvii. [ Google Scholar ]
• Hermens A., Clarke E. (2009). Integrating blended teaching and learning to enhance graduate attributes . Education+ Training , 51 ( 5/6 ), 476–490. [ Google Scholar ]
• Hickey M. T. (2010). Baccalaureate nursing graduates’ perceptions of their clinical instructional experiences and preparation for practice . Journal of Professional Nursing , 26 ( 1 ), 35–41. 10.1016/j.profnurs.2009.03.001 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Hodges C., Moore S., Lockee B., Trust T., Bond A. (2020). The difference between emergency remote teaching and online learning . Educause review , 27 , 1–12. [ Google Scholar ]
• Jackson N. R., Ward A. E. (2012). Curiosity based learning: Impact study in 1st year electronics undergraduates. 2012 International Conference on Information Technology Based Higher Education and Training (ITHET), Istanbul, pp. 1–6. 10.1109/ITHET.2012.6246005. [ CrossRef ] [ Google Scholar ]
• Kolb A. Y., Kolb D. A., Passarelli A., Sharma G. (2014). On becoming an experiential educator: The educator role profile . Simulation & Gaming , 45 ( 2 ), 204–234. 10.1177/1046878114534383 [ CrossRef ] [ Google Scholar ]
• Lairamore C., George-Paschal L., McCullough K., Grantham M., Head D. (2013). A case-based interprofessional education forum improves students’ perspectives on the need for collaboration, teamwork, and communication . MedEdPORTAL, The Journal of Teaching and learning resources , 9 , 10.15766/mep_2374-8265.9484 [ CrossRef ] [ Google Scholar ]
• Majeed F. (2014). Effectiveness of case based teaching of physiology for nursing students . Journal of Taibah University Medical Sciences , 9 ( 4 ), 289–292. 10.1016/j.jtumed.2013.12.005 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Malesela J. M. (2009). Case study as a learning opportunity among nursing students in a university . Health SA Gesondheid (Online) , 14 ( 1 ), 33–38. 10.4102/hsag.v14i1.434 [ CrossRef ] [ Google Scholar ]
• Malmqvist J., Hellberg K., Möllås G., Rose R., Shevlin M. (2019). Conducting the pilot study: A neglected part of the research process? Methodological findings supporting the importance of piloting in qualitative research studies . International Journal of Qualitative Methods , 18 . 10.1177/1609406919878341 [ CrossRef ] [ Google Scholar ]
• Manalo E. (ed.). (2019). Deeper learning, dialogic learning, and critical thinking: Research-based strategies for the classroom . Routledge. [ Google Scholar ]
• Martinez-Rodrigo F., Herrero-De Lucas L. C., De Pablo S., Rey-Boue A. B. (2017). Using PBL to improve educational outcomes and student satisfaction in the teaching of DC/DC and DC/AC converters . IEEE Transactions on Education , 60 ( 3 ), 229–237. 10.1109/TE.2016.2643623 [ CrossRef ] [ Google Scholar ]
• Matua G. A., Seshan V., Akintola A. A., Thanka A. N. (2014). Strategies for providing effective feedback during preceptorship: Perspectives from an Omani Hospital . Journal of Nursing Education and Practice , 4 ( 10 ), 24. 10.5430/jnep.v4n10p24 [ CrossRef ] [ Google Scholar ]
• Morgan D. L., Bottorff J. L. (2010). Advancing our craft: Focus group methods and practice . Qualitative Health Research , 20 ( 5 ), 579–581. 10.1177/1049732310364625 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• MsDade S. A. (1995). Case study pedagogy to advance critical thinking . Teaching psychology , 22 ( 1 ), 9–10. 10.1207/s15328023top2201_3 [ CrossRef ] [ Google Scholar ]
• Muhiuddin G., Jahan N. (2006). Students’ perception towards case study as a method of learning in the field of business administration’ . The Chittagong University Journal of Business Administration , 21 , 25–41. [ Google Scholar ]
• Noblitt L., Vance D. E., Smith M. L. D. (2010). A comparison of case study and traditional teaching methods for improvement of oral communication and critical-thinking skills . Journal of College Science Teaching , 39 ( 5 ), 26–32. [ Google Scholar ]
• Nottingham J. (2017). The learning challenge: How to guide your students through the learning pit to achieve deeper understanding . Corwin Press. [ Google Scholar ]
• Nowell L. S., Norris J. M., White D. E., Moules N. J. (2017). Thematic analysis: Striving to meet the trustworthiness criteria . International Journal of Qualitative Methods , 16 ( 1 ). 10.1177/1609406917733847 [ CrossRef ] [ Google Scholar ]
• Nyumba T., Wilson K., Derrick C. J., Mukherjee N. (2018). The use of focus group discussion methodology: Insights from two decades of application in conservation . Methods in Ecology and evolution , 9 ( 1 ), 20–32. 10.1111/2041-210X.12860 [ CrossRef ] [ Google Scholar ]
• Onweh V. E., Akpan U. T. (2014). Instructional strategies and students academic performance in electrical installation in technical colleges in Akwa Ibom State: Instructional skills for structuring appropriate learning experiences for students . International Journal of Educational Administration and Policy Studies , 6 ( 5 ), 80–86. [ Google Scholar ]
• Park S. J., Choi H. S. (2018). The effect of case-based SBAR communication training program on critical thinking disposition, communication self-efficacy and communication competence of nursing students . Journal of the Korea Academia-Industrial Cooperation Society , 19 ( 11 ), 426–434. 10.5762/KAIS.2018.19.11.426 [ CrossRef ] [ Google Scholar ]
• Parker A., Tritter J. (2006). Focus group method and methodology: Current practice and recent debate . International Journal of Research & Method in Education , 29 ( 1 ), 23–37. 10.1080/01406720500537304 [ CrossRef ] [ Google Scholar ]
• Rezaee R., Mosalanejad L. (2015). The effects of case-based team learning on students’ learning, self-regulation and self-direction . Global Journal of Health Science , 7 ( 4 ), 295. 10.5539/gjhs.v7n4p295 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Rickles J., Zeiser K. L., Yang R., O’Day J., Garet M. S. (2019). Promoting deeper learning in high school: Evidence of opportunities and outcomes . Educational Evaluation and Policy Analysis , 41 ( 2 ), 214–234. [ Google Scholar ]
• Rittle-Johnson B., Star J. R., Durkin K., Loehr A. (2020). Compare and discuss to promote deeper learning. Deeper learning, dialogic learning, and critical thinking: Research-based strategies for the classroom . Routlegde, p. 48. 10.4324/9780429323058-4 [ CrossRef ] [ Google Scholar ]
• Sajjad S. (2010). Effective teaching methods at higher education level . Pakistan Journal of Special Education , 11 , 29–43. [ Google Scholar ]
• Saunders B., Sim J., Kingstone T., Baker S., Waterfield J., Bartlam B., Burroughs H., Jinks C. (2018). Saturation in qualitative research: Exploring its conceptualization and operationalization . Quality & Quantity , 52 ( 4 ), 1893–1907. 10.1007/s11135-017-0574-8 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Savery J. R. (2019). Comparative pedagogical models of problem based learning . The Wiley Handbook of Problem Based Learning , 81–104. 10.1002/9781119173243.ch4 [ CrossRef ] [ Google Scholar ]
• Shirani Bidabadi N., Nasr Isfahani A., Rouhollahi A., Khalili R. (2016). Effective teaching methods in higher education: Requirements and barriers . Journal of Advances in Medical Education & Professionalism , 4 ( 4 ), 170–178. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Streubert H. J., Carpenter D. R. (2011). Qualitative research in nursing: Advancing the humanistic imperative . Wolters Kluwer. [ Google Scholar ]
• Suwono H., Pratiwi H. E., Susanto H., Susilo H. (2017). Enhancement of students’ biological literacy and critical thinking of biology through socio-biological case-based learning . JurnalPendidikan IPA Indonesia , 6 ( 2 ), 213–220. 10.15294/jpii.v6i2.9622 [ CrossRef ] [ Google Scholar ]
• Tiwari A., Lai P., So M., Yuen K. (2006). A comparison of the effects of problem-based learning and lecturing on the development of students’ critical thinking . Medical Education , 40 ( 6 ), 547–554. 10.1111/j.1365-2929.2006.02481.x [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Wang V., Farmer L. (2008). Adult teaching methods in China and bloom's taxonomy . International Journal for the Scholarship of Teaching and Learning , 2 ( 2 ), n2. 10.20429/ijsotl.2008 [ CrossRef ] [ Google Scholar ]
• Wellmon R., Gilin B., Knauss L., Linn M. I. (2012). Changes in student attitudes toward interprofessional learning and collaboration arising from a case-based educational experience . Journal of Allied Health , 41 ( 1 ), 26–34. [ PubMed ] [ Google Scholar ]
• Yajima K., Takahashi S. (2017). Development of evaluation system of AL students . Procedia Computer Science , 112 , 1388–1395. 10.1016/j.procs.2017.08.056 [ CrossRef ] [ Google Scholar ]
• Yang W. P., Chao C. S. C., Lai W. S., Chen C. H., Shih Y. L., Chiu G. L. (2013). Building a bridge for nursing education and clinical care in Taiwan—using action research and confucian tradition to close the gap . Nurse Education Today , 33 ( 3 ), 199–204. 10.1016/j.nedt.2012.02.016 [ PubMed ] [ CrossRef ] [ Google Scholar ]

Encouraging Critical Thinking Using the Case Study Method and Cooperative Learning Techniques

• Robert W. Grossman Kalamazoo College Author

Several workshops presented at the 1991 Lilly Conference on College Teaching provided the inspiration for redesigning an introductory psychology course. This article shows how cooperative learning teams can be taught to analyze case studies. Further, by using alternative conceptual frameworks to analyze these cases, students are encouraged to think more critically about all the theories presented in the course. This approach should be applicable to a variety of other courses.

• Requires Subscription PDF

Access Agreement Journal on Excellence in College Teaching

Before proceeding you must agree to the terms and conditions of usage as outlined below by clicking on the Accept button and/or by both parties’ signatures below. You will have to do this only once. After agreement, you will be redirected back to the main Journal page. A pdf copy of the terms is available for download.

This Access Agreement (the "Agreement") is effective upon processing of payment ("Effective Date") and is entered into by and between the Journal on Excellence in College Teaching (“JECT”) and the Customer (“Customer").

This Agreement constitutes the entire agreement and supersedes and voids all prior communications, understandings, and agreements relating to the Product(s), including any terms of use displayed to Authorized Users via the online site of the Product(s). Alterations to the Agreement and to any Addendum to the Agreement are only valid and binding if they are recorded in writing and signed by both parties

. I. Definitions

"Authorized Users" shall mean individuals who are authorized by the Customer (which shall include those individuals authorized by the Institutions hereunder) to access the Customer's information services whether on-site or off-site via secure authentication and who are affiliated with the Customer as a student (undergraduates and postgraduates), employee (whether on a permanent or temporary basis), or a contractor of the Customer. Individuals who are not a current student, employee, or a contractor of the Customer, but who are permitted to access the Customer's information services from computer terminals within the physical premises of the Customer ("Walk-In Users"), are also deemed to be Authorized Users, but only for the time they are within the physical premises of the Customer. Walk-In Users may not be given means to access the Product(s) when they are not within the physical premises of the Customer.

"Commercial Use" shall mean use for the purpose of monetary reward (whether by or for the Customer or an Authorized User) by means of the sale, resale, loan, transfer, hire, or other form of exploitation of the Product(s). For the avoidance of doubt, neither recovery of direct cost by the Customer from Authorized Users, nor use by the Customer or Authorized Users of the Product(s) in the course of research funded by a commercial organization, shall be deemed to constitute Commercial Use.

"Educational Purposes" shall mean for the purpose of education, teaching, distance learning, private study and/or research as described in Section II below.

"Institutions" shall mean the Customer's participating institutions, if applicable.

"License" shall mean the non-exclusive, non-transferable right to access and use the Product(s) pursuant to the specific terms and conditions set forth in this Agreement.

"Product(s)" shall mean the products, materials and/or information contained therein that are subject to this Agreement. Product(s) include the Journal on Excellence in College Teaching and the archive of the Learning Communities Journal.

"Reasonable Amount" shall be determined based on guidelines set forth by 17 U.S. Code § 107 (Limitations on exclusive rights, Fair use).

"Secure Authentication" shall mean access to the Product(s) by Internet Protocol ("IP") ranges or by another means of authentication agreed between the Publisher and Customer or Institutions (if applicable) from time to time.

II. Authorized Use of Product(s)

Customer, the Institutions (if applicable), and Authorized Users may use the Product(s) for Educational Purposes as follows:

Analysis . Authorized Users shall be permitted to extract or use information contained in the Product(s) for Educational Purposes, including, but not limited to, text and data mining, extraction and manipulation of information for the purposes of illustration, explanation, example, comment, criticism, teaching, research, or analysis.

Course Packs . Customer, the Institutions, and Authorized Users may use a Reasonable Amount of the Product(s) in the preparation of course packs or other educational materials.

Digital Copy . Customer, the Institutions, and Authorized Users may download and digitally copy a Reasonable Amount of the Product(s).

Display . Customer, the Institutions, and Authorized Users shall have the right to electronically display the Product(s) to the extent necessary to further the intent and purpose of this Agreement.

Electronic Reserve . Customer, the Institutions, and Authorized Users may use a Reasonable Amount of each of the Product(s) in connection with specific courses of instruction offered by Customer.

Interlibrary Loan . The Customer and the Institutions shall be permitted to use Reasonable Amounts of the Content to fulfill occasional requests from other, non-participating institutions, a practice commonly called Interlibrary Loan ("ILL"). Customer and the Institutions shall fulfill such requests in compliance with Section 108 of the United States Copyright Law (17 USC S108, "Limitations on exclusive rights: Reproduction by libraries and archives") and the Guidelines for the Proviso of Subsection 108(2g)(2) prepared by the National Commission on New Technological Uses of Copyrighted Works (CONTU).

The electronic form of the Product(s) may be used as a source for ILL. Customer and the Institutions shall include copyright notices on all ILL transmissions. Notwithstanding anything herein to the contrary, in no event shall any non-secure electronic transmission of files be permitted.

Print Copy . Customer, the Institutions, and Authorized Users may print a Reasonable Amount of the Product(s).

Recover Copying Costs . Customer and the Institutions may charge a reasonable fee to cover costs of copying or printing portions of Product(s) for Authorized Users.

Scholarly Sharing . Authorized Users may transmit to a third party colleague in hard copy or electronically, Reasonable Amounts of the Product(s) for personal use, professional use, or Educational Purposes but in no event for Commercial Use. In addition, Authorized Users have the right to use, with appropriate credit, figures, tables, and brief excerpts from the Product(s) in the Authorized User's own scientific, scholarly, and educational works.

Text Mining . Authorized Users may use the licensed material to perform and engage in text mining/data mining activities for legitimate academic research and other Educational Purposes. Those uses beyond educational use shall require permission from the Publisher.

III. Restrictions

Except as provided herein, the institution shall make reasonable efforts to inform its authorized users not to use, alter, decompile, modify, display, or distribute the Product(s) as follows:

Alter Identification . Remove, obscure, or modify copyright notices, text acknowledging, attributions, or other means of identification or disclaimers as they appear. Alter Product(s).

Alter, decompile, adapt, or modify the Product(s), except to the extent necessary to make it perceptible on a computer screen, or as otherwise permitted in this Agreement. Alteration of words or their order is strictly prohibited.

Commercial Use . No Commercial Use of the Product(s) shall be permitted unless the Customer or an Authorized User has been granted prior written consent by an authorized representative of the Product(s). Use of all or any part of the Product(s) for any Commercial Use or for any purpose other than Educational Purposes.

Distribution . Display or distribute any part of the Product(s) on any electronic network, including without limitation, the Internet, and any other distribution medium now in existence or hereinafter created, other than by a Secure Authentication; print and distribute any portion(s) of the Product(s)s to persons or entities other than the Customer or Authorized Users, except as provided in Section II.

JECT acknowledges that the Customer cannot police or control the actions of its students, faculty, and other Authorized Users with respect to their use of the Product(s). In the event of abuse, the institution shall make prompt and reasonable efforts to heal the breach and notify the publisher.

IV. Term and Termination

This agreement shall commence on the Effective Date and shall remain in effect unless and until terminated as permitted herein (the "Term"). There is no perpetual electronic access to content made available during the term of the agreement.

JECT may terminate this Agreement if Customer violates any of the terms and conditions set forth herein. In the event of any termination of access, JECT will promptly notify the Customer of the basis for termination.

The Customer may terminate this Agreement if sufficient funds are not provided or allotted in future government-approved budgets of the Customer (or reasonably available or expected to become available from other sources at the time the Customer’s payment obligation attaches) to permit the Subscriber, in the exercise of its reasonable administrative discretion, to continue this Agreement.

In the event of any unauthorized use of the Product(s) by an Authorized User, Customer shall cooperate with JECT in the investigation of any unauthorized use of the Product(s) of which it is made aware and shall use reasonable efforts to remedy such unauthorized use and prevent its recurrence. JECT may terminate such Authorized User's access to the Product(s) after first providing reasonable notice to the Customer (in no event less than two (2) weeks) and cooperating with the Customer to avoid recurrence of any unauthorized use. In the event of any termination of access, JECT will promptly notify the Customer

. V. Refunds

In the event that a subscription is canceled by the Customer prior to the subscription end date, the following will be used as guidelines for refunds.

Electronic subscriptions . The Customer shall be entitled to a full refund within 14 days of the start of the most recent subscription term. Refunds requested after 14 days but no later than 60 days from the start of the most recent subscription term will be allowed, minus a 30% processing fee. Refunds will not be granted if requested more than 60 days after the start of the most recent subscription term.

Journal on Excellence in College Teaching Center for Teaching Excellence Miami University 317 Laws Hall Oxford, Ohio 45056

All rights reserved | Miami University | The Journal on Excellence in College Teaching

• Open access
• Published: 22 April 2024

Artificial intelligence and medical education: application in classroom instruction and student assessment using a pharmacology & therapeutics case study

• Kannan Sridharan 1 &
• Reginald P. Sequeira 1  

BMC Medical Education volume  24 , Article number:  431 ( 2024 ) Cite this article

354 Accesses

1 Altmetric

Metrics details

Artificial intelligence (AI) tools are designed to create or generate content from their trained parameters using an online conversational interface. AI has opened new avenues in redefining the role boundaries of teachers and learners and has the potential to impact the teaching-learning process.

In this descriptive proof-of- concept cross-sectional study we have explored the application of three generative AI tools on drug treatment of hypertension theme to generate: (1) specific learning outcomes (SLOs); (2) test items (MCQs- A type and case cluster; SAQs; OSPE); (3) test standard-setting parameters for medical students.

Analysis of AI-generated output showed profound homology but divergence in quality and responsiveness to refining search queries. The SLOs identified key domains of antihypertensive pharmacology and therapeutics relevant to stages of the medical program, stated with appropriate action verbs as per Bloom’s taxonomy. Test items often had clinical vignettes aligned with the key domain stated in search queries. Some test items related to A-type MCQs had construction defects, multiple correct answers, and dubious appropriateness to the learner’s stage. ChatGPT generated explanations for test items, this enhancing usefulness to support self-study by learners. Integrated case-cluster items had focused clinical case description vignettes, integration across disciplines, and targeted higher levels of competencies. The response of AI tools on standard-setting varied. Individual questions for each SAQ clinical scenario were mostly open-ended. The AI-generated OSPE test items were appropriate for the learner’s stage and identified relevant pharmacotherapeutic issues. The model answers supplied for both SAQs and OSPEs can aid course instructors in planning classroom lessons, identifying suitable instructional methods, establishing rubrics for grading, and for learners as a study guide. Key lessons learnt for improving AI-generated test item quality are outlined.

Conclusions

AI tools are useful adjuncts to plan instructional methods, identify themes for test blueprinting, generate test items, and guide test standard-setting appropriate to learners’ stage in the medical program. However, experts need to review the content validity of AI-generated output. We expect AIs to influence the medical education landscape to empower learners, and to align competencies with curriculum implementation. AI literacy is an essential competency for health professionals.

Peer Review reports

Artificial intelligence (AI) has great potential to revolutionize the field of medical education from curricular conception to assessment [ 1 ]. AIs used in medical education are mostly generative AI large language models that were developed and validated based on billions to trillions of parameters [ 2 ]. AIs hold promise in the incorporation of history-taking, assessment, diagnosis, and management of various disorders [ 3 ]. While applications of AIs in undergraduate medical training are being explored, huge ethical challenges remain in terms of data collection, maintaining anonymity, consent, and ownership of the provided data [ 4 ]. AIs hold a promising role amongst learners because they can deliver a personalized learning experience by tracking their progress and providing real-time feedback, thereby enhancing their understanding in the areas they are finding difficult [ 5 ]. Consequently, a recent survey has shown that medical students have expressed their interest in acquiring competencies related to the use of AIs in healthcare during their undergraduate medical training [ 6 ].

Pharmacology and Therapeutics (P & T) is a core discipline embedded in the undergraduate medical curriculum, mostly in the pre-clerkship phase. However, the application of therapeutic principles forms one of the key learning objectives during the clerkship phase of the undergraduate medical career. Student assessment in pharmacology & therapeutics (P&T) is with test items such as multiple-choice questions (MCQs), integrated case cluster questions, short answer questions (SAQs), and objective structured practical examination (OSPE) in the undergraduate medical curriculum. It has been argued that AIs possess the ability to communicate an idea more creatively than humans [ 7 ]. It is imperative that with access to billions of trillions of datasets the AI platforms hold promise in playing a crucial role in the conception of various test items related to any of the disciplines in the undergraduate medical curriculum. Additionally, AIs provide an optimized curriculum for a program/course/topic addressing multidimensional problems [ 8 ], although robust evidence for this claim is lacking.

The existing literature has evaluated the knowledge, attitude, and perceptions of adopting AI in medical education. Integration of AIs in medical education is the need of the hour in all health professional education. However, the academic medical fraternity facing challenges in the incorporation of AIs in the medical curriculum due to factors such as inadequate grounding in data analytics, lack of high-quality firm evidence favoring the utility of AIs in medical education, and lack of funding [ 9 ]. Open-access AI platforms are available free to users without any restrictions. Hence, as a proof-of-concept, we chose to explore the utility of three AI platforms to identify specific learning objectives (SLOs) related to pharmacology discipline in the management of hypertension for medical students at different stages of their medical training.

Study design and ethics

The present study is observational, cross-sectional in design, conducted in the Department of Pharmacology & Therapeutics, College of Medicine and Medical Sciences, Arabian Gulf University, Kingdom of Bahrain, between April and August 2023. Ethical Committee approval was not sought given the nature of this study that neither had any interaction with humans, nor collection of any personal data was involved.

Study procedure

We conducted the present study in May-June 2023 with the Poe© chatbot interface created by Quora© that provides access to the following three AI platforms:

Sage Poe [ 10 ]: A generative AI search engine developed by Anthropic © that conceives a response based on the written input provided. Quora has renamed Sage Poe as Assistant © from July 2023 onwards.

Claude-Instant [ 11 ]: A retrieval-based AI search engine developed by Anthropic © that collates a response based on pre-written responses amongst the existing databases.

ChatGPT version 3.5 [ 12 ]: A generative architecture-based AI search engine developed by OpenAI © trained on large and diverse datasets.

We queried the chatbots to generate SLOs, A-type MCQs, integrated case cluster MCQs, integrated SAQs, and OSPE test items in the domain of systemic hypertension related to the P&T discipline. Separate prompts were used to generate outputs for pre-clerkship (preclinical) phase students, and at the time of graduation (before starting residency programs). Additionally, we have also evaluated the ability of these AI platforms to estimate the proportion of students correctly answering these test items. We used the following queries for each of these objectives:

Specific learning objectives

Can you generate specific learning objectives in the pharmacology discipline relevant to undergraduate medical students during their pre-clerkship phase related to anti-hypertensive drugs?

Can you generate specific learning objectives in the pharmacology discipline relevant to undergraduate medical students at the time of graduation related to anti-hypertensive drugs?

A-type MCQs

In the initial query used for A-type of item, we specified the domains (such as the mechanism of action, pharmacokinetics, adverse reactions, and indications) so that a sample of test items generated without any theme-related clutter, shown below:

Write 20 single best answer MCQs with 5 choices related to anti-hypertensive drugs for undergraduate medical students during the pre-clerkship phase of which 5 MCQs should be related to mechanism of action, 5 MCQs related to pharmacokinetics, 5 MCQs related to adverse reactions, and 5 MCQs should be related to indications.

The MCQs generated with the above search query were not based on clinical vignettes. We queried again to generate MCQs using clinical vignettes specifically because most medical schools have adopted problem-based learning (PBL) in their medical curriculum.

Write 20 single best answer MCQs with 5 choices related to anti-hypertensive drugs for undergraduate medical students during the pre-clerkship phase using a clinical vignette for each MCQ of which 5 MCQs should be related to the mechanism of action, 5 MCQs related to pharmacokinetics, 5 MCQs related to adverse reactions, and 5 MCQs should be related to indications.

We attempted to explore whether AI platforms can provide useful guidance on standard-setting. Hence, we used the following search query.

Can you do a simulation with 100 undergraduate medical students to take the above questions and let me know what percentage of students got each MCQ correct?

Integrated case cluster MCQs

Write 20 integrated case cluster MCQs with 2 questions in each cluster with 5 choices for undergraduate medical students during the pre-clerkship phase integrating pharmacology and physiology related to systemic hypertension with a case vignette.

Write 20 integrated case cluster MCQs with 2 questions in each cluster with 5 choices for undergraduate medical students during the pre-clerkship phase integrating pharmacology and physiology related to systemic hypertension with a case vignette. Please do not include ‘none of the above’ as the choice. (This modified search query was used because test items with ‘None of the above’ option were generated with the previous search query).

Write 20 integrated case cluster MCQs with 2 questions in each cluster with 5 choices for undergraduate medical students at the time of graduation integrating pharmacology and physiology related to systemic hypertension with a case vignette.

Integrated short answer questions

Write a short answer question scenario with difficult questions based on the theme of a newly diagnosed hypertensive patient for undergraduate medical students with the main objectives related to the physiology of blood pressure regulation, risk factors for systemic hypertension, pathophysiology of systemic hypertension, pathological changes in the systemic blood vessels in hypertension, pharmacological management, and non-pharmacological treatment of systemic hypertension.

Write a short answer question scenario with moderately difficult questions based on the theme of a newly diagnosed hypertensive patient for undergraduate medical students with the main objectives related to the physiology of blood pressure regulation, risk factors for systemic hypertension, pathophysiology of systemic hypertension, pathological changes in the systemic blood vessels in hypertension, pharmacological management, and non-pharmacological treatment of systemic hypertension.

Write a short answer question scenario with questions based on the theme of a newly diagnosed hypertensive patient for undergraduate medical students at the time of graduation with the main objectives related to the physiology of blood pressure regulation, risk factors for systemic hypertension, pathophysiology of systemic hypertension, pathological changes in the systemic blood vessels in hypertension, pharmacological management, and non-pharmacological treatment of systemic hypertension.

Can you generate 5 OSPE pharmacology and therapeutics prescription writing exercises for the assessment of undergraduate medical students at the time of graduation related to anti-hypertensive drugs?

Can you generate 5 OSPE pharmacology and therapeutics prescription writing exercises containing appropriate instructions for the patients for the assessment of undergraduate medical students during their pre-clerkship phase related to anti-hypertensive drugs?

Can you generate 5 OSPE pharmacology and therapeutics prescription writing exercises containing appropriate instructions for the patients for the assessment of undergraduate medical students at the time of graduation related to anti-hypertensive drugs?

Both authors independently evaluated the AI-generated outputs, and a consensus was reached. We cross-checked the veracity of answers suggested by AIs as per the Joint National Commission Guidelines (JNC-8) and Goodman and Gilman’s The Pharmacological Basis of Therapeutics (2023), a reference textbook [ 13 , 14 ]. Errors in the A-type MCQs were categorized as item construction defects, multiple correct answers, and uncertain appropriateness to the learner’s level. Test items in the integrated case cluster MCQs, SAQs and OSPEs were evaluated with the Preliminary Conceptual Framework for Establishing Content Validity of AI-Generated Test Items based on the following domains: technical accuracy, comprehensiveness, education level, and lack of construction defects (Table  1 ). The responses were categorized as complete and deficient for each domain.

The pre-clerkship phase SLOs identified by Sage Poe, Claude-Instant, and ChatGPT are listed in the electronic supplementary materials 1 – 3 , respectively. In general, a broad homology in SLOs generated by the three AI platforms was observed. All AI platforms identified appropriate action verbs as per Bloom’s taxonomy to state the SLO; action verbs such as describe, explain, recognize, discuss, identify, recommend, and interpret are used to state the learning outcome. The specific, measurable, achievable, relevant, time-bound (SMART) SLOs generated by each AI platform slightly varied. All key domains of antihypertensive pharmacology to be achieved during the pre-clerkship (pre-clinical) years were relevant for graduating doctors. The SLOs addressed current JNC Treatment Guidelines recommended classes of antihypertensive drugs, the mechanism of action, pharmacokinetics, adverse effects, indications/contraindications, dosage adjustments, monitoring therapy, and principles of monotherapy and combination therapy.

The SLOs to be achieved by undergraduate medical students at the time of graduation identified by Sage Poe, Claude-Instant, and ChatGPT listed in electronic supplementary materials 4 – 6 , respectively. The identified SLOs emphasize the application of pharmacology knowledge within a clinical context, focusing on competencies needed to function independently in early residency stages. These SLOs go beyond knowledge recall and mechanisms of action to encompass competencies related to clinical problem-solving, rational prescribing, and holistic patient management. The SLOs generated require higher cognitive ability of the learner: action verbs such as demonstrate, apply, evaluate, analyze, develop, justify, recommend, interpret, manage, adjust, educate, refer, design, initiate & titrate were frequently used.

The MCQs for the pre-clerkship phase identified by Sage Poe, Claude-Instant, and ChatGPT listed in the electronic supplementary materials 7 – 9 , respectively, and those identified with the search query based on the clinical vignette in electronic supplementary materials ( 10 – 12 ).

All MCQs generated by the AIs in each of the four domains specified [mechanism of action (MOA); pharmacokinetics; adverse drug reactions (ADRs), and indications for antihypertensive drugs] are quality test items with potential content validity. The test items on MOA generated by Sage Poe included themes such as renin-angiotensin-aldosterone (RAAS) system, beta-adrenergic blockers (BB), calcium channel blockers (CCB), potassium channel openers, and centrally acting antihypertensives; on pharmacokinetics included high oral bioavailability/metabolism in liver [angiotensin receptor blocker (ARB)-losartan], long half-life and renal elimination [angiotensin converting enzyme inhibitors (ACEI)-lisinopril], metabolism by both liver and kidney (beta-blocker (BB)-metoprolol], rapid onset- short duration of action (direct vasodilator-hydralazine), and long-acting transdermal drug delivery (centrally acting-clonidine). Regarding the ADR theme, dry cough, angioedema, and hyperkalemia by ACEIs in susceptible patients, reflex tachycardia by CCB/amlodipine, and orthostatic hypotension by CCB/verapamil addressed. Clinical indications included the drug of choice for hypertensive patients with concomitant comorbidity such as diabetics (ACEI-lisinopril), heart failure and low ejection fraction (BB-carvedilol), hypertensive urgency/emergency (alpha cum beta receptor blocker-labetalol), stroke in patients with history recurrent stroke or transient ischemic attack (ARB-losartan), and preeclampsia (methyldopa).

Almost similar themes under each domain were identified by the Claude-Instant AI platform with few notable exceptions: hydrochlorothiazide (instead of clonidine) in MOA and pharmacokinetics domains, respectively; under the ADR domain ankle edema/ amlodipine, sexual dysfunction and fatigue in male due to alpha-1 receptor blocker; under clinical indications the best initial monotherapy for clinical scenarios such as a 55-year old male with Stage-2 hypertension; a 75-year-old man Stage 1 hypertension; a 35-year-old man with Stage I hypertension working on night shifts; and a 40-year-old man with stage 1 hypertension and hyperlipidemia.

As with Claude-Instant AI, ChatGPT-generated test items on MOA were mostly similar. However, under the pharmacokinetic domain, immediate- and extended-release metoprolol, the effect of food to enhance the oral bioavailability of ramipril, and the highest oral bioavailability of amlodipine compared to other commonly used antihypertensives were the themes identified. Whereas the other ADR themes remained similar, constipation due to verapamil was a new theme addressed. Notably, in this test item, amlodipine was an option that increased the difficulty of this test item because amlodipine therapy is also associated with constipation, albeit to a lesser extent, compared to verapamil. In the clinical indication domain, the case description asking “most commonly used in the treatment of hypertension and heart failure” is controversial because the options listed included losartan, ramipril, and hydrochlorothiazide but the suggested correct answer was ramipril. This is a good example to stress the importance of vetting the AI-generated MCQ by experts for content validity and to assure robust psychometrics. The MCQ on the most used drug in the treatment of “hypertension and diabetic nephropathy” is more explicit as opposed to “hypertension and diabetes” by Claude-Instant because the therapeutic concept of reducing or delaying nephropathy must be distinguished from prevention of nephropathy, although either an ACEI or ARB is the drug of choice for both indications.

It is important to align student assessment to the curriculum; in the PBL curriculum, MCQs with a clinical vignette are preferred. The modification of the query specifying the search to generate MCQs with a clinical vignette on domains specified previously gave appropriate output by all three AI platforms evaluated (Sage Poe; Claude- Instant; Chat GPT). The scenarios generated had a good clinical fidelity and educational fit for the pre-clerkship student perspective.

The errors observed with AI outputs on the A-type MCQs are summarized in Table  2 . No significant pattern was observed except that Claude-Instant© generated test items in a stereotyped format such as the same choices for all test items related to pharmacokinetics and indications, and all the test items in the ADR domain are linked to the mechanisms of action of drugs. This illustrates the importance of reviewing AI-generated test items by content experts for content validity to ensure alignment with evidence-based medicine and up-to-date treatment guidelines.

The test items generated by ChatGPT had the advantage of explanations supplied rendering these more useful for learners to support self-study. The following examples illustrate this assertion: “ A patient with hypertension is started on a medication that works by blocking beta-1 receptors in the heart (metoprolol)”. Metoprolol is a beta blocker that works by blocking beta-1 receptors in the heart, which reduces heart rate and cardiac output, resulting in a decrease in blood pressure. However, this explanation is incomplete because there is no mention of other less important mechanisms, of beta receptor blockers on renin release. Also, these MCQs were mostly recall type: Which of the following medications is known to have a significant first-pass effect? The explanation reads: propranolol is known to have a significant first pass-effect, meaning that a large portion of the drug is metabolized by the liver before it reaches systemic circulation. Losartan, amlodipine, ramipril, and hydrochlorothiazide do not have significant first-pass effect. However, it is also important to extend the explanation further by stating that the first-pass effect of propranolol does not lead to total loss of pharmacological activity because the metabolite hydroxy propranolol also has potent beta-blocking activity. Another MCQ test item had a construction defect: “A patient with hypertension is started on a medication that can cause photosensitivity. Which of the following medications is most likely responsible?” Options included: losartan, amlodipine, ramipril, hydrochlorothiazide, hydrochlorothiazide/triamterene. The explanation states that hydrochlorothiazide (HCTZ) can cause photosensitivity, which is increased sensitivity to sunlight that can cause skin rashes and sunburns. Losartan, amlodipine, ramipril, and HCTZ/triamterene are less likely to cause photosensitivity. However, it is irrational to claim that HCTZ/triamterene, one of the wrong options, is less likely to cause photosensitivity. The difficulty index of such test items is likely unacceptable in high-stake tests to make equitable psychometric decisions.

The responses from Sage Poe, Claude-Instant, and ChatGPT on guidance for standard-setting methods are listed in the electronic supplementary materials 13 – 15 . Sage Poe offered this information with the caveat that “the results of this simulation were purely hypothetical and do not reflect the actual performance of any real students”. The simulation estimates of examinees correctly answering test items ranged from 60 to 90% for the mechanism of action, 65–85% for pharmacokinetics, 60–80% for adverse reactions, and 75–90% for clinical indications of antihypertensives.

In contrast, Claude-Instant admitted that “Performing a full simulation with 100 medical students is beyond the scope of what I can do as an AI assistant. I can generate sample MCQ questions and answers, but I do not have access to a population of medical students to administer the questions. A true simulation that administers questions to real students and analyzes the results would require access to a database of medical students, a platform to deliver the questions, and code to analyze the responses. As an AI assistant, I lack these capabilities. While I cannot perform a full simulation, here are some considerations for administering MCQ questions to medical students: provide details on the purpose and instructions for the test to ensure students understand what is expected out of them; consider including some easier questions in addition to more challenging ones to boost students’ confidence and motivation; avoid ambiguous wording and ensure each question has only one unambiguously correct answer option; when analyzing results, look for questions factors that may have impacted performance, like unclear wording or answer options that are too similar. Focusing on weak questions can help improve future tests; compare performance across different student groups to identify areas where more teaching may be needed; provide detailed answer explanations and review sessions after the test to help students learn from any errors and reinforce correct concepts”. However, ChatGPT offered the simulation result estimates for the 20 MCQ on antihypertensive drugs: mechanism of action ranged from 67 to 92%, pharmacokinetics 63–86%, adverse effects 65–82%, and clinical indications 64–89%. Furthermore, it also stated that “Overall, the performance of the students was quite good, with most questions having a response rate of over 70%. However, there were some questions where the projected correct response rate was rather low, such as Question #5 (Mechanism of Action of Hydrochlorothiazide; see Electronic Supplementary Material 12 ) and Question 18 (Indications for Verapamil; see Electronic Supplementary Material 10 ). This may suggest areas where students need more focused education or review.”

We asked AI assistants to generate 20 integrated case cluster MCQs with 2 test items in each cluster with five options for undergraduate medical students in the pre-clerkship phase integrating pharmacology and physiology related to systemic hypertension with a case vignette and the responses by Sage Poe, Claude-Instant, and ChatGPT are listed in the electronic supplementary materials ( 16 – 18 ). In all instances, the test items generated had focused case descriptions in the form of a clinical vignette, and horizontal integration across the pathophysiology of hypertension and pharmacology of antihypertensive drugs. These test items mostly targeted the ‘knows (knowledge)’ or ‘knows how (competence)’ level on Miller’s pyramid and are suitable for assessing the clinical competence of pre-clerkship medical students, especially in an integrated PBL curriculum. Both the AI assistants generated excellent clinical vignettes and themes; however, most of the cluster MCQs by ChatGPT had “None of the above” as an option, which is often considered a test item construction flaw. Notwithstanding these limitations, case cluster integrated test items are valuable for learners to integrate their knowledge of different basic medical sciences and their application to clinical sciences. This integrated approach can be used for both instructional and student assessment purposes to make the course more meaningful. Indeed, one of the basic tenets of PBL is curriculum integration.

In the next query, we asked AI assistants to write integrated case cluster MCQs with 2 test items in each cluster with 5 options for undergraduate medical students at the time of graduation integrating pharmacology and physiology of hypertension with a case vignette. The responses are listed in the electronic supplementary materials 19 – 21 . Claude-Instant has provided only partial responses to both the queries. Overall, the information provided in case vignettes requires the learner to interpret the clinical information and make therapeutic decisions. Such test items are appropriate for assessing the minimum competency of medical school graduates to start internship or residency programs; these MCQs are generally comparable to test items used in USMLE Part 2. The evaluation of the integrated case cluster test items provided by all the AI platforms is summarized in Table  3 . In general, the AI outputs had at least one construction defect, and did not include all the core sub-topics to a sufficient degree.

When asked to simulate the above-generated test items to 100 undergraduate medical students at the time of graduation and provide a percentage of students who are likely to correctly answer each item, the AI assistant responded:

“ As an AI language model, I cannot simulate the responses of 100 undergraduate medical students likely to obtain the correct answer for each question. However, I can provide some general information about the difficulty level of each question based on the complexity of the concepts being tested and the level of integration between pharmacology and physiology ”.

AI assistant went on to clarify that questions that require knowledge of basic pharmacology principles, such as the mechanism of action of specific drugs, are likely to be easier for students to answer correctly. Test items that require an understanding of the physiological mechanisms underlying hypertension and correlating with symptoms are likely to be more challenging for students. The AI assistant sorted these test items into two categories accordingly. Overall, the difficulty level of the test item is based on the level of integration between pharmacology and pathophysiology. Test items that require an understanding of both pharmacological and physiological mechanisms are likely to be more challenging for students requiring a strong foundation in both pharmacology and physiology concepts to be able to correctly answer integrated case-cluster MCQs.

Short answer questions

The responses to a search query on generating SAQs appropriate to the pre-clerkship phase Sage Poe, Claude-Instant, and ChatGPT generated items are listed in the electronic supplementary materials 22 – 24 for difficult questions and 25–27 for moderately difficult questions.

It is apparent from these case vignette descriptions that the short answer question format varied. Accordingly, the scope for asking individual questions for each scenario is open-ended. In all instances, model answers are supplied which are helpful for the course instructor to plan classroom lessons, identify appropriate instructional methods, and establish rubrics for grading the answer scripts, and as a study guide for students.

We then wanted to see to what extent AI can differentiate the difficulty of the SAQ by replacing the search term “difficult” with “moderately difficult” in the above search prompt: the changes in the revised case scenarios are substantial. Perhaps the context of learning and practice (and the level of the student in the MD/medical program) may determine the difficulty level of SAQ generated. It is worth noting that on changing the search from cardiology to internal medicine rotation in Sage Poe the case description also changed. Thus, it is essential to select an appropriate AI assistant, perhaps by trial and error, to generate quality SAQs. Most of the individual questions tested stand-alone knowledge and did not require students to demonstrate integration.

The responses of Sage Poe, Claude-Instant, and ChatGPT for the search query to generate SAQs at the time of graduation are listed in the electronic supplementary materials 28 – 30 . It is interesting to note how AI assistants considered the stage of the learner while generating the SAQ. The response by Sage Poe is illustrative for comparison. “You are a newly graduated medical student who is working in a hospital” versus “You are a medical student in your pre-clerkship.”

Some questions were retained, deleted, or modified to align with competency appropriate to the context (Electronic Supplementary Materials 28 – 30 ). Overall, the test items at both levels from all AI platforms were technically accurate and thorough addressing the topics related to different disciplines (Table  3 ). The differences in learning objective transition are summarized in Table  4 . A comparison of learning objectives revealed that almost all objectives remained the same except for a few (Table  5 ).

A similar trend was apparent with test items generated by other AI assistants, such as ChatGPT. The contrasting differences in questions are illustrated by the vertical integration of basic sciences and clinical sciences (Table  6 ).

Taken together, these in-depth qualitative comparisons suggest that AI assistants such as Sage Poe and ChatGPT consider the learner’s stage of training in designing test items, learning outcomes, and answers expected from the examinee. It is critical to state the search query explicitly to generate quality output by AI assistants.

The OSPE test items generated by Claude-Instant and ChatGPT appropriate to the pre-clerkship phase (without mentioning “appropriate instructions for the patients”) are listed in the electronic supplementary materials 31 and 32 and with patient instructions on the electronic supplementary materials 33 and 34 . For reasons unknown, Sage Poe did not provide any response to this search query.

The five OSPE items generated were suitable to assess the prescription writing competency of pre-clerkship medical students. The clinical scenarios identified by the three AI platforms were comparable; these scenarios include patients with hypertension and impaired glucose tolerance in a 65-year-old male, hypertension with chronic kidney disease (CKD) in a 55-year-old woman, resistant hypertension with obstructive sleep apnea in a 45-year-old man, and gestational hypertension at 32 weeks in a 35-year-old (Claude-Instant AI). Incorporating appropriate instructions facilitates the learner’s ability to educate patients and maximize safe and effective therapy. The OSPE item required students to write a prescription with guidance to start conservatively, choose an appropriate antihypertensive drug class (drug) based on the patients’ profile, specifying drug name, dose, dosing frequency, drug quantity to be dispensed, patient name, date, refill, and caution as appropriate, in addition to prescribers’ name, signature, and license number. In contrast, ChatGPT identified clinical scenarios to include patients with hypertension and CKD, hypertension and bronchial asthma, gestational diabetes, hypertension and heart failure, and hypertension and gout (ChatGPT). Guidance for dosage titration, warnings to be aware, safety monitoring, and frequency of follow-up and dose adjustment. These test items are designed to assess learners’ knowledge of P & T of antihypertensives, as well as their ability to provide appropriate instructions to patients. These clinical scenarios for writing prescriptions assess students’ ability to choose an appropriate drug class, write prescriptions with proper labeling and dosing, reflect drug safety profiles, and risk factors, and make modifications to meet the requirements of special populations. The prescription is required to state the drug name, dose, dosing frequency, patient name, date, refills, and cautions or instructions as needed. A conservative starting dose, once or twice daily dosing frequency based on the drug, and instructions to titrate the dose slowly if required.

The responses from Claude-Instant and ChatGPT for the search query related to generating OSPE test items at the time of graduation are listed in electronic supplementary materials 35 and 36 . In contrast to the pre-clerkship phase, OSPEs generated for graduating doctors’ competence assessed more advanced drug therapy comprehension. For example, writing a prescription for:

(1) A 65-year- old male with resistant hypertension and CKD stage 3 to optimize antihypertensive regimen required the answer to include starting ACEI and diuretic, titrating the dosage over two weeks, considering adding spironolactone or substituting ACEI with an ARB, and need to closely monitor serum electrolytes and kidney function closely.

(2) A 55-year-old woman with hypertension and paroxysmal arrhythmia required the answer to include switching ACEI to ARB due to cough, adding a CCB or beta blocker for rate control needs, and adjusting the dosage slowly and monitoring for side effects.

(3) A 45-year-old man with masked hypertension and obstructive sleep apnea require adding a centrally acting antihypertensive at bedtime and increasing dosage as needed based on home blood pressure monitoring and refer to CPAP if not already using one.

(4) A 75-year-old woman with isolated systolic hypertension and autonomic dysfunction to require stopping diuretic and switching to an alpha blocker, upward dosage adjustment and combining with other antihypertensives as needed based on postural blood pressure changes and symptoms.

(5) A 35-year-old pregnant woman with preeclampsia at 29 weeks require doubling methyldopa dose and consider adding labetalol or nifedipine based on severity and educate on signs of worsening and to follow-up immediately for any concerning symptoms.

These case scenarios are designed to assess the ability of the learner to comprehend the complexity of antihypertensive regimens, make evidence-based regimen adjustments, prescribe multidrug combinations based on therapeutic response and tolerability, monitor complex patients for complications, and educate patients about warning signs and follow-up.

A similar output was provided by ChatGPT, with clinical scenarios such as prescribing for patients with hypertension and myocardial infarction; hypertension and chronic obstructive pulmonary airway disease (COPD); hypertension and a history of angina; hypertension and a history of stroke, and hypertension and advanced renal failure. In these cases, wherever appropriate, pharmacotherapeutic issues like taking ramipril after food to reduce side effects such as giddiness; selection of the most appropriate beta-blocker such as nebivolol in patients with COPD comorbidity; the importance of taking amlodipine at the same time every day with or without food; preference for telmisartan among other ARBs in stroke; choosing furosemide in patients with hypertension and edema and taking the medication with food to reduce the risk of gastrointestinal adverse effect are stressed.

The AI outputs on OSPE test times were observed to be technically accurate, thorough in addressing core sub-topics suitable for the learner’s level and did not have any construction defects (Table  3 ). Both AIs provided the model answers with explanatory notes. This facilitates the use of such OSPEs for self-assessment by learners for formative assessment purposes. The detailed instructions are helpful in creating optimized therapy regimens, and designing evidence-based regimens, to provide appropriate instructions to patients with complex medical histories. One can rely on multiple AI sources to identify, shortlist required case scenarios, and OSPE items, and seek guidance on expected model answers with explanations. The model answer guidance for antihypertensive drug classes is more appropriate (rather than a specific drug of a given class) from a teaching/learning perspective. We believe that these scenarios can be refined further by providing a focused case history along with relevant clinical and laboratory data to enhance clinical fidelity and bring a closer fit to the competency framework.

In the present study, AI tools have generated SLOs that comply with the current principles of medical education [ 15 ]. AI tools are valuable in constructing SLOs and so are especially useful for medical fraternities where training in medical education is perceived as inadequate, more so in the early stages of their academic career. Data suggests that only a third of academics in medical schools have formal training in medical education [ 16 ] which is a limitation. Thus, the credibility of alternatives, such as the AIs, is evaluated to generate appropriate course learning outcomes.

We observed that the AI platforms in the present study generated quality test items suitable for different types of assessment purposes. The AI-generated outputs were similar with minor variation. We have used generative AIs in the present study that could generate new content from their training dataset [ 17 ]. Problem-based and interactive learning approaches are referred to as “bottom-up” where learners obtain first-hand experience in solving the cases first and then indulge in discussion with the educators to refine their understanding and critical thinking skills [ 18 ]. We suggest that AI tools can be useful for this approach for imparting the core knowledge and skills related to Pharmacology and Therapeutics to undergraduate medical students. A recent scoping review evaluating the barriers to writing quality test items based on 13 studies has concluded that motivation, time constraints, and scheduling were the most common [ 19 ]. AI tools can be valuable considering the quick generation of quality test items and time management. However, as observed in the present study, the AI-generated test items nevertheless require scrutiny by faculty members for content validity. Moreover, it is important to train faculty in AI technology-assisted teaching and learning. The General Medical Council recommends taking every opportunity to raise the profile of teaching in medical schools [ 20 ]. Hence, both the academic faculty and the institution must consider investing resources in AI training to ensure appropriate use of the technology [ 21 ].

The AI outputs assessed in the present study had errors, particularly with A-type MCQs. One notable observation was that often the AI tools were unable to differentiate the differences between ACEIs and ARBs. AI platforms access several structured and unstructured data, in addition to images, audio, and videos. Hence, the AI platforms can commit errors due to extracting details from unauthenticated sources [ 22 ] created a framework identifying 28 factors for reconstructing the path of AI failures and for determining corrective actions. This is an area of interest for AI technical experts to explore. Also, this further iterates the need for human examination of test items before using them for assessment purposes.

There are concerns that AIs can memorize and provide answers from their training dataset, which they are not supposed to do [ 23 ]. Hence, the use of AIs-generated test items for summative examinations is debatable. It is essential to ensure and enhance the security features of AI tools to reduce or eliminate cross-contamination of test items. Researchers have emphasized that AI tools will only reach their potential if developers and users can access full-text non-PDF formats that help machines comprehend research papers and generate the output [ 24 ].

AI platforms may not always have access to all standard treatment guidelines. However, in the present study, it was observed that all three AI platforms generally provided appropriate test items regarding the choice of medications, aligning with recommendations from contemporary guidelines and standard textbooks in pharmacology and therapeutics. The prompts used in the study were specifically focused on the pre-clerkship phase of the undergraduate medical curriculum (and at the time of their graduation) and assessed fundamental core concepts, which were also reflected in the AI outputs. Additionally, the recommended first-line antihypertensive drug classes have been established for several decades, and information regarding their pharmacokinetics, ADRs, and indications is well-documented in the literature.

Different paradigms and learning theories have been proposed to support AI in education. These paradigms include AI- directed (learner as recipient), AI-supported (learner as collaborator), and AI-empowered (learner as leader) that are based on Behaviorism, Cognitive-Social constructivism, and Connectivism-Complex adaptive systems, respectively [ 25 ]. AI techniques have potential to stimulate and advance instructional and learning sciences. More recently a three- level model that synthesizes and unifies existing learning theories to model the roles of AIs in promoting learning process has been proposed [ 26 ]. The different components of our study rely upon these paradigms and learning theories as the theoretical underpinning.

Strengths and limitations

To the best of our knowledge, this is the first study evaluating the utility of AI platforms in generating test items related to a discipline in the undergraduate medical curriculum. We have evaluated the AI’s ability to generate outputs related to most types of assessment in the undergraduate medical curriculum. The key lessons learnt for improving the AI-generated test item quality from the present study are outlined in Table  7 . We used a structured framework for assessing the content validity of the test items. However, we have demonstrated using a single case study (hypertension) as a pilot experiment. We chose to evaluate anti-hypertensive drugs as it is a core learning objective and one of the most common disorders relevant to undergraduate medical curricula worldwide. It would be interesting to explore the output from AI platforms for other common (and uncommon/region-specific) disorders, non-/semi-core objectives, and disciplines other than Pharmacology and Therapeutics. An area of interest would be to look at the content validity of the test items generated for different curricula (such as problem-based, integrated, case-based, and competency-based) during different stages of the learning process. Also, we did not attempt to evaluate the generation of flowcharts, algorithms, or figures for generating test items. Another potential area for exploring the utility of AIs in medical education would be repeated procedural practices such as the administration of drugs through different routes by trainee residents [ 27 ]. Several AI tools have been identified for potential application in enhancing classroom instructions and assessment purposes pending validation in prospective studies [ 28 ]. Lastly, we did not administer the AI-generated test items to students and assessed their performance and so could not comment on the validity of test item discrimination and difficulty indices. Additionally, there is a need to confirm the generalizability of the findings to other complex areas in the same discipline as well as in other disciplines that pave way for future studies. The conceptual framework used in the present study for evaluating the AI-generated test items needs to be validated in a larger population. Future studies may also try to evaluate the variations in the AI outputs with repetition of the same queries.

Notwithstanding ongoing discussions and controversies, AI tools are potentially useful adjuncts to optimize instructional methods, test blueprinting, test item generation, and guidance for test standard-setting appropriate to learners’ stage in the medical program. However, experts need to critically review the content validity of AI-generated output. These challenges and caveats are to be addressed before the use of widespread use of AIs in medical education can be advocated.

Data availability

All the data included in this study are provided as Electronic Supplementary Materials.

Tolsgaard MG, Pusic MV, Sebok-Syer SS, Gin B, Svendsen MB, Syer MD, Brydges R, Cuddy MM, Boscardin CK. The fundamentals of Artificial Intelligence in medical education research: AMEE Guide 156. Med Teach. 2023;45(6):565–73.

Article   Google Scholar  

Sriwastwa A, Ravi P, Emmert A, Chokshi S, Kondor S, Dhal K, Patel P, Chepelev LL, Rybicki FJ, Gupta R. Generative AI for medical 3D printing: a comparison of ChatGPT outputs to reference standard education. 3D Print Med. 2023;9(1):21.

Azer SA, Guerrero APS. The challenges imposed by artificial intelligence: are we ready in medical education? BMC Med Educ. 2023;23(1):680.

Masters K. Ethical use of Artificial Intelligence in Health Professions Education: AMEE Guide 158. Med Teach. 2023;45(6):574–84.

Nagi F, Salih R, Alzubaidi M, Shah H, Alam T, Shah Z, Househ M. Applications of Artificial Intelligence (AI) in Medical Education: a scoping review. Stud Health Technol Inf. 2023;305:648–51.

Google Scholar  

Mehta N, Harish V, Bilimoria K, et al. Knowledge and attitudes on artificial intelligence in healthcare: a provincial survey study of medical students. MedEdPublish. 2021;10(1):75.

Mir MM, Mir GM, Raina NT, Mir SM, Mir SM, Miskeen E, Alharthi MH, Alamri MMS. Application of Artificial Intelligence in Medical Education: current scenario and future perspectives. J Adv Med Educ Prof. 2023;11(3):133–40.

Garg T. Artificial Intelligence in Medical Education. Am J Med. 2020;133(2):e68.

Matheny ME, Whicher D, Thadaney IS. Artificial intelligence in health care: a report from the National Academy of Medicine. JAMA. 2020;323(6):509–10.

Sage Poe. Available at: https://poe.com/Assistant (Accessed on. 3rd June 2023).

Claude-Instant: Available at: https://poe.com/Claude-instant (Accessed on 3rd. June 2023).

ChatGPT: Available at: https://poe.com/ChatGPT (Accessed on 3rd. June 2023).

James PA, Oparil S, Carter BL, Cushman WC, Dennison-Himmelfarb C, Handler J, Lackland DT, LeFevre ML, MacKenzie TD, Ogedegbe O, Smith SC Jr, Svetkey LP, Taler SJ, Townsend RR, Wright JT Jr, Narva AS, Ortiz E. 2014 evidence-based guideline for the management of high blood pressure in adults: report from the panel members appointed to the Eighth Joint National Committee (JNC 8). JAMA. 2014;311(5):507–20.

Eschenhagen T. Treatment of hypertension. In: Brunton LL, Knollmann BC, editors. Goodman & Gilman’s the pharmacological basis of therapeutics. 14th ed. New York: McGraw Hill; 2023.

Shabatura J. September. Using Bloom’s taxonomy to write effective learning outcomes. https://tips.uark.edu/using-blooms-taxonomy/ (Accessed on 19th 2023).

Trainor A, Richards JB. Training medical educators to teach: bridging the gap between perception and reality. Isr J Health Policy Res. 2021;10(1):75.

Boscardin C, Gin B, Golde PB, Hauer KE. ChatGPT and generative artificial intelligence for medical education: potential and opportunity. Acad Med. 2023. https://doi.org/10.1097/ACM.0000000000005439 . (Published ahead of print).

Duong MT, Rauschecker AM, Rudie JD, Chen PH, Cook TS, Bryan RN, Mohan S. Artificial intelligence for precision education in radiology. Br J Radiol. 2019;92(1103):20190389.

Karthikeyan S, O’Connor E, Hu W. Barriers and facilitators to writing quality items for medical school assessments - a scoping review. BMC Med Educ. 2019;19(1):123.

Developing teachers and trainers in undergraduate medical education. Advice supplementary to Tomorrow’s Doctors. (2009). https://www.gmc-uk.org/-/media/documents/Developing_teachers_and_trainers_in_undergraduate_medical_education___guidance_0815.pdf_56440721.pdf (Accessed on 19th September 2023).

Cooper A, Rodman A. AI and Medical Education - A 21st-Century Pandora’s Box. N Engl J Med. 2023;389(5):385–7.

Chanda SS, Banerjee DN. Omission and commission errors underlying AI failures. AI Soc. 2022;17:1–24.

Narayanan A, Kapoor S. ‘GPT-4 and Professional Benchmarks: The Wrong Answer to the Wrong Question’. Substack newsletter. AI Snake Oil (blog). https://aisnakeoil.substack.com/p/gpt-4-and-professional-benchmarks (Accessed on 19th September 2023).

Brainard J. November. As scientists face a flood of papers, AI developers aim to help. Science, 21 2023. doi.10.1126/science.adn0669.

Ouyang F, Jiao P. Artificial intelligence in education: the three paradigms. Computers Education: Artif Intell. 2021;2:100020.

Gibson D, Kovanovic V, Ifenthaler D, Dexter S, Feng S. Learning theories for artificial intelligence promoting learning processes. Br J Edu Technol. 2023;54(5):1125–46.

Guerrero DT, Asaad M, Rajesh A, Hassan A, Butler CE. Advancing Surgical Education: the Use of Artificial Intelligence in Surgical Training. Am Surg. 2023;89(1):49–54.

Lee S. AI tools for educators. EIT InnoEnergy Master School Teachers Conference. 2023. https://www.slideshare.net/ignatia/ai-toolkit-for-educators?from_action=save (Accessed on 24th September 2023).

Download references

Author information

Authors and affiliations.

Department of Pharmacology & Therapeutics, College of Medicine & Medical Sciences, Arabian Gulf University, Manama, Kingdom of Bahrain

Kannan Sridharan & Reginald P. Sequeira

You can also search for this author in PubMed   Google Scholar

Contributions

RPS– Conceived the idea; KS– Data collection and curation; RPS and KS– Data analysis; RPS and KS– wrote the first draft and were involved in all the revisions.

Corresponding author

Correspondence to Kannan Sridharan .

Ethics declarations

Ethics approval and consent to participate.

Not applicable as neither there was any interaction with humans, nor any personal data was collected in this research study.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Sridharan, K., Sequeira, R.P. Artificial intelligence and medical education: application in classroom instruction and student assessment using a pharmacology & therapeutics case study. BMC Med Educ 24 , 431 (2024). https://doi.org/10.1186/s12909-024-05365-7

Download citation

Received : 26 September 2023

Accepted : 28 March 2024

Published : 22 April 2024

DOI : https://doi.org/10.1186/s12909-024-05365-7

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Medical education
• Pharmacology
• Therapeutics

BMC Medical Education

ISSN: 1472-6920

• Submission enquiries: [email protected]
• General enquiries: [email protected]

• Open access
• Published: 23 April 2024

A mixed methods evaluation of the impact of ECHO ® telementoring model for capacity building of community health workers in India

• Rajmohan Panda 1 ,
• Supriya Lahoti   ORCID: orcid.org/0000-0001-6826-5273 2 ,
• Nivedita Mishra 2 ,
• Rajath R. Prabhu 3 ,
• Kalpana Singh 4 ,
• Apoorva Karan Rai 2 &
• Kumud Rai 2  

Human Resources for Health volume  22 , Article number:  26 ( 2024 ) Cite this article

61 Accesses

1 Altmetric

Metrics details

Introduction

India has the largest cohort of community health workers with one million Accredited Social Health Activists (ASHAs). ASHAs play vital role in providing health education and promoting accessible health care services in the community. Despite their potential to improve the health status of people, they remain largely underutilized because of their limited knowledge and skills. Considering this gap, Extension for Community Healthcare Outcomes (ECHO) ® India, in collaboration with the National Health System Resource Centre (NHSRC), implemented a 15-h (over 6 months) refresher training for ASHAs using a telementoring interface. The present study intends to assess the impact of the training program for improving the knowledge and skills of ASHA workers.

We conducted a pre–post quasi-experimental study using a convergent parallel mixed-method approach. The quantitative survey ( n  = 490) assessed learning competence, performance, and satisfaction of the ASHAs. In addition to the above, in-depth interviews with ASHAs ( n  = 12) and key informant interviews with other stakeholders ( n  = 9) examined the experience and practical applications of the training. Inferences from the quantitative and qualitative approaches were integrated during the reporting stage and presented using an adapted Moore’s Expanded Outcomes Framework.

There was a statistically significant improvement in learning ( p =  0.038) and competence ( p =  0.01) after attending the training. Participants were satisfied with the opportunity provided by the teleECHO™ sessions to upgrade their knowledge. However, internet connectivity, duration and number of participants in the sessions were identified as areas that needed improvement for future training programs. An improvement in confidence to communicate more effectively with the community was reported. Positive changes in the attitudes of ASHAs towards patient and community members were also reported after attending the training. The peer-to-peer learning through case-based discussion approach helped ensure that the training was relevant to the needs and work of the ASHAs.

Conclusions

The ECHO Model ™ was found effective in improving and updating the knowledge and skills of ASHAs across different geographies in India. Efforts directed towards knowledge upgradation of ASHAs are crucial for strengthening the health system at the community level. The findings of this study can be used to guide future training programs.

Trial registration The study has been registered at the Clinical Trials Registry, India (CTRI/2021/10/037189) dated 08/10/2021.

Peer Review reports

The Alma Ata Declaration of 1978 has recognized primary health care as an essential element for improving community health. Community health workers (CHWs) have the potential to complement an overstrained health workforce and enhance primary healthcare access and quality [ 1 ]. Low- and middle-income countries (LMICs) face a triple burden of low density of doctors and nurse-midwives, low government expenditure on health, and disproportionately larger poor health outcomes [ 2 ]. The roles and responsibilities of CHWs vary across LMICs [ 3 ]. A systematic review has documented that the socio-cultural, economic, health system, and political context in which CHW interventions operate in LMICs influence the implementation and success of interventions [ 4 ].

The National Rural Health Mission (NRHM), India introduced Accredited Social Health Activists (ASHAs) as female CHWs in 2005. The ASHAs are women volunteers selected from the local village and were initially conceptualized with a vision to improve maternal and child health in the country; however, over time, they are now involved in different national health programmes [ 5 , 6 ]. Despite their potential to contribute to preventive and promotive healthcare, they remain largely underutilized because of their limited knowledge and skills [ 1 ]. The World Health Organisation (WHO) has suggested ‘regular training and supervision’ for CHWs to fulfil their role successfully [ 7 ]. In India, the health system lacks methods for continuous education and routine upgradation of the ASHA’s skills [ 8 , 9 , 10 ].

In LMICs, digital training programs can help expand the reach of training to large numbers of healthcare workers at a low cost without interfering with the delivery of routine healthcare services [ 11 , 12 ]. An evidence-mapping study of 88 studies that used technology for training CHWs in LMICs found that the focus of trainings was maternal and child health and other high-burden diseases were neglected [ 13 ]. In India, studies evaluating digital trainings for CHWs have focussed on specific diseases or have been limited to specific states in the last decade [ 10 , 14 ]. This study was conducted across multiple states. More such studies with larger sample size are needed on the evaluation of such training initiatives in India [ 13 , 15 , 16 ].

Project Extension for Community Healthcare Outcomes (ECHO) presents an educational opportunity for capacity-building through a telementoring platform that uses video conferencing to create a continuous loop of learning and peer support. The sessions are facilitated by didactic presentation and case-based learning that allows problem-solving through shared best practices [ 17 ]. ECHO India, in collaboration with National Health System Resource Centre (NHSRC), provided refresher training for ASHAs [ 18 ]. There is increasing evidence of the positive effect of ECHO training on medical provider’s learning and self-efficacy. However, its value as a training platform to CHWs in LMICs is limited. Previous studies that evaluated the use of the ECHO Model ™ for CHWs focussed on specific diseases and were conducted in high-income countries (HICs) [ 19 , 20 , 21 ]. For the adoption of digital technology, CHWs in LMICs encounter challenges such as poor proficiency levels in accessing and using digital platforms, limited access to troubleshooting, poor internet connectivity, and in-house support for resolving issues [ 22 ]. The present study was designed to assess the impact of the ECHO telementoring model for improving the knowledge and skills of ASHA workers in delivering comprehensive health services. This will provide new insights for measuring outcomes of digital training programs for CHWs (ASHA workers).

Study design

A pre–post quasi-experimental design using a convergent parallel mixed-method approach [ 23 ] was employed. The quantitative and qualitative data were collected concurrently. Inferences from both approaches were integrated during the reporting stage. This allowed for a comprehensive understanding of the effect of training on the knowledge and skills of ASHAs.

The ECHO training intervention and curriculum

Project ECHO ® designed a 15-h (over 6 months from October 2021 to March 2022), virtual, refresher training program to enhance the capacity of ASHAs to deliver counselling services for comprehensive healthcare in four states ( n  = 2293). Each session lasted for 90 min. The ECHO NHSRC training used a “hub and spoke” structure in which a multidisciplinary team of experts (trainers) based at a regional academic medical centre (the “hub”) engaged with the ASHAs (the “spokes”) [ 24 ] who attended the sessions from dedicated learning sites (PHCs). Each site also had a coordinator who would help facilitate the discussions and questions. The training curriculum was developed based on the NHSRC ‘ASHA training modules’ [ 18 ] in the regional languages in consultation with partners (hub-leaders and trainers). It comprised 10 sessions covering a range of topics, such as maternal health, new-born care, child health, nutrition, reproductive health, violence against women, tuberculosis, vector-borne diseases, non-communicable diseases, COVID-19, palliative care, and mental health. The training presentations included text with visual learning methods, such as images, videos, and links to training resources.

Study settings

The evaluation study was conducted in four states of India, where training sessions were held. These states represented the four geographical regions—northern (Himachal Pradesh) ( n =  499), southern (Tamil Nadu) ( n =  500), eastern (West Bengal) ( n =  618), and north-eastern (Sikkim) ( n =  676). The intervention (training sessions) was completed in March 2022. The end-point data were collected from March 2022 to May 2022.

Study participants and recruitment

Simple random sampling was used to select the ASHAs from each state for the quantitative survey. The participants were recruited from a list of ASHAs who would be receiving the ECHO NHSRC training. To be included, ASHAs had to be enrolled in the refresher training, planning to continue working for the next 10 months, with available contact details and consenting voluntarily. The ASHAs were contacted through mobile phones in each state. Key informant interviews (KIIs) were conducted with hub leaders who were involved in implementing the training, trainers (faculty) who delivered the lectures, and in-depth interviews (IDIs) with ASHAs.

Sample size

The sample size for the quantitative study was estimated by assuming a 25% improvement in knowledge and skills, 80% power, and a design effect factor of 1.7%. An adjustment of 30% loss to follow up and 20% non-response (from previous experience) led to a sample of 591 participants across four states, i.e., 148 participants from each state. For the qualitative study, purposive sampling with maximum variation across age, education, practice sites, and years of work experience was used for the selection of the participants. A total of 12 IDIs were conducted with ASHAs and nine KIIs with stakeholders (Additional file 2 : Appendix S2).

Study tools and data collection

For quantitative data collection, a structured questionnaire was designed through a collaborative approach with the research and program implementation team. The knowledge of ASHAs was assessed by a combination of 18 technical questions and case vignettes. Learning and competence, performance, and satisfaction were assessed with a 5-point Likert scale, using 1 = Strongly Disagree; 2 = Disagree; 3 = Neither Agree nor Disagree; 4 = Agree; and 5 = Strongly Agree. The face validity of the questionnaire was tested with ten ASHAs, separate from those recruited in the study and five primary care experts. The changes related to language, clarity, and relevance were made in the questionnaire based on the feedback from experts and participants. Separate discussion guides were developed for KIIs with trainers (Additional file 3 : Appendix S3) and hub-leaders (Additional file 4 : Appendix S4) and IDIs with ASHAs (Additional file 5 : Appendix S5). The guide focussed on examining the experience and practical applications of the training and was field tested before being administered in the main study. All study tools were translated into the local languages of the states and back-translated to check discrepancies.

The data were collected on the cell phone by experienced and trained researchers from social sciences backgrounds. Due to telephonic data collection, we were unable to capture non-verbal interview data such as emotions or gestures, particularly important in qualitative data. This may affect the richness of data and interpretation of responses. The quantitative tool was designed in the CS Pro software (version 7.5) and data were collected using its smartphone application. The qualitative interviews lasted around 40–50 min and were audio recorded. All interviews were translated and transcribed verbatim.

Data analysis

We summarized the quantitative data using descriptive statistics. Continuous variables were summarized using mean ± SD, and categorical variables were summarized using percentages and frequencies. The responses recorded using the 5-point Likert scale were recategorized during the analysis into three categories, i.e., ‘agree’ (combining strongly agree and agree), ‘disagree’ (combining strongly disagree and disagree), and ‘neutral [ 25 ]. Paired t test was used to find the difference between the pre- and post-scores of learning and competence and the attitude of participants toward ECHO training. McNemar’s test was used to assess changes in pre- and post-test scores for the technical domain. A p value of less than 0.05 was considered significant. STATA 16.0 statistical software was used for the analysis.

Qualitative data were analyzed according to the principles of the Framework approach [ 26 ], which combines inductive and deductive approaches. As a first step, two authors (SL and NM) familiarized themselves with four randomly selected transcripts and independently coded them using initial codes that were developed based on Moore’s framework levels of participation, satisfaction, learning, competence, and performance [ 27 ]. New codes that emerged while undertaking the analysis were included. The discussion and comparison of the double-coded transcripts enabled the development of an agreed set of codes. Any disagreements were discussed and resolved with the help of the third author (RP) to achieve inter-coder agreement. A final codebook was developed and applied to all the transcripts. The codes were combined and categorized into key emerging themes., The themes, including quotes (respondents’ exact words), were included to represent the main findings. Atlas.ti (version 8) software was used for data analysis.

Moore’s level 1—participation

Table 1 represents the baseline demographics of the recruited participants. From 610 participants who completed the pre-training survey, 490 participants completed the post-training survey, resulting in a follow-up rate of 80% (95% CI 76.6, 83.1). A total of 120 (20%, 95% CI 16.8, 23.3) participants were lost to follow up. This was due to a) contact numbers not being operational ( n =  96) and b) refusal due to time considerations ( n =  24). The field investigators attempted three additional phone calls, coordinated with hubs for participants’ alternate contact information, and offered flexible phone appointments to ensure maximum participation in the post-training survey. The majority (68%) of ASHAs were posted at sub-centres. A sub-centre is the most peripheral unit of contact of the health system with the community [ 28 ]. The majority of the participants (75%) had completed their high school (10th) education.

A hub leader described the efforts made by the ECHO to facilitate the participation of the ASHAs in the training.

“ECHO provided a facility where everyone can gather at the nearest block for the training. Physical and online modes [are] both available” (Hub-leader, Himachal Pradesh).

Moore’s level 2—satisfaction

The end-point survey assessed participants’ satisfaction with the ECHO training. The survey included eight items that measured overall training satisfaction and five items that measured satisfaction with factors specific to the telementoring model using close-ended questions. Satisfaction with the training content and environment was measured with four items. Except for one topic area (sharing of additional resources and training material), over 90% of participants were satisfied with almost all of the different components of the ECHO telementoring intervention (Additional file 1 : Appendix S1, Tables S1.1, S1.2, S1.3). While participants found the overall intervention favourable, 54.5% of all participants were dissatisfied with internet connectivity in the training sessions. Around one fourth of the participants faced challenges with the duration (31.2%), frequency (31.2%), and number of participants (28.4%) in the sessions (Additional file 1 : Appendix S1: Table S1.3).

The qualitative findings also show that most of the trainees were satisfied with the learning opportunity provided by the ECHO training.

“After attending these ECHO sessions, I felt we are constantly learning new techniques and it’s a deep sense of satisfaction” (ASHA, Tamil Nadu).

The ASHAs also shared areas or features of the ECHO model that did not meet their requirements and need improvement. They felt that the duration allotted for a session was not sufficient and some topics were covered very fast.

“They rush a lot while teaching over phone. It will be more helpful if they take more time and explain the things in a more detailed manner” (ASHA, WB)

Another ASHA suggested increasing the duration of training to improve their understanding of some topics.

"Increase the time of the training. Topics can be made deeper, and richer for better explanations" (ASHA, Tamil Nadu)

ASHAs described challenges related to connectivity while attending the training.

“The network connection was a problem and video used to lag” (ASHA, Sikkim)

Trainers shared their opinion about aspects of online trainings that did not meet their expectations.

“The problem is that they only join the meeting [online training] and do their own work, they actually do not listen properly.” (Trainer, WB)

A trainer mentioned that the large number of participants in some sessions affected the interaction among participant ASHAs.

“Sometimes a session has too many participants causing coordination efforts to be a challenge in these sessions” (Trainer, TN)

Difficulties in reaching the PHCs were recorded from the state of Sikkim. The geographical location and lack of transport facilities were mentioned by a trainer.

“We have transportation problem, our ASHA comes from rural area and it’s difficult to get taxi, which makes [it] harder to attend classes” (Trainer, Sikkim)

Many participants regarded organizational support as a facilitator for attending the training program. An ASHA from Tamil Nadu described how the issue of distance was resolved through management interventions from the organization.

“Our Block is 30 km away. There is another Block nearby that is 1 km only from here, they sent us there… so there was no problem” (ASHA, TN)

Moore’s level 3—learning

McNemar’s Chi-square statistic showed a significant difference between pre-ECHO and post-ECHO proportions in various aspects of health-related technical knowledge. Before the training, 1% of participants were aware of the correct schedule to be followed in the first week after the delivery of a child, which increased to 40% of participants post-training (p < 0.001). Overall, a statistically significant increase of 6% (95% CI 0.0003, 0.12; p =  0.038) in participants’ technical knowledge after ECHO training was found. After the training, a 7% increase in knowledge of malaria ( p =  0.002) and its symptoms and a 9% increase in knowledge of the right action to be undertaken (p < 0.001) was reported. Knowledge related to some areas such as recommended duration of physical activity or exercise (p < 0.001), immunisation after child birth ( p =  0.001), family planning in women after child birth ( p =  0.002) showed a decrease after attending the training (Additional file 1 : Appendix S1, Table S2). Post ECHO training, ASHAs reported an improvement in their knowledge of using a smartphone (switch on and off, and navigate) ( p =  0.0005) and navigating a mobile application ( p =  0.59). The ASHAs reported a 2% decrease in their knowledge of downloading content in the mobile ( p =  0.07) (Fig.  1 ).

Self-rated ICT knowledge of ASHAs

The qualitative data show that ASHAs who did not have a smartphone found it difficult to download and save content. One of the participants reported receiving additional training content in the form of a pdf file. She also mentioned that those who do not use a smartphone find it challenging to access this additional resource.

“We get the study material in a pdf so that simplifies our work further. But those who do not have a smartphone, find it difficult to get this opportunity” (ASHA, WB)

3A—Declarative learning

Declarative learning assesses how participants articulate the knowledge that the educational activity intended them to know (knowing what). The qualitative findings show that the training had increased the ASHA’s knowledge in specific domains such as breastfeeding during COVID-19.

“The doubt was whether a mother can breastfeed the baby when suffering from COVID-19. I got clarity about that… many such topics were cleared” (ASHA, Himachal Pradesh)

3B—Procedural learning

Procedural learning assesses the participants' articulation of how to do what the educational activity intended them to know (knowing how).

Participants reported that they had gained new skills related to the approach and identification of healthcare issues after attending the ECHO training.

“Earlier we wouldn’t know if ear related issues had a resolution – But following the ear related training we are aware that such issues can be cured or have treatments” (ASHA, Tamil Nadu).

The qualitative interviews revealed additional themes that described the value of the ECHO training program in improving the learning experience of ASHAs.

ASHA workers felt that the case presentations from their peers enhanced their learning experience.

“One ASHA shared a case of an anaemic mother. Based on this case we learned that this could have been prevented if iron tablets are provided from the adolescent stage” (ASHA, Tamil Nadu).

The interactive nature of the sessions and the discussions benefitted the learning experience of the ASHAs.

“Open discussion helped us so much. We can discuss any topics if we haven’t understood and sir used to explain again” (ASHA, Sikkim)

Moore’s level 4—competence

The participants reported significant improvement in their confidence to identify and manage several health conditions like birth asphyxia (for home deliveries) and management with a mucus extractor ( p =  0.01), screen and refer pregnant women ( p =  0.01), disseminate information on domestic violence and sexual harassment ( p =  0.001). Overall, a statistically significant increase of 6% (95% CI 0.01, 0.10; p =  0.01) in participants’ competence after attending the ECHO training was found. Participants reported a decrease in their confidence to track child immunisation ( p  < 0.001), monitor symptoms of COVID (p < 0.001), and clarify concerns of the community ( p  < 0.001) after attending the training (Additional file 1 : Appendix S1, Table S3).

Participants mentioned an improvement in their confidence while communicating with patients and their families.

“Initially we could not talk to people so comfortably, we hesitated at times but after being trained we can talk to people and their families properly and easily now” (ASHA, West Bengal)

An ASHA described a gap in their ability to talk to mothers in the field and suggested including more training content on efficient communication skills.

“We go on field and talk to mothers. There was no training for these, but I feel it will be good if we can have training on how to talk to mothers comfortably” (ASHA, WB)

Moore’s level 5—performance

The study identified a significant improvement in ASHAs’ positive attitude toward maternal and child health issues. Overall, a 5% improvement (95% CI − 0.009, 0.10; p value = 0.09) in participants’ attitudes post-ECHO training was found. Almost all the participants (99%) reported applying the skills learnt during the training at their workplaces. More than 90% of the participants felt that the ECHO training expanded access to healthcare in their community (Fig.  2 ). The ASHAs reported an improvement in their attitude towards inclusion of HIV patients in the community ( p =  0.01) and home visits for new born babies (p < 0.001) (Additional file 1 : Appendix S1, Table S4).

Self-reported performance of ASHAs

The ASHAs shared specific examples where they made changes in their practice or treatment strategies after attending the training.

“[Earlier] the implementation was not proper [correct]. As an example, if a child’s life has to be saved on the spot, we would take the medicines and syringes separately. Now we take the necessary items section wise including the AFI kit. So that’s the change” (ASHA, Tamil Nadu).

The results of this evaluation suggest that Project ECHO provides a suitable and efficacious platform for training for ASHAs. The participants reported an improvement in their knowledge, skills, and practices. They also described improved confidence to communicate more effectively. Some areas in which the ASHAs reported a lack of knowledge and confidence include newborn immunisation and family planning after pregnancy.

The NRHM guidelines for the recruitment of ASHAs require candidates to have at least eight or 10 completed years of formal education. Low literacy and inadequate training of ASHAs have been observed in different states in India [ 30 , 31 ]. However, with the proper training and support, ASHAs can provide comprehensive preventive and promotive healthcare services [ 29 ]. In this study, the majority (75%) of ASHAs across all states had ten or more years of schooling. The ECHO training will bolster their knowledge, skills, and confidence in providing effective services.

The ASHAs receive 23 days of training in the first year, followed by 12 days of training in every subsequent year to keep them updated with the knowledge and skills needed to effectively perform their roles and responsibilities. Previous studies have identified many challenges in the training of ASHAs, such as lack of regular refresher training [ 32 ], shortage of competent trainers, insufficient funds, and use of obsolete health information [ 33 ]. The training programs have mostly been didactic-based and had limitations in the engagement of participants [ 34 ]. The ECHO NHSRC refresher training addresses these limitations by promoting peer-to-peer learning and through a case-based discussion approach [ 35 ].

Our findings report a significant increase in the knowledge of ASHA workers with respect to specific domains like maternal and child health. A randomized controlled trial in Karnataka, India, found a significant improvement in mental health knowledge, attitude, and practice (KAP) scores amongst ASHAs trained by a hybrid training (traditional 1-day in-person classroom training and seven online sessions using the ECHO Model) against conventional classroom training [ 14 ]. This study findings highlight the improvement in knowledge of ASHAs related to oral health and palliative care post-ECHO training. An improvement in knowledge has also been observed in other studies that have evaluated ECHO telementoring interventions in cancer screening [ 36 ], palliative care [ 37 , 38 ], HIV [ 39 ], and chronic pain [ 40 ] In this study, ASHAs reported poor knowledge of the immunisation schedule for a newborn as well as the confidence to record and track immunisation in the community even after the ECHO training. A critical function of ASHAs is to assist ANMs or nurses with all immunisation activities [ 41 ]. A previous study in Karnataka in 2020 found inadequate knowledge among ASHAs about child immunisation. The above study also documented that by increasing the number of days and focusing on child care the ASHAs had a better understanding of interventions related to child healthcare [ 42 ]. As a part of the course structure, ECHO provides one session on new born and post-partum care. An assessment of the number of sessions needed to cover the topics was beyond the scope of our research but would be beneficial.

Previous studies have identified several shortcomings in ASHAs' communication and counselling abilities [ 43 , 44 , 45 ]. The findings of this study revealed that the ASHAs faced communication issues while discussing health matters related to family planning and COVID-19 with the community. Previous research has found that interpersonal communication of ASHAs are influenced by factors such as health system support and community context [ 46 ]. A study exploring the perspectives of ASHAs on a mobile training course in India also found that they encountered barriers in their interactions with beneficiaries such as resistance from family members, fear of poor quality of care, and financial costs of care [ 44 ]. Training programs must therefore, also incorporate how ASHAs can navigate social behaviours and norms to improve the impact of counselling [ 47 , 48 ]. The extent to which the ECHO training can identify and incorporate community hierarchies to improve communication of the ASHAs needs further exploration. In this study, large batch size ( n =  40) and limited use of video by participants during the training hampered the engagement between ASHAs as well as with the trainers. A previous study in the USA suggested that limiting batch size and ensuring face-to-face interactions on the virtual platform ensured a higher level of accountability and made it easier to engage with others in the ECHO training sessions [ 49 ].

CHWs face significant barriers when using digital technology in LMICs, making it challenging for them to access training on digital platforms [ 50 ]. The ASHAs in this study reported an improvement in their ability to use smartphones and navigate mobile applications. Our findings also suggest that ASHAs should be better oriented for accessing content on hand held devices.

The mentorship by trainers added value to participants’ knowledge and helped improve their skills. In this study, participants’ attitudes towards their work changed after attending the ECHO training suggesting that the learning and confidence developed during the training would be transferable to their work in healthcare settings and communities. The ECHO participants of previous studies have also demonstrated similar changes in their practices [ 35 , 40 ]. Our study findings indicate that the ECHO Model is an effective platform that can help foster a virtual community of practice through case-based learning, shared best practices, and online mentorship by experts.

Future directions

There should be more sessions on topics related to post-natal and newborn care as the ASHAs showed poor knowledge and competence in these areas.

There should be more training on counselling and development of communication skills for ASHAs, specially for maternal and child health and COVID-19.

An orientation for ASHAs should be conducted to facilitate the use of technology and the platform for learning. This may also help overcome some of the challenges described by the ASHAs in this study.

Strengths and limitations

The study used a rigorous quasi-experimental design across four different states of India. Our follow-up rate in the study was 80%, indicating a high response from participants completing the pre–post assessment. The presented study has certain limitations. It was not possible to use randomisation and a pure experimental design in this study, and this affects the internal validity of the study. The inclusion of a control group would have strengthened study validity. The self-reported outcomes can be subject to social desirability bias. We did not document the information on attendance and drop outs from the training program. The qualitative results have to be carefully interpreted because of the small sample size of the qualitative study relative to the study sample.

There is increasing recognition of the importance of CHWs globally for promoting a continuum of care and expanding access to health services. ASHA workers constitute critical human resources in the Indian health system and efforts to empower them are crucial for strengthening the health system at the community level. The encouraging results of this study indicate the effectiveness of Project ECHO in building the capacity of ASHA workers across different geographies in the country.

Availability of data and materials

All data generated or analyzed during this study are included in this published article (as Additional files).

Abbreviations

Community health workers

Sustainable development goals

National Rural Health Mission

Accredited social health activists

Digital infrastructure knowledge sharing

Ministry of Human Resource Development

Coronavirus Disease 2019

National Health System Resource Centre

World Health Organization

High-Income Countries

LMICs: Low- and Middle-Income Countries

Extension for Community Healthcare Outcomes

In-depth interviews

Key informant interviews

Continuing medical education

Institutional Ethics Committee

Participant Information Sheet

Jodhpur School of Public Health

Public Health Foundation of India

Hartzler AL, Tuzzio L, Hsu C, Wagner EH. Roles and functions of community health workers in primary care. Ann Fam Med. 2018;16(3):240–5.

Article   PubMed   PubMed Central   Google Scholar  

Olaniran A, Banke-Thomas A, Bar-Zeev S, Madaj B. Not knowing enough, not having enough, not feeling wanted: challenges of community health workers providing maternal and newborn services in Africa and Asia. PLoS ONE. 2022;17(9): e0274110.

Article   CAS   PubMed   PubMed Central   Google Scholar  

O’Donovan J, O’Donovan C, Kuhn I, Sachs SE, Winters N. Ongoing training of community health workers in low-income and middle-income countries: a systematic scoping review of the literature. BMJ Open. 2018;8(4): e021467.

Kok MC, Kane SS, Tulloch O, Ormel H, Theobald S, Dieleman M, et al. How does context influence performance of community health workers in low- and middle-income countries? Evidence from the literature. Health Res Policy Syst. 2015;13(1):13.

Saprii L, Richards E, Kokho P, Theobald S. Community health workers in rural India: analysing the opportunities and challenges accredited social health activists (ASHAs) face in realising their multiple roles. Hum Resour Health. 2015;13(1):95.

Ministry of Health and Family Welfare, Government of India. Non Communicable Disease Control Programmes: National Health Mission. 2023. https://nhm.gov.in/index1.php?lang=1&level=1&sublinkid=1041&lid=614 . Accessed 13 Feb 2023.

World Health Organization. What do we know about community health workers? A systematic review of existing reviews. 2020. https://www.who.int/publications-detail-redirect/what-do-we-know-about-community-health-workers-a-systematic-review-of-existing-reviews . Accessed 13 Feb 2023.

Bajpai N, Dholakia RH. Improving the performance of accredited social health activists in India. Mumbai: Columbia Global Centres South Asia; 2011.

Google Scholar  

Panwar DS, Naidu V, Das E, Verma S, Khan AA. Strengthening support mechanisms for accredited social health activists in order to improve home-based newborn care in Uttar Pradesh, India. BMC Proc. 2012;6(5):O33.

Article   PubMed Central   Google Scholar  

Yadav D, Singh P, Montague K, Kumar V, Sood D, Balaam M, Sharma D, Duggal M, Bartindale T, Varghese D, Olivier P. Sangoshthi: Empowering community health workers through peer learning in rural india. In: Proceedings of the 26th International Conference on World Wide Web 2017 Apr 3 (pp. 499–508).

Labrique AB, Wadhwani C, Williams KA, Lamptey P, Hesp C, Luk R, et al. Best practices in scaling digital health in low and middle income countries. Glob Health. 2018;14(1):103.

Article   Google Scholar  

Bashingwa JJH, Shah N, Mohan D, Scott K, Chamberlain S, Mulder N, et al. Examining the reach and exposure of a mobile phone-based training programme for frontline health workers (ASHAs) in 13 states across India. BMJ Glob Health. 2021;6(Suppl 5): e005299.

Winters N, Langer L, Nduku P, Robson J, O’Donovan J, Maulik P, et al. Using mobile technologies to support the training of community health workers in low-income and middle-income countries: mapping the evidence. BMJ Glob Health. 2019;4(4): e001421.

Nirisha PL, Malathesh BC, Kulal N, Harshithaa NR, Ibrahim FA, Suhas S, et al. Impact of technology driven mental health task-shifting for accredited social health activists (ASHAs): results from a randomised controlled trial of two methods of training. Commun Ment Health J. 2023;59(1):175–84.

Long LA, Pariyo G, Kallander K. Digital technologies for health workforce development in low- and middle-income countries: a scoping review. Glob Health Sci Pract. 2018;6(Supplement 1):S41–8.

Tyagi V, Khan A, Siddiqui S, KakraAbhilashi M, Dhurve P, Tugnawat D, et al. Development of a digital program for training community health workers in the detection and referral of schizophrenia in rural India. Psychiatr Q. 2023;94(2):141–63.

Article   PubMed   Google Scholar  

Arora S, Thornton K, Murata G, Deming P, Kalishman S, Dion D, et al. Outcomes of treatment for Hepatitis C virus infection by primary care providers. N Engl J Med. 2011;364(23):2199–207.

Article   CAS   PubMed   Google Scholar  

Ministry of Health and Family Welfare, Government of India. ASHA Training Modules: National Health Mission. 2022. http://nhm.gov.in/index1.php?lang=1&level=3&sublinkid=184&lid=257 . Accessed 11 Aug 2022.

Zurawski A, Komaromy M, Ceballos V, McAuley C, Arora S. Project ECHO brings innovation to community health worker training and support. J Health Care Poor Underserved. 2016;27(4):53–61.

Komaromy M, Ceballos V, Zurawski A, Bodenheimer T, Thom DH, Arora S. Extension for community healthcare outcomes (ECHO): a new model for community health worker training and support. J Public Health Policy. 2018;39(2):203–16.

Damian AJ, Robinson S, Manzoor F, Lamb M, Rojas A, Porto A, et al. A mixed methods evaluation of the feasibility, acceptability, and impact of a pilot project ECHO for community health workers (CHWs). Pilot Feasibility Stud. 2020;6(1):132.

Feroz AS, Khoja A, Saleem S. Equipping community health workers with digital tools for pandemic response in LMICs. Arch Public Health. 2021;79(1):1.

Creswell JW, Clark VLP. Designing and conducting mixed methods research. Thousand Oaks: SAGE Publications; 2017. p. 521.

Tran L, Feldman R, Riley T III, Jung J. Association of the extension for community healthcare outcomes project with use of direct-acting antiviral treatment among US adults with hepatitis C. JAMA Netw Open. 2021;4(7): e2115523.

Harpe SE. How to analyze Likert and other rating scale data. Curr Pharm Teach Learn. 2015;7(6):836–50.

Hackett A, Strickland K. Using the framework approach to analyse qualitative data: a worked example. Nurse Res. 2018;26(2):8.

Moore DEJ, Green JS, Gallis HA. Achieving desired results and improved outcomes: Integrating planning and assessment throughout learning activities. J Contin Educ Health Prof. 2009;29(1):1.

Chokshi M, Patil B, Khanna R, Neogi SB, Sharma J, Paul VK, et al. Health systems in India. J Perinatol. 2016;36(Suppl 3):S9-12.

National health systems resource centre. ASHA: which way forward. Evaluation of the ASHA Programme. 2010. https://nhm.gov.in/images/pdf/communitisation/asha/Studies/Evaluation_of_ASHA_Program_2010-11_Report.pdf . Accessed 17 Dec 2022.

National health systems resource centre. Tenth common review mission: National health mission. https://nhm.gov.in/images/pdf/monitoring/crm/10th-crm/Report/10th_CRM_Main_Report.pdf . Accessed 29 Dec 2022.

DeRenzi B, Wacksman J, Dell N, Lee S, Lesh N, Borriello G, Ellner A. Closing the feedback loop: a 12-month evaluation of ASTA, a self-tracking application for ASHAs. In: DeRenzi B, editor. Proceedings of the Eighth international conference on information and communication technologies and development. New York: Association for Computing Machinery; 2016. p. 1–10. https://doi.org/10.1145/2909609.2909652 .

Chapter   Google Scholar  

Yadav D. Low-cost mobile learning solutions for community health workers. In: Yadav D, editor. Proceedings of the 26th international on world wide web companion. Geneva: International World Wide Web Conferences Steering Committee; 2017. p. 729–34. https://doi.org/10.1145/3041021.3053377 .

Molapo M, Marsden G. Health education in rural communities with locally produced and locally relevant multimedia content. In: Molapo M, editor. Proceedings of the 3rd ACM symposium on computing for development. New York: Association for Computing Machinery; 2013. p. 1–2. https://doi.org/10.1145/2442882.2442913 .

Bhowmick S, Sorathia K. Findings of the user study conducted to understand the training of rural ASHAs in India. In: Bhowmick S, editor. Proceedings of the tenth international conference on information and communication technologies and development. New York: Association for Computing Machinery; 2019. p. 1–5. https://doi.org/10.1145/3287098.3287150 .

Bikinesi L, O’Bryan G, Roscoe C, Mekonen T, Shoopala N, Mengistu AT, et al. Implementation and evaluation of a Project ECHO telementoring program for the Namibian HIV workforce. Hum Resour Health. 2020;18(1):61.

Adsul P, Nethan ST, deCortina SH, Dhanasekaran K, Hariprasad R. Implementing cancer screening programs by training primary care physicians in India—findings from the national institute of cancer prevention research project ECHO for cancer prevention. Glob Implement Res Appl. 2022;2(1):34–41.

White C, McIlfatrick S, Dunwoody L, Watson M. Supporting and improving community health services—a prospective evaluation of ECHO technology in community palliative care nursing teams. BMJ Support Palliat Care. 2019;9(2):202–8.

Usher R, Payne C, Real S, Carey L. Project ECHO: Enhancing palliative care for primary care occupational therapists and physiotherapists in Ireland. Health Soc Care Commun. 2022;30(3):1143–53.

Wood BR, Unruh KT, Martinez-Paz N, Annese M, Ramers CB, Harrington RD, et al. Impact of a telehealth program that delivers remote consultation and longitudinal mentorship to community HIV providers. Open Forum Infect Dis. 2016;3(3):123.

Katzman JG, Comerci GJ, Boyle JF, Duhigg D, Shelley B, Olivas C, et al. Innovative telementoring for pain management: project ECHO pain. J Contin Educ Health Prof. 2014;34(1):68–75.

Kalne PS, Kalne PS, Mehendale AM. Acknowledging the role of community health workers in providing essential healthcare services in rural india—a review. Cureus. 2022;14(9): e29372.

PubMed   PubMed Central   Google Scholar  

Rohith M, Angadi MM. Evaluation of knowledge and practice of ASHAs, regarding child health services in Vijyapaura district, Karnataka. J Fam Med Prim Care. 2020;9(7):3272–6.

Article   CAS   Google Scholar  

Shrivastava A, Srivastava A. Measuring communication competence and effectiveness of ASHAs (accredited social health activist) in their leadership role at rural settings of Uttar Pradesh (India). Leadersh Health Serv. 2016;29(1):69–81.

Scott K, Ummer O, Chamberlain S, Sharma M, Gharai D, Mishra B, et al. ’[We] learned how to speak with love’: a qualitative exploration of accredited social health activist (ASHA) community health worker experiences of the Mobile Academy refresher training in Rajasthan, India. BMJ Open. 2022;12(6): e050363.

Goel AD, Gosain M, Amarchand R, Sharma H, Rai S, Kapoor SK, et al. Effectiveness of a quality improvement program using difference-in-difference analysis for home based newborn care—results of a community intervention trial. Indian J Pediatr. 2019;86(11):1028–35.

Ved R, Scott K. Counseling is a relationship not just a skill: re-conceptualizing health behavior change communication by India’s accredited social health activists. Glob Health Sci Pract. 2020;8(3):332–4.

Abdel-All M, Putica B, Praveen D, Abimbola S, Joshi R. Effectiveness of community health worker training programmes for cardiovascular disease management in low-income and middle-income countries: a systematic review. BMJ Open. 2017;7(11): e015529.

Smittenaar P, Ramesh BM, Jain M, Blanchard J, Kemp H, Engl E, et al. Bringing greater precision to interactions between community health workers and households to improve maternal and newborn health outcomes in India. Glob Health Sci Pract. 2020;8(3):358–71.

Shimasaki S, Bishop E, Guthrie M, Thomas JF. Strengthening the health workforce through the ECHO stages of participation: participants’ perspectives on key facilitators and barriers. J Med Educ Curric Dev. 2019;6:2382120518820922.

Medhanyie AA, Moser A, Spigt M, Yebyo H, Little A, Dinant G, et al. Mobile health data collection at primary health care in Ethiopia: a feasible challenge. J Clin Epidemiol. 2015;68(1):80–6.

Download references

Acknowledgements

The authors wish to thank all the healthcare workers who kindly participated in this study giving their time, experience, and insights. We also thank Dr. Sourabh Chakraborty (Professor, JSPH), Mr. Swapnil Gupta, and the JSPH data collection team for their contribution to the collection of good quality data in a short time.

The study was funded by Extension for Community Healthcare Outcomes (ECHO) India.

Author information

Authors and affiliations.

Public Health Foundation of India (PHFI), Gurugram, Haryana, India

Rajmohan Panda

Extension for Community Healthcare Outcomes (ECHO) India, Okhla Phase III, New Delhi, India

Supriya Lahoti, Nivedita Mishra, Apoorva Karan Rai & Kumud Rai

HexaHealth, Gurugram, Haryana, India

Rajath R. Prabhu

Hamad Medical Corporation, Doha, Qatar

Kalpana Singh

You can also search for this author in PubMed   Google Scholar

Contributions

R.M. contributed to the conception and design of the study and significant inputs for data analysis and made a significant contribution to the drafting of the discussion and conclusion of the paper. S.L. wrote the first draft of the manuscript. N.M. and S.L. contributed to the implementation of the study and development of interview guides, analysis, and validation of qualitative data. R.P. and K.S. contributed to the analysis and validation of quantitative data. R.M., N.M., R.P., K.S, A.K.R. and K.R. reviewed the manuscript and gave significant inputs for improving the paper. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Supriya Lahoti .

Ethics declarations

Ethics approval and consent to participate.

Ethical clearance was received from the Institutional Ethical Committee (IEC) of the Public Health Foundation of India (PHFI) (ref: TRC-IEC 472/21, dated 26 August 2021). The study has also been registered at the Clinical Trials Registry, India (CTRI/2021/10/037189). All methods were performed in accordance with the relevant guidelines and regulations. A written Participant Information Sheet (PIS) and informed consent form was provided to the participants before conducting the interviews. Verbal informed consent was taken from all participants, and the process of verbal informed consent was approved by the ethics committee (Institutional Ethics Committee (IEC) of the PHFI). Data confidentiality was maintained by coding with the unique identification (ID) of all the participants. The interviews were audio-recorded, and audio files and transcripts were kept in a password-protected folder.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1.

: Appendix S1. Table S1.1. Satisfaction with different factors of the training. Table S1.2. Satisfaction with content and environment of the training. Table S1.3. Challenges faced with respect to ECHO tele-mentoring model. Table S2. Technical knowledge and skills. Table S3. Statements assessing competence. Table S4. Statements assessing attitude and performance.

Additional file 2

: Appendix S2. Participants in qualitative interviews.

Additional file 3

: Appendix S3. Key informant Interview Guide for Trainers End line Evaluation.

Additional file 4

: Appendix S4. Key informant interview guide for Hub leaders End line Evaluation.

Additional file 5

: Appendix S5. In-depth Interview Guide for ASHAs End line Evaluation.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Panda, R., Lahoti, S., Mishra, N. et al. A mixed methods evaluation of the impact of ECHO ® telementoring model for capacity building of community health workers in India. Hum Resour Health 22 , 26 (2024). https://doi.org/10.1186/s12960-024-00907-y

Download citation

Received : 14 March 2023

Accepted : 10 April 2024

Published : 23 April 2024

DOI : https://doi.org/10.1186/s12960-024-00907-y

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Community health workers (CHWs)
• Accredited social health activists (ASHAs)
• Maternal and child health
• Primary healthcare
• Health worker training
• ECHO telementoring
• Mixed-method study

Human Resources for Health

ISSN: 1478-4491

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

• Open access
• Published: 26 April 2024

Clinician and staff experiences with frustrated patients during an electronic health record transition: a qualitative case study

• Sherry L. Ball 1 ,
• Bo Kim 2 , 3 ,
• Sarah L. Cutrona 4 , 5 ,
• Brianne K. Molloy-Paolillo 4 ,
• Ellen Ahlness 6 ,
• Megan Moldestad 6 ,
• George Sayre 6 , 7 &
• Seppo T. Rinne 2 , 8  

BMC Health Services Research volume  24 , Article number:  535 ( 2024 ) Cite this article

Metrics details

Electronic health record (EHR) transitions are known to be highly disruptive, can drastically impact clinician and staff experiences, and may influence patients’ experiences using the electronic patient portal. Clinicians and staff can gain insights into patient experiences and be influenced by what they see and hear from patients. Through the lens of an emergency preparedness framework, we examined clinician and staff reactions to and perceptions of their patients’ experiences with the portal during an EHR transition at the Department of Veterans Affairs (VA).

This qualitative case study was situated within a larger multi-methods evaluation of the EHR transition. We conducted a total of 122 interviews with 30 clinicians and staff across disciplines at the initial VA EHR transition site before, immediately after, and up to 12 months after go-live (September 2020-November 2021). Interview transcripts were coded using a priori and emergent codes. The coded text segments relevant to patient experience and clinician interactions with patients were extracted and analyzed to identify themes. For each theme, recommendations were defined based on each stage of an emergency preparedness framework (mitigate, prepare, respond, recover).

In post-go-live interviews participants expressed concerns about the reliability of communicating with their patients via secure messaging within the new EHR portal. Participants felt ill-equipped to field patients’ questions and frustrations navigating the new portal. Participants learned that patients experienced difficulties learning to use and accessing the portal; when unsuccessful, some had difficulties obtaining medication refills via the portal and used the call center as an alternative. However, long telephone wait times provoked patients to walk into the clinic for care, often frustrated and without an appointment. Patients needing increased in-person attention heightened participants’ daily workload and their concern for patients’ well-being. Recommendations for each theme fit within a stage of the emergency preparedness framework.

Conclusions

Application of an emergency preparedness framework to EHR transitions could help address the concerns raised by the participants, (1) mitigating disruptions by identifying at-risk patients before the transition, (2) preparing end-users by disseminating patient-centered informational resources, (3) responding by building capacity for disrupted services, and (4) recovering by monitoring integrity of the new portal function.

Peer Review reports

Electronic health record (EHR) transitions present significant challenges for healthcare clinicians and staff. These transitions require adjustments in care delivery and may threaten care quality and value. It is critical that healthcare organizations undergoing these changes learn from others who have undergone similar transitions [ 1 , 2 ]. However, the current literature lacks adequate guidance on navigating EHR transitions, especially as they relate to how clinicians and staff interact with patients [ 3 ].

Embedded within EHRs, patient portals facilitate complete, accurate, timely, and unambiguous exchange of information between patients and healthcare workers [ 4 , 5 ]. These portals have become indispensable for completing routine out-of-office-visit tasks, such as medication refills, communicating laboratory results, and addressing patient questions [ 6 ]. In 2003, the VA launched their version of a patient portal, myHealtheVet [ 7 ] and by 2017 69% of Veterans enrolled in healthcare at the VA had registered to access the patient portal [ 8 ]. Similar to other electronic portals, this system allows Veterans to review test results, see upcoming appointments, and communicate privately and securely with their healthcare providers.

EHR transitions can introduce disruptions to patient portal communication that may compromise portal reliability, impacting patient and clinician satisfaction, patients’ active involvement in self-management, and ultimately health outcomes [ 9 ]. During an EHR transition, patients can expect reductions in access to care even when clinician capacity and IT support are increased. Patients will likely need for more assistance navigating the patient portal including and using the portal to communicate with their providers [ 10 ]. Staff must be prepared and understand how the changes in the EHR will affect patients and safeguards must be in place to monitor systems for potential risks to patient safety. Building the capacity to respond to emerging system glitches and identified changes must be included in any transition plan. Although portal disruptions are likely to occur when a new EHR is implemented, we know little about how these disruptions impact healthcare workers’ interactions and care delivery to patients [ 11 , 12 ].

Due to an urgency to raise awareness and promote resolution of these patient portal issues,, we utilized existing data from the first EHR transition site for the Department of Veterans Affairs (VA)’s enterprise-wide transition. We focused on end users’ responses to the question “How Veterans were affected by the transition?”. We used qualitative methods to begin to understand how provider and patient interactions were affected during and by the EHR transition. We explored the impact of the EHR transition on patients through healthcare workers’ vicarious and direct experiences with patients. Due to the high level of disruption in care delivery we draw on insights from an emergency preparedness framework [ 13 ] to generate a set of recommendations to improve healthcare workers’ experiences during EHR transitions. The emergency preparedness framework includes 4 phases of an iterative cycle that include: (1) building capacity to mitigate issues, (2) preparing for the inevitable onset of issues, (3) responding to issues as they emerge, and (4) strategies to recover from any damage incurred.

In early 2020, the VA embarked on an EHR transition from a homegrown, legacy EHR system, developed by VA clinicians and used since the 1990s, to a new commercial system by the Oracle-Cerner Corporation. The primary objectives of this transition were to standardize care and improve interoperability between VA Medical Centers nationwide and the Department of Defense (DoD). Spanning over a decade, this transition plan is scheduled to roll out to all VA medical centers and outpatient clinics.

In this manuscript, we present data from the Mann-Grandstaff VA Medical Center in Spokane, WA, VA’s first EHR transition site. The study uses qualitative methods with clinician and staff interviews as part of a larger multi-method evaluation of the EHR transition. Our overarching goal is to identify and share recommendations to improve VA’s EHR transition efforts; rather than be guided by a theoretical framework our study design including the interview guides [ 14 , 15 ] were based primarily on what was being experienced. An experienced team of ten qualitative methodologists and analysts conducted the study.

This evaluation was designated as non-research/quality improvement work by the VA Bedford Healthcare System Institutional Review Board deeming it exempt from needing an informed consent. Study materials, including interview guides with verbal consent procedures, were reviewed and approved by labor unions and by the VA Bedford Healthcare System Institutional Review Board; all methods were carried out in accordance with local and national VA guidelines and regulations.

Interview guides and an outline of the data collection plans were reviewed and approved by relevant national unions before beginning recruitment.

Recruitment

Recruitment began in July 2020, before the first site implemented the new EHR. Prior to collecting data, we met with site leadership to get buy-in and support for the study, understand local context, determine how the site was approaching the transition, and to obtain the names of clinicians and staff for potential interviews. All potential participants were invited by email to participate in a one-hour voluntary interview conducted on Microsoft Teams® about their experiences with this transition; we used snowball sampling during interviews to expand the pool of interviewees. Verbal permission for audio recording of interviews was obtained immediately prior to the interview. Interview participants were informed that they could skip any questions, pause or stop the recording, and stop the interview at any time and were invited to ask questions before beginning the interview.

Most participants were interviewed at multiple timepoints; these included pre-implementation interviews, brief check-ins, and post-implementation interviews (Table  1 ). At the end of the pre-implementation interview, participants were invited to participate in 3–4 additional, shorter (15–20 min), check-in interviews where information about any changes in the transition process, context, or experience could be discussed. Most initial interviewees, in addition to three new participants, participated in post-implementation interviews (35–60 min; approximately 2–3 months and 10–12 months after the implementation) to reflect on the entire transition process.

Data collection

Experienced qualitative interviewers included PhD trained qualitative methodologist and masters level qualitative analysts (JB, SB, AC, EK, MM, GS) conducted individual interviews with clinicians and staff, aligning to a semi-structured interview guide with follow-up probes using the participant’s words to elicit rich responses grounded in the data [ 16 ]. The guide was designed to inform ongoing efforts to improve the rollout of the new EHR. Six main categories were covered in our interview guides, including (1) attitudes toward the new software, (2) information communicated about the transition, (3) training and education, (4) resources, (5) prior experience with EHRs, and (6) prior experiences with EHR transitions. After piloting the interview guide with a clinician, initial interviews were completed between September and October 2020 and averaged  ∼  45 min in duration. Two-month and one-year post-implementation interview guides included an additional question, “Has the Cerner transition affected Vets?”; data presented here largely draw from responses to this question. Check-ins (October 2020– December 2020) took  ∼  15 min; two-month post-implementation interviews (December 2020– January 2021) and one-year post-implementation interviews (October 2020 - November 2021) took  ∼  45 min. Audio recordings of all interviews were professionally transcribed. To ensure consistency and relationship building, participants were scheduled with the same interviewer for the initial and subsequent interviews whenever feasible (i.e., check-ins and post-implementation interviews). Immediately following each interview, interviewers completed a debrief form where highlights and general reflections were noted.

Throughout the data collection process, interviewers met weekly with the entire qualitative team and the project principal investigators to discuss the recruitment process, interview guide development, and reflections on data collection. To provide timely feedback to leadership within the VA, a matrix analysis [ 17 ] was conducted concurrently with data collection using the following domains: training, roles, barriers, and facilitators. Based on these domains, the team developed categories and subcategories, which formed the foundation of an extensive codebook.

Data analysis

All interviewers also coded the data. We used inductive and deductive content analysis [ 18 ]. Interview transcripts were coded in ATLAS.ti qualitative data analysis software (version 9). A priori codes and categories (based on the overall larger project aims and interview guide questions) and emergent codes and categories were developed to capture concepts that did not fit existing codes or categories [ 18 ]. Codes related to patient experience and clinician interactions with patients were extracted and analyzed using qualitative content analysis to identify themes [ 18 ]. Themes were organized according to their fit within the discrete stages of an emergency preparedness framework to generate recommendations for future rollout. In total, we examined data from 111 interviews with 24 VA clinicians and staff (excluding the initial 11 stakeholder meetings (from the 122 total interviews) that were primarily for stakeholder engagement). We focused on participants’ responses related to their experiences interacting with patients during the EHR transition.

Exemplar quotes primarily came from participants’ responses to the question, “Has the Cerner transition affected Vets?” and addressed issues stemming from use of the patient portal. This included both clinicians’ direct experiences with the portal and indirect experiences when they heard from patients about disruptions when using the portal. We identified four themes related to clinicians’ and staff members’ reported experiences: (1) stress associated with the unreliability of routine portal functions and inaccurate migrated information; (2) concern about patients’ ability to learn to use a new portal (especially older patients and special populations); (3) frustration with apparent inadequate dissemination of patient informational materials along with their own lack of time and resources to educate patients on use of the new portal; and (4) burden of additional tasks on top of their daily workload when patients needed increased in-person attention due to issues with the portal.

Stress associated with the unreliability of routine portal functions and inaccurate migrated information

One participant described the portal changes as, “It’s our biggest stress, it’s the patients’ biggest stress… the vets are definitely frustrated; the clinicians are; so I would hope that would mean that behind the scenes somebody is working on it” (P5, check-in).

Participants expressed significant frustration when they encountered veterans who were suddenly unable to communicate with them using routine secure messaging. These experiences left them wondering whether messages sent to patients were received.

Those that use our secure messaging, which has now changed to My VA Health, or whatever it’s called, [have] difficulty navigating that. Some are able to get in and send the message. When we reply to them, they may or may not get the reply. Which I’ve actually asked one of our patients, ‘Did you get the reply that we took care of this?’ And he was like, ‘No, I did not (P11, 2-months post)

Participants learned that some patients were unable to send secure messages to their care team because the portal contained inaccurate or outdated appointment and primary site information.

I’ve heard people say that the appointments aren’t accurate in there… veterans who have said, ‘yeah, it shows I’m registered,’ and when they go into the new messaging system, it says they are part of a VA that they haven’t gone to in years, and that’s the only area they can message to, they can’t message to the [site] VA, even though that’s where they’ve actively being seen for a while now. (P20, 2-months post)

After the EHR transition, participants noted that obtaining medications through the portal, which was once a routine task, became unreliable. They expressed concern around patients’ ability to obtain their medications through the portal, primarily due to challenges with portal usability and incomplete migration of medication lists from the former to the new EHR.

I think it’s been negative, unfortunately. I try to stay optimistic when I talk to [patients], but they all seem to be all having continued difficulty with their medications, trying to properly reorder and get medications seems to still be a real hassle for them. (P17, one-year post) …the medications, their med list just didn’t transfer over into that list [preventing their ability to refill their medications]. (P13, 2-months post)

Concern about patients’ ability to learn to use a new portal

Clinicians and staff expressed concerns around veterans’ ability to access, learn, and navigate a new portal system. Clinicians noted that even veterans who were adept at using the prior electronic portal or other technologies also faced difficulties using the new portal.

They can’t figure out [the new portal], 99% of them that used to use our [old] portal, the electronic secure messaging or emailing between the team, they just can’t use [the new one]. It’s not functioning. (P13, one-year post) Apparently, there’s a link they have to click on to make the new format work for them, and that’s been confusing for them. But I still am having a lot of them tell me, I had somebody recently, who’s very tech savvy, and he couldn’t figure it out, just how to message us. I know they’re still really struggling with that. (P5, 2-months post) And it does seem like the My Vet [my VA Health, new portal], that used to be MyHealtheVet [prior portal], logging on and getting onto that still remains really challenging for a large number of veterans. Like they’re still just unable to do it. So, I do think that, I mean I want to say that there’s positive things, but really, I struggle (P17, one-year post)

Participants recognized difficulties with the new system and expressed empathy for the veterans struggling to access the portal.

I think that a lot of us, individually, that work here, I think we have more compassion for our veterans, because they’re coming in and they can’t even get onto their portal website. (P24, one-year post)

Participants acknowledged that learning a new system may be especially difficult for older veterans or those with less technology experience.

But, you know, veterans, the general population of them are older, in general. So, their technologic skills are limited, and they got used to a system and now they have to change to a new one. (P13, 2-months post) So, for our more elderly veterans who barely turn on the computer, they’re not getting to this new portal. (P8, check in) And you know, I do keep in mind that this is a group of people who aren’t always technologically advanced, so small things, when it’s not normal to them, stymie them.(P13, one-year post)

Concerns were heightened for veterans who were more dependent on the portal as a key element in their care due to specific challenges. One participant pointed out that there may be populations of patients with special circumstances who depend more heavily on the prior portal, MyHealtheVet.

I have veterans from [specific region], that’s the way they communicate. Hearing impaired people can’t hear on the phone, the robocall thing, it doesn’t work, so they use MyHealtheVet. Well, if that goes away, how is that being communicated to the veteran? Ok? (P18, Check-in)

Frustration with inadequate dissemination of information to veterans about EHR transition and use of new portal

Participants were concerned about poor information dissemination to patients about how to access the new portal. During medical encounters, participants often heard from patients about their frustrations accessing the new portal. Participants noted that they could only give their patients a phone number to call for help using the new system but otherwise lacked the knowledge and the time to help them resolve new portal issues. Some clinicians specifically mentioned feeling ill-equipped to handle their patients’ needs for assistance with the new portal. These experiences exacerbated clinician stress during the transition.

Our veterans were using the MyHealtheVet messaging portal, and when our new system went up, it transitioned to My VA Health, but that wasn’t really communicated to the veterans very well. So, what happened was they would go into their MyHealtheVet like they had been doing for all of these years, to go in and request their medications, and when they pulled it up it’d show that they were assigned to a clinician in [a different state], that they have no active medications. Everything was just messed up. And they didn’t know why because there was no alert or notification that things would be changing. (P8, check in) I field all-day frustration from the veterans. And I love my job, I’m not leaving here even as frustrated as I am, because I’m here for them, not to, I’m here to serve the veterans and I have to advocate for them, and I know it will get better, it can’t stay like this. But I constantly field their frustrations.… So, I give them the 1-800 number to a Cerner help desk that helps with that, and I’ve had multiple [instances of] feedback that it didn’t help. (P13, one-year post) And [the patients are] frequently asking me things about their medication [within the portal], when, you know, I can’t help them with that. So, I have to send them back up to the front desk to try to figure out their medications. (P17, one-year post)

Veteran frustration and the burden of additional tasks due to issues with the portal

Clinicians reported that veterans expressed frustration with alternatives to the portal, including long call center wait times. Some veterans chose to walk into the clinic without an appointment rather than wait on the phone. Clinicians noted an increase in walk-ins by frustrated veterans, which placed added workload on clinics that were not staffed to handle the increase in walk-ins.

It’s been kind of clunky also with trying to get that [new portal] transitioned. And then that’s created more walk-ins here, because one, the vets get frustrated with the phone part of it, and then MyHealtheVet (prior portal) not [working], so they end up walking [into the clinic without an appointment]. (P19, check-in) In terms of messages, they can’t necessarily find the clinician they want to message. We had a veteran who came in recently who wanted to talk to their Rheumatologist, and it’s like, yeah, I typed in their name, and nothing came up. So, they have to try calling or coming in. (P20, 2-months post)

In summary, participants described the new patient portal as a source of stress for both themselves and their patients.

In addition to their own direct experience using a new EHR to communicate with their patients, clinicians and staff can be affected by perceptions of their patients’ experiences during an EHR transition [ 19 ]. At this first VA site to transition to the new EHR, clinicians and staff shared their concerns about their patients’ experiences using the portal. They were particularly troubled by unreliability of the secure messaging system and challenges patients faced learning to use the new system without proper instruction. Moreover, clinicians were alarmed to hear about patients having to make in-person visits– especially unplanned (i.e., walk in) ones– due to challenges with the new portal. Each of these issues needs to be addressed to ensure veteran satisfaction. However, the only solution participants could offer to frustrated patients was the telephone number to the help desk, leaving them with no clear knowledge of a solution strategy or a timeline for resolution of the issues.

We propose applying emergency preparedness actions to future EHR rollouts: mitigate, prepare, respond, and recover (Fig.  1 ) [ 13 ]. By applying these actions, patient portal disruptions may be alleviated and patients’ communication with their clinicians and access to care can be maintained. For example, issues stemming from a disruption in the portal may be mitigated by first identifying and understanding which patients typically use the portal and how they use it. Sites can use this information to prepare for the transition by disseminating instructional materials to staff and patients on how to access the new portal, targeting the most common and critical portal uses. Sites can respond to any expected and emerging portal disruptions by increasing access to alternative mechanisms for tasks disrupted by and typically completed within the portal. After the transition, recovery can begin by testing and demonstrating the accuracy and reliability of functions in the new portal. These actions directly address reported clinician concerns and can help maintain patient-clinician communication, and access to care.

The emergency preparedness framework was applied. This framework includes 4 actions: (1) mitigate, (2) prepare, (3) respond, and (4) recover. These actions can be repeated. Recommendations for how each action (1–4) can be applied to a portal transition are included in each blue quadrant of the circle

Sites could mitigate issues by first understanding which patients will be most affected by the transition, such as those who rely heavily on secure messaging. Reliable use of secure messaging within the VA facilitates positive patient-clinician relationships by providing a mechanism for efficient between-visit communication [ 20 , 21 , 22 , 23 ]. During the EHR transition, clinicians and staff became concerned about the well-being of patients from whom they weren’t receiving messages and those who depended on the portal to complete certain tasks. Since secure messaging is often initiated by patients to clinicians [ 23 ], clinicians will likely be unaware that messages are being missed. Understanding how and which patients currently use the portal and anticipating potential portal needs is a first step toward mitigating potential issues.

Despite efforts to inform Veterans of the EHR transition and patient portal [ 24 ] including information sent to a Veteran by email, direct mail, postings on VA websites, and a town hall, our findings agree with those of Fix and colleagues [ 10 ] and suggest that many Veterans were unprepared for the transition. Our findings suggest that end users heard that more is needed to improve the dissemination of knowledge about the transition and how to navigate the new patient portal to both VA employees and the patients they serve.

Preparations for the transition should prioritize providing VA clinicians and staff with updated information and resources on how to access and use the new portal [ 25 ]. VA clinicians deliver quality care to veterans and many VA employees are proud to serve the nation’s veterans and willing to go the extra mile to support their patients’ needs [ 26 ]. In this study, participants expressed feeling unprepared to assist or even respond to their patients’ questions and concerns about using the new portal. This unpreparedness contributed to increased clinician and staff stress, as they felt ill-equipped to help their patients with portal issues. Such experiences can negatively affect the patient-clinician relationship. Preparing clinicians and patients about an upcoming transition, including technical support for clinicians and patients, may help minimize these potential issues [ 10 , 27 ]. Specialized training about an impending transition, along with detailed instructions on how to gain access to the new system, and a dedicated portal helpline may be necessary to help patients better navigate the transition [ 23 , 28 ].

In addition to a dedicated helpline, our recommendations include responding to potential changes in needed veteran services during the transition. In our study, participants observed more veteran walk-ins due to challenges with the patient portal. Health systems need to anticipate and address this demand by expanding access to in-person services and fortifying other communication channels. For example, sites could use nurses to staff a walk-in clinic to handle increases in walk-in traffic and increase call center capacity to handle increases in telephone calls [ 29 ]. Increased use of walk-in clinics have received heightened attention as a promising strategy for meeting healthcare demands during the COVID-19 pandemic [ 30 ] and can potentially be adapted for meeting care-related needs during an EHR transition. These strategies can fill a gap in communication between clinicians and their patients while patients are learning to access and navigate a new electronic portal.

Finally, there is a need for a recovery mechanism to restore confidence in the reliability of the EHR and the well-being of clinicians and staff. Healthcare workers are experiencing unprecedented levels of stress [ 31 ]. A plan must be in place to improve and monitor the accuracy of data migrated, populated, and processed within the new system [ 2 ]. Knowing that portal function is monitored could help ease clinician and staff concerns and mitigate stress related to the transition.

Limitations

This study has several limitations. First, data collection relied on voluntary participation, which may introduce self-selection response bias. Second, this work was completed at one VA medical center that was the first site in the larger enterprise-wide transition, and experiences at other VAs or healthcare systems might differ substantially. Third, we did not interview veterans and relied entirely on secondhand accounts of patient experiences with the patient portal. Future research should include interviews with veterans during the transition and compare veteran and VA employee experiences.

Despite a current delay in the deployment of the new EHR at additional VA medical centers, findings from this study offer timely lessons that can ensure clinicians and staff are equipped to navigate challenges during the transition. The strategies presented in this paper could help maintain patient-clinician communication and improve veteran experience. Guided by the emergency preparedness framework, recommended strategies to address issues presented here include alerting those patients most affected by the EHR transition, being prepared to address patients’ concerns, increasing staffing for the help desk and walk-in care clinics, and monitoring the accuracy and reliability of the portal to provide assurance to healthcare workers that patients’ needs are being met. These strategies can inform change management at other VA medical centers that will soon undergo EHR transition and may have implications for other healthcare systems undergoing patient portal changes. Further work is needed to directly examine the perspectives of veterans using the portals, as well as the perspectives of both staff and patients in the growing number of healthcare systems beyond VA that are preparing for an EHR-to-EHR transition.

Data availability

Deidentified data analyzed for this study are available from the corresponding author on reasonable request.

Abbreviations

Electronic health record

Department of Veterans Affairs

VA Medical Centers

Department of Defense

Huang C, Koppel R, McGreevey JD 3rd, Craven CK, Schreiber R. Transitions from one Electronic Health record to another: challenges, pitfalls, and recommendations. Appl Clin Inf. 2020;11(5):742–54.

Article   Google Scholar  

Penrod LE. Electronic Health Record Transition considerations. PM R. 2017;9(5S):S13–8.

Article   PubMed   Google Scholar  

Cogan AM, Haltom TM, Shimada SL, Davila JA, McGinn BP, Fix GM. Understanding patients’ experiences during transitions from one electronic health record to another: a scoping review. PEC Innov. 2024;4:100258. https://doi.org/10.1016/j.pecinn.2024.100258 . PMID: 38327990; PMCID: PMC10847675.

Article   PubMed   PubMed Central   Google Scholar  

Powell KR. Patient-perceived facilitators of and barriers to Electronic Portal Use: a systematic review. Comput Inf Nurs. 2017;35(11):565–73.

Google Scholar  

Wilson-Stronks A, Lee KK, Cordero CL, et al. One size does not fit all: meeting the Health Care needs of diverse populations. Oakbrook Terrace, IL: The Joint Commission; 2008.

Carini E, Villani L, Pezzullo AM, Gentili A, Barbara A, Ricciardi W, Boccia S. The Impact of Digital Patient Portals on Health outcomes, System Efficiency, and patient attitudes: updated systematic literature review. J Med Internet Res. 2021;23(9):e26189.

Home -. My HealtheVet - My HealtheVet (va.gov).

Nazi KM, Turvey CL, Klein DM, Hogan TP. A decade of veteran voices: examining patient Portal Enhancements through the Lens of user-centered design. J Med Internet Res. 2018;20(7):e10413. https://doi.org/10.2196/10413 .

Tapuria A, Porat T, Kalra D, Dsouza G, Xiaohui S, Curcin V. Impact of patient access to their electronic health record: systematic review. Inf Health Soc Care. 2021;46:2.

Fix GM, Haltom TM, Cogan AM, et al. Understanding patients’ preferences and experiences during an Electronic Health Record Transition. J GEN INTERN MED. 2023. https://doi.org/10.1007/s11606-023-08338-6 .

Monturo C, Brockway C, Ginev A. Electronic Health Record Transition: the patient experience. CIN: Computers Inf Nurs. 2022;40:1.

Tian D, Hoehner CM, Woeltje KF, Luong L, Lane MA. Disrupted and restored patient experience with transition to New Electronic Health Record System. J Patient Exp. 2021;18:8.

Emergency management programs for healthcare facilities. the four phases of emergency management. US Department of Homeland Security website: https://www.hsdl.org/?view&did=765520 . Accessed 28 Aug 2023.

Ahlness EA, Orlander J, Brunner J, Cutrona SL, Kim B, Molloy-Paolillo BK, Rinne ST, Rucci J, Sayre G, Anderson E. Everything’s so Role-Specific: VA Employee Perspectives’ on Electronic Health Record (EHR) transition implications for roles and responsibilities. J Gen Intern Med. 2023;38(Suppl 4):991–8. Epub 2023 Oct 5. PMID: 37798577; PMCID: PMC10593626.

Rucci JM, Ball S, Brunner J, Moldestad M, Cutrona SL, Sayre G, Rinne S. Like one long battle: employee perspectives of the simultaneous impact of COVID-19 and an Electronic Health Record Transition. J Gen Intern Med. 2023;38(Suppl 4):1040–8. https://doi.org/10.1007/s11606-023-08284-3 . Epub 2023 Oct 5. PMID: 37798583; PMCID: PMC10593661.

Sayre G, Young J. Beyond open-ended questions: purposeful interview guide development to elicit rich, trustworthy data [videorecording]. Seattle (WA): VA Health Services Research & Development HSR&D Cyberseminars; 2018.

Averill JB. Matrix analysis as a complementary analytic strategy in qualitative inquiry. Qual Health Res. 2002;12:6855–66.

Elo S, Kyngäs H. The qualitative content analysis process. J Adv Nurs. 2008;62:1107–15.

Haun JN, Lind JD, Shimada SL, Simon SR. Evaluating Secure Messaging from the veteran perspective: informing the adoption and sustained use of a patient-driven communication platform. Ann Anthropol Pract. 2013;372:57–74.

Kittler AF, Carlson GL, Harris C, Lippincott M, Pizziferri L, Volk LA, et al. Primary care physician attitudes toward using a secure web-based portal designed to facilitate electronic communication with patients. Inf Prim Care. 2004;123:129–38.

Shimada SL, Petrakis BA, Rothendler JA, Zirkle M, Zhao S, Feng H, Fix GM, Ozkaynak M, Martin T, Johnson SA, Tulu B, Gordon HS, Simon SR, Woods SS. An analysis of patient-provider secure messaging at two Veterans Health Administration medical centers: message content and resolution through secure messaging. J Am Med Inf Assoc. 2017;24:5.

Jha AK, Perlin JB, Kizer KW, Dudley RA. Effect of the transformation of the Veterans Affairs Health Care System on the quality of care. N Engl J Med. 2003;348:22.

McAlearney AS, Walker DM, Gaughan A, Moffatt-Bruce S, Huerta TR. Helping patients be better patients: a qualitative study of perceptions about Inpatient Portal Use. Telemed J E Health. 2020;26:9.

https://www.myhealth.va.gov/mhv-portal-web/transitioning-to-my-va-health-learn-more .

Beagley L. Educating patients: understanding barriers, learning styles, and teaching techniques. J Perianesth Nurs. 2011;26:5.

Moldestad M, Stryczek KC, Haverhals L, Kenney R, Lee M, Ball S, et al. Competing demands: Scheduling challenges in being veteran-centric in the setting of Health System initiatives to Improve Access. Mil Med. 2021;186:11–2.

Adusumalli J, Bhagra A, Vitek S, Clark SD, Chon TY. Stress management in staff supporting electronic health record transitions: a novel approach. Explore (NY). 2021;17:6.

Heponiemi T, Gluschkoff K, Vehko T, Kaihlanen AM, Saranto K, Nissinen S, et al. Electronic Health Record implementations and Insufficient Training Endanger nurses’ Well-being: cross-sectional survey study. J Med Internet Res. 2021;23:12e27096.

Laurant M, van der Biezen M, Wijers N, Watananirun K, Kontopantelis E, van Vught AJ. Nurses as substitutes for doctors in primary care. Cochrane Database Syst Rev. 201;7(7):CD001271.

Elnahal S, Kadakia KT, Gondi S, How, U.S. Health systems Can Build Capacity to Handle Demand Surges. Harvard Business Review. 2021. https://hbr.org/2021/10/how-u-s-health-systems-can-build-capacity-to-handle-demand-surges/ Accessed 25 Nov 2022.

George RE, Lowe WA. Well-being and uncertainty in health care practice. Clin Teach. 2019;16:4.

Download references

Acknowledgments

We acknowledge and thank members of the EMPIRIC Evaluation qualitative and supporting team for their contributions to this work including Ellen Ahlness, PhD, Julian Brunner, PhD, Adena Cohen-Bearak, MPH, M.Ed, Leah Cubanski, BA, Christine Firestone, Bo Kim, PhD, Megan Moldestad, MS, and Rachel Smith. We greatly appreciate the staff at the Mann-Grandstaff VA Medical Center and associated community-based outpatient clinics for generously sharing of their time and experiences participating in this study during this challenging time.

The “EHRM Partnership Integrating Rapid Cycle Evaluation to Improve Cerner Implementation (EMPIRIC)” (PEC 20–168) work was supported by funding from the US Department of Veterans Affairs, Veterans Health Administration, Health Services Research & Development Quality Enhancement Research Initiative (QUERI) (PEC 20–168). The findings and conclusions in this article are those of the authors and do not necessarily reflect the views of the Veterans Health Administration, Veterans Affairs, or any participating health agency or funder.

Author information

Authors and affiliations.

VA Northeast Ohio Healthcare System, 10701 East Blvd., Research Service 151, 44106, Cleveland, OH, USA

Sherry L. Ball

Center for Healthcare Organization and Implementation Research, VA Boston Healthcare System, Boston, MA, USA

Bo Kim & Seppo T. Rinne

Department of Psychiatry, Harvard Medical School, Boston, MA, USA

Center for Healthcare Organization and Implementation Research, VA Bedford Healthcare System, Bedford, MA, USA

Sarah L. Cutrona & Brianne K. Molloy-Paolillo

Division of Health Informatics & Implementation Science, Department of Population and Quantitative Health Sciences, University of Massachusetts Chan Medical School, Worcester, MA, USA

Sarah L. Cutrona

Seattle-Denver Center of Innovation for Veteran-Centered and Value-Driven Care, VHA Puget Sound Health Care System, Seattle, WA, USA

Ellen Ahlness, Megan Moldestad & George Sayre

University of Washington School of Public Health, Seattle, WA, USA

George Sayre

Geisel School of Medicine at Dartmouth, Hannover, NH, USA

Seppo T. Rinne

You can also search for this author in PubMed   Google Scholar

Contributions

S.R. designed the larger study. G.S. was the qualitative methodologist who led the qualitative team. S.B., E.A., and M.M. created the interview guides and completed the interviews; Data analysis, data interpretation, and the initial manuscript draft were completed by S.B. and B.K. S.C. and B.M. worked with the qualitative team to finalize the analysis and edit and finalize the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Sherry L. Ball .

Ethics declarations

Ethics approval and consent to participate.

This evaluation was designated as non-research/quality improvement by the VA Bedford Healthcare System Institutional Review Board. All methods were carried out in accordance with local and national VA guidelines and regulations for quality improvement activities. This study included virtual interviews with participants via MS Teams. Employees volunteered to participate in interviews and verbal consent was obtained to record interviews. Study materials, including interview guides with verbal consent procedures, were reviewed and approved by labor unions and determined as non-research by the VA Bedford Healthcare System Institutional Review Board.

Consent for publication

Not applicable.

The findings and conclusions in this paper are those of the authors and do not necessarily represent the official position of the Department of Veterans Affairs.

Prior presentations

Ball S, Kim B, Moldestad M, Molloy-Paolillo B, Cubanski L, Cutrona S, Sayre G, and Rinne S. (2022, June). Electronic Health Record Transition: Providers’ Experiences with Frustrated Patients. Poster presentation at the 2022 AcademyHealth Annual Research Meeting. June 2022.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Ball, S.L., Kim, B., Cutrona, S.L. et al. Clinician and staff experiences with frustrated patients during an electronic health record transition: a qualitative case study. BMC Health Serv Res 24 , 535 (2024). https://doi.org/10.1186/s12913-024-10974-5

Download citation

Received : 29 August 2023

Accepted : 09 April 2024

Published : 26 April 2024

DOI : https://doi.org/10.1186/s12913-024-10974-5

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• EHR transition
• Patient experience
• Clinician experience
• Qualitative analysis

BMC Health Services Research

ISSN: 1472-6963

• General enquiries: [email protected]