implicit bias assignment

Implicit Bias (Unconscious Bias): Definition & Examples

Charlotte Ruhl

Research Assistant & Psychology Graduate

BA (Hons) Psychology, Harvard University

Charlotte Ruhl, a psychology graduate from Harvard College, boasts over six years of research experience in clinical and social psychology. During her tenure at Harvard, she contributed to the Decision Science Lab, administering numerous studies in behavioral economics and social psychology.

Learn about our Editorial Process

Saul Mcleod, PhD

Editor-in-Chief for Simply Psychology

BSc (Hons) Psychology, MRes, PhD, University of Manchester

Saul Mcleod, PhD., is a qualified psychology teacher with over 18 years of experience in further and higher education. He has been published in peer-reviewed journals, including the Journal of Clinical Psychology.

On This Page:

Implicit bias refers to the beliefs and attitudes that affect our understanding, actions and decisions in an unconscious way.

Take-home Messages

• Implicit biases are unconscious attitudes and stereotypes that can manifest in the criminal justice system, workplace, school setting, and in the healthcare system.
• Implicit bias is also known as unconscious bias or implicit social cognition.
• There are many different examples of implicit biases, ranging from categories of race, gender, and sexuality.
• These biases often arise from trying to find patterns and navigate the overwhelming stimuli in this complicated world. Culture, media, and upbringing can also contribute to the development of such biases.
• Removing these biases is a challenge, especially because we often don’t even know they exist, but research reveals potential interventions and provides hope that levels of implicit biases in the United States are decreasing.

The term implicit bias was first coined in 1995 by psychologists Mahzarin Banaji and Anthony Greenwald, who argued that social behavior is largely influenced by unconscious associations and judgments (Greenwald & Banaji, 1995).

So, what is implicit bias?

Specifically, implicit bias refers to attitudes or stereotypes that affect our understanding, actions, and decisions in an unconscious way, making them difficult to control.

Since the mid-90s, psychologists have extensively researched implicit biases, revealing that, without even knowing it, we all possess our own implicit biases.

System 1 and System 2 Thinking

Kahneman (2011) distinguishes between two types of thinking: system 1 and system 2.

• System 1 is the brain’s fast, emotional, unconscious thinking mode. This type of thinking requires little effort, but it is often error-prone. Most everyday activities (like driving, talking, cleaning, etc.) heavily use the type 1 system.
• System 2 is slow, logical, effortful, conscious thought, where reason dominates.

Implicit Bias vs. Explicit Bias

What is meant by implicit bias.

Implicit bias (unconscious bias) refers to attitudes and beliefs outside our conscious awareness and control. Implicit biases are an example of system one thinking, so we are unaware they exist (Greenwald & Krieger, 2006).

An implicit bias may counter a person’s conscious beliefs without realizing it. For example, it is possible to express explicit liking of a certain social group or approval of a certain action while simultaneously being biased against that group or action on an unconscious level.

Therefore, implicit and explicit biases might differ for the same person.

It is important to understand that implicit biases can become explicit biases. This occurs when you become consciously aware of your prejudices and beliefs. They surface in your mind, leading you to choose whether to act on or against them.

What is meant by explicit bias?

Explicit biases are biases we are aware of on a conscious level (for example, feeling threatened by another group and delivering hate speech as a result). They are an example of system 2 thinking.

It is also possible that your implicit and explicit biases differ from your neighbor, friend, or family member. Many factors can control how such biases are developed.

What Are the Implications of Unconscious Bias?

Implicit biases become evident in many different domains of society. On an interpersonal level, they can manifest in simply daily interactions.

This occurs when certain actions (or microaggressions) make others feel uncomfortable or aware of the specific prejudices you may hold against them.

Implicit Prejudice

Implicit prejudice is the automatic, unconscious attitudes or stereotypes that influence our understanding, actions, and decisions. Unlike explicit prejudice, which is consciously controlled, implicit prejudice can occur even in individuals who consciously reject prejudice and strive for impartiality.

Unconscious racial stereotypes are a major example of implicit prejudice. In other words, having an automatic preference for one race over another without being aware of this bias.

This bias can manifest in small interpersonal interactions and has broader implications in society’s legal system and many other important sectors.

Examples may include holding an implicit stereotype that associates Black individuals as violent. As a result, you may cross the street at night when you see a Black man walking in your direction without even realizing why you are crossing the street.

The action taken here is an example of a microaggression. A microaggression is a subtle, automatic, and often nonverbal that communicates hostile, derogatory, or negative prejudicial slights and insults toward any group (Pierce, 1970). Crossing the street communicates an implicit prejudice, even though you might not even be aware.

Another example of an implicit racial bias is if a Latino student is complimented by a teacher for speaking perfect English, but he is a native English speaker. Here, the teacher assumed that English would not be his first language simply because he is Latino.

Gender Stereotypes

Gender biases are another common form of implicit bias. Gender biases are the ways in which we judge men and women based on traditional feminine and masculine assigned traits.

For example, a greater assignment of fame to male than female names (Banaji & Greenwald, 1995) reveals a subconscious bias that holds men at a higher level than their female counterparts. Whether you voice the opinion that men are more famous than women is independent of this implicit gender bias.

Another common implicit gender bias regards women in STEM (science, technology, engineering, and mathematics).

In school, girls are more likely to be associated with language over math. In contrast, males are more likely to be associated with math over language (Steffens & Jelenec, 2011), revealing clear gender-related implicit biases that can ultimately go so far as to dictate future career paths.

Even if you outwardly say men and women are equally good at math, it is possible you subconsciously associate math more strongly with men without even being aware of this association.

Health Care

Healthcare is another setting where implicit biases are very present. Racial and ethnic minorities and women are subject to less accurate diagnoses, curtailed treatment options, less pain management, and worse clinical outcomes (Chapman, Kaatz, & Carnes, 2013).

Additionally, Black children are often not treated as children or given the same compassion or level of care provided for White children (Johnson et al., 2017).

It becomes evident that implicit biases infiltrate the most common sectors of society, making it all the more important to question how we can remove these biases.

LGBTQ+ Community Bias

Similar to implicit racial and gender biases, individuals may hold implicit biases against members of the LGBTQ+ community. Again, that does not necessarily mean that these opinions are voiced outwardly or even consciously recognized by the beholder, for that matter.

Rather, these biases are unconscious. A really simple example could be asking a female friend if she has a boyfriend, assuming her sexuality and that heterosexuality is the norm or default.

Instead, you could ask your friend if she is seeing someone in this specific situation. Several other forms of implicit biases fall into categories ranging from weight to ethnicity to ability that come into play in our everyday lives.

Legal System

Both law enforcement and the legal system shed light on implicit biases. An example of implicit bias functioning in law enforcement is the shooter bias – the tendency among the police to shoot Black civilians more often than White civilians, even when they are unarmed (Mekawi & Bresin, 2015).

This bias has been repeatedly tested in the laboratory setting, revealing an implicit bias against Black individuals. Blacks are also disproportionately arrested and given harsher sentences, and Black juveniles are tried as adults more often than their White peers.

Black boys are also seen as less childlike, less innocent, more culpable, more responsible for their actions, and as being more appropriate targets for police violence (Goff, 2014).

Together, these unconscious stereotypes, which are not rooted in truth, form an array of implicit biases that are extremely dangerous and utterly unjust.

Implicit biases are also visible in the workplace. One experiment that tracked the success of White and Black job applicants found that stereotypically White received 50% more callbacks than stereotypically Black names, regardless of the industry or occupation (Bertrand & Mullainathan, 2004).

This reveals another form of implicit bias: the hiring bias – Anglicized‐named applicants receiving more favorable pre‐interview impressions than other ethnic‐named applicants (Watson, Appiah, & Thornton, 2011).

We’re susceptible to bias because of these tendencies:

We tend to seek out patterns

A key reason we develop such biases is that our brains have a natural tendency to look for patterns and associations to make sense of a very complicated world.

Research shows that even before kindergarten, children already use their group membership (e.g., racial group, gender group, age group, etc.) to guide inferences about psychological and behavioral traits.

At such a young age, they have already begun seeking patterns and recognizing what distinguishes them from other groups (Baron, Dunham, Banaji, & Carey, 2014).

And not only do children recognize what sets them apart from other groups, they believe “what is similar to me is good, and what is different from me is bad” (Cameron, Alvarez, Ruble, & Fuligni, 2001).

Children aren’t just noticing how similar or dissimilar they are to others; dissimilar people are actively disliked (Aboud, 1988).

Recognizing what sets you apart from others and then forming negative opinions about those outgroups (a social group with which an individual does not identify) contributes to the development of implicit biases.

We like to take shortcuts

Another explanation is that the development of these biases is a result of the brain’s tendency to try to simplify the world.

Mental shortcuts make it faster and easier for the brain to sort through all of the overwhelming data and stimuli we are met with every second of the day. And we take mental shortcuts all the time. Rules of thumb, educated guesses, and using “common sense” are all forms of mental shortcuts.

Implicit bias is a result of taking one of these cognitive shortcuts inaccurately (Rynders, 2019). As a result, we incorrectly rely on these unconscious stereotypes to provide guidance in a very complex world.

And especially when we are under high levels of stress, we are more likely to rely on these biases than to examine all of the relevant, surrounding information (Wigboldus, Sherman, Franzese, & Knippenberg, 2004).

Social and Cultural influences

Influences from media, culture, and your individual upbringing can also contribute to the rise of implicit associations that people form about the members of social outgroups. Media has become increasingly accessible, and while that has many benefits, it can also lead to implicit biases.

The way TV portrays individuals or the language journal articles use can ingrain specific biases in our minds.

For example, they can lead us to associate Black people with criminals or females as nurses or teachers. The way you are raised can also play a huge role. One research study found that parental racial attitudes can influence children’s implicit prejudice (Sinclair, Dunn, & Lowery, 2005).

And parents are not the only figures who can influence such attitudes. Siblings, the school setting, and the culture in which you grow up can also shape your explicit beliefs and implicit biases.

Implicit Attitude Test (IAT)

What sets implicit biases apart from other forms is that they are subconscious – we don’t know if we have them.

However, researchers have developed the Implicit Association Test (IAT) tool to help reveal such biases.

The Implicit Attitude Test (IAT) is a psychological assessment to measure an individual’s unconscious biases and associations. The test measures how quickly a person associates concepts or groups (such as race or gender) with positive or negative attributes, revealing biases that may not be consciously acknowledged.

The IAT requires participants to categorize negative and positive words together with either images or words (Greenwald, McGhee, & Schwartz, 1998).

Tests are taken online and must be performed as quickly as possible, the faster you categorize certain words or faces of a category, the stronger the bias you hold about that category.

For example, the Race IAT requires participants to categorize White faces and Black faces and negative and positive words. The relative speed of association of black faces with negative words is used as an indication of the level of anti-black bias.

Professor Brian Nosek and colleagues tested more than 700,000 subjects. They found that more than 70% of White subjects more easily associated White faces with positive words and Black faces with negative words, concluding that this was evidence of implicit racial bias (Nosek, Greenwald, & Banaji, 2007).

Outside of lab testing, it is very difficult to know if we do, in fact, possess these biases. The fact that they are so hard to detect is in the very nature of this form of bias, making them very dangerous in various real-world settings.

How to Reduce Implicit Bias

Because of the harmful nature of implicit biases, it is critical to examine how we can begin to remove them.

Practicing mindfulness is one potential way, as it reduces the stress and cognitive load that otherwise leads to relying on such biases.

A 2016 study found that brief mediation decreased unconscious bias against black people and elderly people (Lueke & Gibson, 2016), providing initial insight into the usefulness of this approach and paving the way for future research on this intervention.

Adjust your perspective

Another method is perspective-taking – looking beyond your own point of view so that you can consider how someone else may think or feel about something.

Researcher Belinda Gutierrez implemented a videogame called “Fair Play,” in which players assume the role of a Black graduate student named Jamal Davis.

As Jamal, players experience subtle race bias while completing “quests” to obtain a science degree.

Gutierrez hypothesized that participants who were randomly assigned to play the game would have greater empathy for Jamal and lower implicit race bias than participants randomized to read narrative text (not perspective-taking) describing Jamal’s experience (Gutierrez, 2014), and her hypothesis was supported, illustrating the benefits of perspective taking in increasing empathy towards outgroup members.

Specific implicit bias training has been incorporated in different educational and law enforcement settings. Research has found that diversity training to overcome biases against women in STEM improved with men (Jackson, Hillard, & Schneider, 2014).

Training programs designed to target and help overcome implicit biases may also be beneficial for police officers (Plant & Peruche, 2005), but there is not enough conclusive evidence to completely support this claim. One pitfall of such training is a potential rebound effect.

Actively trying to inhibit stereotyping actually results in the bias eventually increasing more so than if it had not been initially suppressed in the first place (Macrae, Bodenhausen, Milne, & Jetten, 1994). This is very similar to the white bear problem that is discussed in many psychology curricula.

This concept refers to the psychological process whereby deliberate attempts to suppress certain thoughts make them more likely to surface (Wegner & Schneider, 2003).

Education is crucial. Understanding what implicit biases are, how they can arise how, and how to recognize them in yourself and others are all incredibly important in working towards overcoming such biases.

Learning about other cultures or outgroups and what language and behaviors may come off as offensive is critical as well. Education is a powerful tool that can extend beyond the classroom through books, media, and conversations.

On the bright side, implicit biases in the United States have been improving.

From 2007 to 2016, implicit biases have changed towards neutrality for sexual orientation, race, and skin-tone attitudes (Charlesworth & Banaji, 2019), demonstrating that it is possible to overcome these biases.

Books for further reading

As mentioned, education is extremely important. Here are a few places to get started in learning more about implicit biases:

• Biased: Uncovering the Hidden Prejudice That Shapes What We See Think and Do by Jennifer Eberhardt
• Blindspot by Anthony Greenwald and Mahzarin Banaji
• Implicit Racial Bias Across the Law by Justin Levinson and Robert Smith

Keywords and Terminology

To find materials on implicit bias and related topics, search databases and other tools using the following keywords:

Is unconscious bias the same as implicit bias?

Yes, unconscious bias is the same as implicit bias. Both terms refer to the biases we carry without awareness or conscious control, which can affect our attitudes and actions toward others.

In what ways can implicit bias impact our interactions with others?

Implicit bias can impact our interactions with others by unconsciously influencing our attitudes, behaviors, and decisions. This can lead to stereotyping, prejudice, and discrimination, even when we consciously believe in equality and fairness.

It can affect various domains of life, including workplace dynamics, healthcare provision, law enforcement, and everyday social interactions.

What are some implicit bias examples?

Some examples of implicit biases include assuming a woman is less competent than a man in a leadership role, associating certain ethnicities with criminal behavior, or believing that older people are not technologically savvy.

Other examples include perceiving individuals with disabilities as less capable or assuming that someone who is overweight is lazy or unmotivated.

Aboud, F. E. (1988). Children and prejudice . B. Blackwell.

Banaji, M. R., & Greenwald, A. G. (1995). Implicit gender stereotyping in judgments of fame. Journal of Personality and Social Psychology , 68 (2), 181.

Baron, A. S., Dunham, Y., Banaji, M., & Carey, S. (2014). Constraints on the acquisition of social category concepts. Journal of Cognition and Development , 15 (2), 238-268.

Bertrand, M., & Mullainathan, S. (2004). Are Emily and Greg more employable than Lakisha and Jamal? A field experiment on labor market discrimination. American economic review , 94 (4), 991-1013.

Cameron, J. A., Alvarez, J. M., Ruble, D. N., & Fuligni, A. J. (2001). Children’s lay theories about ingroups and outgroups: Reconceptualizing research on prejudice. Personality and Social Psychology Review , 5 (2), 118-128.

Chapman, E. N., Kaatz, A., & Carnes, M. (2013). Physicians and implicit bias: how doctors may unwittingly perpetuate health care disparities. Journal of general internal medicine , 28 (11), 1504-1510.

Charlesworth, T. E., & Banaji, M. R. (2019). Patterns of implicit and explicit attitudes: I. Long-term change and stability from 2007 to 2016. Psychological science , 30(2), 174-192.

Goff, P. A., Jackson, M. C., Di Leone, B. A. L., Culotta, C. M., & DiTomasso, N. A. (2014). The essence of innocence: consequences of dehumanizing Black children. Journal of personality and socialpsychology,106(4), 526.

Greenwald, A. G., & Banaji, M. R. (1995). Implicit social cognition: attitudes, self-esteem, and stereotypes. Psychological review, 102(1), 4.

Greenwald, A. G., McGhee, D. E., & Schwartz, J. L. (1998). Measuring individual differences in implicit cognition: the implicit association test. Journal of personality and social psychology , 74(6), 1464.

Greenwald, A. G., & Krieger, L. H. (2006). Implicit bias: Scientific foundations. California Law Review , 94 (4), 945-967.

Gutierrez, B., Kaatz, A., Chu, S., Ramirez, D., Samson-Samuel, C., & Carnes, M. (2014). “Fair Play”: a videogame designed to address implicit race bias through active perspective taking. Games for health journal , 3 (6), 371-378.

Jackson, S. M., Hillard, A. L., & Schneider, T. R. (2014). Using implicit bias training to improve attitudes toward women in STEM. Social Psychology of Education , 17 (3), 419-438.

Johnson, T. J., Winger, D. G., Hickey, R. W., Switzer, G. E., Miller, E., Nguyen, M. B., … & Hausmann, L. R. (2017). Comparison of physician implicit racial bias toward adults versus children. Academic pediatrics , 17 (2), 120-126.

Kahneman, D. (2011). Thinking, fast and slow . Macmillan.

Lueke, A., & Gibson, B. (2016). Brief mindfulness meditation reduces discrimination. Psychology of Consciousness: Theory, Research, and Practice , 3 (1), 34.

Macrae, C. N., Bodenhausen, G. V., Milne, A. B., & Jetten, J. (1994). Out of mind but back in sight: Stereotypes on the rebound. Journal of personality and social psychology , 67 (5), 808.

Mekawi, Y., & Bresin, K. (2015). Is the evidence from racial bias shooting task studies a smoking gun? Results from a meta-analysis. Journal of Experimental Social Psychology , 61 , 120-130.

Nosek, B. A., Greenwald, A. G., & Banaji, M. R. (2007). The Implicit Association Test at age 7: A methodological and conceptual review. Automatic processes in social thinking and behavior , 4 , 265-292.

Pierce, C. (1970). Offensive mechanisms. The black seventies , 265-282.

Plant, E. A., & Peruche, B. M. (2005). The consequences of race for police officers’ responses to criminal suspects. Psychological Science , 16 (3), 180-183.

Rynders, D. (2019). Battling Implicit Bias in the IDEA to Advocate for African American Students with Disabilities. Touro L. Rev. , 35 , 461.

Sinclair, S., Dunn, E., & Lowery, B. (2005). The relationship between parental racial attitudes and children’s implicit prejudice. Journal of Experimental Social Psychology , 41 (3), 283-289.

Steffens, M. C., & Jelenec, P. (2011). Separating implicit gender stereotypes regarding math and language: Implicit ability stereotypes are self-serving for boys and men, but not for girls and women. Sex Roles , 64(5-6), 324-335.

Watson, S., Appiah, O., & Thornton, C. G. (2011). The effect of name on pre‐interview impressions and occupational stereotypes: the case of black sales job applicants. Journal of Applied Social Psychology , 41 (10), 2405-2420.

Wegner, D. M., & Schneider, D. J. (2003). The white bear story. Psychological Inquiry , 14 (3-4), 326-329.

Wigboldus, D. H., Sherman, J. W., Franzese, H. L., & Knippenberg, A. V. (2004). Capacity and comprehension: Spontaneous stereotyping under cognitive load. Social Cognition , 22 (3), 292-309.

Further Information

Test yourself for bias.

• Project Implicit (IAT Test) From Harvard University
• Implicit Association Test From the Social Psychology Network
• Test Yourself for Hidden Bias From Teaching Tolerance
• How The Concept Of Implicit Bias Came Into Being With Dr. Mahzarin Banaji, Harvard University. Author of Blindspot: hidden biases of good people5:28 minutes; includes a transcript
• Understanding Your Racial Biases With John Dovidio, Ph.D., Yale University From the American Psychological Association11:09 minutes; includes a transcript
• Talking Implicit Bias in Policing With Jack Glaser, Goldman School of Public Policy, University of California Berkeley21:59 minutes
• Implicit Bias: A Factor in Health Communication With Dr. Winston Wong, Kaiser Permanente19:58 minutes
• Bias, Black Lives and Academic Medicine Dr. David Ansell on Your Health Radio (August 1, 2015)21:42 minutes
• Uncovering Hidden Biases Google talk with Dr. Mahzarin Banaji, Harvard University
• Impact of Implicit Bias on the Justice System 9:14 minutes
• Students Speak Up: What Bias Means to Them 2:17 minutes
• Weight Bias in Health Care From Yale University16:56 minutes
• Gender and Racial Bias In Facial Recognition Technology 4:43 minutes

Journal Articles

• An implicit bias primer Mitchell, G. (2018). An implicit bias primer. Virginia Journal of Social Policy & the Law , 25, 27–59.
• Implicit Association Test at age 7: A methodological and conceptual review Nosek, B. A., Greenwald, A. G., & Banaji, M. R. (2007). The Implicit Association Test at age 7: A methodological and conceptual review. Automatic processes in social thinking and behavior, 4 , 265-292.
• Implicit Racial/Ethnic Bias Among Health Care Professionals and Its Influence on Health Care Outcomes: A Systematic Review Hall, W. J., Chapman, M. V., Lee, K. M., Merino, Y. M., Thomas, T. W., Payne, B. K., … & Coyne-Beasley, T. (2015). Implicit racial/ethnic bias among health care professionals and its influence on health care outcomes: a systematic review. American Journal of public health, 105 (12), e60-e76.
• Reducing Racial Bias Among Health Care Providers: Lessons from Social-Cognitive Psychology Burgess, D., Van Ryn, M., Dovidio, J., & Saha, S. (2007). Reducing racial bias among health care providers: lessons from social-cognitive psychology. Journal of general internal medicine, 22 (6), 882-887.
• Integrating implicit bias into counselor education Boysen, G. A. (2010). Integrating Implicit Bias Into Counselor Education. Counselor Education & Supervision, 49 (4), 210–227.
• Cognitive Biases and Errors as Cause—and Journalistic Best Practices as Effect Christian, S. (2013). Cognitive Biases and Errors as Cause—and Journalistic Best Practices as Effect. Journal of Mass Media Ethics, 28 (3), 160–174.
• Empathy intervention to reduce implicit bias in pre-service teachers Whitford, D. K., & Emerson, A. M. (2019). Empathy Intervention to Reduce Implicit Bias in Pre-Service Teachers. Psychological Reports, 122 (2), 670–688.

• Bipolar Disorder
• Therapy Center
• When To See a Therapist
• Types of Therapy
• Best Online Therapy
• Best Couples Therapy
• Best Family Therapy
• Managing Stress
• Sleep and Dreaming
• Understanding Emotions
• Self-Improvement
• Healthy Relationships
• Student Resources
• Personality Types
• Guided Meditations
• Verywell Mind Insights
• 2023 Verywell Mind 25
• Mental Health in the Classroom
• Editorial Process
• Meet Our Review Board
• Crisis Support

How Does Implicit Bias Influence Behavior?

Strategies to Reduce the Impact of Implicit Bias

Kendra Cherry, MS, is a psychosocial rehabilitation specialist, psychology educator, and author of the "Everything Psychology Book."

Akeem Marsh, MD, is a board-certified child, adolescent, and adult psychiatrist who has dedicated his career to working with medically underserved communities.

Getty Images 

• Measurement
• Discrimination

An implicit bias is an unconscious association, belief, or attitude toward any social group. Implicit biases are one reason why people often attribute certain qualities or characteristics to all members of a particular group, a phenomenon known as stereotyping .

It is important to remember that implicit biases operate almost entirely on an unconscious level . While explicit biases and prejudices are intentional and controllable, implicit biases are less so.

A person may even express explicit disapproval of a certain attitude or belief while still harboring similar biases on a more unconscious level. Such biases do not necessarily align with our own sense of self and personal identity. People can also hold positive or negative associations about their own race, gender, religion, sexuality, or other personal characteristics.

Causes of Implicit Bias

While people might like to believe that they are not susceptible to these implicit biases and stereotypes, the reality is that everyone engages in them whether they like it or not. This reality, however, does not mean that you are necessarily prejudiced or inclined to discriminate against other people. It simply means that your brain is working in a way that makes associations and generalizations.

In addition to the fact that we are influenced by our environment and stereotypes that already exist in the society into which we were born, it is generally impossible to separate ourselves from the influence of society.

You can, however, become more aware of your unconscious thinking and the ways in which society influences you.

It is the natural tendency of the brain to sift, sort, and categorize information about the world that leads to the formation of these implicit biases. We're susceptible to bias because of these tendencies:

• We tend to seek out patterns . Implicit bias occurs because of the brain's natural tendency to look for patterns and associations in the world. Social cognition , or our ability to store, process, and apply information about people in social situations, is dependent on this ability to form associations about the world.
• We like to take shortcuts . Like other cognitive biases , implicit bias is a result of the brain's tendency to try to simplify the world. Because the brain is constantly inundated with more information than it could conceivably process, mental shortcuts make it faster and easier for the brain to sort through all of this data.
• Our experiences and social conditioning play a role . Implicit biases are influenced by experiences, although these attitudes may not be the result of direct personal experience. Cultural conditioning, media portrayals, and upbringing can all contribute to the implicit associations that people form about the members of other social groups.

How Implicit Bias Is Measured

The term implicit bias was first coined by social psychologists Mahzarin Banaji and Tony Greenwald in 1995. In an influential paper introducing their theory of implicit social cognition, they proposed that social behavior was largely influenced by unconscious associations and judgments.

In 1998, Banaji and Greenwald published their now-famous Implicit Association Test (IAT) to support their hypothesis . The test utilizes a computer program to show respondents a series of images and words to determine how long it takes someone to choose between two things.

Subjects might be shown images of faces of different racial backgrounds, for example, in conjunction with either a positive word or a negative word. Subjects would then be asked to click on a positive word when they saw an image of someone from one race and to click on a negative word when they saw someone of another race.

Interpreting the Results

The researchers suggest that when someone clicks quickly, it means that they possess a stronger unconscious association.   If a person quickly clicks on a negative word every time they see a person of a particular race, the researchers suggest that this would indicate that they hold an implicit negative bias toward individuals of that race.

In addition to a test of implicit racial attitudes, the IAT has also been utilized to measure unconscious biases related to gender, weight, sexuality, disability, and other areas. The IAT has grown in popularity and use over the last decade, yet has recently come under fire.

Among the main criticisms are findings that the test results may lack reliability . Respondents may score high on racial bias on one test, and low the next time they are tested.

Also of concern is that scores on the test may not necessarily correlate with individual behavior. People may score high for a type of bias on the IAT, but those results may not accurately predict how they would relate to members of a specific social group.

Link Between Implicit Bias and Discrimination

It is important to understand that implicit bias is not the same thing as racism, although the two concepts are related. Overt racism involves conscious prejudice against members of a particular racial group and can be influenced by both explicit and implicit biases.

Other forms of discrimination that can be influenced by unconscious biases include ageism , sexism, homophobia, and ableism.

One of the benefits of being aware of the potential impact of implicit social biases is that you can take a more active role in overcoming social stereotypes, discrimination, and prejudice.

Effects of Implicit Bias

Implicit biases can influence how people behave toward the members of different social groups. Researchers have found that such bias can have effects in a number of settings, including in school, work, and legal proceedings.

Implicit Bias in School

Implicit bias can lead to a phenomenon known as stereotype threat in which people internalize negative stereotypes about themselves based upon group associations. Research has shown, for example, that young girls often internalize implicit attitudes related to gender and math performance.  

By the age of 9, girls have been shown to exhibit the unconscious beliefs that females have a preference for language over math.   The stronger these implicit beliefs are, the less likely girls and women are to pursue math performance in school. Such unconscious beliefs are also believed to play a role in inhibiting women from pursuing careers in science, technology, engineering, and mathematics (STEM) fields.

Studies have also demonstrated that implicit attitudes can also influence how teachers respond to student behavior, suggesting that implicit bias can have a powerful impact on educational access and academic achievement.

One study, for example, found that Black children—and Black boys in particular—were more likely to be expelled from school for behavioral issues. When teachers were told to watch for challenging behaviors, they were more likely to focus on Black children than on White children.

Implicit Bias In the Workplace

While the Implicit Attitude Test itself may have pitfalls, these problems do not negate the existence of implicit bias. Or the existence and effects of bias, prejudice, and discrimination in the real world. Such prejudices can have very real and potentially devastating consequences.

One study, for example, found that when Black and White job seekers sent out similar resumes to employers, Black applicants were half as likely to be called in for interviews as White job seekers with equal qualifications.

Such discrimination is likely the result of both explicit and implicit biases toward racial groups.

Even when employers strive to eliminate potential bias in hiring, subtle implicit biases may still have an impact on how people are selected for jobs or promoted to advanced positions. Avoiding such biases entirely can be difficult, but being aware of their existence and striving to minimize them can help.

Implicit Bias in Healthcare Settings

Certainly, age, race, or health condition should not play a role in how patients get treated, however, implicit bias can influence quality healthcare and have long-term impacts including suboptimal care, adverse outcomes, and even death.

For example, one study published in the American Journal of Public Health found that physicians with high scores in implicit bias tended to dominate conversations with Black patients and, as a result, the Black patients had less confidence and trust in the provider and rated the quality of their care lower.  

Researchers continue to investigate implicit bias in relation to other ethnic groups as well as specific health conditions, including type 2 diabetes, obesity, mental health, and substance use disorders.

Implicit Bias in Legal Settings

Implicit biases can also have troubling implications in legal proceedings, influencing everything from initial police contact all the way through sentencing. Research has found that there is an overwhelming racial disparity in how Black defendants are treated in criminal sentencing.  

Not only are Black defendants less likely to be offered plea bargains than White defendants charged with similar crimes, but they are also more likely to receive longer and harsher sentences than White defendants.

Strategies to Reduce the Impact of Implict Bias

Implicit biases impact behavior, but there are things that you can do to reduce your own bias. Some ways that you can reduce the influence of implicit bias:

• Focus on seeing people as individuals . Rather than focusing on stereotypes to define people, spend time considering them on a more personal, individual level.
• Work on consciously changing your stereotypes . If you do recognize that your response to a person might be rooted in biases or stereotypes, make an effort to consciously adjust your response.
• Take time to pause and reflect . In order to reduce reflexive reactions, take time to reflect on potential biases and replace them with positive examples of the stereotyped group. 
• Adjust your perspective . Try seeing things from another person's point of view. How would you respond if you were in the same position? What factors might contribute to how a person acts in a particular setting or situation?
• Increase your exposure . Spend more time with people of different racial backgrounds. Learn about their culture by attending community events or exhibits.
• Practice mindfulness . Try meditation, yoga, or focused breathing to increase mindfulness and become more aware of your thoughts and actions.

While implicit bias is difficult to eliminate altogether, there are strategies that you can utilize to reduce its impact. Taking steps such as actively working to overcome your biases , taking other people's perspectives, seeking greater diversity in your life, and building your awareness about your own thoughts are a few ways to reduce the impact of implicit bias.

A Word From Verywell

Implicit biases can be troubling, but they are also a pervasive part of life. Perhaps more troubling, your unconscious attitudes may not necessarily align with your declared beliefs. While people are more likely to hold implicit biases that favor their own in-group, it is not uncommon for people to hold biases against their own social group as well.

The good news is that these implicit biases are not set in stone. Even if you do hold unconscious biases against other groups of people, it is possible to adopt new attitudes, even on the unconscious level.   This process is not necessarily quick or easy, but being aware of the existence of these biases is a good place to start making a change.

Jost JT. The existence of implicit bias is beyond reasonable doubt: A refutation of ideological and methodological objections and executive summary of ten studies that no manager should ignore . Research in Organizational Behavior . 2009;29:39-69. doi:10.1016/j.riob.2009.10.001

Greenwald AG, Mcghee DE, Schwartz JL. Measuring individual differences in implicit cognition: The implicit association test . J Pers Soc Psychol. 1998;74(6):1464-1480. doi:10.1037/0022-3514.74.6.1464

Sabin J, Nosek BA, Greenwald A, Rivara FP. Physicians' implicit and explicit attitudes about race by MD race, ethnicity, and gender . J Health Care Poor Underserved. 2009;20(3):896-913. doi:10.1353/hpu.0.0185

Capers Q, Clinchot D, McDougle L, Greenwald AG. Implicit racial bias in medical school admissions . Acad Med . 2017;92(3):365-369. doi:10.1097/ACM.0000000000001388

Kiefer AK, Sekaquaptewa D. Implicit stereotypes and women's math performance: How implicit gender-math stereotypes influence women's susceptibility to stereotype threat .  Journal of Experimental Social Psychology. 2007;43(5):825-832. doi:10.1016/j.jesp.2006.08.004

Steffens MC, Jelenec P, Noack P. On the leaky math pipeline: Comparing implicit math-gender stereotypes and math withdrawal in female and male children and adolescents .  Journal of Educational Psychology. 2010;102(4):947-963. doi:10.1037/a0019920

Edward Zigler Center in Child Development & Social Policy, Yale School of Medicine. Implicit Bias in Preschool: A Research Study Brief .

Pager D, Western B, Bonikowski B. Discrimination in a low-wage labor market: A field experiment . Am Sociol Rev. 2009;74(5):777-799. doi:10.1177/000312240907400505

Malinen S, Johnston L. Workplace ageism: Discovering hidden bias . Exp Aging Res. 2013;39(4):445-465. doi:10.1080/0361073X.2013.808111

Cooper LA, Roter DL, Carson KA, et al. The associations of clinicians' implicit attitudes about race with medical visit communication and patient ratings of interpersonal care . Am J Public Health . 2012;102(5):979-87. doi:10.2105/AJPH.2011.300558

Leiber MJ, Fox KC. Race and the impact of detention on juvenile justice decision making .  Crime & Delinquency. 2005;51(4):470-497. doi:10.1177/0011128705275976

Van Ryn M, Hardeman R, Phelan SM, et al. Medical school experiences associated with change in implicit racial bias among 3547 students: A medical student CHANGES study report . J Gen Intern Med. 2015;30(12):1748-1756. doi:10.1007/s11606-015-3447-7

By Kendra Cherry, MSEd Kendra Cherry, MS, is a psychosocial rehabilitation specialist, psychology educator, and author of the "Everything Psychology Book."

Implicit Bias

Our biases and assumptions about others can be so automatic that they result in unintended thoughts that contradict our own beliefs. Even given our best intentions, we all hold some form of bias due to socialization and cultural stereotypes. Our implicit biases are essentially bad habits that stem from cultural learnings—they are a byproduct of our socialization and not a moral failing. If we are not aware of our biases, those habits can become activated and applied by default even when they may be undesirable and counteract our intentions. 

The good news is that, like all bad habits, it is possible to break this bad habit of implicit bias, though it will take time and conscious attention. What differentiates those with lower prejudice is their unwillingness to apply stereotypes to a whole group. During the change process, an individual must “not only inhibit automatically activated information but also intentionally replace such activation with nonprejudiced ideas and responses” (Devine, 1998).

It can be difficult to correct our implicit biases because our assumptions often go unnoticed in everyday life. We don’t often receive feedback that confirms or dispels the assumptions we make about others. Our biases continue to live in our minds unless we unearth and intentionally confront them, asking how we know our assumptions are true. 

Why is it important to be aware of implicit bias?

Regardless of how well-intentioned we are as instructors, implicit biases result from automatic thoughts. These can end up negatively impacting students and depriving them of opportunities and learning experiences.

In a study conducted by Moss-Racusin et al. in 2012, faculty members received the same resume and application materials in consideration for a laboratory manager position, with a random assignment of a male or female student name. The faculty members more frequently judged the female student to be less competent and less hireable and offered her a smaller starting salary and less career mentoring than the male student. This bias was independent of the faculty member’s gender, scientific discipline, age, and tenure status, which suggested that the implicit bias towards the female student was “likely unintentional, generated from widespread cultural stereotypes rather than a conscious intention to harm women.”

Interestingly, the faculty members actually reported liking the female student more than the male student. However, this did not translate into similarly positive perceptions of her competence. Faculty members of both genders seemed to be affected by cultural stereotypes about women’s lack of competence in science, despite not intending to dislike the female candidate. This shows the potential negative impacts of implicit bias. Despite good intentions, the continuation of such biases towards any group can have detrimental effects. 

In a similar 2019 study by Eaton et al., they experimentally manipulated the gender and racial identities of CVs for postdoctoral researcher applications that STEM professors in biology and physics would then review. In line with the Moss-Racusin 2012 study, they found a gender bias, in which the physics professors favored the male students, and a racial bias, in which the physics professors perceived Asian and White students as more competent than Black and Latinx candidates. The biology faculty did not exhibit a gender bias, which Eaton et al. theorized might be because biology is a more gender-balanced field than physics. Biology faculty exhibited a racial bias of Asian students being seen as more competent than Black students. This study also found compounded racial and gender biases, where Black and Latina female candidates, as well as Latino male candidates, were rated as less hireable than other candidates. 

Eaton et al. noted less bias in evaluating applicants with exceptionally strong records or clear differences in quality. Implicit bias is more likely to play a role in deciding between moderately and equally qualified candidates. The results of these studies exemplify the impact that implicit biases can have if left unchecked, as the faculty members in both studies did not consciously intend to be biased against these groups. 

How to mitigate bias in the classroom

We can hypothesize how issues similar to those above can arise in the classroom when evaluating student performance on more subjective tasks (e.g., awarding points for class discussion, open-ended writing assignments or projects) or when assigning student grades at the end of the term when a student is close to a letter grade threshold (A/B, B/C, etc.). Additional areas where implicit bias can show up in the classroom include group work and resource allocation, such as opportunities (e.g., undergraduate research) and your time. To help mitigate the impact of implicit bias in the classroom:

• Pay attention to who you mentor and who participates in class. This can show up in class discussions, where our biases can lead us to (unintentionally) respond differently to student comments or call on certain students more than others. Another place our biases can impact the classroom is with participation. Our memory and biases may provide us with false accounts about which students participated the most or least unless there is an objective way to measure who is participating.
• Set criteria in advance. Create rubrics to help reduce bias during grading and share the rubrics with students when the assignments are given. Because you can make grading decisions based on those predetermined criteria, grading will likely be more objective. It provides a clear way for you and the students to be on the same page about what they did well and what they need to improve on an assignment.
• Structure time for making important decisions. It is difficult to be vigilant about bias when you are stressed or tired. Ensure you are well rested before grading exams or giving feedback to students. We also recommend taking a break in between grading multiple assignments. This will help you resist the temptation to make quick decisions, which is especially important for making more objective decisions that will affect others.

Ineffective strategies for reducing bias

• Stereotype suppression . Stereotype suppression involves trying to suppress a stereotype whenever it comes to mind. This strategy is not as effective as it seems because the more you suppress a thought, the more you will think about it. As a result, you may actually be more likely to view others through stereotypes because you are constantly trying to suppress such thoughts.
• Colorblindness . This strategy is the idea of ignoring aspects of another person such as race or gender. Though the intention — to treat someone “normally” — may be good, this is not very effective. Besides not being physically possible to do so, many people also derive pride from such aspects of their background. Additionally, the more you think about treating someone “normally”, the more you actually act differently towards them. This is similar to the idea behind stereotype suppression, where the more you think about trying to act a certain way, the less effective it turns out to be.

Research-based strategies to reduce bias 

Breaking your bias habits is something that has to be practiced over time. Effective, research-based bias reduction strategies include:

• Stereotype replacement . While refraining from judgment, be attentive to patterns manifesting in your thinking. When you encounter an assumption, pause and ask yourself: “How do I know that about the person? Is it from a stereotype I have internalized, or do I have evidence from something actually happening?” By consciously surfacing and questioning your assumptions about others, you are intentionally replacing stereotypes with the individuating information unique to each person.
• Perspective-taking . Consider situational explanations. We tend to assume that an individual’s personal qualities or ability cause their behavior and be less attentive to the aspects of a situation that may have actually caused the person’s behavior. For instance, if a student does poorly on a test, it may not necessarily be because they are not smart or capable. There are many reasons why they may have performed poorly, whether it was lack of sleep, illness, personal problems, or too many exams within a week (to name a few). If you catch yourself attributing situational results to an individual’s internal characteristics, check your assumptions. It is important to think about how you actually know this and consider the possibility of a situational explanation.
• Commit to criteria . Before evaluating applicants or grading assignments, as mentioned in the previous section, it is helpful to have the same predetermined criteria and credentials against which to evaluate students or applicants. Research has found that bias is substantially less prominent when evaluators commit to criteria in advance of doing the evaluation. This allows evaluators to hold each other accountable and creates less room for in-the-moment decisions, creating opportunities for implicit bias and automatic thoughts we don’t intend.  
• Modify the environment. Evaluate what messages are in the environment about who belongs or succeeds, and seek to increase the representation of underrepresented groups. Increasing opportunities for genuine interaction with members of other groups, whether in the classroom or outside of it, can broaden perspectives and recognize individuating information about people.
• Speak up against bias. “Authority” figures can hold a lot of power and act as effective allies. However, non-authority figures can also help speak up against bias and encourage the community to act supportively. For example, a female colleague proposes an idea at a meeting that is later attributed to a male colleague who repeats it. To gently help the meeting attendees realize the misattribution, you could affirm, “Right, as Mary proposed earlier, I think that’s a great idea” or otherwise point out the similarities between the ideas. Another key consideration in speaking up against bias is tone of voice, conveying your intent to understand or clarify rather than ridicule or accuse, which could prompt defensiveness. Additionally, using concrete instances rather than abstract accusations will allow others to be more amenable to the explanations and solutions you offer.

To learn more about research-based strategies to reduce bias, check out the Breaking the Bias Habit learning bundle on Atlas (MIT Touchstone authentication required). 

Devine, P. G. (1989). Stereotypes and prejudice: Their automatic and controlled components. Journal of Personality and Social Psychology, 56 (1), 5-18. 

Devine, P. G., Forscher, P. S., Austin, A. J., & Cox, W. T. (2012). Long-term reduction in implicit race bias: A prejudice habit-breaking intervention. Journal of Experimental Social Psychology, 48 (6), 1267-1278. 

Eaton, A. A., Saunders, J. F., Jacobson, R. K., & West, K. (2019). How Gender and Race Stereotypes Impact the Advancement of Scholars in STEM: Professors’ Biased Evaluations of Physics and Biology Post-Doctoral Candidates. Sex Roles, 82 (3-4), 127-141.

Moss-Racusin, C. A., Dovidio, J. F., Brescoll, V. L., Graham, M. J., & Handelsman, J. (2012). Science faculty’s subtle gender biases favor male students. Proceedings of the National Academy of Sciences, 109 (41), 16474-16479. 

Shaw, Y. & Natisse, K. M. (Host). (2017, June 15). The Culture Inside [Audio podcast episode]. In Invisibilia . NPR. https://www.npr.org/programs/invisibilia/532950995/the-culture-inside.

MIT Teaching and Learning Lab. (2019, Oct 9). Drs. Devine & Cox: Empowering People to Break the Bias Habit [Video]. Available to the MIT community in the Atlas Learning Center: atlas.mit.edu.

Ohio State nav bar

The Ohio State University

• BuckeyeLink
• Find People
• Search Ohio State

Implicit Bias Module Series

Welcome to the Kirwan Institute for the Study of Race & Ethnicity’s Implicit Bias Module Series. At the Kirwan Institute at The Ohio State University, we are committed to the creation of a just and inclusive society, where all people and communities have the opportunity to succeed. Our commitment to this mission is why we work so hard to understand and overcome barriers that prevent access to opportunity in our society, such as implicit bias and racial disparities in our education system.

This course will introduce you to insights about how our minds operate and help you understand the origins of implicit associations. You will also uncover some of your own biases and learn strategies for addressing them. Each module is divided into a short series of lessons, many taking less than 10 minutes to complete. That way, even if you’re pressed for time, you can complete the lessons and modules at your convenience.

We are excited that you are starting this process to explore implicit bias and what its operation means for your decisions and actions. Thank you for joining us!

The Kirwan Institute for the Study of Race and Ethnicity is an interdisciplinary engaged research institute at  The Ohio State University  established in May 2003. As a racial equity research institute, our goal is to connect individuals and communities with opportunities needed for thriving by educating the public, building the capacity of allied social justice organizations, and investing in efforts that support racial equity and inclusion. Here at the Kirwan Institute, we do this through research, engagement, and communication.

Implicit Bias Module Series video transcript

Why focus on implicit rather than explicit bias?

Although our modules focus primarily on implicit bias, Kirwan acknowledges that inclusion and equity efforts must also address explicit bias and discrimination in order to create real change. Our explicit and implicit attitudes are related constructs, and many times peoples’ implicit and explicit attitudes are in alignment.

However, even though concepts are related, they are distinct. Someone can act in a biased manner based on their implicit associations, even if they do not indicate an explicit preference for certain individuals or groups. Learning about implicit bias provides a lens to help examine the causes of racial, gender, or other social disparities, even in the absence of explicit intent to discriminate or cause harm.

How long will it take me to complete these modules?

While everyone works at a different pace, these modules should take participants roughly 45 minutes to 1.5 hours to complete, depending on the time spent on activities and supplemental exercises.

Are these modules supported by research?

Yes. The methods and research shared in these modules is supported by our annual flagship publication, the  State of the Science Implicit Bias Review . Each year, Kirwan researchers compile studies on the subject of implicit bias into an interdisciplinary literature review in a format that is accessible and easy to understand from a wide range of academic and professional backgrounds.

Does implicit bias reflect my beliefs about equity or inclusion?

We will get more into this during Module 1, but our implicit preferences do not necessarily align with our explicit beliefs. For example, one can believe in equality of all people and still hold a pro-self-identity bias. Importantly, some people possess implicit attitudes that do not align with their own held identities.

If I didn’t intend to be biased in the first place, how will learning about implicit bias help?

Becoming aware of what biases you possess and the decisions that are most likely to be influenced by our unconscious processing can help you build interventions and strategies to prevent the expression of bias and unwanted outcomes. Our training also includes information about empirically based interventions to both reduce the expression of bias and alter the associations we possess.

What is the connection between implicit bias and how people act?

Implicit bias has been shown to impact decision-making across a wide array of sectors, including employment, medicine, and education. However, there are limitations to the extent to which unconscious biases can predict individual behavior. People with an implicit preference for one identity may not act on this bias or make biased decisions, much of this depends on the circumstance.

Does this training apply to me if I don’t have implicit bias?

We will touch on this during the modules, but because biases can be activated across a wide-range of identities, we all hold some implicit preferences. However, even people without an implicit preference can still act in ways that produce discriminatory behavior, such as not speaking up when they see bias in their environment. This need to translate knowledge into action is why Kirwan also emphasizes the importance of being an Active Bystander.

Will these modules eliminate bias?

These modules are not designed to eliminate bias. Rather, we hope that an awareness of implicit bias and how it operates will help participants engage in more equitable decision-making practices and behaviors. These strategies to reduce the expression of bias are only one piece of the puzzle and should be complemented by policies and strategies to address institutional and explicit discrimination.

I’m not a K–12 educator, are these modules for me?

This module series includes examples and activities that are uniquely tailored to the experience of those who work in a K–12 education setting or closely-related field. However, much of the content in Modules 1, 3, and 4 are generalizable enough to apply to most audiences.

• Printer-friendly version
• View all sections on a single page

Coming summer 2024: We're developing a new certificate course in partnership with Harvard Online.

Implicit Bias:

Learn the science. make better decisions..

For quick insights, explore our whole library of videos, podcasts, articles, and interactive demos.

For more structure, try these sequences on core concepts in the science of bias.

Browse modules

To learn more about how to use this site, consult the User Guide .

• Interactive
• Action Sheets
• Advertising
• Recruiting & Hiring
• Team Dynamics
• Client Relationships
• Recommended

The Implicit Association Test (IAT)

Can you guess the rule, gender in negotiation, frequently asked questions about the iat, bias in media: a case study, debias the classroom, how to conduct a structured interview, when does life “spark” in a face, bias in healthcare, you predict: where is bias in america headed, how you speak may affect your access to healthcare, what sentence would you give: watch for the anchoring bias, foreigners in their own country: asians in america, changes in bias across america: an explorable map, 4 ways to manage stress, make stress work for you: cognitive reappraisal, should you trust your vision, illusions at work, “the dna is a match”: confirmation bias, the attractiveness halo effect, how race-based facial features can bias inmate sentencing, how well can you read a face, the availability bias, 6 questions that will make you rethink what you know, how good is your memory, the pygmalion effect, race bias in hiring: when both applicant and employer lose, traditional “diversity training” is out. now what, can women be biased against other women, hear me out: accent bias, the standards we choose: the police chief study, shifting standards, what are implicit associations, measuring implicit bias: the implicit association test (iat), moral credentialing, who are we helping, the endowment effect, self-fulfilling prophecies, can you solve the surgeon riddle, about face: how first impressions fool us, the universe inside your mind, blindspot: hidden biases of good people, guided learning.

Our brains evolved to navigate a much different world than the one we live in now, and what was adaptive then isn’t necessarily adaptive today. Learn about some quirks of the human mind and how to outsmart them to make better decisions.

25 - 35 min

What is implicit bias? Where does it come from? How can we measure it? Start here to build a foundation for your future learning.

40 - 55 min

Our vision does a good job of getting us around, but it can be fooled – just look at optical illusions. How do visual perceptions shape our judgments of people? What costs do we bear if we are in error? Given our dependence on visual input, it’s worth knowing.

20 - 30 min

How can the way a person sounds (voice, word choice, accent) spark bias in our minds?

How are our decisions influenced by our expectations rather than by the evidence? Can another person’s expectations of us shape our behavior? Explore the power expectations have at all stages of decision making.

30 - 40 min

Some implicit biases are changing towards neutrality, but others aren’t. Explore models predicting the future and learn why there’s reason to be hopeful.

15 - 20 min

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• HHS Author Manuscripts

Twelve Tips for Teaching Implicit Bias Recognition and Management

Cristina m. gonzalez.

Department of Medicine, Albert Einstein College of Medicine/Montefiore Medical Center, Bronx, New York, USA.

Monica L. Lypson

Professor of Medicine, Department of Medicine, The George Washington University School of Medicine and Health Sciences, Washington, D.C., USA, Adjunct Clinical Professor of Internal Medicine and Learning health Sciences at the University of Michigan Medical School, Ann Arbor, Michigan, USA

Javeed Sukhera

Departments of Psychiatry/Paediatrics and Scientist, Centre for Education Research and Innovation, Schulich School of Medicine and Dentistry, Western University Canada

Implicit biases describe mental associations that affect our actions in an unconscious manner. We can hold certain implicit biases regarding members of certain social groups. Such biases can perpetuate health disparities by widening inequity and decreasing trust in both healthcare and medical education. Despite the widespread discourse about bias in medical education, teaching and learning about the topic should be informed by empirical research and best practice. In this paper, the authors provide a series of twelve tips for teaching implicit bias recognition and management in medical education. Each tip provides a specific and practical strategy that is theoretically and empirically developed through research and evaluation. Ultimately, these twelve tips can assist educators to incorporate implicit bias instruction across the continuum of medical education to improve inequity and advance justice.

Introduction

Implicit bias refers to unconscious and unintentional mental associations that impact our understanding and actions. Such biases, when brought to the clinical encounter, can contribute to health disparities by influencing communication practices and medical decision-making ( Zestcott, Blair, and Stone 2016 ). The effects of bias are pervasive and globally widespread. Research across several cultures and jurisdictions demonstrates the negative effects of bias on healthcare and medical education ( Mellor et al. 2016 ; Gonzalez, Deno, et al. 2018 ). Implicit bias recognition and management (IBRM) training for health professionals provides an approach to addressing biases and reducing health disparities. Accreditation bodies in several countries also require instruction on implicit bias ( Australian Medical Council Limited 2012 ; Committee on Accreditation of Canadian Medical Schools 2018 ; Liaison Committee on Medical Education 2020 ).

Despite increased attention to implicit bias in medical education, approaches to instruction are varied and often limited in scope. Well-intentioned attempts to meet accreditation standards have resulted in curricula often based on opinion, rather than research, thereby lacking appropriate theoretical frameworks and grounding in available evidence ( Sukhera, Watling, and Gonzalez 2020 ). In addition, there are significant gaps in providing opportunities for learners to develop skills and in training faculty ( Gonzalez, Garba, et al. 2018 ; Brooks, Rougas, and George 2016 ; Gonzalez et al. 2020 ). As a consequence of these limitations, questions are being raised regarding both the effectiveness and sustainability of implicit bias related instruction.

Given the importance of this topic and the need for improving instruction and outcomes related to IBRM, the authors identified a need to provide practical and evidence-informed teaching tips regarding IBRM for medical educators. Existing research on IBRM instruction suggests the need to enable learners to recognize when bias may be influencing an encounter and employ skills to manage the effects of bias to mitigate its negative influence ( Sukhera, Watling, and Gonzalez 2020 ). The focus of IBRM is therefore best targeted at recognizing and managing biases within the context of interpersonal encounters in both clinical and non-clinical learning environments. This approach to IBRM recognizes that the elimination of all bias is not the goal of instruction and is distinct from approaches that seek to de-bias or entirely remove the influence of bias on an individual’s communication and decision-making. There is minimal evidence for short-term efficacy and no evidence for actual behavior change ( FitzGerald et al. 2019 ; Forscher et al. 2019 ) for such approaches. The twelve tips in this paper reflect a synthesis of existing literature on bias-related instruction while remaining steeped in learning theory and the authors’ experiences in designing, implementing and evaluating curricula on IBRM.

Tip One: Create a safe learning environment

A safe learning environment is paramount to teaching and learning about emotionally charged topics such as implicit bias. Learners and facilitators need to work together to create a learning environment where there is safety, and room to grow; a “safe but brave” learning environment ( Sotto-Santiago et al. 2020 ). Ground rules must be set to ensure confidentiality, learners must have the psychological safety to grapple with challenging concepts and minimize fear of being labeled as a racist (for example) if they struggle while articulating their thoughts ( Gonzalez et al. 2019 ). In addition, bias related instruction is a challenging topic for faculty to teach, many of whom are experiencing the same worries as the learners ( Gonzalez, Garba, et al. 2018 ). From our experience and research, we have found that explicitly layering instruction by role modeling empathy and vulnerability help to co-create brave spaces for sensitive discussions ( Sukhera, Wodzinski, et al. 2018 ). A practical suggestion to cultivate psychological safety involves using index-sized cards with “honest inquiry” written on one side and “honest reaction” written on the other. Students can hold up the card when they have an honest inquiry and ask a question to learn and grow, but that they may otherwise be nervous to ask ( Zestcott, Blair, and Stone 2016 ). The honest reaction side enables other students to react to the question, explain how it made them feel, and suggest alternate phrasing, if applicable.

Tip Two: Flatten the hierarchy in facilitation

Building on the need for a safe and brave learning environment, flattening the hierarchy in facilitation (disabling the power of rank and differential) serves multiple purposes ( Gonzalez et al. 2019 ). Students have suggested having a senior student facilitate or co-facilitate with a faculty member to ease any tension they feel when discussing implicit bias with someone in an evaluative role. Learning from a senior student may also enhance the relevance of the discussion to their level of training ( Gonzalez et al. 2019 ). Reducing the power differential between learners and teachers aligns with evidence-based approaches to instruction that recognize how learners’ lived experience and prior learning can be activated during instruction ( Merriam 2007 ). Attention to hierarchy also benefits faculty by providing opportunities to pose problems and role model being both a teacher and a lifelong learner ( Freire 1993 ). Using this approach allows teachers and learners to address the discomfort related to the topic and co-construct the ensuing critical reflection and guided discourse. Research suggests that students appreciate when faculty role model vulnerability ( Sukhera, Wodzinski, et al. 2018 ; Gonzalez et al. 2019 ) and faculty may feel more comfortable exposing such vulnerabilities in an environment where the pressure of being “all knowing” is relieved ( Gonzalez, Garba, et al. 2018 ).

Tip Three: Normalize bias while reducing self-blame

Discussions about bias often trigger a defensive and emotional reaction ( Teal et al. 2012 ). When provided feedback about their biases, individuals often experience tensions related to their self-identity as well-intentioned people working hard to take excellent care of patients ( Sukhera, Milne, et al. 2018a ). Normalizing bias while reducing self-blame can effectively address these negative emotions by building trust, enhancing comfort, and increasing engagement ( Gonzalez et al. 2019 ). We suggest emphasizing that although bias is human nature and therefore not anyone’s fault, we must engage in skill development and practice so that it does not influence our clinical practice behaviors. This call to action while avoiding blame can also enhance self-compassion and self-forgiveness ( Sukhera 2018 ; Sukhera 2019 ).

In our experience, there are several practical ways to reduce self-blame within IBRM. For example, presenting data from other industries outside of health professions may enable learners to accept the ubiquitous nature of implicit bias. In addition, video observation and debrief can be helpful for multiple reasons. Practicing observing and discussing behaviors of others eases learners into the idea of discussing their own behaviors. Videos from the lay press or examples within popular culture that involve humor offer an opportunity to recognize bias as part of human nature and bring down defenses ( Gonzalez et al. 2020 ).

Tip Four: Integrate the science behind implicit bias and evidence base for its influence on clinical care

A multi-faceted approach is more likely to engage learners by including reasons to participate in IBRM that they find personally relevant. Describing the neuroscience and cognitive psychology behind implicit bias will ground instruction in content that is similar to much of the basic science taught in medical schools; this approach lends legitimacy to IBRM instruction as well while enhancing relevance ( Merriam 2007 ). By highlighting the influence of bias through relevant clinical examples, resistant or uncertain learners may be motivated to engage in instruction by their desire to provide excellent, high quality care to all patients. Tailoring the evidence, when available, to the clinical specialty or learning context of the audience may further enhance the relevance for differentiated learners further along in their careers ( Sukhera et al. 2020 ).

Tip Five: Create activities that embrace discomfort

Discomfort is an essential ingredient in bias-related instruction. One of the models that may be useful for recognizing and reconciling discomfort comes from the literature on Transformative Learning Theory. Transformative Learning Theory seeks to create dissonance through a “disorienting dilemma” and foster critical reflection, guided discourse, and action through behavior change. ( Mezirow 1997 ; Sukhera, Watling, and Gonzalez 2020 ). Therefore, eliciting discomfort leads to questioning of previous assumptions and catalyzes a paradigm shift for the learner. To create and embrace discomfort, educators can create first-thought exercises; these exercises often demonstrate to participants their own bias in a non-confrontational matter, for example, by recognizing mental images that appear in response to hearing a series of spoken words ( Vela et al. 2008 ). Another option is to use the Implicit Association Test ( Sukhera et al. 2019 ) as an elicitation prompt. In addition, interviewing standardized patients (SP) may provide a direct demonstration of how of bias is perceived during a clinical encounter, while providing an opportunity to debrief the encounter and reconcile their discomfort as they embrace a growth mindset ( Morell, Sharp, and Crandall 2002 ).

Tip Six: Implement Critical Reflection Exercises

Another important component to IBRM fosters both critical reflexivity and critical reflection ( Sukhera, Watling, and Gonzalez 2020 ). Both concepts are distinct processes. While critical reflexivity involves recognizing one’s position in the world to better understand the limitation of one’s knowledge and appreciate the social realities of others, critical reflection involves examining assumptions, power relations, and how these assumptions and relations shape practice ( Ng, Wright, and Kuper 2019 ). Both can be encouraged through the use of well-designed prompts with guiding questions that encourage learners to question assumptions and revisit previous encounters viewing them through a new lens. For example, learners can be encouraged to pause during an emotionally charged encounter that is influenced by bias. The PAUSE model suggests learners should pay attention to the situation, acknowledge their interpretations, judgments, and biases, while understanding other possible interpretations ( Ross 2014 ). In addition, a developmental approach to critical reflection helps move learners from bias awareness towards action for behavior change ( Teal et al. 2010 ; Gonzalez et al. 2020 ).

Tip Seven: Explore the dynamic relationship between experience, implicit, explicit, and structural biases

Any individual’s implicit biases are a product of their lived experience, socialization processes, ( Banaji and Greenwald 2013 ), and reflect biased norms within their personal and professional contexts ( Hernandez et al. 2013 ; Sukhera, Milne, et al. 2018b ). Therefore, exploring the influence of this lived experience may foster acceptance of bias in oneself. Identity or narrative exercises can effectively foster reflection on how an individual’s experiences contribute to their existing biases including the power/oppression/privilege of the system in which they live ( Avant and Gillespie 2019 ; Gonzalez et al. 2020 ). Building on individual experiences also helps learners to recognize how implicit racial biases can become structurally embedded to contribute to concepts such as systemic racism ( Feagin and Bennefield 2014 ; Bailey et al. 2017 ; Payne, Vuletich, and Brown-Iannuzzi 2019 ). Since implicit biases are woven into the fabric of medical organizations and society at large, any educational interventions related to bias must emphasize that individuals alone cannot address implicit biases without addressing structural biases reflected in broader policies and practices ( Byrne and Tanesini 2015 ).

Tip Eight: Perspective-Taking Exercises

Patients experience varied levels of interpersonal and systemic discrimination in society, in addition to their previous experiences within the healthcare system ( Gonzalez, Deno, et al. 2018 ). Recognizing the impact of such lived experience is essential to recognizing and managing biases. Building perspective taking exercises into bias related instruction can serve to enhance learner awareness of how standard procedures may be interpreted differently by patients who are sensitized to experiences of bias, based on their unique lived experience ( Gonzalez et al. 2020 ). Empowering learners with the knowledge that the patient’s reaction occurs in the context of previous experience will enable learners to step back and acknowledge the perceived bias and explain the rationale behind the question or the behavior to the patient, in order to restore rapport with the patient ( Gonzalez, Deno, et al. 2018 ).

Perspective-taking exercises enable learners to practice improved interpersonal communication skills and cultivate empathy ( Riess 2017 ). Effective ways to incorporate perspective taking into IBRM include using videos of lived experience narratives, which have been deemed both relevant and authentic by learners ( Gonzalez et al. 2020 ). Patient narratives also allow individuals to learn and practice ways to restore rapport with a patient if bias is perceived due to a standard behavior ( Gonzalez, Deno, et al. 2018 ).

Tip Nine: Skill-Building Exercises

A lack of skill development and practice is a major gap in IBRM approaches in the literature. Skill development should occur in a graded fashion and encompass skills to address perceived bias from others within the learning environment, as well as addressing biased behaviors by individual learners. For example, role-play and vignettes help to identify strategies to address bias ( Gonzalez et al. 2020 ). Learners can practice verbalizing potential statements they could make to address bias during case-based discussions: A vignette is reviewed with a reflection prompt asking learners to articulate statements they could use to interrupt bias witnessed among others or address bias within themselves. Finally, role-plays that recreate experiences in the learning environment enable learners to develop skills to address perceived bias while in the unscripted role ( Gonzalez et al. 2020 ).

Newly learned skills should also be iteratively practiced and improved upon as part of IBRM. For example, we have previously described our approach of conducting the role-play, engaging learners in a structured debrief, and then providing a “do-over” opportunity so that the same learner has an opportunity to succeed ( Gonzalez et al. 2020 ). Skill-building can also be fostered through the use of SPs. Training SPs to adjust their nonverbal behaviors and/or make statements reflecting perceived bias on anticipated routine questions or behaviors give learners real-time feedback about their efficacy in restoring the rapport within the encounter. These aspects of IBRM instruction should be written into formative SP exercises to provide opportunity for further skill building and practice before any summative assessments are made. In both the role-plays and the SP exercises, learners can identify alternative phrasing and behaviors to enhance their existing communication skills ( Sukhera, Watling, and Gonzalez 2020 ).

Tip Ten: Reinforce IBRM as part of life-long learning

Any attempts at bias-related instruction can be difficult to sustain due to biases perceived within the hidden curriculum and its powerful influence on professional identity formation ( Hafferty and Franks 1994 ; van Ryn et al. 2015 ; Hernandez 2018 ). Therefore, singular teaching sessions or workshops are unlikely to produce sustainable change ( Teal et al. 2012 ; Gonzalez, Garba, et al. 2018 ). We suggest that IBRM must be re-framed as an epistemology of practice that is essential to the professional identity of medical learners to be effective ( Sukhera, Watling, and Gonzalez 2020 ). Medical schools who recognize the personal and professional tensions associated with implicit bias instruction and cultivate a growth mindset around the issue help students make less biased decisions ( Williams RL 2018 ).

The suggestion that IBRM is iterative and cyclical additionally complements efforts to reduce self-blame regarding implicit bias. If all team members have some experience with IBRM, we anticipate this could facilitate discussions during routine clinical and teaching encounters. Team members should be encouraged to seek feedback regarding their own biases from others, debrief encounters where they believe bias may have had an influence, and lean into uncomfortable feedback about their biases in order to grow ( Sukhera 2018 ). Such an approach also addresses student concerns about not feeling empowered to advocate for patients when they perceive bias without risking retribution from supervisors on the team ( Gonzalez et al. 2019 ).

Faculty development programs designed to instruct faculty on debriefing about bias during inpatient rounds or outpatient precepting sessions could equip them with skills to be positive role models about IBRM. In our experience, faculty have suggested a set of stock questions to prompt discussion about potential bias which may serve as a visual aide to highlight the institutional support for these discussions. These ‘pre-reviewed’ questions also provide scaffolding and readily accessible reminders for team members. If discussions of bias are emotionally charged, these questions alleviate one of the many burdens placed on facilitators. Finally, providing dedicated time and space to discuss IBRM and integrate those discussion into existing educational programming such as didactics, journal clubs, and grand rounds, among others, could further serve to normalize and reduce self-blame regarding IBRM and make it another clinical skill to develop and practice.

Tip Eleven: Include Formative and Summative Assessments

“Assessment drives learning” is a common adage. Learners may not value IBRM instruction if it is not assessed ( Gonzalez et al. 2019 ). The educational strategies in this paper reflect existing education approaches that are tailored to IBRM, adapting assessment strategies is also necessary. Formative feedback can be given throughout, and also reinforced in the clinical phases of instruction and trainings as described in Tip Ten. We suggest that assessments should not focus on the idea that bias is can be measured, fixed or eliminated through instruction ( Sukhera et al. 2019 ). Rather, the behavioral focus of IBRM necessitates assessments focused on the knowledge, attitudes, and skills of recognizing the influence of bias in a clinical encounter and then managing this influence to mitigate negative consequences and restore rapport with patients ( Gonzalez, Deno, et al. 2018 ). Formative assessment can be seamlessly woven into all of the tips in this paper. For example, facilitated discussions, following critical reflection as well as the role-play and SP exercises. In addition to formative assessment, we believe opportunities for summative assessment include knowledge tests while assessment of skills in IBRM would naturally fit into objective structured clinical examinations.

Tip Twelve: Obtain explicit support from leadership (formal and informal)

Published frameworks, along with our experience, suggest delivering multiple sessions on IBRM in order to achieve skill development and practice ( Teal et al. 2012 ; Sukhera and Watling 2018 ; Sukhera, Watling, and Gonzalez 2020 ; Gonzalez et al. 2020 ). Integrating multiple sessions into the compulsory curriculum requires central coordination and explicit support from institutional leadership. In addition, support from the leadership may dismantle perceived barriers to instruction and enhance the ability to foster a culture where IBRM is seen as an important part of professional identity formation relevant to all physicians as described in Tip 10. We expect leadership support will allow for central coordination obviating unintentional redundancy, maximizing efficiency, and facilitating the design of a developmentally appropriate curriculum. Institutional support will demonstrate the value of IBRM to faculty, trainees, and students, potentially fostering engagement of learners who are motivated more by professional norms than personal norms ( Hernandez et al. 2013 ). Leadership support should also reflect authentic and meaningful approaches towards structural change as performative statements without action may perpetuate cynicism and mistrust. Lastly, funding and evaluation must be central to leadership support ( Gray et al. 2020 ). IBRM can only produce sustained changes if it is adequately resourced and supported as part of continuous quality improvement initiatives.

The prevalence of implicit bias curricula suggests great interest in the topic. Despite proliferation in IBRM, there are still many challenges for implementation. For example, many existing approaches to foster IBRM often fall short of skill development and practice in IBRM, perpetuating frustration for learners who are faced with knowledge of a problem, but lack skills to adequately address it and improve patient care (unpublished data under review). Our proposed tips were compiled to address this gap while elaborating on actionable tools for educators and curriculum developers. We also sought to address the emotionally charged nature of the subject matter, resistant learners, and unsupportive or unaware institutional climates ( Gonzalez, Garba, et al. 2018 ). With a better understanding of the evidence base and existing frameworks, educators can design curricula in IBRM that engage learners, garner institutional support, and achieve skill development and practice. Integration of such instruction throughout the spectrum of training and practice may empower team members and facilitate meaningful and sustainable learning outcomes over time. Increased individual successes could ease future curriculum development efforts and foster professional growth and life-long learning. In this paper we attempt to provide evidence-informed and practical tips that facilitate awareness and skill development related to IBRM. We recognize that more research is needed bias-related instruction, particularly in clinical learning environments and in relation to patient-centered outcomes. Our hope is that as skills-based instruction in IBRM expands, the required research to investigate its impact on patient outcomes can also be strengthened as we collectively work towards a more just and equitable health system for all.

Contributor Information

Cristina M. Gonzalez, Department of Medicine, Albert Einstein College of Medicine/Montefiore Medical Center, Bronx, New York, USA.

Monica L. Lypson, Professor of Medicine, Department of Medicine, The George Washington University School of Medicine and Health Sciences, Washington, D.C., USA, Adjunct Clinical Professor of Internal Medicine and Learning health Sciences at the University of Michigan Medical School, Ann Arbor, Michigan, USA.

Javeed Sukhera, Departments of Psychiatry/Paediatrics and Scientist, Centre for Education Research and Innovation, Schulich School of Medicine and Dentistry, Western University Canada.

• Australian Medical Council Limited. 2012. ‘ Standards for Assessment and Accreditation of Primary Medical Programs by the Australian Medical Council, 2012 ’, Accessed October 16, 2020. https://www.amc.org.au/wp-content/uploads/2019/10/Standards-for-Assessment-and-Accreditation-of-Primary-Medical-Programs-by-the-Australian-Medical-Council-2012.pdf
• Avant ND, and Gillespie GL. 2019. ‘ Pushing for health equity through structural competency and implicit bias education: A qualitative evaluation of a racial/ethnic health disparities elective course for pharmacy learners ’, Currents in Pharmacy Teaching and Learning . [ PubMed ] [ Google Scholar ]
• Bailey ZD, Krieger N, Agenor M, Graves J, Linos N, and Bassett MT. 2017. ‘ Structural racism and health inequities in the USA: evidence and interventions ’, Lancet , 389 : 1453–63. [ PubMed ] [ Google Scholar ]
• Banaji MR, and Greenwald AG. 2013. Blind Spot: Hidden Biases of Good People (Delacorte Press; ). [ Google Scholar ]
• Brooks KC, Rougas S, and George P. 2016. ‘ When Race Matters on the Wards: Talking About Racial Health Disparities and Racism in the Clinical Setting ’, MedEdPORTAL , 12 : 10523. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Byrne A, and Tanesini A. 2015. ‘ Instilling new habits: addressing implicit bias in healthcare professionals ’, Advances in health sciences education : theory and practice . [ PubMed ] [ Google Scholar ]
• Committee on Accreditation of Canadian Medical Schools. 2018. ‘ CACMS Standards and Elements ’, Accessed October 3, 2020. https://cacms-cafmc.ca/accreditation-documents/2019-2020/standards-and-elements .
• Feagin J, and Bennefield Z. 2014. ‘ Systemic racism and U.S. health care ’, Soc Sci Med , 103 : 7–14. [ PubMed ] [ Google Scholar ]
• FitzGerald C, Martin A, Berner D, and Hurst S. 2019. ‘ Interventions designed to reduce implicit prejudices and implicit stereotypes in real world contexts: a systematic review ’, BMC Psychol , 7 : 29. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Forscher PS, Lai CK, Axt JR, Ebersole CR, Herman M, Devine PG, and Nosek BA. 2019. ‘ A meta-analysis of procedures to change implicit measures ’, Journal of personality and social psychology , 117 : 522–59. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Freire Paulo. 1993. Pedagogy of the Oppressed: 20th Anniversary Ed . (Continuum: New York, NY: ). [ Google Scholar ]
• Gonzalez CM, Deno ML, Kintzer E, Marantz PR, Lypson ML, and McKee MD. 2018. ‘ Patient perspectives on racial and ethnic implicit bias in clinical encounters: Implications for curriculum development ’, Patient Educ Couns , 101 : 1669–75. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• ———. 2019. ‘ A Qualitative Study of New York Medical Student Views on Implicit Bias Instruction: Implications for Curriculum Development ’, Journal of general internal medicine , 34 : 692–98. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Gonzalez CM, Garba RJ, Liguori A, Marantz PR, McKee MD, and Lypson ML. 2018. ‘ How to Make or Break Implicit Bias Instruction: Implications for Curriculum Development ’, Acad Med , 93 : S74–S81. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Gonzalez CM, Walker SA, Rodriguez N, Karp E, and Marantz PR. 2020. ‘ It Can Be Done! A Skills-Based Elective in Implicit Bias Recognition and Management for Preclinical Medical Students ’, Acad Med , 95 : S150–S55. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Gray DM 2nd, Joseph JJ, Glover AR, and Olayiwola JN. 2020. ‘ How academia should respond to racism ’, Nat Rev Gastroenterol Hepatol , 17 : 589–90. [ PubMed ] [ Google Scholar ]
• Hafferty FW, and Franks R. 1994. ‘ The hidden curriculum, ethics teaching, and the structure of medical education ’, Acad Med , 69 : 861–71. [ PubMed ] [ Google Scholar ]
• Hernandez R 2018. ‘ Medical Students’ Implicit Bias and the Communication of Norms in Medical Education ’, Teach Learn Med , 30 : 112–17. [ PubMed ] [ Google Scholar ]
• Hernandez RA, Haidet P, Gill AC, and Teal CR. 2013. ‘ Fostering students’ reflection about bias in healthcare: cognitive dissonance and the role of personal and normative standards ’, Medical teacher , 35 : e1082–9. [ PubMed ] [ Google Scholar ]
• Liaison Committee on Medical Education. 2020. ‘ Functions and structure of a medical school: standards for accreditation of medical education prgrams leading to the M.D. degree ’, Liaison Committee on Medical Education, Accessed June 26, 2020. https://lcme.org/publications/ .
• Mellor D, McCabe M, Ricciardelli L, Mussap A, and Tyler M. 2016. ‘ Toward an Understanding of the Poor Health Status of Indigenous Australian Men ’, Qual Health Res , 26 : 1949–60. [ PubMed ] [ Google Scholar ]
• Merriam S, Caffarella R, Baumgartner L 2007. Learning in Adulthood: A comprehensive guide (John Wiley & Sons, Inc.: San Francisco, CA: ). [ Google Scholar ]
• Mezirow J 1997. ‘ Transformative Learning: Theory to Practice. In Transformative learning in action: insights from practice .’, New Directions for Adult and Continuing Education , 74 : 5–12. [ Google Scholar ]
• Morell VW, Sharp PC, and Crandall SJ. 2002. ‘ Creating student awareness to improve cultural competence: creating the critical incident ’, Medical teacher , 24 : 532–4. [ PubMed ] [ Google Scholar ]
• Ng SL, Wright SR, and Kuper A. 2019. ‘ The Divergence and Convergence of Critical Reflection and Critical Reflexivity: Implications for Health Professions Education ’, Acad Med , 94 : 1122–28. [ PubMed ] [ Google Scholar ]
• Payne BK, Vuletich HA, and Brown-Iannuzzi JL. 2019. ‘ Historical roots of implicit bias in slavery ’, Proceedings of the National Academy of Sciences of the United States of America , 116 : 11693–98. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Riess H 2017. ‘ The Science of Empathy ’, J Patient Exp , 4 : 74–77. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Ross Howard J. 2014. Everyday bias : identifying and navigating unconscious judgments in our daily lives . [ Google Scholar ]
• Sotto-Santiago S, Mac J, Duncan F, and Smith J. 2020. ‘ “I Didn’t Know What to Say”: Responding to Racism, Discrimination, and Microaggressions With the OWTFD Approach ’, MedEdPORTAL , 16 : 10971. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Sukhera J 2019. ‘Empathy and Implicit Bias: Can Empathy Training Improve Equity?’ in, Teaching Empathy in Healthcare (Springer, Cham; ). [ Google Scholar ]
• Sukhera JI 2018. ‘ Bias in the mirror: exploring implicit bias in health professions education . Datawyse / Universitaire Pers Maastricht . 10.26481/dis.20181129js’. [ CrossRef ] [ Google Scholar ]
• Sukhera J, Miller K, Scerbo C, Milne A, Lim R, and Watling C. 2020. ‘ Implicit Stigma Recognition and Management for Health Professionals ’, Acad Psychiatry , 44 : 59–63. [ PubMed ] [ Google Scholar ]
• Sukhera J, Milne A, Teunissen PW, Lingard L, and Watling C. 2018a. ‘ The Actual Versus Idealized Self: Exploring Responses to Feedback About Implicit Bias in Health Professionals ’, Acad Med , 93 : 623–29. [ PubMed ] [ Google Scholar ]
• ———. 2018b. ‘ Adaptive reinventing: implicit bias and the co-construction of social change ’, Advances in health sciences education : theory and practice , 23 : 587–99. [ PubMed ] [ Google Scholar ]
• Sukhera J, and Watling C. 2018. ‘ A Framework for Integrating Implicit Bias Recognition Into Health Professions Education ’, Acad Med , 93 : 35–40. [ PubMed ] [ Google Scholar ]
• Sukhera J, Watling CJ, and Gonzalez CM. 2020. ‘ Implicit Bias in Health Professions: From Recognition to Transformation ’, Acad Med , 95 : 717–23. [ PubMed ] [ Google Scholar ]
• Sukhera J, Wodzinski M, Rehman M, and Gonzalez CM. 2019. ‘ The Implicit Association Test in health professions education: A meta-narrative review ’, Perspect Med Educ , 8 : 267–75. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Sukhera J, Wodzinski M, Teunissen PW, Lingard L, and Watling C. 2018. ‘ Striving While Accepting: Exploring the Relationship Between Identity and Implicit Bias Recognition and Management ’, Acad Med , 93 : S82–S88. [ PubMed ] [ Google Scholar ]
• Teal CR, Gill AC, Green AR, and Crandall S. 2012. ‘ Helping medical learners recognise and manage unconscious bias toward certain patient groups ’, Medical Education , 46 : 80–8. [ PubMed ] [ Google Scholar ]
• Teal CR, Shada RE, Gill AC, Thompson BM, Fruge E, Villarreal GB, and Haidet P. 2010. ‘ When best intentions aren’t enough: helping medical students develop strategies for managing bias about patients ’, Journal of general internal medicine , 25 Suppl 2 : S115–8. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• van Ryn M, Hardeman R, Phelan SM, Dj Ph D, Dovidio JF, Herrin J, Burke SE, Nelson DB, Perry S, Yeazel M, and Przedworski JM. 2015. ‘ Medical School Experiences Associated with Change in Implicit Racial Bias Among 3547 Students: A Medical Student CHANGES Study Report ’, Journal of general internal medicine , 30 : 1748–56. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Vela MB, Kim KE, Tang H, and Chin MH. 2008. ‘ Innovative health care disparities curriculum for incoming medical students ’, Journal of general internal medicine , 23 : 1028–32. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Williams RL, Vasquez CE, Getrich CM, Kano M, Boursaw B, Krabbenhoft C, Sussman A. 2018. ‘ Racial/gender biases in student clinical decision-making: a mixed-method study of medical school attributes associated with lower incidence of biases ’, Journal of general internal medicine . [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Zestcott CA, Blair IV, and Stone J. 2016. ‘ Examining the Presence, Consequences, and Reduction of Implicit Bias in Health Care: A Narrative Review ’, Group Process Intergroup Relat , 19 : 528–42. [ PMC free article ] [ PubMed ] [ Google Scholar ]

NAS Launches Science and Innovation Fund for Ukraine

Review Examines COVID-19 Vaccination and Adverse Events

Science Academies Issue Statements to Inform G7 Talks

Supporting Family Caregivers in STEMM

• Load More...

• Bachelor’s Degrees
• Master’s Degrees
• Doctorate Degrees
• Certificate Programs
• Nursing Degrees
• Cybersecurity
• Human Services
• Science & Mathematics
• Communication
• Liberal Arts
• Social Sciences
• Computer Science
• Admissions Overview
• Tuition and Financial Aid
• Incoming Freshman and Graduate Students
• Transfer Students
• Military Students
• International Students
• Early Access Program
• About Maryville
• Our Faculty
• Our Approach
• Our History
• Accreditation
• Tales of the Brave
• Student Support Overview
• Online Learning Tools
• Infographics

Home / Blog

How to Identify and Overcome Your Implicit Bias

July 20, 2021 

We all have biases — unsupported assumptions we make about people or groups. Implicit bias, also commonly known as unconscious bias, refers to the various social stereotypes and judgments that people unknowingly assign to others based on a variety of factors, such as their age, socioeconomic status, weight, gender, race, or sexual orientation. And while these biases aren’t always negative, they’re shaped by a survival instinct that causes people to associate with people they perceive to be similar to them, because they’re deemed to be “safe.”

Examples of unconscious biases are present throughout our personal and professional lives. In Blink: The Power of Thinking Without Thinking, Malcolm Gladwell notes that in the general population, roughly 3.9% of adult men are 6 foot, 2 inches or taller. Yet among a random sampling of CEOs, he found that nearly a third, or roughly 33.3%, fell into this group.

According to Gladwell, this could be linked to an unconscious belief that height correlates with success. This hypothesis is further underscored by a 2020 Chinese study that found that each centimeter in height above average correlated with a 10% to 13% increase in annual earnings.

In an example from 2018, two Black men walked into a Philadelphia Starbucks to attend a business meeting. The manager asked them to leave, and they declined, saying they were waiting for their associate. The manager called the police, who then arrested the men. In interviews after the arrest, the men said they believed the manager had targeted them because of their race. Starbucks responded by holding companywide training to “address implicit bias, promote conscious inclusion, and prevent discrimination.”

Everyone holds implicit beliefs about various social groups, and these biases can have a negative impact in our social, study, and work environments. Implicit biases are harmful because they influence the way we perceive and interact with others — and can lead us to depersonalize people from different groups based on perceived characteristics. Learning to identify and overcome them is an important step toward overcoming prejudice and social and racial stereotypes.

What Is Implicit Bias?

Whereas explicit biases are those that people express openly (e.g., arguing that mothers of young children shouldn’t hold management positions), implicit biases often lie outside of our conscious awareness.

For example, if a manager assigns a tech-heavy task to a young employee instead of an older one based on the unspoken assumption that younger staff members are better with technology, implicit bias is at play. Unconscious bias can also occur in the classroom; for example, students may marginalize non-native English speakers when choosing work groups, with the unconscious assumption that they may not perform as well as native English-speaking peers.

The insidious nature of bias lies in its unconscious nature, as our implicit biases often contradict the values that we aspire to. And when people aren’t even aware that they’re doing something, it can be difficult to correct.

Types of Implicit Bias

The first step toward addressing implicit biases involves learning to recognize them. Among the various implicit biases prevalent throughout society are some such as race and ethnicity bias, age bias, gender bias, LGBTQIA+ community bias, and ability bias.

Race and Ethnicity Bias

Race and ethnicity bias occurs when people assume certain characteristics about someone based on their race or ethnicity, such as assuming that all Asian students are good at math or that all Hispanic individuals are English-language learners, and then take actions that reinforce those biases — unconsciously overlooking a Hispanic employee for a task that requires strong English communication skills, for example.

Age bias occurs when people make assumptions about others based on their age, such as when a hiring manager looking for a social media-savvy applicant rejects a resume because the graduation date tips off that the applicant is middle-aged, unconsciously assuming that the candidate wouldn’t be adept at social media management.

Gender Bias

Gender bias occurs when people assume one gender is better suited for a particular job — such as welding or babysitting — regardless of an applicant’s experience level.

LGBTQIA+ Community Bias

Assuming that lesbians can’t relate to men, and so reflexively declining to pair them with male teammates; assigning gay men to workplace tasks involving design without thinking of the reasons behind their choice; and unconsciously overlooking bisexuals for leadership positions based on an incorrect assumption that they “can’t make up their minds” are examples of LGBTQIA+ community bias. LGBTQIA+ community bias is also prevalent in the healthcare system. For example, when a nurse practitioner asks a female-presenting woman if she has a boyfriend when discussing her sexual history, implicit bias is at play.

Ability Bias

Ability bias is prevalent throughout society. Examples include hiring managers who are less likely to select a candidate with a disability because they unconsciously assume they’ll be more likely to take sick leave, and individuals who assume that all people who struggle with mental illness are prone to violent or dangerous behavior and so, without knowing they’re doing so, restrict them from certain roles.

Other Types of Bias

Implicit bias can take many other forms, such as:

• Affinity Bias: The tendency for individuals to gravitate toward people similar to themselves.
• Beauty Bias: The tendency for individuals to treat attractive people more favorably.
• Name Bias: The tendency for individuals to judge someone based on their name — and thus perceived background — which can negatively impact a company’s hiring processes.
• Weight Bias: The tendency for individuals to judge someone negatively, or assume negative things about them, if they’re overweight or underweight.

Overcoming Implicit Bias

People can use several different strategies to overcome and address implicit biases, although this is an area that no one can ever fully master. Examples include striving to identify and understand your implicit biases, proactively becoming more inclusive, and spending time with people who are different from you.

Identify and Evaluate Your Own Biases

The first step toward overcoming your implicit biases is to identify them. Reflect on your biases and be proactive in identifying the negative stereotypes you have about others. One way is to take one of Project Implicit’s Implicit Association Tests, which measure topics such as race, gender, weight, and religion.

The American Academy of Family Physicians (AAFP) discusses eight tactics that can be used to reduce implicit biases, using the acronym IMPLICIT:

• Introspection: Set aside time to understand your biases by taking a personal inventory of them. This can be done by taking tests to identify the biases you may have.
• Mindfulness: Once you understand the biases you hold, be mindful that you’re more likely to give in to them when you’re under pressure or need to make quick decisions. If you’re feeling stressed, pause for a minute, collect yourself, and take a few deep breaths.
• Perspective-Taking: If you think you may be stereotyping people or groups, imagine what it would feel like for others to stereotype you.
• Learn to Slow Down: Before jumping to conclusions about others, remind yourself of positive examples of people from their age group, class, ethnicity, or sexual orientation. This can include friends; colleagues; or public figures, such as athletes, members of the clergy, or local leaders.
• Individualization: Remind yourself that all people have individual characteristics that are separate from others within their group. Focus on the things you have in common.
• Check Your Messaging: Instead of telling yourself that you don’t see people based on their color, class, or sexual orientation, learn to use statements that embrace inclusivity. For example, Apple Inc.’s inclusion statement circles around the topic of being different together: “At Apple, we’re not all the same, and that’s our greatest strength.”
• Institutionalize Fairness: In the workplace, learn to embrace and support diversity. The AAFP suggests individuals use the Equity and Empowerment Lens, which is designed to help organizations improve planning and resource allocation to foster more equitable policies.
• Take Two: Overcoming unconscious biases takes time. Understand that this is a lifelong process and that deprogramming your biases requires constant mindfulness and work.

Be Aware and Proactive in Being More Inclusive

Once you’ve identified your personal biases, you can take proactive steps to be more inclusive. For one, check your media bias: Do you find that the blogs you follow, the shows you stream, or the social media accounts you “like” are all produced by people quite similar to you? That kind of affinity is natural, but it also reinforces unconscious biases. Seek out media sources aimed at different groups. You’ll hear challenging opinions and learn how others experience the world.

In the classroom and the workplace, practice intentional inclusion. When asked to form a study or work group, pass over your friends to choose partners from different backgrounds.

Spend Time with People Who Are Different from You

Increasing your contact with different groups can help undermine your subconscious stereotypes.

Societal forces tend to keep us separate from people of different backgrounds and socioeconomic classes. Break out of your usual routine: Join a club sports team or library book group; volunteer with a nonprofit in a different neighborhood; take part in different cultural celebrations (e.g., National Puerto Rican Day, Juneteenth, or Nowruz, the Iranian New Year). This approach is backed by science: Psychological theory suggests that individuals can reduce their prejudices by interacting with people from other races, ethnicities, and backgrounds.

Be Proactive and Take Steps Forward

While implicit bias affects the workplace, school, and social situations, you can work to avoid it through awareness and conscious decision-making. Taking inventory of the biases you have and laying out strategies to overcome them can help lead to a more equitable society for all.

Recommended Reading

7 Tips for Achieving Self-Empowerment

Time Management for Online Students

Job Search Tips: Hard Skills vs. Soft Skills

American Association of Family Physicians, “Eight Tactics to Identify and Reduce Your Implicit Biases”

BuiltIn, “16 Unconscious Bias Examples and How to Avoid Them in the Workplace ”

Catalyst, “11 Harmful Types of Unconscious Bias and How to Interrupt Them”

CIPHR, “What Is Unconscious Bias in the Workplace, and How Can We Tackle It?”

CNN, “Black Men Arrested at Philadelphia Starbucks Reach Agreements”

EnvatoTuts+, “What Is Unconscious Bias? + Top Strategies to Help Avoid It”

Everfi, “How Inclusion in the Workplace Helps Fight Implicit Bias”

Forbes , “Unconscious Bias: How It Affects Us More Than We Know”

Multnomah County, Equity and Empowerment Lens

ONGIG, “10 Examples of Awesome Inclusion Statements”

PLOS ONE, “What Is Creating the Height Premium? New Evidence from a Mendelian Randomization Analysis in China”

Profiles in Diversity Journal, “Overcoming Unconscious Bias Within Organizations”

Project Implicit, Take a Test External link:

Simply Psychology, “Implicit or Unconscious Bias”

ThoughtCo. “What Is the Contact Hypothesis in Psychology?”

VeryWellMind, “How Does Implicit Bias Influence Behavior?”

Vox, “Companies Like Starbucks Love Anti-Bias Training. But It Doesn’t Work and May Backfire”

Bring us your ambition and we’ll guide you along a personalized path to a quality education that’s designed to change your life.

Take Your Next Brave Step

Receive information about the benefits of our programs, the courses you'll take, and what you need to apply.

Implicit Bias Awareness and Mitigation Training (AM)

Group-based bias involves varying degrees of stereotyping (exaggerated beliefs about others), prejudice (dis/liking others), and/or discrimination (unequal treatment of others) that occur above or below conscious awareness. Scholars have labeled the subconscious form of group-based bias implicit or unconscious bias. Implicit bias is primarily seen as an attitude or stereotype held about social groups below conscious awareness. Implicit bias has become very important to understand given systemic inequities and disparate outcomes in our society for different racial groups. Implicit bias is everywhere and affects everyone. We all have implicit bias. The impact of our implicit bias on others, however, significantly depends on our social and professional roles in society. Bias held by social workers, educators, police officers, physicians, prosecutors and criminal court judges can significantly affect the life outcomes of large segments of society.

Research has revealed that many Americans show a positive implicit bias toward White Americans vs. African Americans, young vs. old and fit vs. obese. Showing a preference for or against any particular group does not mean that a person is prejudiced or will discriminate, but it does suggest that s/he has been repeatedly exposed to certain associations between specific groups and specific traits/characteristics and have stored them in memory. These associations are often very strong and difficult to undue without deliberate effort or ongoing training. However, it is possible to implement practices or policies that reduce the likelihood that implicitly biased beliefs will lead to biased behaviors.

IMPLICIT BIAS AWARENESS (IB 101)

IB 101 places emphasis on the individual. It addresses the following questions:

• What is implicit bias?
• What does implicit bias look like in the real world?
• What causes implicit bias?  

CEUs 0.6 Section Number 234CPS006 Instruction Method Online class

You are using an outdated browser. Please upgrade your browser to improve your experience.

ProjectImplicit

• [email protected]
• Take a Test
• Technical Issues
• About the IAT

Ethical Considerations

• Frequently Asked Questions

The purpose of this website is to educate about implicit bias. We give feedback on Implicit Association Test (IAT) performance to raise awareness and encourage self-reflection.

Participation is limited to adults, aged 18 or older.

In order to participate in a Project Implicit study, you must be 18 years of age or older. Our team is currently developing a demonstration site that would be open to participants under the age of 18 years old. If you'd like to be notified of when those materials are available, we encourage you to sign up for the Project Implicit newsletter here .

Participation in scientific research should not be mandatory

The tests at this site are scientific research, and it is not ethical to require that people participate in scientific research. We appreciate that instructors, employers, and others might want to use the site for school or work assignments, and we encourage you to do so. However, it is not ethical to pressure or force people to participate against their will. Instructors who grant course credit for participation should offer an alternative assignment. For example, students who don’t want to participate may read and summarize a scientific or popular press article about implicit bias. And you should also never ask anyone to share their personal feedback with you.

The IAT should not be used for diagnostic purposes

Research shows the IAT is an effective educational tool for raising awareness about implicit bias, but the IAT cannot and should not be used for diagnostic or selection purposes (e.g., hiring or qualification decisions). For example, using the IAT to choose jurors is not justifiable, but it is appropriate to use the IAT to teach jurors about implicit bias.

The IAT does not meet the standards of measurement reliability for diagnostic use. Just as blood pressure readings might change from one doctor's visit to another depending on how stressed and tired you are, and even how much coffee you may have had, IAT results can change from one time to another depending on where you currently are, your recent thoughts or experiences, and deliberate strategies you might use to influence test results.

Using the IAT for research

People might not like their IAT results. Being confronted with IAT results might cause defensiveness or negative emotions. If you are considering using the IAT in your research, your research plan should take this possibility into account.

It is also important to understand that changes in IAT performance over time might reflect increased experience with the test rather than a genuine change in implicit bias. Pre-post research designs (where you administer an IAT, administer some intervention, and then give another IAT) are discouraged unless you have a “control group” that does not complete the intervention.

The IAT has potential for use beyond the scientific laboratory. However, in the absence of relevant scientific expertise, there is potential for misuse. We do not advise its use outside of the safeguards of a research institution.

• Teaching Resources: Commonly Asked Questions about Teaching Practices and Educational Technology

Addressing Bias in Assessment

• Post author: duotl
• Post published: September 22, 2022
• Post category: Assessment / Inclusive Excellence / Teaching Strategies

Written by Dr. Becca Ciancanelli, Director of Inclusive Teaching Practices, and Dr. Stephen Riley, Director of Assessment

Implicit bias can be sneaky. This term refers to the unconscious bias that forms the assumptions that we make about students based on social identity (Imazeki, 2021). These assumptions can be invisible to us, especially in course-level assessment.

For example, when grading papers, professors might be influenced by the student’s perspective on a topic and therefore have less focus on the quality of the argument (Steinke & Fitch, 2017). Confirmation bias might lead a professor to grade a student lower that they perceive as disengaged, or who has not performed well on prior assessments (Malouff et al, 2013). Courses that have a few large stakes assessments, like midterm exams, might block certain students from successfully showing mastery of the content. Taking proactive steps to bring awareness to the assessment structure and grading process will reduce the likelihood of bias in assessment.

Flexible assessment

Bias in assessment can show up in many ways.  One of the prominent aspects of our assessments that can be affected by bias is the way in which we structure our courses toward summative assessments.

Summative assessments are those assignments which we design to see if students have mastered the main learning outcomes of our courses.  In many cases, the assignments take the form of a research paper or standardized test. While both approaches to assessment have merits, there are some implicit biases in them as well. 

For example, the research paper can disproportionately affect those who are English Language Learners (ELL) or those with ADHD who find the research to writing connection especially difficult. Standardized tests often favor students whose recall memory skills and anxiety coping are high while disproportionately affecting those who may have processing disorders and may not be able to easily move information from short term to long term memory.

In such cases, it is important to recognize that Student Learning Outcomes (SLO) are especially helpful in making explicit what information and skills our courses are intending to assess. When we can articulate those outcomes clearly, we may be able to mitigate against some biases in assessment by moving toward flexible assessments as proposed by Universal Design Language (UDL). With flexible assessment, students are given options for how they might demonstrate mastery of a course’s SLO in a summative assessment while still maintaining rigor and integrity. For example, if our summative assessment has been final research paper, we could offer students the opportunity to propose a final project based upon a single rubric that included the important information and skills that aligned with our SLO for the course we were assessing.

There are many excellent examples of how this has been done in different disciplines and the results include increased student engagement and creativity while removing barriers and biases that hinder student learning (Edwards, 2020).

Grading Strategies

Another way bias may present is through our grading of student work.  Researchers have published work about a number of grading biases.  For example, J. M. Malouff has written extensively regarding biases such as the ‘Halo’ effect, where a professor gives grades to students based on their overall impression of each student rather than the actual submitted work (Malouff et al, 2013). Others have pointed out effects such as the ‘Anchor’ effect, where all students are graded based on the work of one superior or creative student, or the ‘Logical Fallacy’ effect, where students are assessed based on tangential criteria to the learning outcomes.  It is easy to see how these and other effects like them could disproportionately affect marginalized and vulnerable students in our courses.

There are ways to mitigate these biases such as turning on anonymous grading in Canvas and having students submit assignments with no identifying information. Another approach for looking at biases in our grading includes using clear and accessible rubric, which you can set up in Canvas For example, using the ACC&U’s VALUE Rubric for Intercultural Knowledge and Competence for grading would ensure students understand the clear guidelines of what is being assessed in an assignment with this outcome. Such a rubric would also give clear actionable steps to meet distinct levels of success based on a nationally normed set of criteria. Finally, working with others to see how they assess our grading, known as interrater reliability, could ensure we are grading assignments with both rigor and equality. This could be achieved by meeting with other members of your team to review scoring practices against the same rubric on an assignment.

Inclusive Approaches

Investigating your language regarding assessment, on your syllabus and in class, will help to interrupt bias. Many students struggle to understand expectations regarding assessment. Being transparent about how your assessment design aligns with your learning outcomes, as well as signaling effective study strategies for mastering your disciplinary content, can reduce barriers to learning for many students. Consider a culturally responsive approach to assessment, such as contract-based grading , which creates a collaborative classroom environment, where students can benefit from working together, and ultimately prepares students for their future workplaces (Jack & Sathy, 2021; Stephens et al, 2012).  Also, encouraging students as they prepare for an assessment can reduce stereotype threat (Learning for Justice, nd).

Please explore the “ Inclusive Assessment ” Module to review key definitions, DEI approaches to assessment and a suggested syllabus statement.

Bringing self-awareness to implicit bias about student performance, which we all have, is a strong step towards creating inclusive environments where all students can thrive.

• Stachowiak, B. (Producer and Host). (2021, October 14). Implicit Bias in Our Teaching with Jennifer Imazeki [Audio podcast]. Teaching in Higher Ed. https://teachinginhighered.com/podcast/implicit-bias-in-our-teaching/
• Steinke, P. & Fitch, P. (2017). Minimizing Bias When Assessing Student Work. Research and Practice in Assessment, 12 (Winter), 87-95.
• Malouff, J.M., Emmerton, A.J., Schutte, N.S. (2013). The Risk of Halo Bias as a Reason to Keep Students Anonymous During Grading. Teaching of Psychology , 40(3): 233-237.
• Jack, J. & Sathy, V. (2021, September 24) It is Time to Cancel the Word ‘Rigor’. The Chronicle of Higher Education . https://www.chronicle.com/article/its-time-to-cancel-the-word-rigor
• Stephens, N.M et al. (2012). Unseen Disadvantage: How American Universities’ Focus on Independence Undermines the Academic Performance of First-Generation College Students. Journal of Personality and Social Psychology , 2012, Vol. 102, No. 6, 1178 –1197.
• Learning for Justice. (n.d.). How Stereotypes Undermine Test Scores. Retrieved September 12 th , 2022. https://www.learningforjustice.org/professional-development/how-stereotypes-undermine-test-scores

2150 E. Evans Ave University of Denver Anderson Academic Commons Room 350 Denver, CO 80208

Copyright ©2019 University of Denver Office of Teaching and Learning | All rights reserved | The University of Denver is an equal opportunity affirmative action institution.

Implicit bias in nursing: Identifying and confronting the issue

As the United States—and hopefully, the nursing profession—becomes increasingly diverse, we must pay more attention to issues of diversity, equity, and inclusion (DEI) in both the classroom and the workplace. DEI terms are often used synonymously but have distinct definitions:

• Diversity is the identification of differences in demographics such as age, gender, race, education level, class, and so on. It is also linked to differences in thoughts, beliefs, and attitudes. 
• Equity is treatment which is fair and impartial. This is unlike equality, in which treatment is standardized, regardless of individual needs. 
• Finally, inclusion is the thoughtful, intentional welcoming of all participants into the work or classroom environment. Inclusive environments are those in which key actors purposefully make everyone feel a part of the team.

DEI may serve as a starting point for acknowledging others’ differences; however, acknowledgment of differences does not always lead to the celebration or embracing of said differences. We must acknowledge our thoughts, attitudes, and beliefs related to our “other” colleagues that lead to personal biases that can serve as barriers to the goal of a more diverse profession—and ultimately, a more diverse healthcare system.

Defining and confronting implicit bias A bias is a generalized preconceived notion about an individual or group. Racism and discrimination are the outcomes of overt biases. An implicit bias, on the other hand, is an unconscious or unintended bias. Micro- and macroaggressions are subtle, covert—implicit—biases. Here are several examples of implicit biases I’ve actually witnessed and the potential unintentional outcomes of these biases.

Identifying bias through the IAT We can subjectively identify our personal biases through introspection and reflection. While this has value, it may be more effective to objectively identify our biases. One way to do this is by utilizing tools like the Harvard Implicit Association Tests (IATs) . The IATs cover a range of topics including race, gender, and age bias. By taking the IAT, nurses can become aware of their own perceptions and can work to increase their cultural awareness and cultural sensitivity when working with diverse populations.

During my fall 2018 introduction to nursing class, I decided to tackle the issue of diversity in nursing. I challenged my freshmen students to:

• Acknowledge the need for increased diversity in nursing
• Identify their personal biases related to age, gender, and race by taking several IATs
• Develop solutions to help increase diversity in nursing education and the nursing profession

Some of the reflections they submitted were very introspective. I was encouraged by their commitment to the assignments and class discussions. One reflection, though, really tugged at my heartstrings. As I was grading Grace’s paper, I held back tears and yelled out, “She gets it!” Here’s a portion of her reflection. I hope it gives everyone something to think about.

“I am a White woman growing up in a middle-class suburban family, attending a private university. I have never experienced discrimination because of my family’s economic status, the color of my skin, and although I can sympathize with my fellow women who fell victim to sexism, I am fortunate enough to be able to say I can’t specifically relate. My innocence and sense of normalcy have continued through life, existing uninterrupted. And although I would never describe myself as racist, sexist, or ageist, I am subconsciously and unavoidably a product of my sheltered environment.

I fully consider myself to be a feminist. Looking back on women’s history, I swell with pride at the leaps and bounds we have made towards equality. Although at times I fear for the progression of feminist America, I am excited to be a part of the long-fought battle not only for gender issues, but also those of race and equality. Yet, my IAT results suggested I have a moderate association between males in high career positions and women in family and home positions, as well as a moderate preference for White people over Black people and young people rather than older people. I was baffled by this outcome. How am I a victim to the very thing I preach against?

After some thinking, I realized it was almost impossible for me to receive results any different. The school district in which I grew up was predominantly White, I hung around people who shared my age and background, and my family reinforced stereotypical gender norms. David Rock, founder and CEO of NeuroLeadership Group, wrote in a Psychology Today article that, “Your brain craves certainty and avoids uncertainty like its pain...Like an addiction to anything, when the craving for certainty is met, there is a sensation of reward”. Although I did not like my IAT results, I cannot argue with the science supporting them.

I am lucky for the ability to recognize the privilege that I have. It is time for people to stop with the pleas of ignorance in the nationwide discussion of inequality. Starting nursing school has given me an entirely new perspective. I hope through the exposure of higher education and people from diverse backgrounds I can learn the stories of those who had a life much different than my own. I plan on taking the information that the IAT has given me and moving forward with it. Continuing with knowledge, growth, and curiosity of those around me, I am optimistic that my natural inclinations will one day mirror the well-rounded, tolerant person I aspire to be.

We can—and must—do more The nursing profession has been a staple in the support of women’s education and work rights. While we acknowledge and celebrate the strides we have made for women, we also need to acknowledge the lack of strides made for young people, men, and people of color. We need to look at our individual beliefs about our minority colleagues—be they age, gender, or race-related. We must be sure we are not contributing to any barriers that prevent increased diversity in our profession. And, we must hold firm to our commitment, through the reciting of the Nightingale Pledge , to “do all in [our] power to maintain and elevate the standard of [our] profession, and... dedicate [ourselves] to devoted service to human welfare.”

Tiffany Montgomery, PhD, RNC-OB, C-EFM, is a Postdoctoral Research Fellow at Drexel University in Philadelphia, Pennsylvania, USA and a member of Sigma’s Nu Eta Chapter at Drexel University. 

Grace Murphy is a junior nursing student at Widener University in Chester, Pennsylvania, USA.

• North America

This site uses cookies to keep track of your information. Learn more here . Accept and close .

GALILEO Open Learning Materials

• < Previous

Home > Psychology, Sociology, Anthropology, and Social Work > Ancillary Materials > 18

Psychology, Sociology, Anthropology, and Social Work Ancillary Materials

Major Graded Assignment on Implicit Bias (PSYC 1101)

J. Sean Callahan , Georgia Highlands College Follow Bentley Gibson , Georgia Highlands College Follow

Material Type

Publication date.

Summer 2020

Course Title

Introduction to General Psychology

Course Number

This major assignment for Introduction to General Psychology (PSYC 1101) was created through a Round 14 ALG Mini-Grant . The assignment guides students in engaging with ideas about diversity and inclusion in ways that require higher-order thinking skills. Students take the Implicit Association Test and the Big Five Test at the beginning and end of the assignment.

Creative Commons License

Recommended Citation

Callahan, J. Sean and Gibson, Bentley, "Major Graded Assignment on Implicit Bias (PSYC 1101)" (2020). Psychology, Sociology, Anthropology, and Social Work Ancillary Materials . 18. https://oer.galileo.usg.edu/psychology-ancillary/18

Since September 15, 2020

• By Institution
• By Material Type
• By Creative Commons License
• By Digital Commons Discipline

Advanced Search

• Notify me via email or RSS
• Frequently Asked Questions

Other Resources

• Affordable Learning Georgia

Home | About | FAQ | My Account | Accessibility Statement

Privacy Copyright

An official website of the Department of Health & Human Services

• Search All AHRQ Sites
• Email Updates

1. Use quotes to search for an exact match of a phrase.

2. Put a minus sign just before words you don't want.

3. Enter any important keywords in any order to find entries where all these terms appear.

• The PSNet Collection
• All Content
• Perspectives
• Current Weekly Issue
• Past Weekly Issues
• Curated Libraries
• Clinical Areas
• Patient Safety 101
• The Fundamentals
• Training and Education
• Continuing Education
• WebM&M: Case Studies
• Training Catalog
• Submit a Case
• Improvement Resources
• Innovations
• Submit an Innovation
• About PSNet
• Editorial Team
• Technical Expert Panel

Implicit bias and patient care: mitigating bias, preventing harm.

Barber Doucet H, Wilson T, Vrablik L, et al. Implicit bias and patient care: mitigating bias, preventing harm. MedEdPORTAL. 2023;19:11343. doi:10.15766/mep_2374-8265.11343.

Addressing racism and implicit biases in healthcare is a patient safety priority . This article describes the evaluation of a simulation training designed to help emergency medicine and pediatric learners identify implicit bias and develop bias mitigation skills. Using standardized participants, trainees were presented with a case involving an 18-month-old African American child with a spiral fracture of the left lower leg (a common ‘toddler fracture’). As part of the simulation, an orthopedic resident on the care team was pushing for involving child welfare and displaying non-specific and biased concerns about the child’s family and welfare. The simulation objectives were for the learners to (1) identify that the case was not concerning for non-accidental trauma, (2) identify that the resident was displaying bias, and (3) communicate with both the resident and family to diffuse the situation and prevent any potential harm. After the simulation, the simulation instructions led a debrief to discuss and reflect on the case.

Between August 2021 and January 2022, seventy-five learners (medical students, residents, fellows, and attending physicians) participated in the simulation training. Participants largely agreed that the simulation was an effective tool for teaching bias communication skills and increasing confidence in identifying and addressing implicit biases and racism.

Implicit bias and caring for diverse populations: pediatric trainee attitudes and gaps in training. September 22, 2021

Medicaid markets and pediatric patient safety in hospitals. September 26, 2007

Universal and serial laboratory testing for SARS-CoV-2 at a long-term care skilled nursing facility for veterans — Los Angeles, California, 2020. June 10, 2020

Learning from latent safety threats identified during simulation to improve patient safety. October 11, 2023

Adopting high reliability organization principles to lead a large scale clinical transformation. April 26, 2023

Promoting health care safety through training high reliability teams. August 24, 2005

Peer support and second victim programs for anesthesia professionals involved in stressful or traumatic clinical events. February 21, 2024

A blinded, prospective study of error detection during physician chart rounds in radiation oncology. October 14, 2020

Closing the disclosure gap: medical errors in pediatrics. March 27, 2019

Determination of health-care teamwork training competencies: a Delphi study. December 2, 2009

Back to basics: checklists in aviation and healthcare. June 17, 2015

Paediatric dosing errors before and after electronic prescribing. August 18, 2010

Methodological variations and their effects on reported medication administration error rates. February 27, 2013

Failure mode and effects analysis outputs: are they valid? August 15, 2012

Failure mode and effects analysis: too little for too much? April 18, 2012

Is failure mode and effect analysis reliable? May 27, 2009

Serious hazards of transfusion (SHOT) haemovigilance and progress is improving transfusion safety. November 20, 2013

Keeping an eye on patient safety using human factors engineering (HFE): a family affair for the hospitalized child. February 3, 2010

What constitutes a prescribing error in paediatrics? October 19, 2005

Does a checklist reduce the number of errors made in nurse-assembled discharge prescriptions? June 14, 2017

Association of primary care clinic appointment time with opioid prescribing. September 11, 2019

Affective influences on clinical reasoning and diagnosis: insights from social psychology and new research opportunities. October 19, 2022

The development of a patient safety program across the continuum of care. November 16, 2005

Practice indicators of suboptimal care and avoidable adverse events: a content analysis of a national qualifying examination. October 16, 2013

Quality of medication use in primary care—mapping the problem, working to a solution: a systematic review of the literature. October 21, 2009

Resident and nurse perspectives on the use of secure text messaging systems. October 19, 2022

Consumers' perspectives on their involvement in recognizing and responding to patient deterioration—developing a model for consumer reporting. July 24, 2019

Association of opioid consumption profiles after hospitalization with risk of adverse health care events. June 2, 2021

Incidence and variables associated with inconsistencies in opioid prescribing at hospital discharge and its associated adverse drug outcomes. May 5, 2021

Exploring the impact of employee engagement and patient safety. September 14, 2022

Making the Patient Safety and Quality Improvement Act of 2005 work. September 26, 2007

The impact of electronic health records on time efficiency of physicians and nurses: a systematic review. November 9, 2005

Opinions of nurses and physicians on a patient, family and visitor activated rapid response system in use across two hospital settings. February 28, 2024

The incidence and nature of prescribing and medication administration errors in paediatric inpatients. February 24, 2010

Inpatient Notes: human factors engineering and inpatient care—new ways to solve old problems. April 26, 2017

Prescription for error: process defects in a community retail pharmacy. December 19, 2007

Inappropriate opioid prescription after surgery. May 29, 2019

Reducing adverse events in blood transfusion. October 5, 2005

Exploring the sociotechnical intersection of patient safety and electronic health record implementation. October 16, 2013

The impact of a closed-loop electronic prescribing and administration system on prescribing errors, administration errors and staff time: a before-and-after study. August 22, 2007

Usability of a computerised drug monitoring programme to detect adverse drug events and non-compliance in outpatient ambulatory care. March 27, 2013

A 2-year study of patient safety competency assessment in 29 clinical laboratories. June 18, 2008

Positive deviance: a different approach to achieving patient safety. August 20, 2014

Cultural diversity: what role does it play in patient safety? August 25, 2010

Technical rationality and the decentring of patients and care delivery: a critique of 'unavoidable' in the context of patient harm. July 25, 2018

Role of nursing home quality on COVID-19 cases and deaths: evidence from Florida nursing homes. September 22, 2021

The relationship between resident physician burnout and its’ effects on patient care, professionalism, and academic achievement: a review of the literature. October 5, 2022

Medication errors with electronic prescribing (eP): two views of the same picture. July 21, 2010

Exploring safety systems for dispensing in community pharmacies: focusing on how staff relate to organizational components. September 17, 2014

Effect of a pharmacist-led educational intervention on inappropriate medication prescriptions in older adults: the D-PRESCRIBE randomized clinical trial. November 21, 2018

Countering cognitive biases in minimising low value care. June 7, 2017

Reporting of Clinical Adverse Events Scale: a measure of doctor and nurse attitudes to adverse event reporting. October 22, 2008

The effect of medical emergency teams on patient outcome: a review of the literature. January 5, 2011

Use of personal electronic devices by nurse anesthetists and the effects on patient safety. May 25, 2016

Health literacy in primary care practice. September 9, 2015

Will human factors restore faith in the GMC? April 17, 2019

Disclosure of adverse events and errors in surgical care: challenges and strategies for improvement. June 4, 2014

The role of language barriers on efficacy of rapid response teams. March 1, 2023

Accuracy of pressure ulcer events in US nursing home ratings. August 24, 2022

Electronic health record interoperability-why electronically discontinued medications are still dispensed. September 22, 2021

Explainable artificial intelligence for safe intraoperative decision support. September 25, 2019

Comparison of physician and computer diagnostic accuracy. October 19, 2016

Evaluation of symptom checkers for self diagnosis and triage: audit study. July 29, 2015

Missed medication doses in hospitalised patients: a descriptive account of quality improvement measures and time series analysis. June 26, 2013

Journal Article

Equity M&M - adaptation of the morbidity and mortality conference to analyze and confront structural inequity in internal medicine

Examining markers of safety in homecare using the international classification for patient safety. June 12, 2013

Educating medical trainees on medication reconciliation: a systematic review. May 13, 2015

Family involvement in managing medications of older patients across transitions of care: a systematic review. June 26, 2019

Factors influencing second victim experiences and support needs of OB/GYN and pediatric healthcare professionals after adverse patient events. January 17, 2024

Impact of patient communication problems on the risk of preventable adverse events in acute care settings. June 25, 2008

Surgical safety checklists: do they improve outcomes? June 20, 2012

Realizing e-prescribing's potential to reduce outpatient psychiatric medication errors. January 31, 2018

Types and effects of feedback for emergency ambulance staff: a systematic mixed studies review and meta-analysis. July 12, 2023

The safety of Australian healthcare: 10 years after QAHCS. April 15, 2005

Improving diagnosis by feedback and deliberate practice: one-on-one coaching for diagnostic maturation. June 2, 2021

How do we learn about error? A cross-sectional study of urology trainees. June 28, 2023

A systems analysis of work-related violence in hospitals: stakeholders, contributory factors, and leverage points. February 22, 2023

Attending to the emotional well-being of the health care workforce in a New York City health system during the COVID-19 pandemic. August 26, 2020

Empowering informal caregivers with health information: OpenNotes as a safety strategy. March 14, 2018

Opioids prescribed after low-risk surgical procedures in the United States, 2004–2012. December 7, 2016

COVID-19 related negative emotions and emotional suppression are associated with greater risk perceptions among emergency nurses: a cross-sectional study. January 25, 2023

Pediatric trainee perspectives on the decision to disclose medical errors. March 30, 2022

Improving self-reporting of adverse drug events in a West Virginia hospital. October 4, 2006

Vital signs: epidemiology of sepsis: prevalence of health care factors and opportunities for prevention. September 21, 2016

Family perceptions of medication administration at school: errors, risk factors, and consequences. April 16, 2008

Evaluating a new rapid response team: NP-led versus intensivist-led comparisons. February 29, 2012

Wristbands as aids to reduce misidentification: an ethnographically guided task analysis. September 28, 2011

The role of feedback in emergency ambulance services: a qualitative interview study. April 6, 2022

Does crew resource management training work? An update, an extension, and some critical needs. August 30, 2006

Using simulation-based training to improve patient safety: what does it take? July 13, 2005

The influence of formulation and medicine delivery system on medication administration errors in care homes for older people. January 30, 2011

A randomised controlled trial of the effect of continuous electronic physiological monitoring on the adverse event rate in high risk medical and surgical patients. December 6, 2006

Systematic review of medication errors in pediatric patients. October 25, 2006

Electronic prescribing reduced prescribing errors in a pediatric renal outpatient clinic. February 13, 2008

Implementing computerized physician order management at a community hospital. February 6, 2008

Importance of safety climate, teamwork climate and demographics: understanding nurses, allied health professionals and clerical staff perceptions of patient safety. January 30, 2019

Integrating and evaluating the data quality and utility of smart pump information in detecting medication administration errors: evaluation study. November 4, 2020

Human-based errors involving smart infusion pumps: a catalog of error types and prevention strategies. September 9, 2020

So many ways to be wrong: completeness and accuracy in a prospective study of OR-to-ICU handoff standardization. July 5, 2023

What do emergency department physicians and nurses feel? A qualitative study of emotions, triggers, regulation strategies, and effects on patient care. April 1, 2020

Let us to the TWISST; Plan, Simulate, Study and Act.

WebM&M Cases

Emergency department volume and delayed diagnosis of serious pediatric conditions. February 28, 2024

Disparities in diagnostic timeliness and outcomes of pediatric appendicitis. February 21, 2024

Using in situ simulation to identify latent safety threats in emergency medicine: a systematic review. November 8, 2023

Preferred language and diagnostic errors in the pediatric emergency department. November 8, 2023

Community validation of an approach to detect delayed diagnosis of appendicitis in big databases. October 11, 2023

Making a move: using simulation to identify latent safety threats before the care of injured patients in a new physical space. September 27, 2023

Outcomes of missed diagnosis of pediatric appendicitis, new-onset diabetic ketoacidosis, and sepsis in five pediatric hospitals. September 13, 2023

Improving emergency medicine clinician awareness of prehospital-administered medications. August 9, 2023

Clinician factors associated with delayed diagnosis of appendicitis. July 5, 2023

In situ simulation as a quality improvement tool to identify and mitigate latent safety threats for emergency department SARS-CoV-2 airway management: a multi-institutional initiative. June 28, 2023

The AHRQ Report on Diagnostic Errors in the Emergency Department: the wrong answer to the wrong question. June 28, 2023

Patient safety in emergency departments: a problem for health care systems? An international survey. June 21, 2023

Non-accidental Injuries in Infants Attending the Emergency Department. May 17, 2023

Types of diagnostic errors reported by paediatric emergency providers in a global paediatric emergency care research network. May 10, 2023

Crowding in the Emergency Department: Challenges for the Care of Children. March 15, 2023

Risk assessment of the acute stroke diagnostic process using failure modes, effects, and criticality analysis. March 1, 2023

In situ simulation as a tool to longitudinally identify and track latent safety threats in a structured quality improvement initiative for SARS-CoV-2 airway management: a single-center study. February 22, 2023

Misdiagnosis in the emergency department: time for a system solution. February 8, 2023

Diagnostic Errors in the Emergency Department: A Systematic Review. December 21, 2022

Jake Tapper shares harrowing story of daughter's near-fatal misdiagnosis. December 15, 2022

Delayed diagnosis of serious paediatric conditions in 13 regional emergency departments. October 26, 2022

Emergency department adverse events detected using the emergency department trigger tool. August 24, 2022

How safe are paediatric emergency departments? A national prospective cohort study. August 3, 2022

Does a suggested diagnosis in a general practitioners' referral question impact diagnostic reasoning: an experimental study. April 27, 2022

Conversations on diagnostic uncertainty and its management among pediatric acute care physicians. April 27, 2022

Use of e-triggers to identify diagnostic errors in the paediatric ED. April 6, 2022

A diagnostic time-out to improve differential diagnosis in pediatric abdominal pain. July 14, 2021

Connect With Us

Sign up for Email Updates

To sign up for updates or to access your subscriber preferences, please enter your email address below.

Agency for Healthcare Research and Quality

5600 Fishers Lane Rockville, MD 20857 Telephone: (301) 427-1364

• Accessibility
• Disclaimers
• Electronic Policies
• HHS Digital Strategy
• HHS Nondiscrimination Notice
• Inspector General
• Plain Writing Act
• Privacy Policy
• Viewers & Players
• U.S. Department of Health & Human Services
• The White House
• Don't have an account? Sign up to PSNet

Submit Your Innovations

Please select your preferred way to submit an innovation.

Continue as a Guest

Track and save your innovation

in My Innovations

Edit your innovation as a draft

Continue Logged In

Please select your preferred way to submit an innovation. Note that even if you have an account, you can still choose to submit an innovation as a guest.

Continue logged in

New users to the psnet site.

Access to quizzes and start earning

CME, CEU, or Trainee Certification.

Get email alerts when new content

matching your topics of interest

in My Innovations.

A Classroom Management Training Helps New Teachers Send Fewer Kids to the Office

• Share article

Understanding and managing a room full of students is often a “trial by fire” for less experienced educators, according to Megan Ryan, the mentor coordinator for teacher professional development at the Louisa County, Va., public schools.

“A lot of new teachers have this slightly romanticized idea about what their classroom is going to look like,” Ryan said. “They were in a wonderfully managed classroom [as student-teachers] with their cooperating teacher, and I don’t think a lot of them got to see the work in the background that went into that. They just feel like all students are going to listen and be engaged—and they don’t.”

That’s why Louisa County and other districts nationwide are exploring ways to help teachers—and particularly novices—better understand students’ social and behavioral cues. With explicit training on classroom management—a skill often underdeveloped in teacher-preparation programs—the thinking goes that teachers will cultivate closer relationships with their students and better manage, or prevent, disruptions that might otherwise result in a student being sent to the office for discipline.

New research suggests teacher training like the program used in Louisa County can significantly improve class climate and reduce disparities that result in students of color being disciplined disproportionately.

Class discipline has proven a thorny problem for schools, particularly amid post-pandemic increases in student behavior problems and disengagement. In an EdWeek Research Center survey last year, 70 percent of educators said students were behaving worse than they did in 2018-19, and 68 percent reported student morale falling during the same period.

But punishments that remove students from class lead to lost instruction and often further disengagement. Civil rights data show Black students, for example, can lose more than three times as much instructional time as their white classmates from exclusionary discipline.

Some efforts to reduce racial discipline gaps have focused on training teachers to avoid implicit, or unconscious, bias when gauging student behavior. But studies suggest anti-bias training has limited benefits and in some cases worsens stereotyping of students. That’s where the classroom management training Louisa County and other districts are working on could prove more effective.

“By focusing teachers’ attention on their interactions with kids and really trying to read kids’ cues correctly, essentially you bypass some of the racial biases by focusing just on behavior,” said Robert Pianta, the director of the Center for Advanced Study of Teaching and Learning at the University of Virginia. “From the kids’ perspective, you are responding more accurately to their cues.”

Take what Pianta calls a “garden-variety misbehavior”: a student doodling or staring into space while a teacher is talking. Prior studies have found implicit racial biases can lead teachers to interpret the same behavior more negatively from students of color than from white students, particularly boys.

Teachers with less effective classroom management often interpret the behavior personally and negatively, Pianta said: “This kid wants to make life difficult for me.”

More experienced teachers are more likely to interpret the behavior neutrally—that the student may be confused or may need more scaffolding to be engaged, for example—and are more likely to ask the student questions about the behavior rather than jumping to discipline.

Classroom-management shortcomings worsen discipline gaps

Teachers who refer high numbers of students out of the classroom for discipline are more likely to be early in their careers, and they’re more likely than other teachers to refer students of color at higher rates than their white peers.

Jing Liu, an assistant professor of education policy at the University of Maryland, College Park, and his colleagues tracked office referrals for 2,900 K-12 teachers at more than 100 urban California schools from 2016 to 2020. They found that roughly 5 percent of teachers were responsible for nearly 35 percent of office discipline referrals.

These high-referring educators—overwhelmingly in their first three years in the field—sent a student to the office for discipline about once every four days, often for subjective misbehavior like defiance, a category found more frequently to be associated with cultural and ethnic bias. By contrast, other teachers on average sent less than one student to the office every other month.

The racial disparities in these high-referring teachers’ disciplinary referrals were so staggering that the study estimated providing better classroom-management training for high-referring teachers could halve racial discipline gaps between Black and white students.

(According to the most recent federal civil rights data , Black students made up 15 percent of all K-12 public school students but accounted for nearly 27 percent of students without disabilities who had to serve an out-of-school suspension at least once in 2020.)

By focusing teachers' attention on their interactions with kids and really trying to read kids' cues correctly, essentially you bypass some of the racial biases by focusing just on behavior.

Such wide discipline gaps “beg the question of the extent to which teacher-preparation programs are addressing issues with aspiring teachers about inequities in the use of discipline strategies in the classroom,” said Heather Peske, the president of the National Council on Teacher Quality, which reviews teacher-preparation programs for the classroom-management strategies they teach by examining program materials and syllabi and, in some cases, interviewing students and staff. “We know that the consequences of poor classroom management are often far worse for students of color. Better training in classroom management could help teachers head off behavior challenges before they become so problematic that a teacher would need to resort to disciplinary referral.”

Limited classroom-management training

In policy, schools have been attempting to move away from discipline that takes students out of the classroom, but most teachers receive little preparation in implementing more positive and inclusive classroom-management approaches.

Inclusive discipline approaches like restorative justice depend on strong teacher-student relationships, but the National Council on Teacher Quality finds many teachers get little preparation in the best practices for building those relationships.

The council’s reviews of teacher-prep programs include an analysis of their emphasis on what NCTQ considers to be five key, evidence-based instructional practices, including praising students’ good behavior, articulating and applying clear consequences for misbehavior, and engaging students through interesting lessons that provide ample opportunities for participation.

“Reinforcing good behavior with praise stands out to us because it’s the least likely to be taught and the least likely to be practiced in teacher prep, even though it has the most research behind its efficacy ,” Peske said.

Only 27 percent of teacher -prep programs required that aspiring teachers learn to reinforce positive classroom behavior, according to NCTQ’s most recent, 2020 review of classroom management practices in teacher prep programs.

In a nationally representative survey of 953 educators conducted from Jan. 31 to March 4 of this year, 39 percent of K-12 educators told the EdWeek Research Center that they had never received explicit classroom-management instruction in their teacher-preparation program. Another 8 percent of educators said they had never participated in a formal teacher-training program.

Even among educators who told the EdWeek Research Center that they did have classroom management training, its quality and usefulness varied widely.

“We had one semester of classroom management,” one teacher recalled in an open-ended section of the survey. “It felt helpful at the time, but didn’t much prepare me for an actual classroom.”

One veteran teacher who participated in classroom-management training in the 1990s, said it focused “primarily about ‘what’ and not necessarily about ‘why’ or ‘how,’” the teacher recalled. “We are now very aware of the importance of creating an inclusive community of learners (the ‘why’) rooted in behavior modeled by adults (the ‘how’) for respectful communication, emotional regulation, and active engagement in the learning process.”

Requirements for teacher prep programs vary significantly by state, field of study, and accrediting group, according to the American Association of Colleges for Teacher Education. The Interstate Teacher Assessment and Support Consortium ‘s 2013 model standards for teacher preparation do call for teachers to understand how to create a positive learning environment and suggest a teacher should understand “how personal identity, worldview, and prior experience affect perceptions and expectations,” and recognize “how they may bias behaviors and interactions with others.”

Building teachers’ self-reflection

Ryan, the mentor coordinator in Louisa County, located northwest of Richmond, is trying to help teachers understand the how’s and why’s behind classroom management.

Since 2018, the district has participated in the My Teaching Partner program, developed at the University of Virginia. Participating teachers learn to record and analyze their own lessons, looking for and analyzing students’ social cues and behavioral triggers.

Ryan said new teachers often don’t take enough time at the start of the school year to get to know their students and explain the purposes behind class procedures and routines.

“They may feel like it’s taking away from instructional time, so we really try to focus on the fact that if they put in that time at the beginning of the year, the benefits and the payoff down the road are going to outweigh losing some instruction time,” she said.

In two-week cycles throughout the year, Ryan records and analyzes lessons with each of her teachers. She seeks three, one-minute clips in which the teacher uses strong, effective, and specific communication with their students, rather than general critiques.

In one clip, Ryan said, a teacher “gave a little grace” on an assignment after learning that a student was late because he had been caring for six siblings. In reviewing another, Ryan called out a strong content discussion, “where the teacher and the student are in a good feedback loop, and they’re having a really deep conversation with good questioning skills.”

Rachel Post, a 6th grade math teacher at William Wirt Middle School in Prince George’s County, Md., who participated in the My Teaching Partner training in 2020, said recording her lessons helped her be “honest” and catch student reactions she would otherwise miss.

“I may be working one-on-one with a student or managing some behavior problem, but there’s so many things that I miss,” she said in one program video. “The thing about watching yourself on video is that you see all of that and you can’t really hide behind excuses.”

After eight of these two-week cycles, a study found teachers who participated in the mentoring program were referring fewer students for discipline outside the classroom and had no discipline gaps between Black and white students. Participating teachers’ students, across all races and achievement levels, also had higher engagement.

“You watch these shifts in the teacher’s behavior and then you also watch shifts in the kids’ behavior,” Pianta said. “They’re paying attention more. They’re looking like they’re enjoying the classroom more, participating in the classroom to a greater extent. … It’s like the classroom becomes a more active environment.”

Coverage of whole-child approaches to learning is supported in part by a grant from the Chan Zuckerberg Initiative, at www.chanzuckerberg.com . Education Week retains sole editorial control over the content of this coverage.

Sign Up for EdWeek Update

Edweek top school jobs.

Sign Up & Sign In

• Open access
• Published: 15 April 2024

Demuxafy : improvement in droplet assignment by integrating multiple single-cell demultiplexing and doublet detection methods

• Drew Neavin 1 ,
• Anne Senabouth 1 ,
• Himanshi Arora 1 , 2 ,
• Jimmy Tsz Hang Lee 3 ,
• Aida Ripoll-Cladellas 4 ,
• sc-eQTLGen Consortium ,
• Lude Franke 5 ,
• Shyam Prabhakar 6 , 7 , 8 ,
• Chun Jimmie Ye 9 , 10 , 11 , 12 ,
• Davis J. McCarthy 13 , 14 ,
• Marta Melé 4 ,
• Martin Hemberg 15 &
• Joseph E. Powell   ORCID: orcid.org/0000-0002-5070-4124 1 , 16  

Genome Biology volume  25 , Article number:  94 ( 2024 ) Cite this article

133 Accesses

Metrics details

Recent innovations in single-cell RNA-sequencing (scRNA-seq) provide the technology to investigate biological questions at cellular resolution. Pooling cells from multiple individuals has become a common strategy, and droplets can subsequently be assigned to a specific individual by leveraging their inherent genetic differences. An implicit challenge with scRNA-seq is the occurrence of doublets—droplets containing two or more cells. We develop Demuxafy, a framework to enhance donor assignment and doublet removal through the consensus intersection of multiple demultiplexing and doublet detecting methods. Demuxafy significantly improves droplet assignment by separating singlets from doublets and classifying the correct individual.

Droplet-based single-cell RNA sequencing (scRNA-seq) technologies have provided the tools to profile tens of thousands of single-cell transcriptomes simultaneously [ 1 ]. With these technological advances, combining cells from multiple samples in a single capture is common, increasing the sample size while simultaneously reducing batch effects, cost, and time. In addition, following cell capture and sequencing, the droplets can be demultiplexed—each droplet accurately assigned to each individual in the pool [ 2 , 3 , 4 , 5 , 6 , 7 ].

Many scRNA-seq experiments now capture upwards of 20,000 droplets, resulting in ~16% (3,200) doublets [ 8 ]. Current demultiplexing methods can also identify doublets—droplets containing two or more cells—from different individuals (heterogenic doublets). These doublets can significantly alter scientific conclusions if they are not effectively removed. Therefore, it is essential to remove doublets from droplet-based single-cell captures.

However, demultiplexing methods cannot identify droplets containing multiple cells from the same individual (homogenic doublets) and, therefore, cannot identify all doublets in a single capture. If left in the dataset, those doublets could appear as transitional cells between two distinct cell types or a completely new cell type. Accordingly, additional methods have been developed to identify heterotypic doublets (droplets that contain two cells from different cell types) by comparing the transcriptional profile of each droplet to doublets simulated from the dataset [ 9 , 10 , 11 , 12 , 13 , 14 , 15 ]. It is important to recognise that demultiplexing methods achieve two functions—segregation of cells from different donors and separation of singlets from doublets—while doublet detecting methods solely classify singlets versus doublets.

Therefore, demultiplexing and transcription-based doublet detecting methods provide complementary information to improve doublet detection, providing a cleaner dataset and more robust scientific results. There are currently five genetic-based demultiplexing [ 2 , 3 , 4 , 5 , 6 , 7 , 16 ] and seven transcription-based doublet-detecting methods implemented in various languages [ 9 , 10 , 11 , 12 , 13 , 14 , 15 ]. Under different scenarios, each method is subject to varying performance and, in some instances, biases in their ability to accurately assign cells or detect doublets from certain conditions. The best combination of methods is currently unclear but will undoubtedly depend on the dataset and research question.

Therefore, we set out to identify the best combination of genetic-based demultiplexing and transcription-based doublet-detecting methods to remove doublets and partition singlets from different donors correctly. In addition, we have developed a software platform ( Demuxafy ) that performs these intersectional methods and provides additional commands to simplify the execution and interpretation of results for each method (Fig. 1 a).

Study design and qualitative method classifications. a  Demuxafy is a platform to perform demultiplexing and doublet detecting with consistent documentation. Demuxafy also provides wrapper scripts to quickly summarize the results from each method and assign clusters to each individual with reference genotypes when a reference-free demultiplexing method is used. Finally, Demuxafy provides a script to easily combine the results from multiple different methods into a single data frame and it provides a final assignment for each droplet based on the combination of multiple methods. In addition, Demuxafy provides summaries of the number of droplets classified as singlets or doublets by each method and a summary of the number of droplets assigned to each individual by each of the demultiplexing methods. b  Two datasets are included in this analysis - a PBMC dataset and a fibroblast dataset. The PBMC dataset contains 74 pools that captured approximately 20,000 droplets each with 12-16 donor cells multiplexed per pool. The fibroblast dataset contains 11 pools of roughly 7,000 droplets per pool with sizes ranging from six to eight donors per pool. All pools were processed by all demultiplexing and doublet detecting methods and the droplet and donor classifications were compared between the methods and between the PBMCs and fibroblasts. Then the PBMC droplets that were classified as singlets by all methods were taken as ‘true singlets’ and used to generate new pools in silico. Those pools were then processed by each of the demultiplexing and doublet detecting methods and intersectional combinations of demultiplexing and doublet detecting methods were tested for different experimental designs

To compare the demultiplexing and doublet detecting methods, we utilised two large, multiplexed datasets—one that contained ~1.4 million peripheral blood mononuclear cells (PBMCs) from 1,034 donors [ 17 ] and one with ~94,000 fibroblasts from 81 donors [ 18 ]. We used the true singlets from the PBMC dataset to generate new in silico pools to assess the performance of each method and the multi-method intersectional combinations (Fig. 1 b).

Here, we compare 14 demultiplexing and doublet detecting methods with different methodological approaches, capabilities, and intersectional combinations. Seven of those are demultiplexing methods ( Demuxalot [ 6 ], Demuxlet [ 3 ], Dropulation [ 5 ], Freemuxlet [ 16 ], ScSplit [ 7 ], Souporcell [ 4 ], and Vireo [ 2 ]) which leverage the common genetic variation between individuals to identify cells that came from each individual and to identify heterogenic doublets. The seven remaining methods ( DoubletDecon [ 9 ], DoubletDetection [ 14 ], DoubletFinder [ 10 ], ScDblFinder [ 11 ], Scds [ 12 ], Scrublet [ 13 ], and Solo [ 15 ]) identify doublets based on their similarity to simulated doublets generated by adding the transcriptional profiles of two randomly selected droplets in the dataset. These methods assume that the proportion of real doublets in the dataset is low, so combining any two droplets will likely represent the combination of two singlets.

We identify critical differences in the performance of demultiplexing and doublet detecting methods to classify droplets correctly. In the case of the demultiplexing techniques, their performance depends on their ability to identify singlets from doublets and assign a singlet to the correct individual. For doublet detecting methods, the performance is based solely on their ability to differentiate a singlet from a doublet. We identify limitations in identifying specific doublet types and cell types by some methods. In addition, we compare the intersectional combinations of these methods for multiple experimental designs and demonstrate that intersectional approaches significantly outperform all individual techniques. Thus, the intersectional methods provide enhanced singlet classification and doublet removal—a critical but often under-valued step of droplet-based scRNA-seq processing. Our results demonstrate that intersectional combinations of demultiplexing and doublet detecting software provide significant advantages in droplet-based scRNA-seq preprocessing that can alter results and conclusions drawn from the data. Finally, to provide easy implementation of our intersectional approach, we provide Demuxafy ( https://demultiplexing-doublet-detecting-docs.readthedocs.io/en/latest/index.html ) a complete platform to perform demultiplexing and doublet detecting intersectional methods (Fig. 1 a).

Study design

To evaluate demultiplexing and doublet detecting methods, we developed an experimental design that applies the different techniques to empirical pools and pools generated in silico from the combination of true singlets—droplets identified as singlets by every method (Fig. 1 a). For the first phase of this study, we used two empirical multiplexed datasets—the peripheral blood mononuclear cell (PBMC) dataset containing ~1.4 million cells from 1034 donors and a fibroblast dataset of ~94,000 cells from 81 individuals (Additional file 1 : Table S1). We chose these two cell systems to assess the methods in heterogeneous (PBMC) and homogeneous (fibroblast) cell types.

Demultiplexing and doublet detecting methods perform similarly for heterogeneous and homogeneous cell types

We applied the demultiplexing methods ( Demuxalot , Demuxlet , Dropulation , Freemuxlet , ScSplit , Souporcell , and Vireo ) and doublet detecting methods ( DoubletDecon , DoubletDetection , DoubletFinder , ScDblFinder , Scds , Scrublet , and Solo ) to the two datasets and assessed the results from each method. We first compared the droplet assignments by identifying the number of singlets and doublets identified by a given method that were consistently annotated by all methods (Fig. 2 a–d). We also identified the percentage of droplets that were annotated consistently between pairs of methods (Additional file 2 : Fig S1). In the cases where two demultiplexing methods were compared to one another, both the droplet type (singlet or doublet) and the assignment of the droplet to an individual had to match to be considered in agreement. In all other comparisons (i.e. demultiplexing versus doublet detecting and doublet detecting versus doublet detecting), only the droplet type (singlet or doublet) was considered for agreement since doublet detecting methods cannot annotate donor assignment. We found that the two method types were more similar to other methods of the same type (i.e., demultiplexing versus demultiplexing and doublet detecting versus doublet detecting) than they were to methods from a different type (demultiplexing methods versus doublet detecting methods; Supplementary Fig 1). We found that the similarity of the demultiplexing and doublet detecting methods was consistent in the PBMC and fibroblast datasets (Pearson correlation R = 0.78, P -value < 2×10 −16 ; Fig S1a-c). In addition, demultiplexing methods were more similar than doublet detecting methods for both the PBMC and fibroblast datasets (Wilcoxon rank-sum test: P < 0.01; Fig. 2 a–b and Additional file 2 : Fig S1).

Demultiplexing and Doublet Detecting Method Performance Comparison. a  The proportion of droplets classified as singlets and doublets by each method in the PBMCs. b  The number of other methods that classified the singlets and doublets identified by each method in the PBMCs. c  The proportion of droplets classified as singlets and doublets by each method in the fibroblasts. d The number of other methods that classified the singlets and doublets identified by each method in the fibroblasts. e - f The performance of each method when the majority classification of each droplet is considered the correct annotation in the PBMCs ( e ) and fibroblasts ( f ). g - h  The number of droplets classified as singlets (box plots) and doublets (bar plots) by all methods in the PBMC ( g ) and fibroblast ( h ) pools. i - j  The number of donors that were not identified by each method in each pool for PBMCs ( i ) and fibroblasts ( j ). PBMC: peripheral blood mononuclear cell. MCC: Matthew’s correlationcoefficient

The number of unique molecular identifiers (UMIs) and genes decreased in droplets that were classified as singlets by a larger number of methods while the mitochondrial percentage increased in both PBMCs and fibroblasts (Additional file 2 : Fig S2).

We next interrogated the performance of each method using the Matthew’s correlation coefficient (MCC) to calculate the consistency between Demuxify and true droplet classification. We identified consistent trends in the MCC scores for each method between the PBMCs (Fig. 2 e) and fibroblasts (Fig. 2 f). These data indicate that the methods behave similarly, relative to one another, for heterogeneous and homogeneous datasets.

Next, we sought to identify the droplets concordantly classified by all demultiplexing and doublet detecting methods in the PBMC and fibroblast datasets. On average, 732 singlets were identified for each individual by all the methods in the PBMC dataset. Likewise, 494 droplets were identified as singlets for each individual by all the methods in the fibroblast pools. However, the concordance of doublets identified by all methods was very low for both datasets (Fig. 2 e–f). Notably, the consistency of classifying a droplet as a doublet by all methods was relatively low (Fig. 2 b,d,g, and h). This suggests that doublet identification is not consistent between all the methods. Therefore, further investigation is required to identify the reasons for these inconsistencies between methods. It also suggests that combining multiple methods for doublet classification may be necessary for more complete doublet removal. Further, some methods could not identify all the individuals in each pool (Fig. 2 i–j). The non-concordance between different methods demonstrates the need to effectively test each method on a dataset where the droplet types are known.

Computational resources vary for demultiplexing and doublet detecting methods

We recorded each method’s computational resources for the PBMC pools, with ~20,000 cells captured per pool (Additional file 1 : Table S1). Of the demultiplexing methods, ScSplit took the most time (multiple days) and required the most steps, but Demuxalot , Demuxlet , and Freemuxlet used the most memory. Solo took the longest time (median 13 h) and most memory to run for the doublet detecting methods but is the only method built to be run directly from the command line, making it easy to implement (Additional file 2 : Fig S3).

Generate pools with known singlets and doublets

However, there is no gold standard to identify which droplets are singlets or doublets. Therefore, in the second phase of our experimental design (Fig. 1 b), we used the PBMC droplets classified as singlets by all methods to generate new pools in silico. We chose to use the PBMC dataset since our first analyses indicated that method performance is similar for homogeneous (fibroblast) and heterogeneous (PBMC) cell types (Fig. 2 and Additional file 2 : Fig S1) and because we had many more individuals available to generate in silico pools from the PBMC dataset (Additional file 1 : Table S1).

We generated 70 pools—10 each of pools that included 2, 4, 8, 16, 32, 64, or 128 individuals (Additional file 1 : Table S2). We assume a maximum 20% doublet rate as it is unlikely researchers would use a technology that has a higher doublet rate (Fig. 3 a).

In silico Pool Doublet Annotation and Method Performance. a  The percent of singlets and doublets in the in -silico pools - separated by the number of multiplexed individuals per pool. b  The percentage and number of doublets that are heterogenic (detectable by demultiplexing methods), heterotypic (detectable by doublet detecting methods), both (detectable by either method category) and neither (not detectable with current methods) for each multiplexed pool size. c  Percent of droplets that each of the demultiplexing and doublet detecting methods classified correctly for singlets and doublet subtypes for different multiplexed pool sizes. d  Matthew’s Correlation Coefficient (MCC) for each of the methods for each of the multiplexed pool sizes. e  Balanced accuracy for each of the methods for each of the multiplexed pool sizes

We used azimuth to classify the PBMC cell types for each droplet used to generate the in silico pools [ 19 ] (Additional file 2 : Fig S4). As these pools have been generated in silico using empirical singlets that have been well annotated, we next identified the proportion of doublets in each pool that were heterogenic, heterotypic, both, and neither. This approach demonstrates that a significant percentage of doublets are only detectable by doublet detecting methods (homogenic and heterotypic) for pools with 16 or fewer donors multiplexed (Fig. 3 b).

While the total number of doublets that would be missed if only using demultiplexing methods appears small for fewer multiplexed individuals (Fig. 3 b), it is important to recognise that this is partly a function of the ~732 singlet cells per individual used to generate these pools. Hence, the in silico pools with fewer individuals also have fewer cells. Therefore, to obtain numbers of doublets that are directly comparable to one another, we calculated the number of each doublet type that would be expected to be captured with 20,000 cells when 2, 4, 8, 16, or 32 individuals were multiplexed (Additional file 2 : Fig S5). These results demonstrate that many doublets would be falsely classified as singlets since they are homogenic when just using demultiplexing methods for a pool of 20,000 cells captured with a 16% doublet rate (Additional file 2 : Fig S5). However, as more individuals are multiplexed, the number of droplets that would not be detectable by demultiplexing methods (homogenic) decreases. This suggests that typical workflows that use only one demultiplexing method to remove doublets from pools that capture 20,000 droplets with 16 or fewer multiplexed individuals fail to adequately remove between 173 (16 multiplexed individuals) and 1,325 (2 multiplexed individuals) doublets that are homogenic and heterotypic which could be detected by doublet detecting methods (Additional file 2 : Fig S5). Therefore, a technique that uses both demultiplexing and doublet detecting methods in parallel will complement more complete doublet removal methods. Consequently, we next set out to identify the demultiplexing and doublet detecting methods that perform the best on their own and in concert with other methods.

Doublet and singlet droplet classification effectiveness varies for demultiplexing and doublet detecting methods

Demultiplexing methods fail to classify homogenic doublets.

We next investigated the percentage of the droplets that were correctly classified by each demultiplexing and doublet detecting method. In addition to the seven demultiplexing methods, we also included Demuxalot with the additional steps to refine the genotypes that can then be used for demultiplexing— Demuxalot (refined). Demultiplexing methods correctly classify a large portion of the singlets and heterogenic doublets (Fig. 3 c). This pattern is highly consistent across different cell types, with the notable exceptions being decreased correct classifications for erythrocytes and platelets when greater than 16 individuals are multiplexed (Additional file 2 : Fig S6).

However, Demuxalot consistently demonstrates the highest correct heterogenic doublet classification. Further, the percentage of the heterogenic doublets classified correctly by Souporcell decreases when large numbers of donors are multiplexed. ScSplit is not as effective as the other demultiplexing methods at classifying heterogenic doublets, partly due to the unique doublet classification method, which assumes that the doublets will generate a single cluster separate from the donors (Table 1 ). Importantly, the demultiplexing methods identify almost none of the homogenic doublets for any multiplexed pool size—demonstrating the need to include doublet detecting methods to supplement the demultiplexing method doublet detection.

Doublet detecting method classification performances vary greatly

In addition to assessing each of the methods with default settings, we also evaluated ScDblFinder with ‘known doublets’ provided. This method can take already known doublets and use them when detecting doublets. For these cases, we used the droplets that were classified as doublets by all the demultiplexing methods as ‘known doublets’.

Most of the methods classified a similarly high percentage of singlets correctly, with the exceptions of DoubletDecon and DoubletFinder for all pool sizes (Fig. 3 c). However, unlike the demultiplexing methods, there are explicit cell-type-specific biases for many of the doublet detecting methods (Additional file 2 : Fig S7). These differences are most notable for cell types with fewer cells (i.e. ASDC and cDC2) and proliferating cells (i.e. CD4 Proliferating, CD8 Proliferating, and NK Proliferating). Further, all of the softwares demonstrate high correct percentages for some cell types including CD4 Naïve and CD8 Naïve (Additional file 2 : Fig S7).

As expected, all doublet detecting methods identified heterotypic doublets more effectively than homotypic doublets (Fig. 3 c). However, ScDblFinder and Scrublet classified the most doublets correctly across all doublet types for pools containing 16 individuals or fewer. Solo was more effective at identifying doublets than Scds for pools containing more than 16 individuals. It is also important to note that it was not feasible to run DoubletDecon for the largest pools containing 128 multiplexed individuals and an average of 115,802 droplets (range: 113,594–119,126 droplets). ScDblFinder performed similarly when executed with and without known doublets (Pearson correlation P = 2.5 × 10 -40 ). This suggests that providing known doublets to ScDblFinder does not offer an added benefit.

Performances vary between demultiplexing and doublet detecting method and across the number of multiplexed individuals

We assessed the overall performance of each method with two metrics: the balanced accuracy and the MCC. We chose to use balanced accuracy since, with unbalanced group sizes, it is a better measure of performance than accuracy itself. Further, the MCC has been demonstrated as a more reliable statistical measure of performance since it considers all possible categories—true singlets (true positives), false singlets (false positives), true doublets (true negatives), and false doublets (false negatives). Therefore, a high score on the MCC scale indicates high performance in each metric. However, we provide additional performance metrics for each method (Additional file 1 : Table S3). For demultiplexing methods, both the droplet type (singlet or doublet) and the individual assignment were required to be considered a ‘true singlet’. In contrast, only the droplet type (singlet or doublet) was needed for doublet detection methods.

The MCC and balanced accuracy metrics are similar (Spearman’s ⍴ = 0.87; P < 2.2 × 10 -308 ). Further, the performance of Souporcell decreases for pools with more than 32 individuals multiplexed for both metrics (Student’s t -test for MCC: P < 1.1 × 10 -9 and balanced accuracy: P < 8.1 × 10 -11 ). Scds , ScDblFinder , and Scrublet are among the top-performing doublet detecting methods Fig. 3 d–e).

Overall, between 0.4 and 78.8% of droplets were incorrectly classified by the demultiplexing or doublet detecting methods depending on the technique and the multiplexed pool size (Additional file 2 : Fig S8). Demuxalot (refined) and DoubletDetection demonstrated the lowest percentage of incorrect droplets with about 1% wrong in the smaller pools (two multiplexed individuals) and about 3% incorrect in pools with at least 16 multiplexed individuals. Since some transitional states and cell types are present in low percentages in total cell populations (i.e. ASDCs at 0.02%), incorrect classification of droplets could alter scientific interpretations of the data, and it is, therefore, ideal for decreasing the number of erroneous assignments as much as possible.

False singlets and doublets demonstrate different metrics than correctly classified droplets

We next asked whether specific cell metrics might contribute to false singlet and doublet classifications for different methods. Therefore, we compared the number of genes, number of UMIs, mitochondrial percentage and ribosomal percentage of the false singlets and doublets to equal numbers of correctly classified cells for each demultiplexing and doublet detecting method.

The number of UMIs (Additional file 2 : Fig S9 and Additional file 1 : Table S4) and genes (Additional file 2 : Fig S10 and Additional file 1 : Table S5) demonstrated very similar distributions for all comparisons and all methods (Spearman ⍴ = 0.99, P < 2.2 × 10 -308 ). The number of UMIs and genes were consistently higher in false singlets and lower in false doublets for most demultiplexing methods except some smaller pool sizes (Additional file 2 : Fig S9a and Additional file 2 : Fig S10a; Additional file 1 : Table S4 and Additional file 1 : Table S5). The number of UMIs and genes was consistently higher in droplets falsely classified as singlets by the doublet detecting methods than the correctly identified droplets (Additional file 2 : Fig S9b and Additional file 2 : Fig S10b; Additional file 1 : Table S4 and Additional file 1 : Table S5). However, there was less consistency in the number of UMIs and genes detected in false singlets than correctly classified droplets between the different doublet detecting methods (Additional file 2 : Fig S9b and Additional file 2 : Fig S10b; Additional file 1 : Table S4 and Additional file 1 : Table S5).

The ribosomal percentage of the droplets falsely classified as singlets or doublets is similar to the correctly classified droplets for most methods—although they are statistically different for larger pool sizes (Additional file 2 : Fig S11a and Additional file 1 : Table S6). However, the false doublets classified by some demultiplexing methods ( Demuxalot , Demuxalot (refined), Demuxlet , ScSplit , Souporcell , and Vireo ) demonstrated higher ribosomal percentages. Some doublet detecting methods ( ScDblFinder , ScDblFinder with known doublets and Solo) demonstrated higher ribosomal percentages for the false doublets while other demonstrated lower ribosomal percentages ( DoubletDecon , DoubletDetection , and DoubletFinder ; Additional file 2 : Fig S11b and Additional file 1 : Table S6).

Like the ribosomal percentage, the mitochondrial percentage in false singlets is also relatively similar to correctly classified droplets for both demultiplexing (Additional file 2 : Fig S12a and Additional file 1 : Table S7) and doublet detecting methods (Additional file 2 : Fig S12b). The mitochondrial percentage for false doublets is statistically lower than the correctly classified droplets for a few larger pools for Freemuxlet , ScSplit , and Souporcell . The doublet detecting method Solo also demonstrates a small but significant decrease in mitochondrial percentage in the false doublets compared to the correctly annotated droplets. However, other doublet detecting methods including DoubletFinder and the larger pools of most other methods demonstrated a significant increase in mitochondrial percent in the false doublets compared to the correctly annotated droplets (Additional file 2 : Fig S12b).

Overall, these results demonstrate a strong relationship between the number of genes and UMIs and limited influence of ribosomal or mitochondrial percentage in a droplet and false classification, suggesting that the number of genes and UMIs can significantly bias singlet and doublet classification by demultiplexing and doublet detecting methods.

Ambient RNA, number of reads per cell, and uneven pooling impact method performance

To further quantify the variables that impact the performance of each method, we simulated four conditions that could occur with single-cell RNA-seq experiments: (1) decreased number of reads (reduced 50%), (2) increased ambient RNA (10%, 20% and 50%), (3) increased mitochondrial RNA (5%, 10% and 25%) and 4) uneven donor pooling from single donor spiking (0.5 or 0.75 proportion of pool from one donor). We chose these scenarios because they are common technical effects that can occur.

We observed a consistent decrease in the demultiplexing method performance when the number of reads were decreased by 50% but the degree of the effect varied for each method and was larger in pools containing more multiplexed donors (Additional file 2 : Fig S13a and Additional file 1 : Table S8). Decreasing the number of reads did not have a detectable impact on the performance of the doublet detecting methods.

Simulating additional ambient RNA (10%, 20%, or 50%) decreased the performance of all the demultiplexing methods (Additional file 2 : Fig S13b and Additional file 1 : Table S9) but some were unimpacted in pools that had 16 or fewer individuals multiplexed ( Souporcell and Vireo ). The performance of some of the doublet detecting methods were impacted by the ambient RNA but the performance of most methods did not decrease. Scrublet and ScDblFinder were the doublet detecting methods most impacted by ambient RNA but only in pools with at least 32 multiplexed donors (Additional file 2 : Fig S13b and Additional file 1 : Table S9).

Increased mitochondrial percent did not impact the performance of demultiplexing or doublet detecting methods (Additional file 2 : Fig S13c and Additional file 1 : Table S10).

We also tested whether experimental designs that pooling uneven proportions of donors would alter performance. We tested scenarios where either half the pool was composed of a single donor (0.5 spiked donor proportion) or where three quarters of the pool was composed of a single donor. This experimental design significantly reduced the demultiplexing method performance (Additional file 2 : Fig S13d and Additional file 1 : Table S11) with the smallest influence on Freemuxlet . The performance of most of the doublet detecting methods were unimpacted except for DoubletDetection that demonstrated significant decreases in performance in pools where at least 16 donors were multiplexed. Intriguingly, the performance of Solo increased with the spiked donor pools when the pools consisted of 16 donors or less.

Our results demonstrate significant differences in overall performance between different demultiplexing and doublet detecting methods. We further noticed some differences in the use of the methods. Therefore, we have accumulated these results and each method’s unique characteristics and benefits in a heatmap for visual interpretation (Fig. 4 ).

Assessment of each of the demultiplexing and doublet detecting methods. Assessments of a variety of metrics for each of the demultiplexing (top) and doublet detecting (bottom) methods

Framework for improving singlet classifications via method combinations

After identifying the demultiplexing and doublet detecting methods that performed well individually, we next sought to test whether using intersectional combinations of multiple methods would enhance droplet classifications and provide a software platform— Demuxafy —capable of supporting the execution of these intersectional combinations.

We recognise that different experimental designs will be required for each project. As such, we considered this when testing combinations of methods. We considered multiple experiment designs and two different intersectional methods: (1) more than half had to classify a droplet as a singlet to be called a singlet and (2) at least half of the methods had to classify a droplet as a singlet to be called a singlet. Significantly, these two intersectional methods only differ when an even number of methods are being considered. For combinations that include demultiplexing methods, the individual called by the majority of the methods is the individual used for that droplet. When ties occur, the individual is considered ‘unassigned’.

Combining multiple doublet detecting methods improve doublet removal for non-multiplexed experimental designs

For the non-multiplexed experimental design, we considered all possible method combinations (Additional file 1 : Table S12). We identified important differences depending on the number of droplets captured and have provided recommendations accordingly. We identified that DoubletFinder , Scrublet , ScDblFinder and Scds is the ideal combination for balanced droplet calling when less than 2,000 droplets are captured. Scds and ScDblFinder or Scrublet , Scds and ScDblFinder is the best combination when 2,000–10,000 droplets are captured. Scds , Scrublet, ScDblFinder and DoubletDetection is the best combination when 10,000–20,000 droplets are captured and Scrublet , Scds , DoubletDetection and ScDblFinder . It is important to note that even a slight increase in the MCC significantly impacts the number of true singlets and true doublets classified with the degree of benefit highly dependent on the original method performance. The combined method increases the MCC compared to individual doublet detecting methods on average by 0.11 and up to 0.33—a significant improvement in the MCC ( t -test FDR < 0.05 for 95% of comparisons). For all combinations, the intersectional droplet method requires more than half of the methods to consider the droplet a singlet to classify it as a singlet (Fig. 5 ).

Recommended Method Combinations Dependent on Experimental Design. Method combinations are provided for different experimental designs, including those that are not multiplexed (left) and multiplexed (right), including experiments that have reference SNP genotypes available vs those that do not and finally, multiplexed experiments with different numbers of individuals multiplexed. The each bar represents either a single method (shown with the coloured icon above the bar) or a combination of methods (shown with the addition of the methods and an arrow indicating the bar). The proportion of true singlets, true doublets, false singlets and false doublets for each method or combination of methods is shown with the filled barplot and the MCC is shown with the black points overlaid on the barplot. MCC: Matthew’s Correlation Coefficient

Demuxafy performs better than Chord

Chord is an ensemble machine learning doublet detecting method that uses Scds and DoubletFinder to identify doublets. We compared Demuxafy using Scds and DoubletFinder to Chord and identified that Demuxafy outperformed Chord in pools that contained at least eight donors and was equivalent in pools that contained less than eight donors (Additional file 2 : Fig S14). This is because Chord classifies more droplets as false singlets and false doublets than Demuxafy . In addition, Chord failed to complete for two of the pools that contained 128 multiplexed donors.

Combining multiple demultiplexing and doublet detecting methods improve doublet removal for multiplexed experimental designs

For experiments where 16 or fewer individuals are multiplexed with reference SNP genotypes available, we considered all possible combinations between the demultiplexing and doublet detecting methods except ScDblFinder with known doublets due to its highly similar performance to ScDblFinder (Fig 3 ; Additional file 1 : Table S13). The best combinations are DoubletFinder , Scds , ScDblFinder , Vireo and Demuxalot (refined) (<~5 donors) and Scrublet , ScDblFinder , DoubletDetection , Dropulation and Demuxalot (refined) (Fig. 5 ). These intersectional methods increase the MCC compared to the individual methods ( t -test FDR < 0.05), generally resulting in increased true singlets and doublets compared to the individual methods. The improvement in MCC depends on every single method’s performance but, on average, increases by 0.22 and up to 0.71. For experiments where the reference SNP genotypes are unknown, the individuals multiplexed in the pool with 16 or fewer individuals multiplexed, DoubletFinder , ScDblFinder, Souporcell and Vireo (<~5 donors) and Scds , ScDblFinder , DoubletDetection , Souporcell and Vireo are the ideal methods (Fig. 5 ). These intersectional methods again significantly increase the MCC up to 0.87 compared to any of the individual techniques that could be used for this experimental design ( t -test FDR < 0.05 for 94.2% of comparisons). In both cases, singlets should only be called if more than half of the methods in the combination classify the droplet as a singlet.

Combining multiple demultiplexing methods improves doublet removal for large multiplexed experimental designs

For experiments that multiplex more than 16 individuals, we considered the combinations between all demultiplexing methods (Additional file 1 : Table S14) since only a small proportion of the doublets would be undetectable by demultiplexing methods (droplets that are homogenic; Fig 3 b). To balance doublet removal and maintain true singlets, we recommend the combination of Demuxalot (refined) and Dropulation . These method combinations significantly increase the MCC by, on average, 0.09 compared to all the individual methods ( t -test FDR < 0.05). This substantially increases true singlets and true doublets relative to the individual methods. If reference SNP genotypes are not available for the individuals multiplexed in the pools, Vireo performs the best (≥ 16 multiplexed individuals; Fig. 5 ). This is the only scenario in which executing a single method is advantageous to a combination of methods. This is likely due to the fact that most of the methods perform poorly for larger pool sizes (Fig. 3 c).

These results collectively demonstrate that, regardless of the experimental design, demultiplexing and doublet detecting approaches that intersect multiple methods significantly enhance droplet classification. This is consistent across different pool sizes and will improve singlet annotation.

Demuxafy improves doublet removal and improves usability

To make our intersectional approaches accessible to other researchers, we have developed Demuxafy ( https://demultiplexing-doublet-detecting-docs.readthedocs.io/en/latest/index.html ) - an easy-to-use software platform powered by Singularity. This platform provides the requirements and instructions to execute each demultiplexing and doublet detecting methods. In addition, Demuxafy provides wrapper scripts that simplify method execution and effectively summarise results. We also offer tools that help estimate expected numbers of doublets and provide method combination recommendations based on scRNA-seq pool characteristics. Demuxafy also combines the results from multiple different methods, provides classification combination summaries, and provides final integrated combination classifications based on the intersectional techniques selected by the user. The significant advantages of Demuxafy include a centralised location to execute each of these methods, simplified ways to combine methods with an intersectional approach, and summary tables and figures that enable practical interpretation of multiplexed datasets (Fig. 1 a).

Demultiplexing and doublet detecting methods have made large-scale scRNA-seq experiments achievable. However, many demultiplexing and doublet detecting methods have been developed in the recent past, and it is unclear how their performances compare. Further, the demultiplexing techniques best detect heterogenic doublets while doublet detecting methods identify heterotypic doublets. Therefore, we hypothesised that demultiplexing and doublet detecting methods would be complementary and be more effective at removing doublets than demultiplexing methods alone.

Indeed, we demonstrated the benefit of utilising a combination of demultiplexing and doublet detecting methods. The optimal intersectional combination of methods depends on the experimental design and capture characteristics. Our results suggest super loaded captures—where a high percentage of doublets is expected—will benefit from multiplexing. Further, when many donors are multiplexed (>16), doublet detecting is not required as there are few doublets that are homogenic and heterotypic.

We have provided different method combination recommendations based on the experimental design. This decision is highly dependent on the research question.

Conclusions

Overall, our results provide researchers with important demultiplexing and doublet detecting performance assessments and combinatorial recommendations. Our software platform, Demuxafy ( https://demultiplexing-doublet-detecting-docs.readthedocs.io/en/latest/index.html ), provides a simple implementation of our methods in any research lab around the world, providing cleaner scRNA-seq datasets and enhancing interpretation of results.

PBMC scRNA-seq data

Blood samples were collected and processed as described previously [ 17 ]. Briefly, mononuclear cells were isolated from whole blood samples and stored in liquid nitrogen until thawed for scRNA-seq capture. Equal numbers of cells from 12 to 16 samples were multiplexed per pool and single-cell suspensions were super loaded on a Chromium Single Cell Chip A (10x Genomics) to capture 20,000 droplets per pool. Single-cell libraries were processed per manufacturer instructions and the 10× Genomics Cell Ranger Single Cell Software Suite (v 2.2.0) was used to process the data and map it to GRCh38. Cellbender v0.1.0 was used to identify empty droplets. Almost all droplets reported by Cell Ranger were identified to contain cells by Cellbender (mean: 99.97%). The quality control metrics of each pool are demonstrated in Additional file 2 : Fig S15.

PBMC DNA SNP genotyping

SNP genotype data were prepared as described previously [ 17 ]. Briefly, DNA was extracted from blood with the QIAamp Blood Mini kit and genotyped on the Illumina Infinium Global Screening Array. SNP genotypes were processed with Plink and GCTA before imputing on the Michigan Imputation Server using Eagle v2.3 for phasing and Minimac3 for imputation based on the Haplotype Reference Consortium panel (HRCr1.1). SNP genotypes were then lifted to hg38 and filtered for > 1% minor allele frequency (MAF) and an R 2 > 0.3.

Fibroblast scRNA-seq data

The fibroblast scRNA-seq data has been described previously [ 18 ]. Briefly, human skin punch biopsies from donors over the age of 18 were cultured in DMEM high glucose supplemented with 10% fetal bovine serum (FBS), L-glutamine, 100 U/mL penicillin and 100 μg/mL (Thermo Fisher Scientific, USA).

For scRNA-seq, viable cells were flow sorted and single cell suspensions were loaded onto a 10× Genomics Single Cell 3’ Chip and were processed per 10× instructions and the Cell Ranger Single Cell Software Suite from 10× Genomics was used to process the sequencing data into transcript count tables as previously described [ 18 ]. Cellbender v0.1.0 was used to identify empty droplets. Almost all droplets reported by Cell Ranger were identified to contain cells by Cellbender (mean: 99.65%). The quality control metrics of each pool are demonstrated in Additional file 2 : Fig S16.

Fibroblast DNA SNP genotyping

The DNA SNP genotyping for fibroblast samples has been described previously [ 18 ]. Briefly, DNA from each donor was genotyped on an Infinium HumanCore-24 v1.1 BeadChip (Illumina). GenomeStudioTM V2.0 (Illumina), Plink and GenomeStudio were used to process the SNP genotypes. Eagle V2.3.5 was used to phase the SNPs and it was imputed with the Michigan Imputation server using minimac3 and the 1000 genome phase 3 reference panel as described previously [ 18 ].

Demultiplexing methods

All the demultiplexing methods were built and run from a singularity image.

Demuxalot [ 6 ] is a genotype reference-based single cell demultiplexing method. Demualot v0.2.0 was used in python v3.8.5 to annotate droplets. The likelihoods, posterior probabilities and most likely donor for each droplet were estimated using the Demuxalot Demultiplexer.predict_posteriors function. We also used Demuxalot Demultiplexer.learn_genotypes function to refine the genotypes before estimating the likelihoods, posterior probabilities and likely donor of each droplet with the refined genotypes as well.

The Popscle v0.1-beta suite [ 16 ] for population genomics in single cell data was used for Demuxlet and Freemuxlet demultiplexing methods. The popscle dsc-pileup function was used to create a pileup of variant calls at known genomic locations from aligned sequence reads in each droplet with default arguments.

Demuxlet [ 3 ] is a SNP genotype reference-based single cell demultiplexing method. Demuxlet was run with a genotype error coefficient of 1 and genotype error offset rate of 0.05 and the other default parameters using the popscle demuxlet command from Popscle (v0.1-beta).

Freemuxlet [ 16 ] is a SNP genotype reference-free single cell demultiplexing method. Freemuxlet was run with default parameters including the number of samples included in the pool using the popscle freemuxlet command from Popscle (v0.1-beta).

Dropulation

Dropulation [ 5 ] is a SNP genotype reference-based single cell demultiplexing method that is part of the Drop-seq software. Dropulation from Drop-seq v2.5.1 was implemented for this manuscript. In addition, the method for calling singlets and doublets was provided by the Dropulation developer and implemented in a custom R script available on Github and Zenodo (see “Availability of data and materials”).

ScSplit v1.0.7 [ 7 ] was downloaded from the ScSplit github and the recommended steps for data filtering quality control prior to running ScSplit were followed. Briefly, reads that had read quality lower than 10, were unmapped, were secondary alignments, did not pass filters, were optical PCR duplicates or were duplicate reads were removed. The resulting bam file was then sorted and indexed followed by freebayes to identify single nucleotide variants (SNVs) in the dataset. The resulting SNVs were filtered for quality scores greater than 30 and for variants present in the reference SNP genotype vcf. The resulting filtered bam and vcf files were used as input for the s cSplit count command with default settings to count the number of reference and alternative alleles in each droplet. Next the allele matrices were used to demultiplex the pool and assign cells to different clusters using the scSplit run command including the number of individuals ( -n ) option and all other options set to default. Finally, the individual genotypes were predicted for each cluster using the scSplit genotype command with default parameters.

Souporcell [ 4 ] is a SNP genotype reference-free single cell demultiplexing method. The Souporcell v1.0 singularity image was downloaded via instructions from the gihtub page. The Souporcell pipeline was run using the souporcell_pipeline.py script with default options and the option to include known variant locations ( --common_variants ).

Vireo [ 2 ] is a single cell demultiplexing method that can be used with reference SNP genotypes or without them. For this assessment, Vireo was used with reference SNP genotypes. Per Vireo recommendations, we used model 1 of the cellSNP [ 20 ] version 0.3.2 to make a pileup of SNPs for each droplet with the recommended options using the genotyped reference genotype file as the list of common known SNP and filtered with SNP locations that were covered by at least 20 UMIs and had at least 10% minor allele frequency across all droplets. Vireo version 0.4.2 was then used to demultiplex using reference SNP genotypes and indicating the number of individuals in the pools.

Doublet detecting methods

All doublet detecting methods were built and run from a singularity image.

DoubletDecon

DoubletDecon [ 9 ] is a transcription-based deconvolution method for identifying doublets. DoubletDecon version 1.1.6 analysis was run in R version 3.6.3. SCTransform [ 21 ] from Seurat [ 22 ] version 3.2.2 was used to preprocess the scRNA-seq data and then the Improved_Seurat_Pre_Process function was used to process the SCTransformed scRNA-seq data. Clusters were identified using Seurat function FindClusters with resolution 0.2 and 30 principal components (PCs). Then the Main_Doublet_Decon function was used to deconvolute doublets from singlets for six different rhops—0.6, 0.7, 0.8, 0.9, 1.0 and 1.1. We used a range of rhop values since the doublet annotation by DoubletDecon is dependent on the rhop parameter which is selected by the user. The rhop that resulted in the closest number of doublets to the expected number of doublets was selected on a per-pool basis and used for all subsequent analysis. Expected number of doublets were estimated with the following equation:

where N is the number of droplets captured and D is the number of expected doublets.

DoubletDetection

DoubletDetection [ 14 ] is a transcription-based method for identifying doublets. DoubletDetection version 2.5.2 analysis was run in python version 3.6.8. Droplets without any UMIs were removed before analysis with DoubletDetection . Then the doubletdetection.BoostClassifier function was run with 50 iterations with use_phenograph set to False and standard_scaling set to True. The predicted number of doublets per iteration was visualised across all iterations and any pool that did not converge after 50 iterations, it was run again with increasing numbers of iterations until they reached convergence.

DoubletFinder

DoubletFinder [ 10 ] is a transcription-based doublet detecting method. DoubletFinder version 2.0.3 was implemented in R version 3.6.3. First, droplets that were more than 3 median absolute deviations (mad) away from the median for mitochondrial per cent, ribosomal per cent, number of UMIs or number of genes were removed per developer recommendations. Then the data was normalised with SCTransform followed by cluster identification using FindClusters with resolution 0.3 and 30 principal components (PCs). Then, pKs were selected by the pK that resulted in the largest BC MVN as recommended by DoubletFinder. The pK vs BC MVN relationship was visually inspected for each pool to ensure an effective BC MVN was selected for each pool. Finally, the homotypic doublet proportions were calculated and the number of expected doublets with the highest doublet proportion were classified as doublets per the following equation:

ScDblFinder

ScDblFinder [ 11 ] is a transcription-based method for detecting doublets from scRNA-seq data. ScDblFinder 1.3.25 was implemented in R version 4.0.3. ScDblFinder was implemented with two sets of options. The first included implementation with the expected doublet rate as calculated by:

where N is the number of droplets captured and R is the expected doublet rate. The second condition included the same expected number of doublets and included the doublets that had already been identified by all the demultiplexing methods.

Scds [ 12 ] is a transcription-based doublet detecting method. Scds version 1.1.2 analysis was completed in R version 3.6.3. Scds was implemented with the cxds function and bcds functions with default options followed by the cxds_bcds_hybrid with estNdbl set to TRUE so that doublets will be estimated based on the values from the cxds and bcds functions.

Scrublet [ 13 ] is a transcription-based doublet detecting method for single-cell RNA-seq data. Scrublet was implemented in python version 3.6.3. Scrublet was implemented per developer recommendations with at least 3 counts per droplet, 3 cells expressing a given gene, 30 PCs and a doublet rate based on the following equation:

where N is the number of droplets captured and R is the expected doublet rate. Four different minimum number of variable gene percentiles: 80, 85, 90 and 95. Then, the best variable gene percentile was selected based on the distribution of the simulated doublet scores and the location of the doublet threshold selection. In the case that the selected threshold does not fall between a bimodal distribution, those pools were run again with a manual threshold set.

Solo [ 15 ] is a transcription-based method for detecting doublets in scRNA-seq data. Solo was implemented with default parameters and an expected number of doublets based on the following equation:

where N is the number of droplets captured and D is the number of expected doublets. Solo was additionally implemented in a second run for each pool with the doublets that were identified by all the demultiplexing methods as known doublets to initialize the model.

In silico pool generation

Cells that were identified as singlets by all methods were used to simulate pools. Ten pools containing 2, 4, 8, 16, 32, 64 and 128 individuals were simulated assuming a maximum 20% doublet rate as it is unlikely researchers would use a technology that has a higher doublet rate. The donors for each simulated pool were randomly selected using a custom R script which is available on Github and Zenodo (see ‘Availability of data and materials’). A separate bam for the cell barcodes for each donor was generated using the filterbarcodes function from the sinto package (v0.8.4). Then, the GenerateSyntheticDoublets function provided by the Drop-seq [ 5 ] package was used to simulate new pools containing droplets with known singlets and doublets.

Twenty-one total pools—three pools from each of the different simulated pool sizes (2, 4, 8, 16, 32, 64 and 128 individuals) —were used to simulate different experimental scenarios that may be more challenging for demultiplexing and doublet detecting methods. These include simulating higher ambient RNA, higher mitochondrial percent, decreased read coverage and imbalanced donor proportions as described subsequently.

High ambient RNA simulations

Ambient RNA was simulated by changing the barcodes and UMIs on a random selection of reads for 10, 20 or 50% of the total UMIs. This was executed with a custom R script that is available in Github and Zenodo (see ‘Availability of data and materials’).

High mitochondrial percent simulations

High mitochondrial percent simulations were produced by replacing reads in 5, 10 or 25% of the randomly selected cells with mitochondrial reads. The number of reads to replace was derived from a normal distribution with an average of 30 and a standard deviation of 3. This was executed with a custom R script available in Github and Zenodo (see ‘Availability of data and materials’).

Imbalanced donor simulations

We simulated pools that contained uneven proportions of the donors in the pools to identify if some methods are better at demultiplexing pools containing uneven proportions of each donor in the pool. We simulated pools where 50, 75 or 95% of the pool contained cells from a single donor and the remainder of the pool was even proportions of the remaining donors in the pool. This was executed with a custom R script available in Github and Zenodo (see ‘Availability of data and materials’).

Decrease read coverage simulations

Decreased read coverage of pools was simulated by down-sampling the reads by two-thirds of the original coverage.

Classification annotation

Demultiplexing methods classifications were considered correct if the droplet annotation (singlet or doublet) and the individual annotation was correct. If the droplet type was correct but the individual annotation was incorrect (i.e. classified as a singlet but annotated as the wrong individual), then the droplet was incorrectly classified.

Doublet detecting methods were considered to have correct classifications if the droplet annotation matched the known droplet type.

All downstream analyses were completed in R version 4.0.2.

Availability of data and materials

All data used in this manuscript is publicly available. The PBMC data is available on GEO (Accession: GSE196830) [ 23 ] as originally described in [ 17 ]. The fibroblast data is available on ArrayExpress (Accession Number: E-MTAB-10060) [ 24 ] and as originally described in [ 18 ]. The code used for the analyses in this manuscript are provided on Github ( https://github.com/powellgenomicslab/Demuxafy_manuscript/tree/v4 ) and Zenodo ( https://zenodo.org/records/10813452 ) under an MIT Open Source License [ 25 , 26 ]. Demuxafy is provided as a package with source code available on Github ( https://github.com/drneavin/Demultiplexing_Doublet_Detecting_Docs ) and instructions on ReadTheDocs ( https://demultiplexing-doublet-detecting-docs.readthedocs.io/en/latest/ ) under an MIT Open Source License [ 27 ]. Demuxafy is also available on Zenodo with the link https://zenodo.org/records/10870989 [ 28 ].

Zheng GXY, Terry JM, Belgrader P, Ryvkin P, Bent ZW, Wilson R, et al. Massively parallel digital transcriptional profiling of single cells. Nat Commun. 2017;8:1–12.

Article   Google Scholar  

Huang Y, McCarthy DJ, Stegle O. Vireo: Bayesian demultiplexing of pooled single-cell RNA-seq data without genotype reference. Genome Biol. 2019;20:273.

Article   PubMed   PubMed Central   Google Scholar  

Kang HM, Subramaniam M, Targ S, Nguyen M, Maliskova L, McCarthy E, et al. Multiplexed droplet single-cell RNA-sequencing using natural genetic variation. Nat Biotechnol. 2018;36:89–94.

Article   CAS   PubMed   Google Scholar  

Heaton H, Talman AM, Knights A, Imaz M, Gaffney DJ, Durbin R, et al. Souporcell: robust clustering of single-cell RNA-seq data by genotype without reference genotypes. Nat Methods. 2020;17:615–20.

Wells MF, Nemesh J, Ghosh S, Mitchell JM, Salick MR, Mello CJ, et al. Natural variation in gene expression and viral susceptibility revealed by neural progenitor cell villages. Cell Stem Cell. 2023;30:312–332.e13.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Rogozhnikov A, Ramkumar P, Shah K, Bedi R, Kato S, Escola GS. Demuxalot: scaled up genetic demultiplexing for single-cell sequencing. bioRxiv. 2021;2021.05.22.443646.

Xu J, Falconer C, Nguyen Q, Crawford J, McKinnon BD, Mortlock S, et al. Genotype-free demultiplexing of pooled single-cell RNA-seq. Genome Biol. 2019;20:290.

What is the maximum number of cells that can be profiled?. Available from: https://kb.10xgenomics.com/hc/en-us/articles/360001378811-What-is-the-maximum-number-of-cells-that-can-be-profiled -

DePasquale EAK, Schnell DJ, Van Camp PJ, Valiente-Alandí Í, Blaxall BC, Grimes HL, et al. DoubletDecon: deconvoluting doublets from single-cell RNA-sequencing data. Cell Rep. 2019;29:1718–1727.e8.

McGinnis CS, Murrow LM, Gartner ZJ. DoubletFinder: doublet detection in single-cell RNA sequencing data using artificial nearest neighbors. Cell Syst. 2019;8:329–337.e4.

Germain P-L, Lun A, Meixide CG, Macnair W, Robinson MD. Doublet identification in single-cell sequencing data. 2022;

Bais AS, Kostka D. Scds: Computational annotation of doublets in single-cell RNA sequencing data. Bioinformatics. 2020;36:1150–8.

Wolock SL, Lopez R, Klein AM. Scrublet: computational identification of cell doublets in single-cell transcriptomic data. Cell Syst. 2019;8:281–291.e9.

Shor, Jonathan. DoubletDetection. Available from: https://github.com/JonathanShor/DoubletDetection .

Bernstein NJ, Fong NL, Lam I, Roy MA, Hendrickson DG, Kelley DR. Solo: doublet identification in single-cell RNA-Seq via semi-supervised deep learning. Cell Syst. 2020;11:95–101.e5.

popscle. Available from: https://github.com/statgen/popscle .

Yazar S, Alquicira-Hernandez J, Wing K, Senabouth A, Gordon MG, Andersen S, et al. Single-cell eQTL mapping identifies cell type–specific genetic control of autoimmune disease. Science. 2022;376:eabf3041.

Neavin D, Nguyen Q, Daniszewski MS, Liang HH, Chiu HS, Senabouth A, et al. Single cell eQTL analysis identifies cell type-specific genetic control of gene expression in fibroblasts and reprogrammed induced pluripotent stem cells. Genome Biol. 2021;1–19.

Hao Y, Hao S, Andersen-Nissen E, Mauck WM, Zheng S, Butler A, et al. Integrated analysis of multimodal single-cell data. Cell. 2021;184:3573–3587.e29.

Huang X, Huang Y. Cellsnp-lite: an efficient tool for genotyping single cells. bioRxiv. 2021;2020.12.31.424913.

Hafemeister C, Satija R. Normalization and variance stabilization of single-cell RNA-seq data using regularized negative binomial regression. bioRxiv. 2019;576827.

Stuart T, Butler A, Hoffman P, Hafemeister C, Papalexi E, Mauck WM, et al. Comprehensive integration of single-cell data. Cell. 2019;177:1888–1902.e21.

Powell JE. Single-cell eQTL mapping identifies cell type specific genetic control of autoimmune disease. Datasets. Gene Expression Omnibus. 2022. https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE196830 .

Nguyen Q, Powell JE. scRNA-seq in 79 fibroblast cell lines and 31 reprogrammed induced pluripotent stem cell lines for sceQTL analysis. Datasets. ArrayExpress. 2021. https://www.ebi.ac.uk/biostudies/arrayexpress/studies/E-MTAB-10060?query=E-MTAB-10060 .

Neavin DR. Demuxafy analyses. Github. 2024. https://github.com/powellgenomicslab/Demuxafy_manuscript/tree/v4 .

Neavin DR. Demuxafy analyses. Zenodo. 2024. https://zenodo.org/records/10813452 .

Neavin D. Demuxafy. Github. 2024. https://github.com/drneavin/Demultiplexing_Doublet_Detecting_Docs .

Neavin D. Demuxafy. Zenodo. 2024.  https://zenodo.org/records/10870989 .

McCaughey T, Liang HH, Chen C, Fenwick E, Rees G, Wong RCB, et al. An interactive multimedia approach to improving informed consent for induced pluripotent stem cell research. Cell Stem Cell. 2016;18:307–8.

Download references

Authors’ Twitter handles

Twitter handles: @drneavin (Drew Neavin), @thjimmylee (Jimmy Tsz Hang Lee), @marta_mele_m (Marta Melé)

Peer review information

Wenjing She was the primary editor of this article and managed its editorial process and peer review in collaboration with the rest of the editorial team.

Review history

The review history is available as Additional file 3 .

This work was funded by the National Health and Medical Research Council (NHMRC) Investigator grant (1175781), and funding from the Goodridge foundation. J.E.P is also supported by a fellowship from the Fok Foundation.

Author information

Authors and affiliations.

Garvan-Weizmann Centre for Cellular Genomics, Garvan Institute for Medical Research, Darlinghurst, NSW, Australia

Drew Neavin, Anne Senabouth, Himanshi Arora & Joseph E. Powell

Present address: Statewide Genomics at NSW Health Pathology, Sydney, NSW, Australia

Himanshi Arora

Wellcome Sanger Institute, Wellcome Genome Campus, Hinxton, UK

Jimmy Tsz Hang Lee

Life Sciences Department, Barcelona Supercomputing Center, Barcelona, Catalonia, Spain

Aida Ripoll-Cladellas & Marta Melé

Department of Genetics, University of Groningen, University Medical Center Groningen, Groningen, The Netherlands

Lude Franke

Spatial and Single Cell Systems Domain, Genome Institute of Singapore (GIS), Agency for Science, Technology and Research (A*STAR), Singapore, Republic of Singapore

Shyam Prabhakar

Population and Global Health, Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore, Republic of Singapore

Cancer Science Institute of Singapore, National University of Singapore, Singapore, Republic of Singapore

Bakar Institute for Computational Health Sciences, University of California, San Francisco, CA, USA

Chun Jimmie Ye

Institute for Human Genetics, University of California, San Francisco, San Francisco, CA, USA

Division of Rheumatology, Department of Medicine, University of California, San Francisco, San Francisco, CA, USA

Chan Zuckerberg Biohub, San Francisco, CA, USA

Bioinformatics and Cellular Genomics, St Vincent’s Institute of Medical Research, Fitzroy, Australia

Davis J. McCarthy

Melbourne Integrative Genomics, School of BioSciences–School of Mathematics & Statistics, Faculty of Science, University of Melbourne, Melbourne, Australia

Present address: The Gene Lay Institute of Immunology and Inflammation, Brigham and Women’s Hospital and Harvard Medical School, Boston, MA, USA

Martin Hemberg

UNSW Cellular Genomics Futures Institute, University of New South Wales, Kensington, NSW, Australia

Joseph E. Powell

You can also search for this author in PubMed   Google Scholar

sc-eQTLGen Consortium

Contributions.

DRN and JEP conceived the project idea and study design. JTHL, AR, LF, SP, CJY, DJM, MM and MH provided feedback on experimental design. DRN carried out analyses with support on coding from AS. JTHL and AR tested Demuxafy and provided feedback. DRN and JEP wrote the manuscript. All authors reviewed and provided feedback on the manuscript.

Corresponding authors

Correspondence to Drew Neavin or Joseph E. Powell .

Ethics declarations

Ethics approval and consent to participate.

Briefly, all work was approved by the Royal Hobart Hospital, the Hobart Eye Surgeons Clinic, Human Research Ethics Committees of the Royal Victorian Eye and Ear Hospital (11/1031), University of Melbourne (1545394) and University of Tasmania (H0014124) in accordance with the requirements of the National Health & Medical Research Council of Australia (NHMRC) and conformed with the Declaration of Helsinki [ 29 ].

Consent for publication

No personal data for any individual requiring consent for publication was included in this manuscript.

Competing interests

C.J.Y. is founder for and holds equity in DropPrint Genomics (now ImmunAI) and Survey Genomics, a Scientific Advisory Board member for and holds equity in Related Sciences and ImmunAI, a consultant for and holds equity in Maze Therapeutics, and a consultant for TReX Bio, HiBio, ImYoo, and Santa Ana. Additionally, C.J.Y is also newly an Innovation Investigator for the Arc Institute. C.J.Y. has received research support from Chan Zuckerberg Initiative, Chan Zuckerberg Biohub, Genentech, BioLegend, ScaleBio and Illumina.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1: supplementary tables and legends., additional file 2: supplementary figures and legends., additional file 3..

Review history.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Neavin, D., Senabouth, A., Arora, H. et al. Demuxafy : improvement in droplet assignment by integrating multiple single-cell demultiplexing and doublet detection methods. Genome Biol 25 , 94 (2024). https://doi.org/10.1186/s13059-024-03224-8

Download citation

Received : 07 March 2023

Accepted : 25 March 2024

Published : 15 April 2024

DOI : https://doi.org/10.1186/s13059-024-03224-8

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Single-cell analysis
• Genetic demultiplexing
• Doublet detecting

Genome Biology

ISSN: 1474-760X

• Submission enquiries: [email protected]
• General enquiries: [email protected]