review of related literature in educational research

• Library databases
• Library website

Education Literature Review: Education Literature Review

What does this guide cover.

Writing the literature review is a long, complex process that requires you to use many different tools, resources, and skills.

This page provides links to the guides, tutorials, and webinars that can help you with all aspects of completing your literature review.

The Basic Process

These resources provide overviews of the entire literature review process. Start here if you are new to the literature review process.

• Literature Reviews Overview : Writing Center
• How to do a Literature Review : Library
• Video: Common Errors Made When Conducting a Lit Review (YouTube)  

The Role of the Literature Review

Your literature review gives your readers an understanding of the evolution of scholarly research on your topic.

In your literature review you will:

• survey the scholarly landscape
• provide a synthesis of the issues, trends, and concepts
• possibly provide some historical background

Review the literature in two ways:

• Section 1: reviews the literature for the Problem
• Section 3: reviews the literature for the Project

The literature review is NOT an annotated bibliography. Nor should it simply summarize the articles you've read. Literature reviews are organized thematically and demonstrate synthesis of the literature.

For more information, view the Library's short video on searching by themes:

Short Video: Research for the Literature Review

(4 min 10 sec) Recorded August 2019 Transcript 

Search for Literature

The iterative process of research:

• Find an article.
• Read the article and build new searches using keywords and names from the article.
• Mine the bibliography for other works.
• Use “cited by” searches to find more recent works that reference the article.
• Repeat steps 2-4 with the new articles you find.

These are the main skills and resources you will need in order to effectively search for literature on your topic:

• Subject Research: Education by Jon Allinder Last Updated Aug 7, 2023 3726 views this year
• Keyword Searching: Finding Articles on Your Topic by Lynn VanLeer Last Updated Sep 12, 2023 18405 views this year
• Google Scholar by Jon Allinder Last Updated Aug 16, 2023 12315 views this year
• Quick Answer: How do I find books and articles that cite an article I already have?
• Quick Answer: How do I find a measurement, test, survey or instrument?

Video: Education Databases and Doctoral Research Resources

(6 min 04 sec) Recorded April 2019 Transcript 

Staying Organized

The literature review requires organizing a variety of information. The following resources will help you develop the organizational systems you'll need to be successful.

• Organize your research
• Citation Management Software

You can make your search log as simple or complex as you would like.  It can be a table in a word document or an excel spread sheet.  Here are two examples.  The word document is a basic table where you can keep track of databases, search terms, limiters, results and comments.  The Excel sheet is more complex and has additional sheets for notes, Google Scholar log; Journal Log, and Questions to ask the Librarian.  

• Search Log Example Sample search log in Excel
• Search Log Example Sample search log set up as a table in a word document.
• Literature Review Matrix with color coding Sample template for organizing and synthesizing your research

Writing the Literature Review

The following resources created by the Writing Center and the Academic Skills Center support the writing process for the dissertation/project study. 

• Critical Reading
• What is Synthesis 
• Walden Templates
• Quick Answer: How do I find Walden EdD (Doctor of Education) studies?
• Quick Answer: How do I find Walden PhD dissertations?

Beyond the Literature Review

The literature review isn't the only portion of a dissertation/project study that requires searching. The following resources can help you identify and utilize a theory, methodology, measurement instruments, or statistics.

• Education Theory by Jon Allinder Last Updated May 17, 2024 413 views this year
• Tests & Measures in Education by Kimberly Burton Last Updated Nov 18, 2021 43 views this year
• Education Statistics by Jon Allinder Last Updated Feb 22, 2022 58 views this year
• Office of Research and Doctoral Services

Books and Articles about the Lit Review

The following articles and books outline the purpose of the literature review and offer advice for successfully completing one.

• Chen, D. T. V., Wang, Y. M., & Lee, W. C. (2016). Challenges confronting beginning researchers in conducting literature reviews. Studies in Continuing Education, 38(1), 47-60. https://doi.org/10.1080/0158037X.2015.1030335 Proposes a framework to conceptualize four types of challenges students face: linguistic, methodological, conceptual, and ontological.
• Randolph, J.J. (2009). A guide to writing the dissertation literature review. Practical Assessment, Research & Evaluation 14(13), 1-13. Provides advice for writing a quantitative or qualitative literature review, by a Walden faculty member.
• Torraco, R. J. (2016). Writing integrative literature reviews: Using the past and present to explore the future. Human Resource Development Review, 15(4), 404–428. https://doi.org/10.1177/1534484316671606 This article presents the integrative review of literature as a distinctive form of research that uses existing literature to create new knowledge.
• Wee, B. V., & Banister, D. (2016). How to write a literature review paper?. Transport Reviews, 36(2), 278-288. http://doi.org/10.1080/01441647.2015.1065456 Discusses how to write a literature review with a focus on adding value rather and suggests structural and contextual aspects found in outstanding literature reviews.
• Winchester, C. L., & Salji, M. (2016). Writing a literature review. Journal of Clinical Urology, 9(5), 308-312. https://doi.org/10.1177/2051415816650133 Reviews the use of different document types to add structure and enrich your literature review and the skill sets needed in writing the literature review.
• Xiao, Y., & Watson, M. (2017). Guidance on conducting a systematic literature review. Journal of Planning Education and Research. https://doi.org/10.1177/0739456X17723971 Examines different types of literature reviews and the steps necessary to produce a systematic review in educational research.

• Office of Student Disability Services

Walden Resources

Departments.

• Academic Residencies
• Academic Skills
• Career Planning and Development
• Customer Care Team
• Field Experience
• Military Services
• Student Success Advising
• Writing Skills

Centers and Offices

• Center for Social Change
• Office of Academic Support and Instructional Services
• Office of Degree Acceleration
• Office of Student Affairs

Student Resources

• Doctoral Writing Assessment
• Form & Style Review
• Quick Answers
• ScholarWorks
• SKIL Courses and Workshops
• Walden Bookstore
• Walden Catalog & Student Handbook
• Student Safety/Title IX
• Legal & Consumer Information
• Website Terms and Conditions
• Cookie Policy
• Accessibility
• Accreditation
• State Authorization
• Net Price Calculator
• Contact Walden

Walden University is a member of Adtalem Global Education, Inc. www.adtalem.com Walden University is certified to operate by SCHEV © 2024 Walden University LLC. All rights reserved.

• Library Search
• View Library Account
• The Claremont Colleges Library
• Research Guides
• Conducting a Literature Review
• Getting Started With Research
• Finding Scholarly Articles
• Finding Books & Papers
• Finding Education Data & Testing Resources

What is a Literature Review?

Basics of a literature review, types of literature reviews.

• Citing Your Information (Attribution)

A Literature Review is a systematic and comprehensive analysis of books, scholarly articles and other sources relevant to a specific topic providing a base of knowledge on a topic. Literature reviews are designed to identify and critique the existing literature on a topic to justify your research by exposing gaps in current research .  This investigation should provide a description, summary, and critical evaluation of works related to the research problem and should also add to the overall knowledge of the topic as well as demonstrating how your research will fit within a larger field of study.  A literature review should offer critical analysis of the current research on a topic and that analysis should direct your research objective. This should not be confused with a book review or an annotated bibliography both research tools but very different in purpose and scope.  A Literature Review can be a stand alone element or part of a larger end product, know your assignment.  Key to a good Literature Review is to document your process. For more information see:

Planning a Literature Review .

There are many different ways to organize your references in a literature review, but most reviews contain certain basic elements.

• Objective of the literature review - Clearly describe the purpose of the paper and state your objectives in completing the literature review.
•   Overview of the subject, issue or theory under consideration – Give an overview of your research topic and what prompted it.
• Categorization of sources – Grouping your research either historic, chronologically or thematically
• Organization of Subtopics – Subtopics should be grouped and presented in a logical order starting with the most prominent or significant and moving to the least significant
• Discussion – Provide analysis of both the uniqueness of each source and its similarities with other source
• Conclusion   - Summary of your analysis and evaluation of the reviewed works and how it is related to its parent discipline, scientific endeavor or profession

As Kennedy (2007) notes*, it is important to think of knowledge in a given field as consisting of three layers. First, there are the primary studies that researchers conduct and publish. Second are the reviews of those studies that summarize and offer new interpretations built from and often extending beyond the original studies. Third, there are the perceptions, conclusions, opinion, and interpretations that are shared informally that become part of the lore of field. In composing a literature review, it is important to note that it is often this third layer of knowledge that is cited as "true" even though it often has only a loose relationship to the primary studies and secondary literature reviews.

Given this, while literature reviews are designed to provide an overview and synthesis of pertinent sources you have explored, there are several approaches to how they can be done, depending upon the type of analysis underpinning your study. Listed below are definitions of types of literature reviews:

Argumentative Review      This form examines literature selectively in order to support or refute an argument, deeply imbedded assumption, or philosophical problem already established in the literature. The purpose is to develop a body of literature that establishes a contrarian viewpoint. Given the value-laden nature of some social science research [e.g., educational reform; immigration control], argumentative approaches to analyzing the literature can be a legitimate and important form of discourse. However, note that they can also introduce problems of bias when they are used to to make summary claims of the sort found in systematic reviews.

Integrative Review      Considered a form of research that reviews, critiques, and synthesizes representative literature on a topic in an integrated way such that new frameworks and perspectives on the topic are generated. The body of literature includes all studies that address related or identical hypotheses. A well-done integrative review meets the same standards as primary research in regard to clarity, rigor, and replication.

Historical Review      Few things rest in isolation from historical precedent. Historical reviews are focused on examining research throughout a period of time, often starting with the first time an issue, concept, theory, phenomena emerged in the literature, then tracing its evolution within the scholarship of a discipline. The purpose is to place research in a historical context to show familiarity with state-of-the-art developments and to identify the likely directions for future research.

Methodological Review      A review does not always focus on what someone said [content], but how they said it [method of analysis]. This approach provides a framework of understanding at different levels (i.e. those of theory, substantive fields, research approaches and data collection and analysis techniques), enables researchers to draw on a wide variety of knowledge ranging from the conceptual level to practical documents for use in fieldwork in the areas of ontological and epistemological consideration, quantitative and qualitative integration, sampling, interviewing, data collection and data analysis, and helps highlight many ethical issues which we should be aware of and consider as we go through our study.

Systematic Review      This form consists of an overview of existing evidence pertinent to a clearly formulated research question, which uses pre-specified and standardized methods to identify and critically appraise relevant research, and to collect, report, and analyse data from the studies that are included in the review. Typically it focuses on a very specific empirical question, often posed in a cause-and-effect form, such as "To what extent does A contribute to B?"

Theoretical Review      The purpose of this form is to concretely examine the corpus of theory that has accumulated in regard to an issue, concept, theory, phenomena. The theoretical literature review help establish what theories already exist, the relationships between them, to what degree the existing theories have been investigated, and to develop new hypotheses to be tested. Often this form is used to help establish a lack of appropriate theories or reveal that current theories are inadequate for explaining new or emerging research problems. The unit of analysis can focus on a theoretical concept or a whole theory or framework.

* Kennedy, Mary M. "Defining a Literature." Educational Researcher 36 (April 2007): 139-147.

All content is from The Literature Review created by Dr. Robert Larabee USC

• << Previous: Finding Education Data & Testing Resources
• Next: Citing Your Information (Attribution) >>
• : Aug 31, 2023 2:11 PM
• URL: https://libguides.libraries.claremont.edu/education

• USC Libraries
• Research Guides

Organizing Your Social Sciences Research Paper

• 5. The Literature Review
• Purpose of Guide
• Design Flaws to Avoid
• Independent and Dependent Variables
• Glossary of Research Terms
• Reading Research Effectively
• Narrowing a Topic Idea
• Broadening a Topic Idea
• Extending the Timeliness of a Topic Idea
• Academic Writing Style
• Applying Critical Thinking
• Choosing a Title
• Making an Outline
• Paragraph Development
• Research Process Video Series
• Executive Summary
• The C.A.R.S. Model
• Background Information
• The Research Problem/Question
• Theoretical Framework
• Citation Tracking
• Content Alert Services
• Evaluating Sources
• Primary Sources
• Secondary Sources
• Tiertiary Sources
• Scholarly vs. Popular Publications
• Qualitative Methods
• Quantitative Methods
• Insiderness
• Using Non-Textual Elements
• Limitations of the Study
• Common Grammar Mistakes
• Writing Concisely
• Avoiding Plagiarism
• Footnotes or Endnotes?
• Further Readings
• Generative AI and Writing
• USC Libraries Tutorials and Other Guides
• Bibliography

A literature review surveys prior research published in books, scholarly articles, and any other sources relevant to a particular issue, area of research, or theory, and by so doing, provides a description, summary, and critical evaluation of these works in relation to the research problem being investigated. Literature reviews are designed to provide an overview of sources you have used in researching a particular topic and to demonstrate to your readers how your research fits within existing scholarship about the topic.

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . Fourth edition. Thousand Oaks, CA: SAGE, 2014.

Importance of a Good Literature Review

A literature review may consist of simply a summary of key sources, but in the social sciences, a literature review usually has an organizational pattern and combines both summary and synthesis, often within specific conceptual categories . A summary is a recap of the important information of the source, but a synthesis is a re-organization, or a reshuffling, of that information in a way that informs how you are planning to investigate a research problem. The analytical features of a literature review might:

• Give a new interpretation of old material or combine new with old interpretations,
• Trace the intellectual progression of the field, including major debates,
• Depending on the situation, evaluate the sources and advise the reader on the most pertinent or relevant research, or
• Usually in the conclusion of a literature review, identify where gaps exist in how a problem has been researched to date.

Given this, the purpose of a literature review is to:

• Place each work in the context of its contribution to understanding the research problem being studied.
• Describe the relationship of each work to the others under consideration.
• Identify new ways to interpret prior research.
• Reveal any gaps that exist in the literature.
• Resolve conflicts amongst seemingly contradictory previous studies.
• Identify areas of prior scholarship to prevent duplication of effort.
• Point the way in fulfilling a need for additional research.
• Locate your own research within the context of existing literature [very important].

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper. 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . Los Angeles, CA: SAGE, 2011; Knopf, Jeffrey W. "Doing a Literature Review." PS: Political Science and Politics 39 (January 2006): 127-132; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012.

Types of Literature Reviews

It is important to think of knowledge in a given field as consisting of three layers. First, there are the primary studies that researchers conduct and publish. Second are the reviews of those studies that summarize and offer new interpretations built from and often extending beyond the primary studies. Third, there are the perceptions, conclusions, opinion, and interpretations that are shared informally among scholars that become part of the body of epistemological traditions within the field.

In composing a literature review, it is important to note that it is often this third layer of knowledge that is cited as "true" even though it often has only a loose relationship to the primary studies and secondary literature reviews. Given this, while literature reviews are designed to provide an overview and synthesis of pertinent sources you have explored, there are a number of approaches you could adopt depending upon the type of analysis underpinning your study.

Argumentative Review This form examines literature selectively in order to support or refute an argument, deeply embedded assumption, or philosophical problem already established in the literature. The purpose is to develop a body of literature that establishes a contrarian viewpoint. Given the value-laden nature of some social science research [e.g., educational reform; immigration control], argumentative approaches to analyzing the literature can be a legitimate and important form of discourse. However, note that they can also introduce problems of bias when they are used to make summary claims of the sort found in systematic reviews [see below].

Integrative Review Considered a form of research that reviews, critiques, and synthesizes representative literature on a topic in an integrated way such that new frameworks and perspectives on the topic are generated. The body of literature includes all studies that address related or identical hypotheses or research problems. A well-done integrative review meets the same standards as primary research in regard to clarity, rigor, and replication. This is the most common form of review in the social sciences.

Historical Review Few things rest in isolation from historical precedent. Historical literature reviews focus on examining research throughout a period of time, often starting with the first time an issue, concept, theory, phenomena emerged in the literature, then tracing its evolution within the scholarship of a discipline. The purpose is to place research in a historical context to show familiarity with state-of-the-art developments and to identify the likely directions for future research.

Methodological Review A review does not always focus on what someone said [findings], but how they came about saying what they say [method of analysis]. Reviewing methods of analysis provides a framework of understanding at different levels [i.e. those of theory, substantive fields, research approaches, and data collection and analysis techniques], how researchers draw upon a wide variety of knowledge ranging from the conceptual level to practical documents for use in fieldwork in the areas of ontological and epistemological consideration, quantitative and qualitative integration, sampling, interviewing, data collection, and data analysis. This approach helps highlight ethical issues which you should be aware of and consider as you go through your own study.

Systematic Review This form consists of an overview of existing evidence pertinent to a clearly formulated research question, which uses pre-specified and standardized methods to identify and critically appraise relevant research, and to collect, report, and analyze data from the studies that are included in the review. The goal is to deliberately document, critically evaluate, and summarize scientifically all of the research about a clearly defined research problem . Typically it focuses on a very specific empirical question, often posed in a cause-and-effect form, such as "To what extent does A contribute to B?" This type of literature review is primarily applied to examining prior research studies in clinical medicine and allied health fields, but it is increasingly being used in the social sciences.

Theoretical Review The purpose of this form is to examine the corpus of theory that has accumulated in regard to an issue, concept, theory, phenomena. The theoretical literature review helps to establish what theories already exist, the relationships between them, to what degree the existing theories have been investigated, and to develop new hypotheses to be tested. Often this form is used to help establish a lack of appropriate theories or reveal that current theories are inadequate for explaining new or emerging research problems. The unit of analysis can focus on a theoretical concept or a whole theory or framework.

NOTE : Most often the literature review will incorporate some combination of types. For example, a review that examines literature supporting or refuting an argument, assumption, or philosophical problem related to the research problem will also need to include writing supported by sources that establish the history of these arguments in the literature.

Baumeister, Roy F. and Mark R. Leary. "Writing Narrative Literature Reviews."  Review of General Psychology 1 (September 1997): 311-320; Mark R. Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Kennedy, Mary M. "Defining a Literature." Educational Researcher 36 (April 2007): 139-147; Petticrew, Mark and Helen Roberts. Systematic Reviews in the Social Sciences: A Practical Guide . Malden, MA: Blackwell Publishers, 2006; Torracro, Richard. "Writing Integrative Literature Reviews: Guidelines and Examples." Human Resource Development Review 4 (September 2005): 356-367; Rocco, Tonette S. and Maria S. Plakhotnik. "Literature Reviews, Conceptual Frameworks, and Theoretical Frameworks: Terms, Functions, and Distinctions." Human Ressource Development Review 8 (March 2008): 120-130; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.

Structure and Writing Style

I.  Thinking About Your Literature Review

The structure of a literature review should include the following in support of understanding the research problem :

• An overview of the subject, issue, or theory under consideration, along with the objectives of the literature review,
• Division of works under review into themes or categories [e.g. works that support a particular position, those against, and those offering alternative approaches entirely],
• An explanation of how each work is similar to and how it varies from the others,
• Conclusions as to which pieces are best considered in their argument, are most convincing of their opinions, and make the greatest contribution to the understanding and development of their area of research.

The critical evaluation of each work should consider :

• Provenance -- what are the author's credentials? Are the author's arguments supported by evidence [e.g. primary historical material, case studies, narratives, statistics, recent scientific findings]?
• Methodology -- were the techniques used to identify, gather, and analyze the data appropriate to addressing the research problem? Was the sample size appropriate? Were the results effectively interpreted and reported?
• Objectivity -- is the author's perspective even-handed or prejudicial? Is contrary data considered or is certain pertinent information ignored to prove the author's point?
• Persuasiveness -- which of the author's theses are most convincing or least convincing?
• Validity -- are the author's arguments and conclusions convincing? Does the work ultimately contribute in any significant way to an understanding of the subject?

II.  Development of the Literature Review

Four Basic Stages of Writing 1.  Problem formulation -- which topic or field is being examined and what are its component issues? 2.  Literature search -- finding materials relevant to the subject being explored. 3.  Data evaluation -- determining which literature makes a significant contribution to the understanding of the topic. 4.  Analysis and interpretation -- discussing the findings and conclusions of pertinent literature.

Consider the following issues before writing the literature review: Clarify If your assignment is not specific about what form your literature review should take, seek clarification from your professor by asking these questions: 1.  Roughly how many sources would be appropriate to include? 2.  What types of sources should I review (books, journal articles, websites; scholarly versus popular sources)? 3.  Should I summarize, synthesize, or critique sources by discussing a common theme or issue? 4.  Should I evaluate the sources in any way beyond evaluating how they relate to understanding the research problem? 5.  Should I provide subheadings and other background information, such as definitions and/or a history? Find Models Use the exercise of reviewing the literature to examine how authors in your discipline or area of interest have composed their literature review sections. Read them to get a sense of the types of themes you might want to look for in your own research or to identify ways to organize your final review. The bibliography or reference section of sources you've already read, such as required readings in the course syllabus, are also excellent entry points into your own research. Narrow the Topic The narrower your topic, the easier it will be to limit the number of sources you need to read in order to obtain a good survey of relevant resources. Your professor will probably not expect you to read everything that's available about the topic, but you'll make the act of reviewing easier if you first limit scope of the research problem. A good strategy is to begin by searching the USC Libraries Catalog for recent books about the topic and review the table of contents for chapters that focuses on specific issues. You can also review the indexes of books to find references to specific issues that can serve as the focus of your research. For example, a book surveying the history of the Israeli-Palestinian conflict may include a chapter on the role Egypt has played in mediating the conflict, or look in the index for the pages where Egypt is mentioned in the text. Consider Whether Your Sources are Current Some disciplines require that you use information that is as current as possible. This is particularly true in disciplines in medicine and the sciences where research conducted becomes obsolete very quickly as new discoveries are made. However, when writing a review in the social sciences, a survey of the history of the literature may be required. In other words, a complete understanding the research problem requires you to deliberately examine how knowledge and perspectives have changed over time. Sort through other current bibliographies or literature reviews in the field to get a sense of what your discipline expects. You can also use this method to explore what is considered by scholars to be a "hot topic" and what is not.

III.  Ways to Organize Your Literature Review

Chronology of Events If your review follows the chronological method, you could write about the materials according to when they were published. This approach should only be followed if a clear path of research building on previous research can be identified and that these trends follow a clear chronological order of development. For example, a literature review that focuses on continuing research about the emergence of German economic power after the fall of the Soviet Union. By Publication Order your sources by publication chronology, then, only if the order demonstrates a more important trend. For instance, you could order a review of literature on environmental studies of brown fields if the progression revealed, for example, a change in the soil collection practices of the researchers who wrote and/or conducted the studies. Thematic [“conceptual categories”] A thematic literature review is the most common approach to summarizing prior research in the social and behavioral sciences. Thematic reviews are organized around a topic or issue, rather than the progression of time, although the progression of time may still be incorporated into a thematic review. For example, a review of the Internet’s impact on American presidential politics could focus on the development of online political satire. While the study focuses on one topic, the Internet’s impact on American presidential politics, it would still be organized chronologically reflecting technological developments in media. The difference in this example between a "chronological" and a "thematic" approach is what is emphasized the most: themes related to the role of the Internet in presidential politics. Note that more authentic thematic reviews tend to break away from chronological order. A review organized in this manner would shift between time periods within each section according to the point being made. Methodological A methodological approach focuses on the methods utilized by the researcher. For the Internet in American presidential politics project, one methodological approach would be to look at cultural differences between the portrayal of American presidents on American, British, and French websites. Or the review might focus on the fundraising impact of the Internet on a particular political party. A methodological scope will influence either the types of documents in the review or the way in which these documents are discussed.

Other Sections of Your Literature Review Once you've decided on the organizational method for your literature review, the sections you need to include in the paper should be easy to figure out because they arise from your organizational strategy. In other words, a chronological review would have subsections for each vital time period; a thematic review would have subtopics based upon factors that relate to the theme or issue. However, sometimes you may need to add additional sections that are necessary for your study, but do not fit in the organizational strategy of the body. What other sections you include in the body is up to you. However, only include what is necessary for the reader to locate your study within the larger scholarship about the research problem.

Here are examples of other sections, usually in the form of a single paragraph, you may need to include depending on the type of review you write:

• Current Situation : Information necessary to understand the current topic or focus of the literature review.
• Sources Used : Describes the methods and resources [e.g., databases] you used to identify the literature you reviewed.
• History : The chronological progression of the field, the research literature, or an idea that is necessary to understand the literature review, if the body of the literature review is not already a chronology.
• Selection Methods : Criteria you used to select (and perhaps exclude) sources in your literature review. For instance, you might explain that your review includes only peer-reviewed [i.e., scholarly] sources.
• Standards : Description of the way in which you present your information.
• Questions for Further Research : What questions about the field has the review sparked? How will you further your research as a result of the review?

IV.  Writing Your Literature Review

Once you've settled on how to organize your literature review, you're ready to write each section. When writing your review, keep in mind these issues.

Use Evidence A literature review section is, in this sense, just like any other academic research paper. Your interpretation of the available sources must be backed up with evidence [citations] that demonstrates that what you are saying is valid. Be Selective Select only the most important points in each source to highlight in the review. The type of information you choose to mention should relate directly to the research problem, whether it is thematic, methodological, or chronological. Related items that provide additional information, but that are not key to understanding the research problem, can be included in a list of further readings . Use Quotes Sparingly Some short quotes are appropriate if you want to emphasize a point, or if what an author stated cannot be easily paraphrased. Sometimes you may need to quote certain terminology that was coined by the author, is not common knowledge, or taken directly from the study. Do not use extensive quotes as a substitute for using your own words in reviewing the literature. Summarize and Synthesize Remember to summarize and synthesize your sources within each thematic paragraph as well as throughout the review. Recapitulate important features of a research study, but then synthesize it by rephrasing the study's significance and relating it to your own work and the work of others. Keep Your Own Voice While the literature review presents others' ideas, your voice [the writer's] should remain front and center. For example, weave references to other sources into what you are writing but maintain your own voice by starting and ending the paragraph with your own ideas and wording. Use Caution When Paraphrasing When paraphrasing a source that is not your own, be sure to represent the author's information or opinions accurately and in your own words. Even when paraphrasing an author’s work, you still must provide a citation to that work.

V.  Common Mistakes to Avoid

These are the most common mistakes made in reviewing social science research literature.

• Sources in your literature review do not clearly relate to the research problem;
• You do not take sufficient time to define and identify the most relevant sources to use in the literature review related to the research problem;
• Relies exclusively on secondary analytical sources rather than including relevant primary research studies or data;
• Uncritically accepts another researcher's findings and interpretations as valid, rather than examining critically all aspects of the research design and analysis;
• Does not describe the search procedures that were used in identifying the literature to review;
• Reports isolated statistical results rather than synthesizing them in chi-squared or meta-analytic methods; and,
• Only includes research that validates assumptions and does not consider contrary findings and alternative interpretations found in the literature.

Cook, Kathleen E. and Elise Murowchick. “Do Literature Review Skills Transfer from One Course to Another?” Psychology Learning and Teaching 13 (March 2014): 3-11; Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . London: SAGE, 2011; Literature Review Handout. Online Writing Center. Liberty University; Literature Reviews. The Writing Center. University of North Carolina; Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: SAGE, 2016; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012; Randolph, Justus J. “A Guide to Writing the Dissertation Literature Review." Practical Assessment, Research, and Evaluation. vol. 14, June 2009; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016; Taylor, Dena. The Literature Review: A Few Tips On Conducting It. University College Writing Centre. University of Toronto; Writing a Literature Review. Academic Skills Centre. University of Canberra.

Writing Tip

Break Out of Your Disciplinary Box!

Thinking interdisciplinarily about a research problem can be a rewarding exercise in applying new ideas, theories, or concepts to an old problem. For example, what might cultural anthropologists say about the continuing conflict in the Middle East? In what ways might geographers view the need for better distribution of social service agencies in large cities than how social workers might study the issue? You don’t want to substitute a thorough review of core research literature in your discipline for studies conducted in other fields of study. However, particularly in the social sciences, thinking about research problems from multiple vectors is a key strategy for finding new solutions to a problem or gaining a new perspective. Consult with a librarian about identifying research databases in other disciplines; almost every field of study has at least one comprehensive database devoted to indexing its research literature.

Frodeman, Robert. The Oxford Handbook of Interdisciplinarity . New York: Oxford University Press, 2010.

Another Writing Tip

Don't Just Review for Content!

While conducting a review of the literature, maximize the time you devote to writing this part of your paper by thinking broadly about what you should be looking for and evaluating. Review not just what scholars are saying, but how are they saying it. Some questions to ask:

• How are they organizing their ideas?
• What methods have they used to study the problem?
• What theories have been used to explain, predict, or understand their research problem?
• What sources have they cited to support their conclusions?
• How have they used non-textual elements [e.g., charts, graphs, figures, etc.] to illustrate key points?

When you begin to write your literature review section, you'll be glad you dug deeper into how the research was designed and constructed because it establishes a means for developing more substantial analysis and interpretation of the research problem.

Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1 998.

Yet Another Writing Tip

When Do I Know I Can Stop Looking and Move On?

Here are several strategies you can utilize to assess whether you've thoroughly reviewed the literature:

• Look for repeating patterns in the research findings . If the same thing is being said, just by different people, then this likely demonstrates that the research problem has hit a conceptual dead end. At this point consider: Does your study extend current research?  Does it forge a new path? Or, does is merely add more of the same thing being said?
• Look at sources the authors cite to in their work . If you begin to see the same researchers cited again and again, then this is often an indication that no new ideas have been generated to address the research problem.
• Search Google Scholar to identify who has subsequently cited leading scholars already identified in your literature review [see next sub-tab]. This is called citation tracking and there are a number of sources that can help you identify who has cited whom, particularly scholars from outside of your discipline. Here again, if the same authors are being cited again and again, this may indicate no new literature has been written on the topic.

Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: Sage, 2016; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.

• << Previous: Theoretical Framework
• Next: Citation Tracking >>
• Last Updated: May 20, 2024 9:47 AM
• URL: https://libguides.usc.edu/writingguide

• Research Guides
• CECH Library

Education Basics

Literature review overview.

• Article and Media Sources
• Quick Stats and Reference
• OAE and Praxis Core
• Citation & Annotation

There are eight general steps in conducting an education literature review. Please follow the eight numbered boxes, starting below.

Please note that the general framework for this guide is derived from the work of Joyce P. Gall, M.D. Gall, and Walter R. Borg in Applying Educational Research: a Practical Guide (5th ed., 2005). Also, much of the information on framing the research question comes from Emily Grimm's Selected Reference Sources for Graduate Students in Education and Education Related Areas (1995).

Step 1: Frame Your Research Question(s)

Basic Questions

• What do I want to know?  For what purpose? Consider subject terms, synonyms, related concepts and approaches.
• What do I know already?
• Who else might have performed similar research and why? Consider individuals, institutions, governmental agencies and other groups.
• What summarizing or descriptive information is already available? Consider the secondary sources found below.

Time Questions

• For which time span(s) do I need information?
• Would recurrent or temporal events in education affect my research?  For example: school terms, budget hearings, conference proceedings, legislative sessions, policy decisions, elections, administrative procedural changes.

Limitation(s) Questions

• Do I have other limitations?  For example:  language, age group, grade level, type of student, type of school, type of district, geography, curricular area, or style of teaching.

Aspect Questions

• What aspects of education interest me?  For example:  financial, administrative, teaching, legislative, gender, parental, theoretical, research, developmental, practical or other.

Subjective Aspect Questions

• What are my values, prejudices, biases, and areas of ignorance in regard to my research question(s)?
• Will I let these prejudices limit my research?
• Will I let these prejudices influence my note taking, choice of vocabulary and indexing terms, selection of data, evaluations of the work of other researchers, inclusion of conflicting theories, reporting of data, or my conclusions?

Step 2: Contact Experts to Get Answers or for Guidance to Relevant Publications

Consider consulting other educators, faculty or government officials who may specialize in your research area.

You may also want to consult the American Educational Research Association SIG (Special Interest Group) website for the names of groups and individuals who have expertise in different educational areas.  AERA provides the names, addresses, e-mail addresses, and phone numbers of individuals doing research in a variety of areas.

Step 3: Read Secondary Sources to Gain a Broad Overview of the Literature Related to Your Research Area

Use secondary sources to further define your research question and to expand your literature search.  Secondary sources include encyclopedias, handbooks, dictionaries, and thesauri. Secondary sources are resources that review research that others have done.  They provide a general overview, will give you ideas for key search terms, and often include useful bibliographies for further reading.

Here are some key secondary sources and books on doing educational research:

• Review of Educational Research The Review of Educational Research (RER) publishes critical, integrative reviews of research literature bearing on education, including conceptualizations, interpretations, and syntheses of literature and scholarly work in a field broadly relevant to education and educational research.
• Educational Psychology Review Educational Psychology Review is an international forum for the publication of peer-reviewed integrative review articles, special thematic issues, reflections or comments on previous research or new research directions, interviews, and research-based advice for practitioners.
• Doing educational research : a guide to first-time researchers CECH Prof Ed LB1028 .D65 2004
• Effective action research: developing reflective thinking and practice Electronic (2011)
• Encyclopedia of Education Electronic and Langsam Library Reference, LB 15 .E47 2003
• Encyclopedia of Special Education [electronic resource] : a Reference for the Education of Children, Adolescents, and Adults with Disabilities and other Exceptional Individuals Electronic, 2007.
• Handbook of research on educational communications and technology CECH Library Reference, LB 1028.3 . H355 2008
• Handbook of research on multicultural education CECH Library Reference, LC 1099.3 .H35 2004
• Handbook of research on teaching CECH Library Reference, LB1028 .S39 2001
• How to design and evaluate research in education CECH Reserves LB1028 .F665 2012
• Methods in educational research: from theory to practice Electronic (2010)
• The Phi Delta Kappan [electronic resource] Electronic, Contains many articles that cite research and analyze practical implications.
• The Routledge International Encyclopedia of Education CECH Library Reference, LB 15 .R633 2008

Step 4: Select Preliminary Sources that Index Relevant Research Literature

Preliminary sources index primary research resources such as journal articles, conference proceeding papers, technical reports, government documents, dissertations and more.  The CECH Library has created several specialized library guides on topics such as special education, instructional design & technology, and teaching STEM related topics that list which resources are most helpful for doing research in these areas. See below for key databases in education:

Step 5: Identify Subject Terms, or Descriptors, and Use Them to Search Preliminary Sources

Choosing the most appropriate subject search terms, or descriptors, for searching indexes and catalogs can greatly influence your search results.  A good place to start is ERIC's thesaurus of descriptors:

Step 6: Read and Evaluate Primary Sources Discovered Through Indexes

For assistance in obtaining copies of primary sources, please consult your liaison librarian .

As you print out copies of articles, review copies of books or reports, remember to look in the sources for bibliographies, names of individuals or groups who have done research on the topic, and for additional subject terms to help you narrow or broaden your research.

Step 7: Classify the Publications You Have Reviewed into Meaningful Categories

As you review the sources you find, classify them into meaningful categories.  This will help you prioritize reading them and may indicate useful ways to synthesize what you discover.  You may want to create a simple code for the different categories.

Step 8: Prepare Your Literature Review Report

See the following resources for advice on preparing a literature review report:

• << Previous: OAE and Praxis Core
• Next: Citation & Annotation >>
• Last Updated: Apr 30, 2024 11:57 AM
• URL: https://guides.libraries.uc.edu/edbasics

University of Cincinnati Libraries

PO Box 210033 Cincinnati, Ohio 45221-0033

Phone: 513-556-1424

Contact Us | Staff Directory

University of Cincinnati

Alerts | Clery and HEOA Notice | Notice of Non-Discrimination | eAccessibility Concern | Privacy Statement | Copyright Information

© 2021 University of Cincinnati

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• CBE Life Sci Educ
• v.21(3); Fall 2022

Literature Reviews, Theoretical Frameworks, and Conceptual Frameworks: An Introduction for New Biology Education Researchers

Julie a. luft.

† Department of Mathematics, Social Studies, and Science Education, Mary Frances Early College of Education, University of Georgia, Athens, GA 30602-7124

Sophia Jeong

‡ Department of Teaching & Learning, College of Education & Human Ecology, Ohio State University, Columbus, OH 43210

Robert Idsardi

§ Department of Biology, Eastern Washington University, Cheney, WA 99004

Grant Gardner

∥ Department of Biology, Middle Tennessee State University, Murfreesboro, TN 37132

Associated Data

To frame their work, biology education researchers need to consider the role of literature reviews, theoretical frameworks, and conceptual frameworks as critical elements of the research and writing process. However, these elements can be confusing for scholars new to education research. This Research Methods article is designed to provide an overview of each of these elements and delineate the purpose of each in the educational research process. We describe what biology education researchers should consider as they conduct literature reviews, identify theoretical frameworks, and construct conceptual frameworks. Clarifying these different components of educational research studies can be helpful to new biology education researchers and the biology education research community at large in situating their work in the broader scholarly literature.

INTRODUCTION

Discipline-based education research (DBER) involves the purposeful and situated study of teaching and learning in specific disciplinary areas ( Singer et al. , 2012 ). Studies in DBER are guided by research questions that reflect disciplines’ priorities and worldviews. Researchers can use quantitative data, qualitative data, or both to answer these research questions through a variety of methodological traditions. Across all methodologies, there are different methods associated with planning and conducting educational research studies that include the use of surveys, interviews, observations, artifacts, or instruments. Ensuring the coherence of these elements to the discipline’s perspective also involves situating the work in the broader scholarly literature. The tools for doing this include literature reviews, theoretical frameworks, and conceptual frameworks. However, the purpose and function of each of these elements is often confusing to new education researchers. The goal of this article is to introduce new biology education researchers to these three important elements important in DBER scholarship and the broader educational literature.

The first element we discuss is a review of research (literature reviews), which highlights the need for a specific research question, study problem, or topic of investigation. Literature reviews situate the relevance of the study within a topic and a field. The process may seem familiar to science researchers entering DBER fields, but new researchers may still struggle in conducting the review. Booth et al. (2016b) highlight some of the challenges novice education researchers face when conducting a review of literature. They point out that novice researchers struggle in deciding how to focus the review, determining the scope of articles needed in the review, and knowing how to be critical of the articles in the review. Overcoming these challenges (and others) can help novice researchers construct a sound literature review that can inform the design of the study and help ensure the work makes a contribution to the field.

The second and third highlighted elements are theoretical and conceptual frameworks. These guide biology education research (BER) studies, and may be less familiar to science researchers. These elements are important in shaping the construction of new knowledge. Theoretical frameworks offer a way to explain and interpret the studied phenomenon, while conceptual frameworks clarify assumptions about the studied phenomenon. Despite the importance of these constructs in educational research, biology educational researchers have noted the limited use of theoretical or conceptual frameworks in published work ( DeHaan, 2011 ; Dirks, 2011 ; Lo et al. , 2019 ). In reviewing articles published in CBE—Life Sciences Education ( LSE ) between 2015 and 2019, we found that fewer than 25% of the research articles had a theoretical or conceptual framework (see the Supplemental Information), and at times there was an inconsistent use of theoretical and conceptual frameworks. Clearly, these frameworks are challenging for published biology education researchers, which suggests the importance of providing some initial guidance to new biology education researchers.

Fortunately, educational researchers have increased their explicit use of these frameworks over time, and this is influencing educational research in science, technology, engineering, and mathematics (STEM) fields. For instance, a quick search for theoretical or conceptual frameworks in the abstracts of articles in Educational Research Complete (a common database for educational research) in STEM fields demonstrates a dramatic change over the last 20 years: from only 778 articles published between 2000 and 2010 to 5703 articles published between 2010 and 2020, a more than sevenfold increase. Greater recognition of the importance of these frameworks is contributing to DBER authors being more explicit about such frameworks in their studies.

Collectively, literature reviews, theoretical frameworks, and conceptual frameworks work to guide methodological decisions and the elucidation of important findings. Each offers a different perspective on the problem of study and is an essential element in all forms of educational research. As new researchers seek to learn about these elements, they will find different resources, a variety of perspectives, and many suggestions about the construction and use of these elements. The wide range of available information can overwhelm the new researcher who just wants to learn the distinction between these elements or how to craft them adequately.

Our goal in writing this paper is not to offer specific advice about how to write these sections in scholarly work. Instead, we wanted to introduce these elements to those who are new to BER and who are interested in better distinguishing one from the other. In this paper, we share the purpose of each element in BER scholarship, along with important points on its construction. We also provide references for additional resources that may be beneficial to better understanding each element. Table 1 summarizes the key distinctions among these elements.

Comparison of literature reviews, theoretical frameworks, and conceptual reviews

This article is written for the new biology education researcher who is just learning about these different elements or for scientists looking to become more involved in BER. It is a result of our own work as science education and biology education researchers, whether as graduate students and postdoctoral scholars or newly hired and established faculty members. This is the article we wish had been available as we started to learn about these elements or discussed them with new educational researchers in biology.

LITERATURE REVIEWS

Purpose of a literature review.

A literature review is foundational to any research study in education or science. In education, a well-conceptualized and well-executed review provides a summary of the research that has already been done on a specific topic and identifies questions that remain to be answered, thus illustrating the current research project’s potential contribution to the field and the reasoning behind the methodological approach selected for the study ( Maxwell, 2012 ). BER is an evolving disciplinary area that is redefining areas of conceptual emphasis as well as orientations toward teaching and learning (e.g., Labov et al. , 2010 ; American Association for the Advancement of Science, 2011 ; Nehm, 2019 ). As a result, building comprehensive, critical, purposeful, and concise literature reviews can be a challenge for new biology education researchers.

Building Literature Reviews

There are different ways to approach and construct a literature review. Booth et al. (2016a) provide an overview that includes, for example, scoping reviews, which are focused only on notable studies and use a basic method of analysis, and integrative reviews, which are the result of exhaustive literature searches across different genres. Underlying each of these different review processes are attention to the s earch process, a ppraisa l of articles, s ynthesis of the literature, and a nalysis: SALSA ( Booth et al. , 2016a ). This useful acronym can help the researcher focus on the process while building a specific type of review.

However, new educational researchers often have questions about literature reviews that are foundational to SALSA or other approaches. Common questions concern determining which literature pertains to the topic of study or the role of the literature review in the design of the study. This section addresses such questions broadly while providing general guidance for writing a narrative literature review that evaluates the most pertinent studies.

The literature review process should begin before the research is conducted. As Boote and Beile (2005 , p. 3) suggested, researchers should be “scholars before researchers.” They point out that having a good working knowledge of the proposed topic helps illuminate avenues of study. Some subject areas have a deep body of work to read and reflect upon, providing a strong foundation for developing the research question(s). For instance, the teaching and learning of evolution is an area of long-standing interest in the BER community, generating many studies (e.g., Perry et al. , 2008 ; Barnes and Brownell, 2016 ) and reviews of research (e.g., Sickel and Friedrichsen, 2013 ; Ziadie and Andrews, 2018 ). Emerging areas of BER include the affective domain, issues of transfer, and metacognition ( Singer et al. , 2012 ). Many studies in these areas are transdisciplinary and not always specific to biology education (e.g., Rodrigo-Peiris et al. , 2018 ; Kolpikova et al. , 2019 ). These newer areas may require reading outside BER; fortunately, summaries of some of these topics can be found in the Current Insights section of the LSE website.

In focusing on a specific problem within a broader research strand, a new researcher will likely need to examine research outside BER. Depending upon the area of study, the expanded reading list might involve a mix of BER, DBER, and educational research studies. Determining the scope of the reading is not always straightforward. A simple way to focus one’s reading is to create a “summary phrase” or “research nugget,” which is a very brief descriptive statement about the study. It should focus on the essence of the study, for example, “first-year nonmajor students’ understanding of evolution,” “metacognitive prompts to enhance learning during biochemistry,” or “instructors’ inquiry-based instructional practices after professional development programming.” This type of phrase should help a new researcher identify two or more areas to review that pertain to the study. Focusing on recent research in the last 5 years is a good first step. Additional studies can be identified by reading relevant works referenced in those articles. It is also important to read seminal studies that are more than 5 years old. Reading a range of studies should give the researcher the necessary command of the subject in order to suggest a research question.

Given that the research question(s) arise from the literature review, the review should also substantiate the selected methodological approach. The review and research question(s) guide the researcher in determining how to collect and analyze data. Often the methodological approach used in a study is selected to contribute knowledge that expands upon what has been published previously about the topic (see Institute of Education Sciences and National Science Foundation, 2013 ). An emerging topic of study may need an exploratory approach that allows for a description of the phenomenon and development of a potential theory. This could, but not necessarily, require a methodological approach that uses interviews, observations, surveys, or other instruments. An extensively studied topic may call for the additional understanding of specific factors or variables; this type of study would be well suited to a verification or a causal research design. These could entail a methodological approach that uses valid and reliable instruments, observations, or interviews to determine an effect in the studied event. In either of these examples, the researcher(s) may use a qualitative, quantitative, or mixed methods methodological approach.

Even with a good research question, there is still more reading to be done. The complexity and focus of the research question dictates the depth and breadth of the literature to be examined. Questions that connect multiple topics can require broad literature reviews. For instance, a study that explores the impact of a biology faculty learning community on the inquiry instruction of faculty could have the following review areas: learning communities among biology faculty, inquiry instruction among biology faculty, and inquiry instruction among biology faculty as a result of professional learning. Biology education researchers need to consider whether their literature review requires studies from different disciplines within or outside DBER. For the example given, it would be fruitful to look at research focused on learning communities with faculty in STEM fields or in general education fields that result in instructional change. It is important not to be too narrow or too broad when reading. When the conclusions of articles start to sound similar or no new insights are gained, the researcher likely has a good foundation for a literature review. This level of reading should allow the researcher to demonstrate a mastery in understanding the researched topic, explain the suitability of the proposed research approach, and point to the need for the refined research question(s).

The literature review should include the researcher’s evaluation and critique of the selected studies. A researcher may have a large collection of studies, but not all of the studies will follow standards important in the reporting of empirical work in the social sciences. The American Educational Research Association ( Duran et al. , 2006 ), for example, offers a general discussion about standards for such work: an adequate review of research informing the study, the existence of sound and appropriate data collection and analysis methods, and appropriate conclusions that do not overstep or underexplore the analyzed data. The Institute of Education Sciences and National Science Foundation (2013) also offer Common Guidelines for Education Research and Development that can be used to evaluate collected studies.

Because not all journals adhere to such standards, it is important that a researcher review each study to determine the quality of published research, per the guidelines suggested earlier. In some instances, the research may be fatally flawed. Examples of such flaws include data that do not pertain to the question, a lack of discussion about the data collection, poorly constructed instruments, or an inadequate analysis. These types of errors result in studies that are incomplete, error-laden, or inaccurate and should be excluded from the review. Most studies have limitations, and the author(s) often make them explicit. For instance, there may be an instructor effect, recognized bias in the analysis, or issues with the sample population. Limitations are usually addressed by the research team in some way to ensure a sound and acceptable research process. Occasionally, the limitations associated with the study can be significant and not addressed adequately, which leaves a consequential decision in the hands of the researcher. Providing critiques of studies in the literature review process gives the reader confidence that the researcher has carefully examined relevant work in preparation for the study and, ultimately, the manuscript.

A solid literature review clearly anchors the proposed study in the field and connects the research question(s), the methodological approach, and the discussion. Reviewing extant research leads to research questions that will contribute to what is known in the field. By summarizing what is known, the literature review points to what needs to be known, which in turn guides decisions about methodology. Finally, notable findings of the new study are discussed in reference to those described in the literature review.

Within published BER studies, literature reviews can be placed in different locations in an article. When included in the introductory section of the study, the first few paragraphs of the manuscript set the stage, with the literature review following the opening paragraphs. Cooper et al. (2019) illustrate this approach in their study of course-based undergraduate research experiences (CUREs). An introduction discussing the potential of CURES is followed by an analysis of the existing literature relevant to the design of CUREs that allows for novel student discoveries. Within this review, the authors point out contradictory findings among research on novel student discoveries. This clarifies the need for their study, which is described and highlighted through specific research aims.

A literature reviews can also make up a separate section in a paper. For example, the introduction to Todd et al. (2019) illustrates the need for their research topic by highlighting the potential of learning progressions (LPs) and suggesting that LPs may help mitigate learning loss in genetics. At the end of the introduction, the authors state their specific research questions. The review of literature following this opening section comprises two subsections. One focuses on learning loss in general and examines a variety of studies and meta-analyses from the disciplines of medical education, mathematics, and reading. The second section focuses specifically on LPs in genetics and highlights student learning in the midst of LPs. These separate reviews provide insights into the stated research question.

Suggestions and Advice

A well-conceptualized, comprehensive, and critical literature review reveals the understanding of the topic that the researcher brings to the study. Literature reviews should not be so big that there is no clear area of focus; nor should they be so narrow that no real research question arises. The task for a researcher is to craft an efficient literature review that offers a critical analysis of published work, articulates the need for the study, guides the methodological approach to the topic of study, and provides an adequate foundation for the discussion of the findings.

In our own writing of literature reviews, there are often many drafts. An early draft may seem well suited to the study because the need for and approach to the study are well described. However, as the results of the study are analyzed and findings begin to emerge, the existing literature review may be inadequate and need revision. The need for an expanded discussion about the research area can result in the inclusion of new studies that support the explanation of a potential finding. The literature review may also prove to be too broad. Refocusing on a specific area allows for more contemplation of a finding.

It should be noted that there are different types of literature reviews, and many books and articles have been written about the different ways to embark on these types of reviews. Among these different resources, the following may be helpful in considering how to refine the review process for scholarly journals:

• Booth, A., Sutton, A., & Papaioannou, D. (2016a). Systemic approaches to a successful literature review (2nd ed.). Los Angeles, CA: Sage. This book addresses different types of literature reviews and offers important suggestions pertaining to defining the scope of the literature review and assessing extant studies.
• Booth, W. C., Colomb, G. G., Williams, J. M., Bizup, J., & Fitzgerald, W. T. (2016b). The craft of research (4th ed.). Chicago: University of Chicago Press. This book can help the novice consider how to make the case for an area of study. While this book is not specifically about literature reviews, it offers suggestions about making the case for your study.
• Galvan, J. L., & Galvan, M. C. (2017). Writing literature reviews: A guide for students of the social and behavioral sciences (7th ed.). Routledge. This book offers guidance on writing different types of literature reviews. For the novice researcher, there are useful suggestions for creating coherent literature reviews.

THEORETICAL FRAMEWORKS

Purpose of theoretical frameworks.

As new education researchers may be less familiar with theoretical frameworks than with literature reviews, this discussion begins with an analogy. Envision a biologist, chemist, and physicist examining together the dramatic effect of a fog tsunami over the ocean. A biologist gazing at this phenomenon may be concerned with the effect of fog on various species. A chemist may be interested in the chemical composition of the fog as water vapor condenses around bits of salt. A physicist may be focused on the refraction of light to make fog appear to be “sitting” above the ocean. While observing the same “objective event,” the scientists are operating under different theoretical frameworks that provide a particular perspective or “lens” for the interpretation of the phenomenon. Each of these scientists brings specialized knowledge, experiences, and values to this phenomenon, and these influence the interpretation of the phenomenon. The scientists’ theoretical frameworks influence how they design and carry out their studies and interpret their data.

Within an educational study, a theoretical framework helps to explain a phenomenon through a particular lens and challenges and extends existing knowledge within the limitations of that lens. Theoretical frameworks are explicitly stated by an educational researcher in the paper’s framework, theory, or relevant literature section. The framework shapes the types of questions asked, guides the method by which data are collected and analyzed, and informs the discussion of the results of the study. It also reveals the researcher’s subjectivities, for example, values, social experience, and viewpoint ( Allen, 2017 ). It is essential that a novice researcher learn to explicitly state a theoretical framework, because all research questions are being asked from the researcher’s implicit or explicit assumptions of a phenomenon of interest ( Schwandt, 2000 ).

Selecting Theoretical Frameworks

Theoretical frameworks are one of the most contemplated elements in our work in educational research. In this section, we share three important considerations for new scholars selecting a theoretical framework.

The first step in identifying a theoretical framework involves reflecting on the phenomenon within the study and the assumptions aligned with the phenomenon. The phenomenon involves the studied event. There are many possibilities, for example, student learning, instructional approach, or group organization. A researcher holds assumptions about how the phenomenon will be effected, influenced, changed, or portrayed. It is ultimately the researcher’s assumption(s) about the phenomenon that aligns with a theoretical framework. An example can help illustrate how a researcher’s reflection on the phenomenon and acknowledgment of assumptions can result in the identification of a theoretical framework.

In our example, a biology education researcher may be interested in exploring how students’ learning of difficult biological concepts can be supported by the interactions of group members. The phenomenon of interest is the interactions among the peers, and the researcher assumes that more knowledgeable students are important in supporting the learning of the group. As a result, the researcher may draw on Vygotsky’s (1978) sociocultural theory of learning and development that is focused on the phenomenon of student learning in a social setting. This theory posits the critical nature of interactions among students and between students and teachers in the process of building knowledge. A researcher drawing upon this framework holds the assumption that learning is a dynamic social process involving questions and explanations among students in the classroom and that more knowledgeable peers play an important part in the process of building conceptual knowledge.

It is important to state at this point that there are many different theoretical frameworks. Some frameworks focus on learning and knowing, while other theoretical frameworks focus on equity, empowerment, or discourse. Some frameworks are well articulated, and others are still being refined. For a new researcher, it can be challenging to find a theoretical framework. Two of the best ways to look for theoretical frameworks is through published works that highlight different frameworks.

When a theoretical framework is selected, it should clearly connect to all parts of the study. The framework should augment the study by adding a perspective that provides greater insights into the phenomenon. It should clearly align with the studies described in the literature review. For instance, a framework focused on learning would correspond to research that reported different learning outcomes for similar studies. The methods for data collection and analysis should also correspond to the framework. For instance, a study about instructional interventions could use a theoretical framework concerned with learning and could collect data about the effect of the intervention on what is learned. When the data are analyzed, the theoretical framework should provide added meaning to the findings, and the findings should align with the theoretical framework.

A study by Jensen and Lawson (2011) provides an example of how a theoretical framework connects different parts of the study. They compared undergraduate biology students in heterogeneous and homogeneous groups over the course of a semester. Jensen and Lawson (2011) assumed that learning involved collaboration and more knowledgeable peers, which made Vygotsky’s (1978) theory a good fit for their study. They predicted that students in heterogeneous groups would experience greater improvement in their reasoning abilities and science achievements with much of the learning guided by the more knowledgeable peers.

In the enactment of the study, they collected data about the instruction in traditional and inquiry-oriented classes, while the students worked in homogeneous or heterogeneous groups. To determine the effect of working in groups, the authors also measured students’ reasoning abilities and achievement. Each data-collection and analysis decision connected to understanding the influence of collaborative work.

Their findings highlighted aspects of Vygotsky’s (1978) theory of learning. One finding, for instance, posited that inquiry instruction, as a whole, resulted in reasoning and achievement gains. This links to Vygotsky (1978) , because inquiry instruction involves interactions among group members. A more nuanced finding was that group composition had a conditional effect. Heterogeneous groups performed better with more traditional and didactic instruction, regardless of the reasoning ability of the group members. Homogeneous groups worked better during interaction-rich activities for students with low reasoning ability. The authors attributed the variation to the different types of helping behaviors of students. High-performing students provided the answers, while students with low reasoning ability had to work collectively through the material. In terms of Vygotsky (1978) , this finding provided new insights into the learning context in which productive interactions can occur for students.

Another consideration in the selection and use of a theoretical framework pertains to its orientation to the study. This can result in the theoretical framework prioritizing individuals, institutions, and/or policies ( Anfara and Mertz, 2014 ). Frameworks that connect to individuals, for instance, could contribute to understanding their actions, learning, or knowledge. Institutional frameworks, on the other hand, offer insights into how institutions, organizations, or groups can influence individuals or materials. Policy theories provide ways to understand how national or local policies can dictate an emphasis on outcomes or instructional design. These different types of frameworks highlight different aspects in an educational setting, which influences the design of the study and the collection of data. In addition, these different frameworks offer a way to make sense of the data. Aligning the data collection and analysis with the framework ensures that a study is coherent and can contribute to the field.

New understandings emerge when different theoretical frameworks are used. For instance, Ebert-May et al. (2015) prioritized the individual level within conceptual change theory (see Posner et al. , 1982 ). In this theory, an individual’s knowledge changes when it no longer fits the phenomenon. Ebert-May et al. (2015) designed a professional development program challenging biology postdoctoral scholars’ existing conceptions of teaching. The authors reported that the biology postdoctoral scholars’ teaching practices became more student-centered as they were challenged to explain their instructional decision making. According to the theory, the biology postdoctoral scholars’ dissatisfaction in their descriptions of teaching and learning initiated change in their knowledge and instruction. These results reveal how conceptual change theory can explain the learning of participants and guide the design of professional development programming.

The communities of practice (CoP) theoretical framework ( Lave, 1988 ; Wenger, 1998 ) prioritizes the institutional level , suggesting that learning occurs when individuals learn from and contribute to the communities in which they reside. Grounded in the assumption of community learning, the literature on CoP suggests that, as individuals interact regularly with the other members of their group, they learn about the rules, roles, and goals of the community ( Allee, 2000 ). A study conducted by Gehrke and Kezar (2017) used the CoP framework to understand organizational change by examining the involvement of individual faculty engaged in a cross-institutional CoP focused on changing the instructional practice of faculty at each institution. In the CoP, faculty members were involved in enhancing instructional materials within their department, which aligned with an overarching goal of instituting instruction that embraced active learning. Not surprisingly, Gehrke and Kezar (2017) revealed that faculty who perceived the community culture as important in their work cultivated institutional change. Furthermore, they found that institutional change was sustained when key leaders served as mentors and provided support for faculty, and as faculty themselves developed into leaders. This study reveals the complexity of individual roles in a COP in order to support institutional instructional change.

It is important to explicitly state the theoretical framework used in a study, but elucidating a theoretical framework can be challenging for a new educational researcher. The literature review can help to identify an applicable theoretical framework. Focal areas of the review or central terms often connect to assumptions and assertions associated with the framework that pertain to the phenomenon of interest. Another way to identify a theoretical framework is self-reflection by the researcher on personal beliefs and understandings about the nature of knowledge the researcher brings to the study ( Lysaght, 2011 ). In stating one’s beliefs and understandings related to the study (e.g., students construct their knowledge, instructional materials support learning), an orientation becomes evident that will suggest a particular theoretical framework. Theoretical frameworks are not arbitrary , but purposefully selected.

With experience, a researcher may find expanded roles for theoretical frameworks. Researchers may revise an existing framework that has limited explanatory power, or they may decide there is a need to develop a new theoretical framework. These frameworks can emerge from a current study or the need to explain a phenomenon in a new way. Researchers may also find that multiple theoretical frameworks are necessary to frame and explore a problem, as different frameworks can provide different insights into a problem.

Finally, it is important to recognize that choosing “x” theoretical framework does not necessarily mean a researcher chooses “y” methodology and so on, nor is there a clear-cut, linear process in selecting a theoretical framework for one’s study. In part, the nonlinear process of identifying a theoretical framework is what makes understanding and using theoretical frameworks challenging. For the novice scholar, contemplating and understanding theoretical frameworks is essential. Fortunately, there are articles and books that can help:

• Creswell, J. W. (2018). Research design: Qualitative, quantitative, and mixed methods approaches (5th ed.). Los Angeles, CA: Sage. This book provides an overview of theoretical frameworks in general educational research.
• Ding, L. (2019). Theoretical perspectives of quantitative physics education research. Physical Review Physics Education Research , 15 (2), 020101-1–020101-13. This paper illustrates how a DBER field can use theoretical frameworks.
• Nehm, R. (2019). Biology education research: Building integrative frameworks for teaching and learning about living systems. Disciplinary and Interdisciplinary Science Education Research , 1 , ar15. https://doi.org/10.1186/s43031-019-0017-6 . This paper articulates the need for studies in BER to explicitly state theoretical frameworks and provides examples of potential studies.
• Patton, M. Q. (2015). Qualitative research & evaluation methods: Integrating theory and practice . Sage. This book also provides an overview of theoretical frameworks, but for both research and evaluation.

CONCEPTUAL FRAMEWORKS

Purpose of a conceptual framework.

A conceptual framework is a description of the way a researcher understands the factors and/or variables that are involved in the study and their relationships to one another. The purpose of a conceptual framework is to articulate the concepts under study using relevant literature ( Rocco and Plakhotnik, 2009 ) and to clarify the presumed relationships among those concepts ( Rocco and Plakhotnik, 2009 ; Anfara and Mertz, 2014 ). Conceptual frameworks are different from theoretical frameworks in both their breadth and grounding in established findings. Whereas a theoretical framework articulates the lens through which a researcher views the work, the conceptual framework is often more mechanistic and malleable.

Conceptual frameworks are broader, encompassing both established theories (i.e., theoretical frameworks) and the researchers’ own emergent ideas. Emergent ideas, for example, may be rooted in informal and/or unpublished observations from experience. These emergent ideas would not be considered a “theory” if they are not yet tested, supported by systematically collected evidence, and peer reviewed. However, they do still play an important role in the way researchers approach their studies. The conceptual framework allows authors to clearly describe their emergent ideas so that connections among ideas in the study and the significance of the study are apparent to readers.

Constructing Conceptual Frameworks

Including a conceptual framework in a research study is important, but researchers often opt to include either a conceptual or a theoretical framework. Either may be adequate, but both provide greater insight into the research approach. For instance, a research team plans to test a novel component of an existing theory. In their study, they describe the existing theoretical framework that informs their work and then present their own conceptual framework. Within this conceptual framework, specific topics portray emergent ideas that are related to the theory. Describing both frameworks allows readers to better understand the researchers’ assumptions, orientations, and understanding of concepts being investigated. For example, Connolly et al. (2018) included a conceptual framework that described how they applied a theoretical framework of social cognitive career theory (SCCT) to their study on teaching programs for doctoral students. In their conceptual framework, the authors described SCCT, explained how it applied to the investigation, and drew upon results from previous studies to justify the proposed connections between the theory and their emergent ideas.

In some cases, authors may be able to sufficiently describe their conceptualization of the phenomenon under study in an introduction alone, without a separate conceptual framework section. However, incomplete descriptions of how the researchers conceptualize the components of the study may limit the significance of the study by making the research less intelligible to readers. This is especially problematic when studying topics in which researchers use the same terms for different constructs or different terms for similar and overlapping constructs (e.g., inquiry, teacher beliefs, pedagogical content knowledge, or active learning). Authors must describe their conceptualization of a construct if the research is to be understandable and useful.

There are some key areas to consider regarding the inclusion of a conceptual framework in a study. To begin with, it is important to recognize that conceptual frameworks are constructed by the researchers conducting the study ( Rocco and Plakhotnik, 2009 ; Maxwell, 2012 ). This is different from theoretical frameworks that are often taken from established literature. Researchers should bring together ideas from the literature, but they may be influenced by their own experiences as a student and/or instructor, the shared experiences of others, or thought experiments as they construct a description, model, or representation of their understanding of the phenomenon under study. This is an exercise in intellectual organization and clarity that often considers what is learned, known, and experienced. The conceptual framework makes these constructs explicitly visible to readers, who may have different understandings of the phenomenon based on their prior knowledge and experience. There is no single method to go about this intellectual work.

Reeves et al. (2016) is an example of an article that proposed a conceptual framework about graduate teaching assistant professional development evaluation and research. The authors used existing literature to create a novel framework that filled a gap in current research and practice related to the training of graduate teaching assistants. This conceptual framework can guide the systematic collection of data by other researchers because the framework describes the relationships among various factors that influence teaching and learning. The Reeves et al. (2016) conceptual framework may be modified as additional data are collected and analyzed by other researchers. This is not uncommon, as conceptual frameworks can serve as catalysts for concerted research efforts that systematically explore a phenomenon (e.g., Reynolds et al. , 2012 ; Brownell and Kloser, 2015 ).

Sabel et al. (2017) used a conceptual framework in their exploration of how scaffolds, an external factor, interact with internal factors to support student learning. Their conceptual framework integrated principles from two theoretical frameworks, self-regulated learning and metacognition, to illustrate how the research team conceptualized students’ use of scaffolds in their learning ( Figure 1 ). Sabel et al. (2017) created this model using their interpretations of these two frameworks in the context of their teaching.

Conceptual framework from Sabel et al. (2017) .

A conceptual framework should describe the relationship among components of the investigation ( Anfara and Mertz, 2014 ). These relationships should guide the researcher’s methods of approaching the study ( Miles et al. , 2014 ) and inform both the data to be collected and how those data should be analyzed. Explicitly describing the connections among the ideas allows the researcher to justify the importance of the study and the rigor of the research design. Just as importantly, these frameworks help readers understand why certain components of a system were not explored in the study. This is a challenge in education research, which is rooted in complex environments with many variables that are difficult to control.

For example, Sabel et al. (2017) stated: “Scaffolds, such as enhanced answer keys and reflection questions, can help students and instructors bridge the external and internal factors and support learning” (p. 3). They connected the scaffolds in the study to the three dimensions of metacognition and the eventual transformation of existing ideas into new or revised ideas. Their framework provides a rationale for focusing on how students use two different scaffolds, and not on other factors that may influence a student’s success (self-efficacy, use of active learning, exam format, etc.).

In constructing conceptual frameworks, researchers should address needed areas of study and/or contradictions discovered in literature reviews. By attending to these areas, researchers can strengthen their arguments for the importance of a study. For instance, conceptual frameworks can address how the current study will fill gaps in the research, resolve contradictions in existing literature, or suggest a new area of study. While a literature review describes what is known and not known about the phenomenon, the conceptual framework leverages these gaps in describing the current study ( Maxwell, 2012 ). In the example of Sabel et al. (2017) , the authors indicated there was a gap in the literature regarding how scaffolds engage students in metacognition to promote learning in large classes. Their study helps fill that gap by describing how scaffolds can support students in the three dimensions of metacognition: intelligibility, plausibility, and wide applicability. In another example, Lane (2016) integrated research from science identity, the ethic of care, the sense of belonging, and an expertise model of student success to form a conceptual framework that addressed the critiques of other frameworks. In a more recent example, Sbeglia et al. (2021) illustrated how a conceptual framework influences the methodological choices and inferences in studies by educational researchers.

Sometimes researchers draw upon the conceptual frameworks of other researchers. When a researcher’s conceptual framework closely aligns with an existing framework, the discussion may be brief. For example, Ghee et al. (2016) referred to portions of SCCT as their conceptual framework to explain the significance of their work on students’ self-efficacy and career interests. Because the authors’ conceptualization of this phenomenon aligned with a previously described framework, they briefly mentioned the conceptual framework and provided additional citations that provided more detail for the readers.

Within both the BER and the broader DBER communities, conceptual frameworks have been used to describe different constructs. For example, some researchers have used the term “conceptual framework” to describe students’ conceptual understandings of a biological phenomenon. This is distinct from a researcher’s conceptual framework of the educational phenomenon under investigation, which may also need to be explicitly described in the article. Other studies have presented a research logic model or flowchart of the research design as a conceptual framework. These constructions can be quite valuable in helping readers understand the data-collection and analysis process. However, a model depicting the study design does not serve the same role as a conceptual framework. Researchers need to avoid conflating these constructs by differentiating the researchers’ conceptual framework that guides the study from the research design, when applicable.

Explicitly describing conceptual frameworks is essential in depicting the focus of the study. We have found that being explicit in a conceptual framework means using accepted terminology, referencing prior work, and clearly noting connections between terms. This description can also highlight gaps in the literature or suggest potential contributions to the field of study. A well-elucidated conceptual framework can suggest additional studies that may be warranted. This can also spur other researchers to consider how they would approach the examination of a phenomenon and could result in a revised conceptual framework.

It can be challenging to create conceptual frameworks, but they are important. Below are two resources that could be helpful in constructing and presenting conceptual frameworks in educational research:

• Maxwell, J. A. (2012). Qualitative research design: An interactive approach (3rd ed.). Los Angeles, CA: Sage. Chapter 3 in this book describes how to construct conceptual frameworks.
• Ravitch, S. M., & Riggan, M. (2016). Reason & rigor: How conceptual frameworks guide research . Los Angeles, CA: Sage. This book explains how conceptual frameworks guide the research questions, data collection, data analyses, and interpretation of results.

CONCLUDING THOUGHTS

Literature reviews, theoretical frameworks, and conceptual frameworks are all important in DBER and BER. Robust literature reviews reinforce the importance of a study. Theoretical frameworks connect the study to the base of knowledge in educational theory and specify the researcher’s assumptions. Conceptual frameworks allow researchers to explicitly describe their conceptualization of the relationships among the components of the phenomenon under study. Table 1 provides a general overview of these components in order to assist biology education researchers in thinking about these elements.

It is important to emphasize that these different elements are intertwined. When these elements are aligned and complement one another, the study is coherent, and the study findings contribute to knowledge in the field. When literature reviews, theoretical frameworks, and conceptual frameworks are disconnected from one another, the study suffers. The point of the study is lost, suggested findings are unsupported, or important conclusions are invisible to the researcher. In addition, this misalignment may be costly in terms of time and money.

Conducting a literature review, selecting a theoretical framework, and building a conceptual framework are some of the most difficult elements of a research study. It takes time to understand the relevant research, identify a theoretical framework that provides important insights into the study, and formulate a conceptual framework that organizes the finding. In the research process, there is often a constant back and forth among these elements as the study evolves. With an ongoing refinement of the review of literature, clarification of the theoretical framework, and articulation of a conceptual framework, a sound study can emerge that makes a contribution to the field. This is the goal of BER and education research.

Supplementary Material

• Allee, V. (2000). Knowledge networks and communities of learning . OD Practitioner , 32 ( 4 ), 4–13. [ Google Scholar ]
• Allen, M. (2017). The Sage encyclopedia of communication research methods (Vols. 1–4 ). Los Angeles, CA: Sage. 10.4135/9781483381411 [ CrossRef ] [ Google Scholar ]
• American Association for the Advancement of Science. (2011). Vision and change in undergraduate biology education: A call to action . Washington, DC. [ Google Scholar ]
• Anfara, V. A., Mertz, N. T. (2014). Setting the stage . In Anfara, V. A., Mertz, N. T. (eds.), Theoretical frameworks in qualitative research (pp. 1–22). Sage. [ Google Scholar ]
• Barnes, M. E., Brownell, S. E. (2016). Practices and perspectives of college instructors on addressing religious beliefs when teaching evolution . CBE—Life Sciences Education , 15 ( 2 ), ar18. https://doi.org/10.1187/cbe.15-11-0243 [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Boote, D. N., Beile, P. (2005). Scholars before researchers: On the centrality of the dissertation literature review in research preparation . Educational Researcher , 34 ( 6 ), 3–15. 10.3102/0013189x034006003 [ CrossRef ] [ Google Scholar ]
• Booth, A., Sutton, A., Papaioannou, D. (2016a). Systemic approaches to a successful literature review (2nd ed.). Los Angeles, CA: Sage. [ Google Scholar ]
• Booth, W. C., Colomb, G. G., Williams, J. M., Bizup, J., Fitzgerald, W. T. (2016b). The craft of research (4th ed.). Chicago, IL: University of Chicago Press. [ Google Scholar ]
• Brownell, S. E., Kloser, M. J. (2015). Toward a conceptual framework for measuring the effectiveness of course-based undergraduate research experiences in undergraduate biology . Studies in Higher Education , 40 ( 3 ), 525–544. https://doi.org/10.1080/03075079.2015.1004234 [ Google Scholar ]
• Connolly, M. R., Lee, Y. G., Savoy, J. N. (2018). The effects of doctoral teaching development on early-career STEM scholars’ college teaching self-efficacy . CBE—Life Sciences Education , 17 ( 1 ), ar14. https://doi.org/10.1187/cbe.17-02-0039 [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Cooper, K. M., Blattman, J. N., Hendrix, T., Brownell, S. E. (2019). The impact of broadly relevant novel discoveries on student project ownership in a traditional lab course turned CURE . CBE—Life Sciences Education , 18 ( 4 ), ar57. https://doi.org/10.1187/cbe.19-06-0113 [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Creswell, J. W. (2018). Research design: Qualitative, quantitative, and mixed methods approaches (5th ed.). Los Angeles, CA: Sage. [ Google Scholar ]
• DeHaan, R. L. (2011). Education research in the biological sciences: A nine decade review (Paper commissioned by the NAS/NRC Committee on the Status, Contributions, and Future Directions of Discipline Based Education Research) . Washington, DC: National Academies Press. Retrieved May 20, 2022, from www7.nationalacademies.org/bose/DBER_Mee ting2_commissioned_papers_page.html [ Google Scholar ]
• Ding, L. (2019). Theoretical perspectives of quantitative physics education research . Physical Review Physics Education Research , 15 ( 2 ), 020101. [ Google Scholar ]
• Dirks, C. (2011). The current status and future direction of biology education research . Paper presented at: Second Committee Meeting on the Status, Contributions, and Future Directions of Discipline-Based Education Research, 18–19 October (Washington, DC). Retrieved May 20, 2022, from http://sites.nationalacademies.org/DBASSE/BOSE/DBASSE_071087 [ Google Scholar ]
• Duran, R. P., Eisenhart, M. A., Erickson, F. D., Grant, C. A., Green, J. L., Hedges, L. V., Schneider, B. L. (2006). Standards for reporting on empirical social science research in AERA publications: American Educational Research Association . Educational Researcher , 35 ( 6 ), 33–40. [ Google Scholar ]
• Ebert-May, D., Derting, T. L., Henkel, T. P., Middlemis Maher, J., Momsen, J. L., Arnold, B., Passmore, H. A. (2015). Breaking the cycle: Future faculty begin teaching with learner-centered strategies after professional development . CBE—Life Sciences Education , 14 ( 2 ), ar22. https://doi.org/10.1187/cbe.14-12-0222 [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Galvan, J. L., Galvan, M. C. (2017). Writing literature reviews: A guide for students of the social and behavioral sciences (7th ed.). New York, NY: Routledge. https://doi.org/10.4324/9781315229386 [ Google Scholar ]
• Gehrke, S., Kezar, A. (2017). The roles of STEM faculty communities of practice in institutional and departmental reform in higher education . American Educational Research Journal , 54 ( 5 ), 803–833. https://doi.org/10.3102/0002831217706736 [ Google Scholar ]
• Ghee, M., Keels, M., Collins, D., Neal-Spence, C., Baker, E. (2016). Fine-tuning summer research programs to promote underrepresented students’ persistence in the STEM pathway . CBE—Life Sciences Education , 15 ( 3 ), ar28. https://doi.org/10.1187/cbe.16-01-0046 [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Institute of Education Sciences & National Science Foundation. (2013). Common guidelines for education research and development . Retrieved May 20, 2022, from www.nsf.gov/pubs/2013/nsf13126/nsf13126.pdf
• Jensen, J. L., Lawson, A. (2011). Effects of collaborative group composition and inquiry instruction on reasoning gains and achievement in undergraduate biology . CBE—Life Sciences Education , 10 ( 1 ), 64–73. https://doi.org/10.1187/cbe.19-05-0098 [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Kolpikova, E. P., Chen, D. C., Doherty, J. H. (2019). Does the format of preclass reading quizzes matter? An evaluation of traditional and gamified, adaptive preclass reading quizzes . CBE—Life Sciences Education , 18 ( 4 ), ar52. https://doi.org/10.1187/cbe.19-05-0098 [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Labov, J. B., Reid, A. H., Yamamoto, K. R. (2010). Integrated biology and undergraduate science education: A new biology education for the twenty-first century? CBE—Life Sciences Education , 9 ( 1 ), 10–16. https://doi.org/10.1187/cbe.09-12-0092 [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Lane, T. B. (2016). Beyond academic and social integration: Understanding the impact of a STEM enrichment program on the retention and degree attainment of underrepresented students . CBE—Life Sciences Education , 15 ( 3 ), ar39. https://doi.org/10.1187/cbe.16-01-0070 [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Lave, J. (1988). Cognition in practice: Mind, mathematics and culture in everyday life . New York, NY: Cambridge University Press. [ Google Scholar ]
• Lo, S. M., Gardner, G. E., Reid, J., Napoleon-Fanis, V., Carroll, P., Smith, E., Sato, B. K. (2019). Prevailing questions and methodologies in biology education research: A longitudinal analysis of research in CBE — Life Sciences Education and at the Society for the Advancement of Biology Education Research . CBE—Life Sciences Education , 18 ( 1 ), ar9. https://doi.org/10.1187/cbe.18-08-0164 [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Lysaght, Z. (2011). Epistemological and paradigmatic ecumenism in “Pasteur’s quadrant:” Tales from doctoral research . In Official Conference Proceedings of the Third Asian Conference on Education in Osaka, Japan . Retrieved May 20, 2022, from http://iafor.org/ace2011_offprint/ACE2011_offprint_0254.pdf
• Maxwell, J. A. (2012). Qualitative research design: An interactive approach (3rd ed.). Los Angeles, CA: Sage. [ Google Scholar ]
• Miles, M. B., Huberman, A. M., Saldaña, J. (2014). Qualitative data analysis (3rd ed.). Los Angeles, CA: Sage. [ Google Scholar ]
• Nehm, R. (2019). Biology education research: Building integrative frameworks for teaching and learning about living systems . Disciplinary and Interdisciplinary Science Education Research , 1 , ar15. https://doi.org/10.1186/s43031-019-0017-6 [ Google Scholar ]
• Patton, M. Q. (2015). Qualitative research & evaluation methods: Integrating theory and practice . Los Angeles, CA: Sage. [ Google Scholar ]
• Perry, J., Meir, E., Herron, J. C., Maruca, S., Stal, D. (2008). Evaluating two approaches to helping college students understand evolutionary trees through diagramming tasks . CBE—Life Sciences Education , 7 ( 2 ), 193–201. https://doi.org/10.1187/cbe.07-01-0007 [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Posner, G. J., Strike, K. A., Hewson, P. W., Gertzog, W. A. (1982). Accommodation of a scientific conception: Toward a theory of conceptual change . Science Education , 66 ( 2 ), 211–227. [ Google Scholar ]
• Ravitch, S. M., Riggan, M. (2016). Reason & rigor: How conceptual frameworks guide research . Los Angeles, CA: Sage. [ Google Scholar ]
• Reeves, T. D., Marbach-Ad, G., Miller, K. R., Ridgway, J., Gardner, G. E., Schussler, E. E., Wischusen, E. W. (2016). A conceptual framework for graduate teaching assistant professional development evaluation and research . CBE—Life Sciences Education , 15 ( 2 ), es2. https://doi.org/10.1187/cbe.15-10-0225 [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Reynolds, J. A., Thaiss, C., Katkin, W., Thompson, R. J. Jr. (2012). Writing-to-learn in undergraduate science education: A community-based, conceptually driven approach . CBE—Life Sciences Education , 11 ( 1 ), 17–25. https://doi.org/10.1187/cbe.11-08-0064 [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Rocco, T. S., Plakhotnik, M. S. (2009). Literature reviews, conceptual frameworks, and theoretical frameworks: Terms, functions, and distinctions . Human Resource Development Review , 8 ( 1 ), 120–130. https://doi.org/10.1177/1534484309332617 [ Google Scholar ]
• Rodrigo-Peiris, T., Xiang, L., Cassone, V. M. (2018). A low-intensity, hybrid design between a “traditional” and a “course-based” research experience yields positive outcomes for science undergraduate freshmen and shows potential for large-scale application . CBE—Life Sciences Education , 17 ( 4 ), ar53. https://doi.org/10.1187/cbe.17-11-0248 [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Sabel, J. L., Dauer, J. T., Forbes, C. T. (2017). Introductory biology students’ use of enhanced answer keys and reflection questions to engage in metacognition and enhance understanding . CBE—Life Sciences Education , 16 ( 3 ), ar40. https://doi.org/10.1187/cbe.16-10-0298 [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Sbeglia, G. C., Goodridge, J. A., Gordon, L. H., Nehm, R. H. (2021). Are faculty changing? How reform frameworks, sampling intensities, and instrument measures impact inferences about student-centered teaching practices . CBE—Life Sciences Education , 20 ( 3 ), ar39. https://doi.org/10.1187/cbe.20-11-0259 [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Schwandt, T. A. (2000). Three epistemological stances for qualitative inquiry: Interpretivism, hermeneutics, and social constructionism . In Denzin, N. K., Lincoln, Y. S. (Eds.), Handbook of qualitative research (2nd ed., pp. 189–213). Los Angeles, CA: Sage. [ Google Scholar ]
• Sickel, A. J., Friedrichsen, P. (2013). Examining the evolution education literature with a focus on teachers: Major findings, goals for teacher preparation, and directions for future research . Evolution: Education and Outreach , 6 ( 1 ), 23. https://doi.org/10.1186/1936-6434-6-23 [ Google Scholar ]
• Singer, S. R., Nielsen, N. R., Schweingruber, H. A. (2012). Discipline-based education research: Understanding and improving learning in undergraduate science and engineering . Washington, DC: National Academies Press. [ Google Scholar ]
• Todd, A., Romine, W. L., Correa-Menendez, J. (2019). Modeling the transition from a phenotypic to genotypic conceptualization of genetics in a university-level introductory biology context . Research in Science Education , 49 ( 2 ), 569–589. https://doi.org/10.1007/s11165-017-9626-2 [ Google Scholar ]
• Vygotsky, L. S. (1978). Mind in society: The development of higher psychological processes . Cambridge, MA: Harvard University Press. [ Google Scholar ]
• Wenger, E. (1998). Communities of practice: Learning as a social system . Systems Thinker , 9 ( 5 ), 2–3. [ Google Scholar ]
• Ziadie, M. A., Andrews, T. C. (2018). Moving evolution education forward: A systematic analysis of literature to identify gaps in collective knowledge for teaching . CBE—Life Sciences Education , 17 ( 1 ), ar11. https://doi.org/10.1187/cbe.17-08-0190 [ PMC free article ] [ PubMed ] [ Google Scholar ]

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

Chapter 1: Introduction

Learning objectives.

At the conclusion of this chapter, you will be able to:

• Identify the purpose of the literature review in  the research process
• Distinguish between different types of literature reviews

1.1 What is a Literature Review?

Pick up nearly any book on research methods and you will find a description of a literature review.  At a basic level, the term implies a survey of factual or nonfiction books, articles, and other documents published on a particular subject.  Definitions may be similar across the disciplines, with new types and definitions continuing to emerge.  Generally speaking, a literature review is a:

• “comprehensive background of the literature within the interested topic area…” ( O’Gorman & MacIntosh, 2015, p. 31 ).
• “critical component of the research process that provides an in-depth analysis of recently published research findings in specifically identified areas of interest.” ( House, 2018, p. 109 ).
• “written document that presents a logically argued case founded on a comprehensive understanding of the current state of knowledge about a topic of study” ( Machi & McEvoy,  2012, p. 4 ).

As a foundation for knowledge advancement in every discipline, it is an important element of any research project.  At the graduate or doctoral level, the literature review is an essential feature of thesis and dissertation, as well as grant proposal writing.  That is to say, “A substantive, thorough, sophisticated literature review is a precondition for doing substantive, thorough, sophisticated research…A researcher cannot perform significant research without first understanding the literature in the field.” ( Boote & Beile, 2005, p. 3 ).  It is by this means, that a researcher demonstrates familiarity with a body of knowledge and thereby establishes credibility with a reader.  An advanced-level literature review shows how prior research is linked to a new project, summarizing and synthesizing what is known while identifying gaps in the knowledge base, facilitating theory development, closing areas where enough research already exists, and uncovering areas where more research is needed. ( Webster & Watson, 2002, p. xiii )

A graduate-level literature review is a compilation of the most significant previously published research on your topic. Unlike an annotated bibliography or a research paper you may have written as an undergraduate, your literature review will outline, evaluate and synthesize relevant research and relate those sources to your own thesis or research question. It is much more than a summary of all the related literature.

It is a type of writing that demonstrate the importance of your research by defining the main ideas and the relationship between them. A good literature review lays the foundation for the importance of your stated problem and research question.

Literature reviews:

• define a concept
• map the research terrain or scope
• systemize relationships between concepts
• identify gaps in the literature ( Rocco & Plathotnik, 2009, p. 128 )

The purpose of a literature review is to demonstrate that your research question  is meaningful. Additionally, you may review the literature of different disciplines to find deeper meaning and understanding of your topic. It is especially important to consider other disciplines when you do not find much on your topic in one discipline. You will need to search the cognate literature before claiming there is “little previous research” on your topic.

Well developed literature reviews involve numerous steps and activities. The literature review is an iterative process because you will do at least two of them: a preliminary search to learn what has been published in your area and whether there is sufficient support in the literature for moving ahead with your subject. After this first exploration, you will conduct a deeper dive into the literature to learn everything you can about the topic and its related issues.

Literature Review Tutorial

1.2 Literature Review Basics

An effective literature review must:

• Methodologically analyze and synthesize quality literature on a topic
• Provide a firm foundation to a topic or research area
• Provide a firm foundation for the selection of a research methodology
• Demonstrate that the proposed research contributes something new to the overall body of knowledge of advances the research field’s knowledge base. ( Levy & Ellis, 2006 ).

All literature reviews, whether they are qualitative, quantitative or both, will at some point:

• Introduce the topic and define its key terms
• Establish the importance of the topic
• Provide an overview of the amount of available literature and its types (for example: theoretical, statistical, speculative)
• Identify gaps in the literature
• Point out consistent finding across studies
• Arrive at a synthesis that organizes what is known about a topic
• Discusses possible implications and directions for future research

1.3 Types of Literature Reviews

There are many different types of literature reviews, however there are some shared characteristics or features.  Remember a comprehensive literature review is, at its most fundamental level, an original work based on an extensive critical examination and synthesis of the relevant literature on a topic. As a study of the research on a particular topic, it is arranged by key themes or findings, which may lead up to or link to the  research question.  In some cases, the research question will drive the type of literature review that is undertaken.

The following section includes brief descriptions of the terms used to describe different literature review types with examples of each.   The included citations are open access, Creative Commons licensed or copyright-restricted.

1.3.1 Types of Review

1.3.1.1 conceptual.

Guided by an understanding of basic issues rather than a research methodology. You are looking for key factors, concepts or variables and the presumed relationship between them. The goal of the conceptual literature review is to categorize and describe concepts relevant to your study or topic and outline a relationship between them. You will include relevant theory and empirical research.

Examples of a Conceptual Review:

• Education : The formality of learning science in everyday life: A conceptual literature review. ( Dohn, 2010 ).
• Education : Are we asking the right questions? A conceptual review of the educational development literature in higher education. ( Amundsen & Wilson, 2012 ).

1.3.1.2 Empirical

An empirical literature review collects, creates, arranges, and analyzes numeric data reflecting the frequency of themes, topics, authors and/or methods found in existing literature. Empirical literature reviews present their summaries in quantifiable terms using descriptive and inferential statistics.

Examples of an Empirical Review:

• Nursing : False-positive findings in Cochrane meta-analyses with and without application of trial sequential analysis: An empirical review. ( Imberger, Thorlund, Gluud, & Wettersley, 2016 ).
• Education : Impediments of e-learning adoption in higher learning institutions of Tanzania: An empirical review ( Mwakyusa & Mwalyagile, 2016 ).

1.3.1.3 Exploratory

Unlike a synoptic literature review, the purpose here is to provide a broad approach to the topic area. The aim is breadth rather than depth and to get a general feel for the size of the topic area. A graduate student might do an exploratory review of the literature before beginning a synoptic, or more comprehensive one.

Examples of an Exploratory Review:

• Education : University research management: An exploratory literature review. ( Schuetzenmeister, 2010 ).
• Education : An exploratory review of design principles in constructivist gaming learning environments. ( Rosario & Widmeyer, 2009 ).

1.3.1.4 Focused

A type of literature review limited to a single aspect of previous research, such as methodology. A focused literature review generally will describe the implications of choosing a particular element of past research, such as methodology in terms of data collection, analysis and interpretation.

Examples of a Focused Review:

• Nursing : Clinical inertia in the management of type 2 diabetes mellitus: A focused literature review. ( Khunti, Davies, & Khunti, 2015 ).
• Education : Language awareness: Genre awareness-a focused review of the literature. ( Stainton, 1992 ).

1.3.1.5 Integrative

Critiques past research and draws overall conclusions from the body of literature at a specified point in time. Reviews, critiques, and synthesizes representative literature on a topic in an integrated way. Most integrative reviews are intended to address mature topics or  emerging topics. May require the author to adopt a guiding theory, a set of competing models, or a point of view about a topic.  For more description of integrative reviews, see Whittemore & Knafl (2005).

Examples of an Integrative Review:

• Nursing : Interprofessional teamwork and collaboration between community health workers and healthcare teams: An integrative review. ( Franklin,  Bernhardt, Lopez, Long-Middleton, & Davis, 2015 ).
• Education : Exploring the gap between teacher certification and permanent employment in Ontario: An integrative literature review. ( Brock & Ryan, 2016 ).

1.3.1.6 Meta-analysis

A subset of a  systematic review, that takes findings from several studies on the same subject and analyzes them using standardized statistical procedures to pool together data. Integrates findings from a large body of quantitative findings to enhance understanding, draw conclusions, and detect patterns and relationships. Gather data from many different, independent studies that look at the same research question and assess similar outcome measures. Data is combined and re-analyzed, providing a greater statistical power than any single study alone. It’s important to note that not every systematic review includes a meta-analysis but a meta-analysis can’t exist without a systematic review of the literature.

Examples of a Meta-Analysis:

• Education : Efficacy of the cooperative learning method on mathematics achievement and attitude: A meta-analysis research. ( Capar & Tarim, 2015 ).
• Nursing : A meta-analysis of the effects of non-traditional teaching methods on the critical thinking abilities of nursing students. ( Lee, Lee, Gong, Bae, & Choi, 2016 ).
• Education : Gender differences in student attitudes toward science: A meta-analysis of the literature from 1970 to 1991. ( Weinburgh, 1995 ).

1.3.1.7 Narrative/Traditional

An overview of research on a particular topic that critiques and summarizes a body of literature. Typically broad in focus. Relevant past research is selected and synthesized into a coherent discussion. Methodologies, findings and limits of the existing body of knowledge are discussed in narrative form. Sometimes also referred to as a traditional literature review. Requires a sufficiently focused research question. The process may be subject to bias that supports the researcher’s own work.

Examples of a Narrative/Traditional Review:

• Nursing : Family carers providing support to a person dying in the home setting: A narrative literature review. ( Morris, King, Turner, & Payne, 2015 ).
• Education : Adventure education and Outward Bound: Out-of-class experiences that make a lasting difference. ( Hattie, Marsh, Neill, & Richards, 1997 ).
• Education : Good quality discussion is necessary but not sufficient in asynchronous tuition: A brief narrative review of the literature. ( Fear & Erikson-Brown, 2014 ).
• Nursing : Outcomes of physician job satisfaction: A narrative review, implications, and directions for future research. ( Williams & Skinner, 2003 ).

1.3.1.8 Realist

Aspecific type of literature review that is theory-driven and interpretative and is intended to explain the outcomes of a complex intervention program(s).

Examples of a Realist Review:

• Nursing : Lean thinking in healthcare: A realist review of the literature. ( Mazzacato, Savage, Brommels, 2010 ).
• Education : Unravelling quality culture in higher education: A realist review. ( Bendermacher, Egbrink, Wolfhagen, & Dolmans, 2017 ).

1.3.1.9 Scoping

Tend to be non-systematic and focus on breadth of coverage conducted on a topic rather than depth. Utilize a wide range of materials; may not evaluate the quality of the studies as much as count the number. One means of understanding existing literature. Aims to identify nature and extent of research; preliminary assessment of size and scope of available research on topic. May include research in progress.

Examples of a Scoping Review:

• Nursing : Organizational interventions improving access to community-based primary health care for vulnerable populations: A scoping review. ( Khanassov, Pluye, Descoteaux, Haggerty,  Russell, Gunn, & Levesque, 2016 ).
• Education : Interdisciplinary doctoral research supervision: A scoping review. ( Vanstone, Hibbert, Kinsella, McKenzie, Pitman, & Lingard, 2013 ).
• Nursing : A scoping review of the literature on the abolition of user fees in health care services in Africa. ( Ridde, & Morestin, 2011 ).

1.3.1.10 Synoptic

Unlike an exploratory review, the purpose is to provide a concise but accurate overview of all material that appears to be relevant to a chosen topic. Both content and methodological material is included. The review should aim to be both descriptive and evaluative. Summarizes previous studies while also showing how the body of literature could be extended and improved in terms of content and method by identifying gaps.

Examples of a Synoptic Review:

• Education : Theoretical framework for educational assessment: A synoptic review. ( Ghaicha, 2016 ).
• Education : School effects research: A synoptic review of past efforts and some suggestions for the future. ( Cuttance, 1981 ).

1.3.1.11 Systematic Review

A rigorous review that follows a strict methodology designed with a presupposed selection of literature reviewed.  Undertaken to clarify the state of existing research, the evidence, and possible implications that can be drawn from that.  Using comprehensive and exhaustive searching of the published and unpublished literature, searching various databases, reports, and grey literature.  Transparent and reproducible in reporting details of time frame, search and methods to minimize bias.  Must include a team of at least 2-3 and includes the critical appraisal of the literature.  For more description of systematic reviews, including links to protocols, checklists, workflow processes, and structure see “ A Young Researcher’s Guide to a Systematic Review “.

Examples of a Systematic Review:

• Education : The potentials of using cloud computing in schools: A systematic literature review ( Hartmann, Braae, Pedersen, & Khalid, 2017 )
• Nursing : Is butter back? A systematic review and meta-analysis of butter consumption and risk of cardiovascular disease, diabetes, and total mortality. ( Pimpin, Wu, Haskelberg, Del Gobbo, & Mozaffarian, 2016 ).
• Education : The use of research to improve professional practice: a systematic review of the literature. ( Hemsley-Brown & Sharp, 2003 ).
• Nursing : Using computers to self-manage type 2 diabetes. ( Pal, Eastwood, Michie, Farmer, Barnard, Peacock, Wood, Inniss, & Murray, 2013 ).

1.3.1.12 Umbrella/Overview of Reviews

Compiles evidence from multiple systematic reviews into one document. Focuses on broad condition or problem for which there are competing interventions and highlights reviews that address those interventions and their effects. Often used in recommendations for practice.

Examples of an Umbrella/Overview Review:

• Education : Reflective practice in healthcare education: An umbrella review. ( Fragknos, 2016 ).
• Nursing : Systematic reviews of psychosocial interventions for autism: an umbrella review. ( Seida, Ospina, Karkhaneh, Hartling, Smith, & Clark, 2009 ).

For a brief discussion see “ Not all literature reviews are the same ” (Thomson, 2013).

1.4 Why do a Literature Review?

The purpose of the literature review is the same regardless of the topic or research method. It tests your own research question against what is already known about the subject.

1.4.1 First – It’s part of the whole. Omission of a literature review chapter or section in a graduate-level project represents a serious void or absence of critical element in the research process.

The outcome of your review is expected to demonstrate that you:

• can systematically explore the research in your topic area
• can read and critically analyze the literature in your discipline and then use it appropriately to advance your own work
• have sufficient knowledge in the topic to undertake further investigation

1.4.2 Second – It’s good for you!

• You improve your skills as a researcher
• You become familiar with the discourse of your discipline and learn how to be a scholar in your field
• You learn through writing your ideas and finding your voice in your subject area
• You define, redefine and clarify your research question for yourself in the process

1.4.3 Third – It’s good for your reader. Your reader expects you to have done the hard work of gathering, evaluating and synthesizes the literature.  When you do a literature review you:

• Set the context for the topic and present its significance
• Identify what’s important to know about your topic – including individual material, prior research, publications, organizations and authors.
• Demonstrate relationships among prior research
• Establish limitations of existing knowledge
• Analyze trends in the topic’s treatment and gaps in the literature

1.4.4 Why do a literature review?

• To locate gaps in the literature of your discipline
• To avoid reinventing the wheel
• To carry on where others have already been
• To identify other people working in the same field
• To increase your breadth of knowledge in your subject area
• To find the seminal works in your field
• To provide intellectual context for your own work
• To acknowledge opposing viewpoints
• To put your work in perspective
• To demonstrate you can discover and retrieve previous work in the area

1.5 Common Literature Review Errors

Graduate-level literature reviews are more than a summary of the publications you find on a topic.  As you have seen in this brief introduction, literature reviews are a very specific type of research, analysis, and writing.  We will explore these topics more in the next chapters.  Some things to keep in mind as you begin your own research and writing are ways to avoid the most common errors seen in the first attempt at a literature review.  For a quick review of some of the pitfalls and challenges a new researcher faces when he/she begins work, see “ Get Ready: Academic Writing, General Pitfalls and (oh yes) Getting Started! ”.

As you begin your own graduate-level literature review, try to avoid these common mistakes:

• Accepts another researcher’s finding as valid without evaluating methodology and data
• Contrary findings and alternative interpretations are not considered or mentioned
• Findings are not clearly related to one’s own study, or findings are too general
• Insufficient time allowed to define best search strategies and writing
• Isolated statistical results are simply reported rather than synthesizing the results
• Problems with selecting and using most relevant keywords, subject headings and descriptors
• Relies too heavily on secondary sources
• Search methods are not recorded or reported for transparency
• Summarizes rather than synthesizes articles

In conclusion, the purpose of a literature review is three-fold:

• to survey the current state of knowledge or evidence in the area of inquiry,
• to identify key authors, articles, theories, and findings in that area, and
• to identify gaps in knowledge in that research area.

A literature review is commonly done today using computerized keyword searches in online databases, often working with a trained librarian or information expert. Keywords can be combined using the Boolean operators, “and”, “or” and sometimes “not”  to narrow down or expand the search results. Once a list of articles is generated from the keyword and subject heading search, the researcher must then manually browse through each title and abstract, to determine the suitability of that article before a full-text article is obtained for the research question.

Literature reviews should be reasonably complete, and not restricted to a few journals, a few years, or a specific methodology or research design. Reviewed articles may be summarized in the form of tables, and can be further structured using organizing frameworks such as a concept matrix.

A well-conducted literature review should indicate whether the initial research questions have already been addressed in the literature, whether there are newer or more interesting research questions available, and whether the original research questions should be modified or changed in light of findings of the literature review.

The review can also provide some intuitions or potential answers to the questions of interest and/or help identify theories that have previously been used to address similar questions and may provide evidence to inform policy or decision-making. ( Bhattacherjee, 2012 ).

Read Abstract 1.  Refer to Types of Literature Reviews.  What type of literature review do you think this study is and why?  See the Answer Key for the correct response.

Nursing : To describe evidence of international literature on the safe care of the hospitalised child after the World Alliance for Patient Safety and list contributions of the general theoretical framework of patient safety for paediatric nursing.

An integrative literature review between 2004 and 2015 using the databases PubMed, Cumulative Index of Nursing and Allied Health Literature (CINAHL), Scopus, Web of Science and Wiley Online Library, and the descriptors Safety or Patient safety, Hospitalised child, Paediatric nursing, and Nursing care.

Thirty-two articles were analysed, most of which were from North American, with a descriptive approach. The quality of the recorded information in the medical records, the use of checklists, and the training of health workers contribute to safe care in paediatric nursing and improve the medication process and partnerships with parents.

General information available on patient safety should be incorporated in paediatric nursing care. ( Wegner, Silva, Peres, Bandeira, Frantz, Botene, & Predebon, 2017 ).

Read Abstract 2.  Refer to Types of Literature Reviews.  What type of lit review do you think this study is and why?  See the Answer Key for the correct response.

Education : The focus of this paper centers around timing associated with early childhood education programs and interventions using meta-analytic methods. At any given assessment age, a child’s current age equals starting age, plus duration of program, plus years since program ended. Variability in assessment ages across the studies should enable everyone to identify the separate effects of all three time-related components. The project is a meta-analysis of evaluation studies of early childhood education programs conducted in the United States and its territories between 1960 and 2007. The population of interest is children enrolled in early childhood education programs between the ages of 0 and 5 and their control-group counterparts. Since the data come from a meta-analysis, the population for this study is drawn from many different studies with diverse samples. Given the preliminary nature of their analysis, the authors cannot offer conclusions at this point. ( Duncan, Leak, Li, Magnuson, Schindler, & Yoshikawa, 2011 ).

Test Yourself

See Answer Key for the correct responses.

The purpose of a graduate-level literature review is to summarize in as many words as possible everything that is known about my topic.

A literature review is significant because in the process of doing one, the researcher learns to read and critically assess the literature of a discipline and then uses it appropriately to advance his/her own research.

Read the following abstract and choose the correct type of literature review it represents.

Nursing: E-cigarette use has become increasingly popular, especially among the young. Its long-term influence upon health is unknown. Aim of this review has been to present the current state of knowledge about the impact of e-cigarette use on health, with an emphasis on Central and Eastern Europe. During the preparation of this narrative review, the literature on e-cigarettes available within the network PubMed was retrieved and examined. In the final review, 64 research papers were included. We specifically assessed the construction and operation of the e-cigarette as well as the chemical composition of the e-liquid; the impact that vapor arising from the use of e-cigarette explored in experimental models in vitro; and short-term effects of use of e-cigarettes on users’ health. Among the substances inhaled by the e-smoker, there are several harmful products, such as: formaldehyde, acetaldehyde, acroleine, propanal, nicotine, acetone, o-methyl-benzaldehyde, carcinogenic nitrosamines. Results from experimental animal studies indicate the negative impact of e-cigarette exposure on test models, such as ascytotoxicity, oxidative stress, inflammation, airway hyper reactivity, airway remodeling, mucin production, apoptosis, and emphysematous changes. The short-term impact of e-cigarettes on human health has been studied mostly in experimental setting. Available evidence shows that the use of e-cigarettes may result in acute lung function responses (e.g., increase in impedance, peripheral airway flow resistance) and induce oxidative stress. Based on the current available evidence, e-cigarette use is associated with harmful biologic responses, although it may be less harmful than traditional cigarettes. (J ankowski, Brożek, Lawson, Skoczyński, & Zejda, 2017 ).

• Meta-analysis
• Exploratory

Education: In this review, Mary Vorsino writes that she is interested in keeping the potential influences of women pragmatists of Dewey’s day in mind while presenting modern feminist re readings of Dewey. She wishes to construct a narrowly-focused and succinct literature review of thinkers who have donned a feminist lens to analyze Dewey’s approaches to education, learning, and democracy and to employ Dewey’s works in theorizing on gender and education and on gender in society. This article first explores Dewey as both an ally and a problematic figure in feminist literature and then investigates the broader sphere of feminist pragmatism and two central themes within it: (1) valuing diversity, and diverse experiences; and (2) problematizing fixed truths. ( Vorsino, 2015 ).

Image Attributions

Literature Reviews for Education and Nursing Graduate Students Copyright © by Linda Frederiksen is licensed under a Creative Commons Attribution 4.0 International License , except where otherwise noted.

Share This Book

Literature Reviews

• Getting started

What is a literature review?

Why conduct a literature review, stages of a literature review, lit reviews: an overview (video), check out these books.

• Types of reviews
• 1. Define your research question
• 2. Plan your search
• 3. Search the literature
• 4. Organize your results
• 5. Synthesize your findings
• 6. Write the review
• Artificial intelligence (AI) tools
• Thompson Writing Studio This link opens in a new window
• Need to write a systematic review? This link opens in a new window

Contact a Librarian

Ask a Librarian

Definition: A literature review is a systematic examination and synthesis of existing scholarly research on a specific topic or subject.

Purpose: It serves to provide a comprehensive overview of the current state of knowledge within a particular field.

Analysis: Involves critically evaluating and summarizing key findings, methodologies, and debates found in academic literature.

Identifying Gaps: Aims to pinpoint areas where there is a lack of research or unresolved questions, highlighting opportunities for further investigation.

Contextualization: Enables researchers to understand how their work fits into the broader academic conversation and contributes to the existing body of knowledge.

tl;dr  A literature review critically examines and synthesizes existing scholarly research and publications on a specific topic to provide a comprehensive understanding of the current state of knowledge in the field.

What is a literature review NOT?

❌ An annotated bibliography

❌ Original research

❌ A summary

❌ Something to be conducted at the end of your research

❌ An opinion piece

❌ A chronological compilation of studies

The reason for conducting a literature review is to:

Literature Reviews: An Overview for Graduate Students

While this 9-minute video from NCSU is geared toward graduate students, it is useful for anyone conducting a literature review.

Writing the literature review: A practical guide

Available 3rd floor of Perkins

Writing literature reviews: A guide for students of the social and behavioral sciences

Available online!

So, you have to write a literature review: A guided workbook for engineers

Telling a research story: Writing a literature review

The literature review: Six steps to success

Systematic approaches to a successful literature review

Request from Duke Medical Center Library

Doing a systematic review: A student's guide

• Next: Types of reviews >>
• Last Updated: May 17, 2024 8:42 AM
• URL: https://guides.library.duke.edu/litreviews

Services for...

• Faculty & Instructors
• Graduate Students
• Undergraduate Students
• International Students
• Patrons with Disabilities

• Harmful Language Statement
• Re-use & Attribution / Privacy
• Support the Libraries

• AERA Open Editors
• AERJ Editors
• EEPA Editors
• ER Issues and Archives
• JEBS Editors
• JSTOR Online Archives
• RER Editors
• RRE Editors
• AERA Examination and Desk Copies
• Mail/Fax Book Order Form
• International Distribution
• Books & Publications
• Merchandise
• Search The Store
• Online Paper Repository
• Inaugural Presentations in the i-Presentation Gallery
• Research Points
• AERA Journal Advertising Rate Cards
• Publications Permissions
• Publications FAQs

Share 

The  Review of Educational Research  ( RER , bimonthly, begun in 1931) publishes critical, integrative reviews of research literature bearing on education. Such reviews should include conceptualizations, interpretations, and syntheses of literature and scholarly work in a field broadly relevant to education and educational research.  RER  encourages the submission of research relevant to education from any discipline, such as reviews of research in psychology, sociology, history, philosophy, political science, economics, computer science, statistics, anthropology, and biology, provided that the review bears on educational issues.  RER  does not publish original empirical research unless it is incorporated in a broader integrative review.  RER  will occasionally publish solicited, but carefully refereed, analytic reviews of special topics, particularly from disciplines infrequently represented.

Impact Factor : 11.2 5-Year Impact Factor : 16.6 Ranking : 1/263 in Education & Educational Research

• Search Menu
• Browse content in Arts and Humanities
• Browse content in Archaeology
• Anglo-Saxon and Medieval Archaeology
• Archaeological Methodology and Techniques
• Archaeology by Region
• Archaeology of Religion
• Archaeology of Trade and Exchange
• Biblical Archaeology
• Contemporary and Public Archaeology
• Environmental Archaeology
• Historical Archaeology
• History and Theory of Archaeology
• Industrial Archaeology
• Landscape Archaeology
• Mortuary Archaeology
• Prehistoric Archaeology
• Underwater Archaeology
• Urban Archaeology
• Zooarchaeology
• Browse content in Architecture
• Architectural Structure and Design
• History of Architecture
• Residential and Domestic Buildings
• Theory of Architecture
• Browse content in Art
• Art Subjects and Themes
• History of Art
• Industrial and Commercial Art
• Theory of Art
• Biographical Studies
• Byzantine Studies
• Browse content in Classical Studies
• Classical History
• Classical Philosophy
• Classical Mythology
• Classical Literature
• Classical Reception
• Classical Art and Architecture
• Classical Oratory and Rhetoric
• Greek and Roman Epigraphy
• Greek and Roman Law
• Greek and Roman Papyrology
• Greek and Roman Archaeology
• Late Antiquity
• Religion in the Ancient World
• Digital Humanities
• Browse content in History
• Colonialism and Imperialism
• Diplomatic History
• Environmental History
• Genealogy, Heraldry, Names, and Honours
• Genocide and Ethnic Cleansing
• Historical Geography
• History by Period
• History of Emotions
• History of Agriculture
• History of Education
• History of Gender and Sexuality
• Industrial History
• Intellectual History
• International History
• Labour History
• Legal and Constitutional History
• Local and Family History
• Maritime History
• Military History
• National Liberation and Post-Colonialism
• Oral History
• Political History
• Public History
• Regional and National History
• Revolutions and Rebellions
• Slavery and Abolition of Slavery
• Social and Cultural History
• Theory, Methods, and Historiography
• Urban History
• World History
• Browse content in Language Teaching and Learning
• Language Learning (Specific Skills)
• Language Teaching Theory and Methods
• Browse content in Linguistics
• Applied Linguistics
• Cognitive Linguistics
• Computational Linguistics
• Forensic Linguistics
• Grammar, Syntax and Morphology
• Historical and Diachronic Linguistics
• History of English
• Language Acquisition
• Language Evolution
• Language Reference
• Language Variation
• Language Families
• Lexicography
• Linguistic Anthropology
• Linguistic Theories
• Linguistic Typology
• Phonetics and Phonology
• Psycholinguistics
• Sociolinguistics
• Translation and Interpretation
• Writing Systems
• Browse content in Literature
• Bibliography
• Children's Literature Studies
• Literary Studies (Asian)
• Literary Studies (European)
• Literary Studies (Eco-criticism)
• Literary Studies (Romanticism)
• Literary Studies (American)
• Literary Studies (Modernism)
• Literary Studies - World
• Literary Studies (1500 to 1800)
• Literary Studies (19th Century)
• Literary Studies (20th Century onwards)
• Literary Studies (African American Literature)
• Literary Studies (British and Irish)
• Literary Studies (Early and Medieval)
• Literary Studies (Fiction, Novelists, and Prose Writers)
• Literary Studies (Gender Studies)
• Literary Studies (Graphic Novels)
• Literary Studies (History of the Book)
• Literary Studies (Plays and Playwrights)
• Literary Studies (Poetry and Poets)
• Literary Studies (Postcolonial Literature)
• Literary Studies (Queer Studies)
• Literary Studies (Science Fiction)
• Literary Studies (Travel Literature)
• Literary Studies (War Literature)
• Literary Studies (Women's Writing)
• Literary Theory and Cultural Studies
• Mythology and Folklore
• Shakespeare Studies and Criticism
• Browse content in Media Studies
• Browse content in Music
• Applied Music
• Dance and Music
• Ethics in Music
• Ethnomusicology
• Gender and Sexuality in Music
• Medicine and Music
• Music Cultures
• Music and Religion
• Music and Media
• Music and Culture
• Music Education and Pedagogy
• Music Theory and Analysis
• Musical Scores, Lyrics, and Libretti
• Musical Structures, Styles, and Techniques
• Musicology and Music History
• Performance Practice and Studies
• Race and Ethnicity in Music
• Sound Studies
• Browse content in Performing Arts
• Browse content in Philosophy
• Aesthetics and Philosophy of Art
• Epistemology
• Feminist Philosophy
• History of Western Philosophy
• Metaphysics
• Moral Philosophy
• Non-Western Philosophy
• Philosophy of Science
• Philosophy of Language
• Philosophy of Mind
• Philosophy of Perception
• Philosophy of Action
• Philosophy of Law
• Philosophy of Religion
• Philosophy of Mathematics and Logic
• Practical Ethics
• Social and Political Philosophy
• Browse content in Religion
• Biblical Studies
• Christianity
• East Asian Religions
• History of Religion
• Judaism and Jewish Studies
• Qumran Studies
• Religion and Education
• Religion and Health
• Religion and Politics
• Religion and Science
• Religion and Law
• Religion and Art, Literature, and Music
• Religious Studies
• Browse content in Society and Culture
• Cookery, Food, and Drink
• Cultural Studies
• Customs and Traditions
• Ethical Issues and Debates
• Hobbies, Games, Arts and Crafts
• Lifestyle, Home, and Garden
• Natural world, Country Life, and Pets
• Popular Beliefs and Controversial Knowledge
• Sports and Outdoor Recreation
• Technology and Society
• Travel and Holiday
• Visual Culture
• Browse content in Law
• Arbitration
• Browse content in Company and Commercial Law
• Commercial Law
• Company Law
• Browse content in Comparative Law
• Systems of Law
• Competition Law
• Browse content in Constitutional and Administrative Law
• Government Powers
• Judicial Review
• Local Government Law
• Military and Defence Law
• Parliamentary and Legislative Practice
• Construction Law
• Contract Law
• Browse content in Criminal Law
• Criminal Procedure
• Criminal Evidence Law
• Sentencing and Punishment
• Employment and Labour Law
• Environment and Energy Law
• Browse content in Financial Law
• Banking Law
• Insolvency Law
• History of Law
• Human Rights and Immigration
• Intellectual Property Law
• Browse content in International Law
• Private International Law and Conflict of Laws
• Public International Law
• IT and Communications Law
• Jurisprudence and Philosophy of Law
• Law and Politics
• Law and Society
• Browse content in Legal System and Practice
• Courts and Procedure
• Legal Skills and Practice
• Primary Sources of Law
• Regulation of Legal Profession
• Medical and Healthcare Law
• Browse content in Policing
• Criminal Investigation and Detection
• Police and Security Services
• Police Procedure and Law
• Police Regional Planning
• Browse content in Property Law
• Personal Property Law
• Study and Revision
• Terrorism and National Security Law
• Browse content in Trusts Law
• Wills and Probate or Succession
• Browse content in Medicine and Health
• Browse content in Allied Health Professions
• Arts Therapies
• Clinical Science
• Dietetics and Nutrition
• Occupational Therapy
• Operating Department Practice
• Physiotherapy
• Radiography
• Speech and Language Therapy
• Browse content in Anaesthetics
• General Anaesthesia
• Neuroanaesthesia
• Browse content in Clinical Medicine
• Acute Medicine
• Cardiovascular Medicine
• Clinical Genetics
• Clinical Pharmacology and Therapeutics
• Dermatology
• Endocrinology and Diabetes
• Gastroenterology
• Genito-urinary Medicine
• Geriatric Medicine
• Infectious Diseases
• Medical Toxicology
• Medical Oncology
• Pain Medicine
• Palliative Medicine
• Rehabilitation Medicine
• Respiratory Medicine and Pulmonology
• Rheumatology
• Sleep Medicine
• Sports and Exercise Medicine
• Clinical Neuroscience
• Community Medical Services
• Critical Care
• Emergency Medicine
• Forensic Medicine
• Haematology
• History of Medicine
• Browse content in Medical Dentistry
• Oral and Maxillofacial Surgery
• Paediatric Dentistry
• Restorative Dentistry and Orthodontics
• Surgical Dentistry
• Browse content in Medical Skills
• Clinical Skills
• Communication Skills
• Nursing Skills
• Surgical Skills
• Medical Ethics
• Medical Statistics and Methodology
• Browse content in Neurology
• Clinical Neurophysiology
• Neuropathology
• Nursing Studies
• Browse content in Obstetrics and Gynaecology
• Gynaecology
• Occupational Medicine
• Ophthalmology
• Otolaryngology (ENT)
• Browse content in Paediatrics
• Neonatology
• Browse content in Pathology
• Chemical Pathology
• Clinical Cytogenetics and Molecular Genetics
• Histopathology
• Medical Microbiology and Virology
• Patient Education and Information
• Browse content in Pharmacology
• Psychopharmacology
• Browse content in Popular Health
• Caring for Others
• Complementary and Alternative Medicine
• Self-help and Personal Development
• Browse content in Preclinical Medicine
• Cell Biology
• Molecular Biology and Genetics
• Reproduction, Growth and Development
• Primary Care
• Professional Development in Medicine
• Browse content in Psychiatry
• Addiction Medicine
• Child and Adolescent Psychiatry
• Forensic Psychiatry
• Learning Disabilities
• Old Age Psychiatry
• Psychotherapy
• Browse content in Public Health and Epidemiology
• Epidemiology
• Public Health
• Browse content in Radiology
• Clinical Radiology
• Interventional Radiology
• Nuclear Medicine
• Radiation Oncology
• Reproductive Medicine
• Browse content in Surgery
• Cardiothoracic Surgery
• Gastro-intestinal and Colorectal Surgery
• General Surgery
• Neurosurgery
• Paediatric Surgery
• Peri-operative Care
• Plastic and Reconstructive Surgery
• Surgical Oncology
• Transplant Surgery
• Trauma and Orthopaedic Surgery
• Vascular Surgery
• Browse content in Science and Mathematics
• Browse content in Biological Sciences
• Aquatic Biology
• Biochemistry
• Bioinformatics and Computational Biology
• Developmental Biology
• Ecology and Conservation
• Evolutionary Biology
• Genetics and Genomics
• Microbiology
• Molecular and Cell Biology
• Natural History
• Plant Sciences and Forestry
• Research Methods in Life Sciences
• Structural Biology
• Systems Biology
• Zoology and Animal Sciences
• Browse content in Chemistry
• Analytical Chemistry
• Computational Chemistry
• Crystallography
• Environmental Chemistry
• Industrial Chemistry
• Inorganic Chemistry
• Materials Chemistry
• Medicinal Chemistry
• Mineralogy and Gems
• Organic Chemistry
• Physical Chemistry
• Polymer Chemistry
• Study and Communication Skills in Chemistry
• Theoretical Chemistry
• Browse content in Computer Science
• Artificial Intelligence
• Computer Architecture and Logic Design
• Game Studies
• Human-Computer Interaction
• Mathematical Theory of Computation
• Programming Languages
• Software Engineering
• Systems Analysis and Design
• Virtual Reality
• Browse content in Computing
• Business Applications
• Computer Security
• Computer Games
• Computer Networking and Communications
• Digital Lifestyle
• Graphical and Digital Media Applications
• Operating Systems
• Browse content in Earth Sciences and Geography
• Atmospheric Sciences
• Environmental Geography
• Geology and the Lithosphere
• Maps and Map-making
• Meteorology and Climatology
• Oceanography and Hydrology
• Palaeontology
• Physical Geography and Topography
• Regional Geography
• Soil Science
• Urban Geography
• Browse content in Engineering and Technology
• Agriculture and Farming
• Biological Engineering
• Civil Engineering, Surveying, and Building
• Electronics and Communications Engineering
• Energy Technology
• Engineering (General)
• Environmental Science, Engineering, and Technology
• History of Engineering and Technology
• Mechanical Engineering and Materials
• Technology of Industrial Chemistry
• Transport Technology and Trades
• Browse content in Environmental Science
• Applied Ecology (Environmental Science)
• Conservation of the Environment (Environmental Science)
• Environmental Sustainability
• Environmentalist Thought and Ideology (Environmental Science)
• Management of Land and Natural Resources (Environmental Science)
• Natural Disasters (Environmental Science)
• Nuclear Issues (Environmental Science)
• Pollution and Threats to the Environment (Environmental Science)
• Social Impact of Environmental Issues (Environmental Science)
• History of Science and Technology
• Browse content in Materials Science
• Ceramics and Glasses
• Composite Materials
• Metals, Alloying, and Corrosion
• Nanotechnology
• Browse content in Mathematics
• Applied Mathematics
• Biomathematics and Statistics
• History of Mathematics
• Mathematical Education
• Mathematical Finance
• Mathematical Analysis
• Numerical and Computational Mathematics
• Probability and Statistics
• Pure Mathematics
• Browse content in Neuroscience
• Cognition and Behavioural Neuroscience
• Development of the Nervous System
• Disorders of the Nervous System
• History of Neuroscience
• Invertebrate Neurobiology
• Molecular and Cellular Systems
• Neuroendocrinology and Autonomic Nervous System
• Neuroscientific Techniques
• Sensory and Motor Systems
• Browse content in Physics
• Astronomy and Astrophysics
• Atomic, Molecular, and Optical Physics
• Biological and Medical Physics
• Classical Mechanics
• Computational Physics
• Condensed Matter Physics
• Electromagnetism, Optics, and Acoustics
• History of Physics
• Mathematical and Statistical Physics
• Measurement Science
• Nuclear Physics
• Particles and Fields
• Plasma Physics
• Quantum Physics
• Relativity and Gravitation
• Semiconductor and Mesoscopic Physics
• Browse content in Psychology
• Affective Sciences
• Clinical Psychology
• Cognitive Psychology
• Cognitive Neuroscience
• Criminal and Forensic Psychology
• Developmental Psychology
• Educational Psychology
• Evolutionary Psychology
• Health Psychology
• History and Systems in Psychology
• Music Psychology
• Neuropsychology
• Organizational Psychology
• Psychological Assessment and Testing
• Psychology of Human-Technology Interaction
• Psychology Professional Development and Training
• Research Methods in Psychology
• Social Psychology
• Browse content in Social Sciences
• Browse content in Anthropology
• Anthropology of Religion
• Human Evolution
• Medical Anthropology
• Physical Anthropology
• Regional Anthropology
• Social and Cultural Anthropology
• Theory and Practice of Anthropology
• Browse content in Business and Management
• Business Strategy
• Business Ethics
• Business History
• Business and Government
• Business and Technology
• Business and the Environment
• Comparative Management
• Corporate Governance
• Corporate Social Responsibility
• Entrepreneurship
• Health Management
• Human Resource Management
• Industrial and Employment Relations
• Industry Studies
• Information and Communication Technologies
• International Business
• Knowledge Management
• Management and Management Techniques
• Operations Management
• Organizational Theory and Behaviour
• Pensions and Pension Management
• Public and Nonprofit Management
• Strategic Management
• Supply Chain Management
• Browse content in Criminology and Criminal Justice
• Criminal Justice
• Criminology
• Forms of Crime
• International and Comparative Criminology
• Youth Violence and Juvenile Justice
• Development Studies
• Browse content in Economics
• Agricultural, Environmental, and Natural Resource Economics
• Asian Economics
• Behavioural Finance
• Behavioural Economics and Neuroeconomics
• Econometrics and Mathematical Economics
• Economic Systems
• Economic History
• Economic Methodology
• Economic Development and Growth
• Financial Markets
• Financial Institutions and Services
• General Economics and Teaching
• Health, Education, and Welfare
• History of Economic Thought
• International Economics
• Labour and Demographic Economics
• Law and Economics
• Macroeconomics and Monetary Economics
• Microeconomics
• Public Economics
• Urban, Rural, and Regional Economics
• Welfare Economics
• Browse content in Education
• Adult Education and Continuous Learning
• Care and Counselling of Students
• Early Childhood and Elementary Education
• Educational Equipment and Technology
• Educational Strategies and Policy
• Higher and Further Education
• Organization and Management of Education
• Philosophy and Theory of Education
• Schools Studies
• Secondary Education
• Teaching of a Specific Subject
• Teaching of Specific Groups and Special Educational Needs
• Teaching Skills and Techniques
• Browse content in Environment
• Applied Ecology (Social Science)
• Climate Change
• Conservation of the Environment (Social Science)
• Environmentalist Thought and Ideology (Social Science)
• Natural Disasters (Environment)
• Social Impact of Environmental Issues (Social Science)
• Browse content in Human Geography
• Cultural Geography
• Economic Geography
• Political Geography
• Browse content in Interdisciplinary Studies
• Communication Studies
• Museums, Libraries, and Information Sciences
• Browse content in Politics
• African Politics
• Asian Politics
• Chinese Politics
• Comparative Politics
• Conflict Politics
• Elections and Electoral Studies
• Environmental Politics
• European Union
• Foreign Policy
• Gender and Politics
• Human Rights and Politics
• Indian Politics
• International Relations
• International Organization (Politics)
• International Political Economy
• Irish Politics
• Latin American Politics
• Middle Eastern Politics
• Political Methodology
• Political Communication
• Political Philosophy
• Political Sociology
• Political Behaviour
• Political Economy
• Political Institutions
• Political Theory
• Politics and Law
• Public Administration
• Public Policy
• Quantitative Political Methodology
• Regional Political Studies
• Russian Politics
• Security Studies
• State and Local Government
• UK Politics
• US Politics
• Browse content in Regional and Area Studies
• African Studies
• Asian Studies
• East Asian Studies
• Japanese Studies
• Latin American Studies
• Middle Eastern Studies
• Native American Studies
• Scottish Studies
• Browse content in Research and Information
• Research Methods
• Browse content in Social Work
• Addictions and Substance Misuse
• Adoption and Fostering
• Care of the Elderly
• Child and Adolescent Social Work
• Couple and Family Social Work
• Developmental and Physical Disabilities Social Work
• Direct Practice and Clinical Social Work
• Emergency Services
• Human Behaviour and the Social Environment
• International and Global Issues in Social Work
• Mental and Behavioural Health
• Social Justice and Human Rights
• Social Policy and Advocacy
• Social Work and Crime and Justice
• Social Work Macro Practice
• Social Work Practice Settings
• Social Work Research and Evidence-based Practice
• Welfare and Benefit Systems
• Browse content in Sociology
• Childhood Studies
• Community Development
• Comparative and Historical Sociology
• Economic Sociology
• Gender and Sexuality
• Gerontology and Ageing
• Health, Illness, and Medicine
• Marriage and the Family
• Migration Studies
• Occupations, Professions, and Work
• Organizations
• Population and Demography
• Race and Ethnicity
• Social Theory
• Social Movements and Social Change
• Social Research and Statistics
• Social Stratification, Inequality, and Mobility
• Sociology of Religion
• Sociology of Education
• Sport and Leisure
• Urban and Rural Studies
• Browse content in Warfare and Defence
• Defence Strategy, Planning, and Research
• Land Forces and Warfare
• Military Administration
• Military Life and Institutions
• Naval Forces and Warfare
• Other Warfare and Defence Issues
• Peace Studies and Conflict Resolution
• Weapons and Equipment

• < Previous chapter
• Next chapter >

3 Conducting a Review of Related Literature

• Published: February 2023
• Cite Icon Cite
• Permissions Icon Permissions

This chapter explores approaches for reviewing and synthesizing research literature. Authors conduct reviews of literature in order to provide a context within which to place their study. A literature review can provide background information necessary for a reader to understand a study. Furthermore, syntheses of findings from related literature will reveal what is known about a topic and allow researchers to identify the gaps or weaknesses in the knowledge base, which can help to establish a rationale for a study. Quantitative, qualitative, and action research studies always include a review of related literature; historical and philosophical studies may not. Modern search engines make finding related literature easy but correspondingly difficult in that they return so many possibilities that it can take considerable time and effort to sort through and identify the most relevant sources. Using a reference management system can save many hours of labor in organizing and formatting references. Organizing information under various headings based on important elements of a study will help the reader understand how previous studies are related to the current study. A literature review should also entail critique; researchers are expected to point out weaknesses in data collection or analysis or discrepancies among various findings. Most important, a researcher must demonstrate how previous research is connected to their own. Perhaps they are attempting to fill in gaps in the literature, extend previous findings, or contradict prevailing notions.

Signed in as

Institutional accounts.

• Google Scholar Indexing
• GoogleCrawler [DO NOT DELETE]

Personal account

• Sign in with email/username & password
• Get email alerts
• Save searches
• Purchase content
• Activate your purchase/trial code
• Add your ORCID iD

Institutional access

Sign in with a library card.

• Sign in with username/password
• Recommend to your librarian
• Institutional account management
• Get help with access

Access to content on Oxford Academic is often provided through institutional subscriptions and purchases. If you are a member of an institution with an active account, you may be able to access content in one of the following ways:

IP based access

Typically, access is provided across an institutional network to a range of IP addresses. This authentication occurs automatically, and it is not possible to sign out of an IP authenticated account.

Sign in through your institution

Choose this option to get remote access when outside your institution. Shibboleth/Open Athens technology is used to provide single sign-on between your institution’s website and Oxford Academic.

• Click Sign in through your institution.
• Select your institution from the list provided, which will take you to your institution's website to sign in.
• When on the institution site, please use the credentials provided by your institution. Do not use an Oxford Academic personal account.
• Following successful sign in, you will be returned to Oxford Academic.

If your institution is not listed or you cannot sign in to your institution’s website, please contact your librarian or administrator.

Enter your library card number to sign in. If you cannot sign in, please contact your librarian.

Society Members

Society member access to a journal is achieved in one of the following ways:

Sign in through society site

Many societies offer single sign-on between the society website and Oxford Academic. If you see ‘Sign in through society site’ in the sign in pane within a journal:

• Click Sign in through society site.
• When on the society site, please use the credentials provided by that society. Do not use an Oxford Academic personal account.

If you do not have a society account or have forgotten your username or password, please contact your society.

Sign in using a personal account

Some societies use Oxford Academic personal accounts to provide access to their members. See below.

A personal account can be used to get email alerts, save searches, purchase content, and activate subscriptions.

Some societies use Oxford Academic personal accounts to provide access to their members.

Viewing your signed in accounts

Click the account icon in the top right to:

• View your signed in personal account and access account management features.
• View the institutional accounts that are providing access.

Signed in but can't access content

Oxford Academic is home to a wide variety of products. The institutional subscription may not cover the content that you are trying to access. If you believe you should have access to that content, please contact your librarian.

For librarians and administrators, your personal account also provides access to institutional account management. Here you will find options to view and activate subscriptions, manage institutional settings and access options, access usage statistics, and more.

Our books are available by subscription or purchase to libraries and institutions.

• About Oxford Academic
• Publish journals with us
• University press partners
• What we publish
• New features  
• Open access
• Rights and permissions
• Accessibility
• Advertising
• Media enquiries
• Oxford University Press
• Oxford Languages
• University of Oxford

Oxford University Press is a department of the University of Oxford. It furthers the University's objective of excellence in research, scholarship, and education by publishing worldwide

• Copyright © 2024 Oxford University Press
• Cookie settings
• Cookie policy
• Privacy policy
• Legal notice

This Feature Is Available To Subscribers Only

Sign In or Create an Account

This PDF is available to Subscribers Only

For full access to this pdf, sign in to an existing account, or purchase an annual subscription.

A Review of the Literature on Teacher Effectiveness and Student Outcomes

• Open Access
• First Online: 24 May 2019

Cite this chapter

You have full access to this open access chapter

• Nathan Burroughs 25 ,
• Jacqueline Gardner 26 ,
• Youngjun Lee 27 ,
• Siwen Guo 28 ,
• Israel Touitou 29 ,
• Kimberly Jansen 30 &
• William Schmidt 31  

Part of the book series: IEA Research for Education ((IEAR,volume 6))

148k Accesses

42 Citations

74 Altmetric

Researchers agree that teachers are one of the most important school-based resources in determining students’ future academic success and lifetime outcomes, yet have simultaneously had difficulties in defining what teacher characteristics make for an effective teacher. This chapter reviews the large body of literature on measures of teacher effectiveness, underscoring the diversity of methods by which the general construct of “teacher quality” has been explored, including experience, professional knowledge, and opportunity to learn. Each of these concepts comprises a number of different dimensions and methods of operationalizing. Single-country research (and particularly research from the United States) is distinguished from genuinely comparative work. Despite a voluminous research literature on the question of teacher quality, evidence for the impact of teacher characteristics (experience and professional knowledge) on student outcomes remains quite limited. There is a smaller, but more robust set of findings for the effect of teacher support on opportunity to learn. Five measures may be associated with higher student achievement: teacher experience (measured by years of teaching), teacher professional knowledge (measured by education and self-reported preparation to teach mathematics), and teacher provision of opportunity to learn (measured by time on mathematics and content coverage). These factors provide the basis for a comparative cross-country model.

You have full access to this open access chapter,  Download chapter PDF

• Opportunity to learn
• Teacher education
• Teacher experience
• Teacher quality
• Trends in International Mathematics and Science Study (TIMSS)

2.1 Defining Teacher Effectiveness

Researchers agree that teachers are one of the most important school-based resources in determining students’ future academic success and lifetime outcomes (Chetty et al. 2014 ; Rivkin et al. 2005 ; Rockoff 2004 ). As a consequence, there has been a strong emphasis on improving teacher effectiveness as a means to enhancing student learning. Goe ( 2007 ), among others, defined teacher effectiveness in terms of growth in student learning, typically measured by student standardized assessment results. Chetty et al. ( 2014 ) found that students taught by highly effective teachers, as defined by the student growth percentile (SGPs) and value-added measures (VAMs), were more likely to attend college, earn more, live in higher-income neighborhoods, save more money for retirement, and were less likely to have children during their teenage years. This potential of a highly effective teacher to significantly enhance the lives of their students makes it essential that researchers and policymakers properly understand the factors that contribute to a teacher’s effectiveness. However, as we will discuss in more detail later in this report, studies have found mixed results regarding the relationships between specific teacher characteristics and student achievement (Wayne and Youngs 2003 ). In this chapter, we explore these findings, focusing on the three main categories of teacher effectiveness identified and examined in the research literature: namely, teacher experience, teacher knowledge, and teacher behavior. Here we emphasize that much of the existing body of research is based on studies from the United States, and so the applicability of such national research to other contexts remains open to discussion.

2.2 Teacher Experience

Teacher experience refers to the number of years that a teacher has worked as a classroom teacher. Many studies show a positive relationship between teacher experiences and student achievement (Wayne and Youngs 2003 ). For example, using data from 4000 teachers in North Carolina, researchers found that teacher experience was positively related to student achievement in both reading and mathematics (Clotfelter et al. 2006 ). Rice ( 2003 ) found that the relationship between teacher experience and student achievement was most pronounced for students at the secondary level. Additional work in schools in the United States by Wiswall ( 2013 ), Papay and Kraft ( 2015 ), and Ladd and Sorenson ( 2017 ), and a Dutch twin study by Gerritsen et al. ( 2014 ), also indicated that teacher experience had a cumulative effect on student outcomes.

Meanwhile, other studies have failed to identify consistent and statistically significant associations between student achievement and teacher experience (Blomeke et al. 2016 ; Gustaffsson and Nilson 2016 ; Hanushek and Luque 2003 ; Luschei and Chudgar 2011 ; Wilson and Floden 2003 ). Some research from the United States has indicated that experience matters very much early on in a teacher’s career, but that, in later years, there were little to no additional gains (Boyd et al. 2006 ; Rivkin et al. 2005 ; Staiger and Rockoff 2010 ). In the first few years of a teacher’s career, accruing more years of experience seems to be more strongly related to student achievement (Rice 2003 ). Rockoff ( 2004 ) found that, when comparing teacher effectiveness (understood as value-added) to student test scores in reading and mathematics, teacher experience was positively related to student mathematics achievement; however, such positive relationships leveled off after teachers had gained two years of teaching experience. Drawing on data collected from teachers of grades four to eight between 2000 and 2008 within a large urban school district in the United States, Papay and Kraft ( 2015 ) confirmed previous research on the benefits experience can add to a novice teacher’s career. They found that student outcomes increased most rapidly during their teachers’ first few years of employment. They also found some further student gains due to additional years of teaching experience beyond the first five years. The research of Pil and Leana ( 2009 ) adds additional nuance; they found that acquiring teacher experience at the same grade level over a number of years, not just teacher experience in general (i.e. at multiple grades), was positively related to student achievement.

2.3 Teacher Professional Knowledge

A teacher’s professional knowledge refers to their subject-matter knowledge, curricular knowledge, and pedagogical knowledge (Collinson 1999 ). This professional knowledge is influenced by the undergraduate degrees earned by a teacher, the college attended, graduate studies undertaken, and opportunities to engage with on-the job training, commonly referred to as professional development (Collinson 1999 ; Rice 2003 ; Wayne and Youngs 2003 ). After undertaking in-depth quantitative analyses of the United States’ 1993–1994 Schools and Staffing Survey (SASS) and National Assessment of Educational Progress (NAEP) data sets, Darling-Hammond ( 2000 ) argued that measures of teacher preparation and certification were by far the strongest correlates of student achievement in reading and mathematics, after controlling for student poverty levels and language status.

As with experience, research on the impact of teacher advanced degrees, subject specializations, and certification has been inconclusive, with several studies (Aaronson et al. 2007 ; Blomeke et al. 2016 ; Hanushek and Luque 2003 ; Harris and Sass 2011 ; Luschei and Chudgar 2011 ) suggesting weak, inconsistent, or non-significant relationships with student achievement. However, several international studies comparing country means found that teacher degrees (Akiba et al. 2007 ; Gustaffsson and Nilson 2016 ; Montt 2011 ) were related to student outcomes, as did Woessman’s ( 2003 ) student-level study of multiple countries.

2.3.1 Undergraduate Education

In their meta-analysis of teacher effectiveness, Wayne and Youngs ( 2003 ) found three studies that showed some relationship between the quality of the undergraduate institution that a teacher attended and their future students’ success in standardized tests. In a thorough review of the research on teacher effectiveness attributes, Rice ( 2003 ) found that the selectivity of undergraduate institution and the teacher preparation program may be related to student achievement for students at the high school level and for high-poverty students.

In terms of teacher preparation programs, Boyd et al. ( 2009 ) found that overall these programs varied in their effectiveness. In their study of 31 teacher preparation programs designed to prepare teachers for the New York City School District, Boyd et al. ( 2009 ) drew from data based on document analyses, interviews, surveys of teacher preparation instructors, surveys of participants and graduates, and student value-added scores. They found that if a program was effective in preparing teachers to teach one subject, it tended to also have success in preparing teachers to teach other subjects as well. They also found that teacher preparation programs that focused on the practice of teaching and the classroom, and provided opportunities for teachers to study classroom practices, tended to prepare more effective teachers. Finally, they found that programs that included some sort of final project element (such as a personal research paper, or portfolio presentation) tended to prepare more effective teachers.

Beyond the institution a teacher attends, the coursework they choose to take within that program may also be related to their future students’ achievement. These associations vary by subject matter. A study by Rice ( 2003 ) indicated that, for teachers teaching at the secondary level, subject-specific coursework had a greater impact on their future students’ achievement. Similarly Goe ( 2007 ) found that, for mathematics, an increase in the amount of coursework undertaken by a trainee teacher was positively related to their future students’ achievement. By contrast, the meta-analysis completed by Wayne and Youngs ( 2003 ) found that, for history and English teachers, there was no evidence of a relationship between a teacher’s undergraduate coursework and their future students’ achievement in those subjects.

2.3.2 Graduate Education

In a review of 14 studies, Wilson and Floden ( 2003 ) were unable to identify consistent relationships between a teacher’s level of education and their students’ achievement. Similarly, in their review of data from 4000 teachers in North Carolina, Clotfelter et al. ( 2006 ) found that teachers who held a master’s degree were associated with lower student achievement. However, specifically in terms of mathematics instruction, teachers with higher degrees and who undertook more coursework during their education seem to be positively related to their students’ mathematics achievement (Goe 2007 ). Likewise, Harris and Sass ( 2011 ) found that there was a positive relationship between teachers who had obtained an advanced degree during their teaching career and their students’ achievement in middle school mathematics. They did not find any significant relationships between advanced degrees and student achievement in any other subject area. Further, using data from the United States’ Early Childhood Longitudinal Study (ECLS-K), Phillips ( 2010 ) found that subject-specific graduate degrees in elementary or early-childhood education were positively related to students’ reading achievement gains.

2.3.3 Certification Status

Another possible indicator of teacher effectiveness could be whether or not a teacher holds a teaching certificate. Much of this research has focused on the United States, which uses a variety of certification approaches, with lower grades usually having multi-subject general certifications and higher grades requiring certification in specific subjects. Wayne and Youngs ( 2003 ) found no clear relationship between US teachers’ certification status and their students’ achievement, with the exception of the subject area of mathematics, where students tended have higher test scores when their teachers had a standard mathematics certification. Rice ( 2003 ) also found that US teacher certification was related to high school mathematics achievement, and also found that there was some evidence of a relationship between certification status and student achievement in lower grades. Meanwhile, in their study of grade one students, Palardy and Rumberger ( 2008 ) also found evidence that students made greater gains in reading ability when taught by fully certified teachers.

In a longitudinal study using data from teachers teaching grades four and five and their students in the Houston School District in Texas, Darling-Hammond et al. ( 2005 ) found that those teachers who had completed training that resulted in a recognized teaching certificate were more effective that those who had no dedicated teaching qualifications. The study results suggested that teachers without recognized US certification or with non-standard certifications generally had negative effects on student achievement after controlling for student characteristics and prior achievement, as well as the teacher’s experience and degrees. The effects of teacher certification on student achievement were generally much stronger than the effects for teacher experience. Conversely, analyzing data from the ECLS-K, Phillips ( 2010 ) found that grade one students tended to have lower mathematics achievement gains when they had teachers with standard certification. In sum, the literature the influence of teacher certification remains deeply ambiguous.

2.3.4 Professional Development

Although work by Desimone et al. ( 2002 , 2013 ) suggested that professional development may influence the quality of instruction, most researchers found that teachers’ professional development experiences showed only limited associations with their effectiveness, although middle- and high-school mathematics teachers who undertook more content-focused training may be the exception (Blomeke et al. 2016 ; Harris and Sass 2011 ). In their meta-analysis of the effects of professional development on student achievement, Blank and De Las Alas ( 2009 ) found that 16 studies reported significant and positive relationships between professional development and student achievement. For mathematics, the average effect size of studies using a pre-post assessment design was 0.21 standard deviations.

Analyzing the data from six data sets, two from the Beginning Teacher Preparation Survey conducted in Connecticut and Tennessee, and four from the United States National Center for Education Statistics’ National Assessment of Educational Progress (NAEP), Wallace ( 2009 ) used structural equation modeling to find that professional development had a very small, but occasionally statistically significant effect on student achievement. She found, for example, that for NAEP mathematics data from the year 2000, 1.2 additional hours of professional development per year were related to an increase in average student scores of 0.62 points, and for reading, an additional 1.1 h of professional development were related to an average increase in student scores of 0.24 points. Overall, Wallace ( 2009 ) identified professional development had moderate effects on teacher practice and some small effects on student achievement when mediated by teacher practice.

2.3.5 Teacher Content Knowledge

Of course, characteristics like experience and education may be imperfect proxies for teacher content knowledge; unfortunately, content knowledge is difficult to assess directly. However, there is a growing body of work suggesting that teacher content knowledge may associated with student learning. It should be noted that there is an important distinction between general content knowledge about a subject (CK) and pedagogical content knowledge (PCK) specifically related to teaching that subject, each of which may be independently related to student outcomes (Baumert et al. 2010 ).

Studies from the United States (see for example, Chingos and Peterson 2011 ; Clotfelter et al. 2006 ; Constantine et al. 2009 ; Hill et al. 2005 ; Shuls and Trivitt 2015 ) have found some evidence that higher teacher cognitive skills in mathematics are associated with higher student scores. Positive associations between teacher content knowledge and student outcomes were also found in studies based in Germany (Baumert et al. 2010 ) and Peru (Metzler and Woessman 2012 ), and in a comparative study using Programme for the International Assessment of Adult Competencies (PIAAC) data undertaken by Hanushek et al. ( 2018 ). These findings are not universal, however, other studies from the United States (Blazar 2015 ; Garet et al. 2016 ; Rockoff et al. 2011 ) failed to find a statistically significant association between teacher content knowledge and student learning.

The studies we have discussed all used some direct measure of teacher content knowledge. An alternative method of assessing mathematics teacher content knowledge is self-reported teacher preparation to teach mathematics topics. Both TIMSS and IEA’s Teacher Education and Development Study in Mathematics (TEDS-M, conducted in 2007–2008) have included many questions, asking teachers to report on their preparedness to teach particular topics. Although Luschei and Chudgar ( 2011 ) and Gustafsson and Nilson ( 2016 ) found that these items had a weak direct relationship to student achievement across countries, other studies have suggested that readiness is related to instructional quality (Blomeke et al. 2016 ), as well as content knowledge and content preparation (Schmidt et al. 2017 ), suggesting that instructional quality may have an indirect effect on student learning.

2.4 Teacher Behaviors and Opportunity to Learn

Although the impact of teacher characteristics (experience, education, and preparedness to teach) on student outcomes remains an open question, there is much a much more consistent relationship between student achievement and teacher behaviors (instructional time and instructional content), especially behaviors related instructional content. Analyzing TIMSS, Schmidt et al. ( 2001 ) found an association between classroom opportunity to learn (OTL), interpreted narrowly as student exposure to instructional content, and student achievement. In a later study using student-level PISA data, Schmidt et al. ( 2015 ) identified a robust relationship between OTL and mathematics literacy across 62 different educational systems. The importance of instructional content has been recognized by national policymakers, and has helped motivate standards-based reform in an effort to improve student achievement, such as the Common Core in the United States (Common Core Standards Initiative 2018 ). However, we found that there was little research on whether teacher instructional content that aligned with national standards had improved student learning; the only study that we were able to identify found that such alignment had only very weak associations with student mathematics scores (Polikoff and Porter 2014 ). Student-reported data indicates that instructional time (understood as classroom time on a particular subject) does seem to be related to mathematics achievement (Cattaneo et al. 2016 ; Jerrim et al. 2017 ; Lavy 2015 ; Rivkin and Schiman 2015 ; Woessman 2003 ).

2.5 Conclusion

This review of the literature simply brushes the surface of the exceptional body of work on the relationship between student achievement and teacher characteristics and behaviors. Whether analyzing US-based, international, or the (limited) number of comparative studies, the associations between easily measurable teacher characteristics, like experience and education, and student outcomes in mathematics, remains debatable. In contrast, there is more evidence to support the impact of teacher behaviors, such as instructional content and time on task, on student achievement. Our goal was to incorporate all these factors into a comparative model across countries, with the aim of determining what an international cross-national study like TIMSS could reveal about the influence of teachers on student outcomes in mathematics. The analysis that follows draws on the existing body of literature on teacher effectiveness, which identified key teacher factors that may be associated with higher student achievement: teacher experience, teacher professional knowledge (measured by education and self-reported preparation to teach mathematics), and teacher provision of opportunity to learn (time on mathematics and content coverage).

Aaronson, D., Barrow, L., & Sander, W. (2007). Teachers and student achievement in the Chicago public high schools. Journal of Labor Economics, 25 (1), 95–135.

Article   Google Scholar  

Akiba, M., LeTendre, G., & Scribner, J. (2007). Teacher quality, opportunity gap, and national achievement in 46 countries. Educational Researcher, 36 (7), 369–387.

Baumert, J., Kunter, M., Blum, W., Brunner, M., Voss, T., Jordan, A., et al. (2010). Teachers’ mathematical knowledge, cognitive activation in the classroom, and student progress. American Educational Research Journal, 47 (1), 133–180.

Blank, R. K., & De Las Alas, N. (2009). The effects of teacher professional development on gains in student achievement: How meta analysis provides scientific evidence useful to education leaders . Washington, DC: Council of Chief State School Officers. Retrieved from https://files.eric.ed.gov/fulltext/ED544700.pdf .

Blazar, D. (2015). Effective teaching in elementary mathematics: Identifying classroom practices that support student achievement. Economics of Education Review, 48, 16–29.

Blomeke, S., Olsen, R., & Suhl, U. (2016). Relation of student achievement to the quality of their teachers and instructional quality. In T. Nilson & J. Gustafsson (Eds.), Teacher quality, instructional quality and student outcomes . IEA Research for Education (Vol. 2, pp. 21–50). Cham, Switzerland: Springer. Retrieved from https://link.springer.com/chapter/10.1007/978-3-319-41252-8_2 .

Boyd, D., Grossman, P., Lankford, H., & Loeb, S. (2006). How changes in entry requirements alter the teacher workforce and affect student achievement. Education Finance and Policy, 1 (2), 176–216.

Boyd, D. J., Grossman, P. L., Lankford, H., Loeb, S., & Wyckoff, J. (2009). Teacher preparation and student achievement. Educational Evaluation and Policy Analysis, 31 (4), 416–440.

Cattaneo, M., Oggenfuss, C., & Wolter, S. (2016). The more, the better? The impact of instructional time on student performance. Institute for the Study of Labor (IZA) Discussion Paper No. 9797. Bonn, Germany: Forschungsinstitut zur Zukunft der Arbeit/Institute for the Study of Labor. Retrieved from ftp.iza.org/dp9797.pdf .

Chetty, R., Friedman, J. N., & Rockoff, J. E. (2014). Measuring the impacts of teachers II: Teacher value-added and student outcomes in adulthood. American Economic Review, 104 (9), 2633–2679.

Chingos, M., & Peterson, P. (2011). It’s easier to pick a good teacher than to train one: Familiar and new results on the correlates of teacher effectiveness. Economics of Education Review, 30 (3), 449–465.

Clotfelter, C. T., Ladd, H. F., & Vigdor, J. L. (2006). Teacher-student matching and the assessment of teacher effectiveness. Journal of Human Resources, 41 (4), 778–820.

Collinson, V. (1999). Redefining teacher excellence. Theory Into Practice , 38 (1), 4–11. Retrieved from https://doi.org/10.1080/00405849909543824 .

Common Core Standards Initiative. (2018). Preparing America’s students for success [www document]. Retrieved from http://www.corestandards.org/ .

Constantine, J., Player, D., Silva, T., Hallgren, K., Grider, M., & Deke, J. (2009). An evaluation of teachers trained through different routes to certification. National Center for Education Evaluation and Regional Assistance paper 2009-4043. Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, US Department of Education. Retrieved from https://ies.ed.gov/ncee/pubs/20094043/ .

Darling-Hammond, L. (2000). Teacher quality and student achievement. Education Policy Analysis Archives , 8, 1. Retrieved from https://epaa.asu.edu/ojs/article/viewFile/392/515 .

Darling-Hammond, L., Holtzman, D. J., Gatlin, S. J., & Vasquez Heilig, J. (2005). Does teacher preparation matter? Evidence about teacher certification, Teach for America, and teacher effectiveness. Education Policy Analysis Archives, 13, 42. Retrieved from https://epaa.asu.edu/ojs/article/view/147 .

Desimone, L., Porter, A., Garet, M., Yoon, K., & Birman, B. (2002). Effects of professional development on teachers’ instruction: Results from a three-year longitudinal study. Education Evaluation and Policy Analysis, 24 (2), 81–112.

Desimone, L., Smith, T., & Phillips, K. (2013). Teacher and administrator responses to standards-based reform. Teachers College Record, 115 (8), 1–53.

Google Scholar  

Garet, M. S., Heppen, J. B., Walters, K., Parkinson, J., Smith, T. M., Song, M., et al. (2016). Focusing on mathematical knowledge: The impact of content-intensive teacher professional development . National Center for Education Evaluation and Regional Assistance paper 2016-4010. Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, US Department of Education. Retrieved from https://ies.ed.gov/ncee/pubs/20094010/ .

Gerritsen, S., Plug, E., & Webbink, D. (2014). Teacher quality and student achievement: Evidence from a Dutch sample of twins . CPB discussion paper 294. The Hague, The Netherlands: Central Plan Bureau/Netherlands Bureau for Economic Policy Analysis. Retrieved from https://ideas.repec.org/p/cpb/discus/294.html .

Goe, L. (2007). The link between teacher quality and student outcomes: A research synthesis . NCCTQ Report. Washington, DC: National Comprehensive Center for Teacher Quality. Retrieved from http://www.gtlcenter.org/sites/default/files/docs/LinkBetweenTQandStudentOutcomes.pdf .

Gustafsson, J., & Nilson, T. (2016). The impact of school climate and teacher quality on mathematics achievement: A difference-in-differences approach. In T. Nilson & J. Gustafsson (Eds.), Teacher quality, instructional quality and student outcomes , IEA Research for Education (Vol. 2, pp. 81–95). Cham, Switzerland: Springer. Retrieved from https://link.springer.com/chapter/10.1007/978-3-319-41252-8_4 .

Hanushek, E., & Luque, J. (2003). Efficiency and equity in schools around the world. Economics of Education Review, 22 (5), 481–502.

Hanushek, E., Piopiunik, M., & Wiederhold, S. (2018). The value of smarter teachers: International evidence on teacher cognitive skills and student performance . Journal of Human Resources (in press). https://doi.org/10.3368/jhr.55.1.0317.8619r1 .

Harris, D. N., & Sass, T. R. (2011). Teacher training, teacher quality and student achievement. Journal of Public Economics, 95 (7–8), 798–812.

Hill, H., Rowan, B., & Ball, D. (2005). Effects of teachers’ mathematical knowledge for teaching on student achievement. American Educational Research Journal, 42 (2), 371–406.

Jerrim, J., Lopez-Agudo, L., Marcenaro-Gutierrez, O., & Shure, N. (2017). What happens when econometrics and psychometrics collide? An example using the PISA data. Economics of Education Review, 61, 51–58.

Ladd, H. F., & Sorenson, L. C. (2017). Returns to teacher experience: Student achievement and motivation in middle school. Education Finance and Policy, 12 (2), 241–279. Retrieved from https://www.mitpressjournals.org/doi/10.1162/EDFP_a_00194 .

Lavy, V. (2015). Do differences in schools’ instruction time explain international achievement gaps? Evidence from developed and developing countries. The Economic Journal, 125 (11), 397–424.

Luschei, T., & Chudgar, A. (2011). Teachers, student achievement, and national income: A cross-national examination of relationships and interactions. Prospects, 41, 507–533.

Metzler, J., & Woessman, L. (2012). The impact of teacher subject knowledge on student achievement: Evidence from within-teacher within-student variation. Journal of Development Economics, 99 (2), 486–496.

Montt, G. (2011). Cross-national differences in educational achievement inequality. Sociology of Education, 84 (1), 49–68.

Palardy, G. J., & Rumberger, R. W. (2008). Teacher effectiveness in first grade: The importance of background qualifications, attitudes, and instructional practices for student learning. Educational Evaluation and Policy Analysis, 30 (2), 111–140.

Papay, J., & Kraft, M. (2015). Productivity returns to experience in the teacher labor market: Methodological challenges and new evidence on long-term career improvement. Journal of Public Economics, 130, 105–119.

Phillips, K. J. (2010). What does ‘highly qualified’ mean for student achievement? Evaluating the relationships between teacher quality indicators and at-risk students’ mathematics and reading achievement gains in first grade. The Elementary School Journal, 110 (4), 464–493.

Pil, F. K., & Leana, C. (2009). Applying organizational research to public school reform: The effects of teacher human and social capital on student performance. Academy of Management Journal , 52 (6), 1101–1124.

Polikoff, M., & Porter, A. (2014). Instructional alignment as a measure of teaching quality. Educational Evaluation and Policy Analysis, 36 (4), 399–416.

Rice, J. K. (2003). Teacher quality: Understanding the effectiveness of teacher attributes . Washington DC: Economic Policy Institute.

Rivkin, S., Hanushek, E., & Kain, J. (2005). Teachers, schools, and academic achievement. Econometrica, 73 (2), 417–458.

Rivkin, S., & Schiman, J. (2015). Instruction time, classroom quality, and academic achievement. The Economic Journal, 125 (11), 425–448.

Rockoff, J. (2004). The impact of individual teachers on student achievement: Evidence from panel data. The American Economic Review, 94 (2), 247–252.

Rockoff, J. E., Jacob, B. A., Kane, T. J., & Staiger, D. O. (2011). Can you recognize an effective teacher when you recruit one? Education Finance and Policy, 6 (1), 43–74.

Schmidt, W., Burroughs, N., Cogan, L., & Houang, R. (2017). The role of subject-matter content in teacher preparation: An international perspective for mathematics. Journal of Curriculum Studies, 49 (2), 111–131.

Schmidt, W., Burroughs, N., Zoido, P., & Houang, R. (2015). The role of schooling in perpetuating educational inequality: An international perspective. Education Researcher, 44 (4), 371–386.

Schmidt, W., McKnight, C., Houang, R., Wang, H., Wiley, D., Cogan, L., et al. (2001). Why schools matter: A cross-national comparison of curriculum and learning . San Francisco, CA: Jossey-Bass.

Shuls, J., & Trivitt, J. (2015). Teacher effectiveness: An analysis of licensure screens. Educational Policy, 29 (4), 645–675.

Staiger, D., & Rockoff, J. (2010). Searching for effective teachers with imperfect information. Journal of Economic Perspectives, 24 (3), 97–118.

Wallace, M. R. (2009). Making sense of the links: Professional development, teacher practices, and student achievement. Teachers College Record, 111 (2), 573–596.

Wayne, A. J., & Youngs, P. (2003). Teacher characteristics and student achievement gains: A review. Review of Educational Research, 73 (1), 89–122.

Wilson, S. M., & Floden, R. E. (2003). Creating effective teachers: Concise answers for hard questions. An addendum to the report “Teacher preparation research: Current knowledge, gaps, and recommendations . Washington, DC: AACTE Publications.

Wiswall, M. (2013). The dynamics of teacher quality. Journal of Public Economics, 100, 61–78.

Woessman, L. (2003). Schooling resources, educational institutions, and student performance: The international evidence. Oxford Bulletin of Economics and Statistics, 65 (2), 117–170.

Download references

Author information

Authors and affiliations.

CREATE for STEM, Michigan State University, East Lansing, MI, USA

Nathan Burroughs

Office of K-12 Outreach, Michigan State University, East Lansing, MI, USA

Jacqueline Gardner

Michigan State University, East Lansing, MI, USA

Youngjun Lee

College of Education, Michigan State University, East Lansing, MI, USA

Israel Touitou

Kimberly Jansen

Center for the Study of Curriculum Policy, Michigan State University, East Lansing, MI, USA

William Schmidt

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Nathan Burroughs .

Rights and permissions

Open Access This chapter is licensed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (http://creativecommons.org/licenses/by-nc/4.0/), which permits any noncommercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

Reprints and permissions

Copyright information

© 2019 International Association for the Evaluation of Educational Achievement (IEA)

About this chapter

Burroughs, N. et al. (2019). A Review of the Literature on Teacher Effectiveness and Student Outcomes. In: Teaching for Excellence and Equity. IEA Research for Education, vol 6. Springer, Cham. https://doi.org/10.1007/978-3-030-16151-4_2

Download citation

DOI : https://doi.org/10.1007/978-3-030-16151-4_2

Published : 24 May 2019

Publisher Name : Springer, Cham

Print ISBN : 978-3-030-16150-7

Online ISBN : 978-3-030-16151-4

eBook Packages : Education Education (R0)

Share this chapter

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

• Open access
• Published: 17 May 2024

Towards inclusive learning environments in post-graduate medical education: stakeholder-driven strategies in Dutch GP-specialty training

• N.M. van Moppes   ORCID: orcid.org/0000-0003-3457-7724 1 ,
• M. Nasori   ORCID: orcid.org/0000-0001-8559-1791 1 ,
• J. Bont   ORCID: orcid.org/0000-0002-5358-0235 1 ,
• J.M. van Es 1 ,
• M.R.M. Visser 1 &
• M.E.T.C. van den Muijsenbergh   ORCID: orcid.org/0000-0002-4994-4008 2 , 3  

BMC Medical Education volume  24 , Article number:  550 ( 2024 ) Cite this article

92 Accesses

Metrics details

A recent study found that ethnic minority General Practice (GP)-trainees receive more negative assessments than their majority peers. Previous qualitative research suggested that learning climate-related factors play a pivotal role in unequal opportunities for trainees in post-graduate medical settings, indicating that insufficient inclusivity had put minority students at risk of failure and dropout.

Study objectives

We aimed to develop broadly supported strategies for an inclusive learning climate in Dutch GP-specialty training.

We employed Participatory Action Research (PAR)-methods, incorporating Participatory Learning and Action (PLA)-techniques to ensure equal voices for all stakeholders in shaping Diversity, Equity, and Inclusion (DEI)-strategies for GP-specialty training. Our approach engaged stakeholders within two pilot GP-specialty training institutes across diverse roles, including management, support staff, in-faculty teachers, in-clinic supervisors, and trainees, representing ethnic minorities and the majority population. Purposeful convenience sampling formed stakeholder- and co-reader groups in two Dutch GP-specialty training institutes. Stakeholder discussion sessions were based on experiences and literature, including two relevant frameworks, and explored perspectives on the dynamics of potential ethnic minority trainees’ disadvantages and opportunities for inclusive strategies. A co-reader group commented on discussion outcomes. Consequently, a management group prioritized suggested strategies based on expected feasibility and compatibility.

Input from twelve stakeholder group sessions and thirteen co-readers led to implementation guidance for seven inclusive learning environment strategies, of which the management group prioritized three:

• Provide DEI-relevant training programs to all GP-specialty training stakeholders;

• Appoint DEI ambassadors in all layers of GP-specialty training;

• Give a significant voice to minority GP-trainees in their education.

The study’s participatory approach engaged representatives of all GP-specialty training stakeholders and identified seven inclusive learning climate strategies, of which three were prioritized for implementation in two training institutions.

Peer Review reports

Introduction

Following international migration trends [ 1 , 2 ], diversity among students and trainees is growing [ 3 , 4 ], with each of them bringing their specific cultural values, family- and migration histories [ 5 ]. However, postgraduate medical ethnic minority GP-trainees still face underrepresentation [ 3 , 4 ] and may encounter unequal opportunities for success compared to their majority peers [ 6 , 7 , 8 , 9 ]. Learning climate-related factors, notably those related to lacking inclusiveness, likely play a pivotal role in this discrepancy [ 10 , 11 , 12 ].

Educational opportunities in GP-specialty training primarily rely on in-clinic learning, encompassing formal and informal contexts. Formal learning, characterized by structured, planned, and accredited activities within educational institutions, coexists with less structured informal learning, which is self-directed and arises from in-clinic everyday experiences and interactions, often susceptible to unspoken norms. While both approaches complement each other in providing a well-rounded education, the informal context might inadvertently reflect dominant cultural values and attitudes, potentially affecting in-classroom learning [ 13 , 14 , 15 ]. Particularly for ethnic minority GP-trainees, this lacking transparency may contribute to an increased risk of facing underperformance assessments, as these unspoken norms and values may not be self-evidently familiar to them [ 10 , 11 , 12 ].

Learning environments are subject to complex dynamics. Understanding the interconnected constructs of these dynamics is crucial for implementing transformative changes [ 16 ]. Accordingly, changes for inclusive learning opportunities require input from all organizational layers [ 17 ].

With this study, we aimed to develop broadly supported recommendations for an inclusive learning climate in Dutch GP-specialty training.

We used a qualitative Participatory Action Research (PAR) approach [ 18 , 19 ], applying Participatory Learning and Action (PLA) techniques in stakeholder groups combined with insights from literature (Appendix) along with GP-trainees’ experiences related to inclusive education, to actively engage stakeholders in an inclusive dialogue [ 20 , 21 , 22 ]. This approach supported co-ownership, promoted compatibility with the organization’s actual needs, and facilitated successful implementation [ 23 ].

We employed two conceptual frameworks to shape the topic lists for stakeholder groups and guide result analysis.

The Building Equity Taxonomy (BET) framework for Diversity, Equity, and Inclusion (DEI), addressing students’ needs for equal educational opportunities, and covering the areas of physical integration, social-emotional engagement, equal learning opportunities, instructional excellence, and fostering inspired learners [ 24 , 25 ] (Fig.  1 ). This framework is relevant to various educational settings, including GP-specialty training [ 12 , 26 , 27 , 28 , 29 ].

Building Equity Taxonomy [ 24 ] compared to Maslow’s hierarchy of needs [ 25 ]

The Wensing & Grol framework implementation guidance, equivalent to the internationally recognized Consolidated Framework for Implementation Research (CFIR) [ 30 ]. It provides implementation guidance for complex organizations, including clinical healthcare and educational settings [ 30 , 31 ] (Fig.  2 ). This framework underpinned our implementation guidance, which the management team used for prioritization.

The Wensing & Groll model for implementation guidance [ 30 ]

This study took place at Amsterdam UMC’s two GP-specialty training institutes (AMC and VUmc). These institutes have demonstrated commitment to inclusiveness in their 2020–2022 annual reports, and they collaborate with the six other Dutch GP-specialty training institutes under GP-specialty Training Netherlands (HN).

One in three medical graduates in the Netherlands aims to enter GP-specialty training. In response to national medical demands, HN annually expands its acceptance of new trainees, projecting 921 in 2023 and an anticipated 1,190 in 2024, distributed across eight training institutes. About 17% of these trainees belong to ethnic minority groups, with most having completed pre-training at Dutch Medical Schools and a smaller group having graduated abroad [ 7 ]. Due to General Data Protection Regulation (GDPR) restrictions, the precise distribution of minority trainees across the eight national institutes remains undisclosed. However, a prior quantitative study indicated that by 2023, our pilot institutes showed a relatively proportional representation of Dutch GP-specialty training [ 7 ]. However, it is essential to note that qualitative research emphasizes a thorough description of the setting to enrich readers’ contextual understanding rather than strict representativeness.

The Dutch GP-specialty training program is a three-year dual-track program, supporting professional growth by combining in-clinic experience learning with one-day-a-week in-faculty education. Entry assessments aspire to guarantee the applicants’ knowledge, motivation, and Dutch proficiency. The program includes protocolled assessments, such as practical observations, systematic testing, and reviews of learning objectives.

Study population

Acknowledging the essential need of broad support for inclusive organizational changes, we engaged participants from all backgrounds represented within the organization. Our study population encompassed the ethnic majority background as well as diverse ethnic backgrounds across all organizational layers (ranging from support personnel, management, educational staff (comprising both faculty and clinical educators), and trainees themselves), divided into two stakeholder groups, one co-reader group, and a management team group (Fig.  3 ).

Participant groups

Aiming to prevent eligible participants from experiencing researchers’ pressure, researchers sent information letters to team leaders, requesting them to forward in-faculty teachers, in-clinic supervisors, supporting bureau and management personnel, and trainees. From those interested, we purposefully selected twelve participants (six in each stakeholder group), striving for diversity regarding the position in the institute, age, gender, and ethnicity [ 32 ]. Stakeholders ranged from supporting bureau and management personnel (further in this text referred to as ‘staff’) to trainees, in-faculty teachers, and in-clinic supervisors representing diverse minority backgrounds as well as the majority background.

Stakeholder groups, each representing one GP-specialty training institute, provided input for inclusive strategies. Additionally, a co-reader group comprising interested individuals not in the stakeholder groups provided further insights through written comments. These groups represented diverse organizational layers, cultural backgrounds, ages, and gender. Representatives from management teams then evaluated and prioritized the suggested strategies.

Data collection

Data collection and analysis took place from January 2021 to December 2022. Two researchers (MN, NvM) familiar with PLA-techniques facilitated six 90-minute PLA-based sessions for each stakeholder group. The sessions focused on inclusive learning environments and GP-specialty training’s inclusivity. In a cyclical process [ 33 ](Fig.  4 ), participants engaged in PLA techniques such as ice-breaking, flexible brainstorming, free-associating, direct ranking, mind-mapping, and visual evaluation. These methods facilitated sharing experiences and opinions and aligning these with relevant literature (Appendix, Table  1 ) to identify suitable inclusive strategies. After each stakeholder group session, the facilitator-researchers held debriefing sessions to reflect on their roles and identify areas for improvement. Independently, they summarized the key findings from each session and reached consensus through discussions. They presented these summaries in subsequent sessions for a member check and made adjustments based on participants’ feedback. To ensure a broader perspective, the co-reader group commented anonymously on these approved summaries, allowing them to contribute their personal perspectives, opinions, and experiences freely. Stakeholder groups then discussed and implemented these comments in their final session (Fig.  3 ).

Cyclic phases until consensus of stakeholder groups’ processes [ 33 ]

The stakeholder group topic list focused on:

Exploring :

The initial educational context;

Potential learning climate-related disparities;

Out-of-the-box wishes and key elements for an inclusive learning climate;

Strategy developing and preparing for implementation:

Recommendations for inclusive GP-specialty training;

Mapping onto the BET framework’s hierarchical levels of DEI [ 24 ](Fig.  1 );

Translating recommendations into actionable strategies.

Identifying Wensing & Grol conditions and requirements for implementation [ 30 ](Fig.  2 ).

Due to the Covid-19 pandemic, we adapted the study’s in-person design to online methods for creative brainstorming. In these virtual sessions, physical distance and potential distractions of personal environments challenged trust and commitment, especially for GP-trainees who felt vulnerable sharing ideas with in-faculty teachers, in-clinic supervisors, and staff, who might also be their assessors in daily educational contexts [ 34 ]. To address this risk, we dedicated extra time, and utilized online tools: Zoom 5.13.11 for breakout rooms, Padlet 200.0.0 for visualizing PLA techniques, and concise PowerPoint presentations for member check summaries and goal-setting [ 35 ].

The facilitator-researchers (NvM, MN) collected audio recordings and written co-reader comments. An external bureau transcribed audio-recordings verbatim.

One researcher (NvM) regularly presented our findings during periodic staff meetings. These presentations not only aimed to keep the entire team informed but also played a crucial role in garnering broader support and incorporating diverse opinions for our project.

Data analysis

Within three days after each session, we (NvM, MN, and MV) analyzed the transcribed audio recordings and written co-readers’ comments, and discussed our analyses until consensus.

To provide actionable qualitative insights while responding to ongoing participant feedback, we adopted an inductive rapid qualitative data analysis approach inspired by Hamilton’s model [ 36 , 37 , 38 , 39 ]. This method prioritizes identifying key elements and mechanisms over extensive theoretical insights. Through structured data collection using topic lists and Participatory Learning and Action (PLA) techniques, along with expedited transcription, we efficiently analyzed ideas and condensed findings into concise formats like post-interview notes and matrix summaries. Although not a traditional thematic or framework analysis, we employed theme-informed and framework-informed codes to organize data, considering context and group dynamics, which allowed us to explore interactional group dynamics and communication styles in the participants’ discourse and its points of consensus or contention within specific statements [ 40 ]. We anticipated this method, aligned with the literature, to yield qualitative outcomes as consistent and rich as traditional in-depth transcription coding while facilitating the analysis of interconnected sessions [ 36 , 41 , 42 ].

We analyzed the stakeholders’ ideas, recommendations, and their identifyed Wensing & Grol conditions and requirements for implementation to create implementation guidance [ 30 ]. This guidance encompassed analyzing organizational structure, identifying change potential and barriers, defining the target population, describing tailored DEI-strategies, estimating timelines for internalization processes and implementation, and designing evaluation methods (Fig.  2 ). Subsequently, we invited management group participants for hybrid (online and in-person) meetings, where they engaged in substantive discussions to evaluate this guidance and prioritize recommended strategies, based on the expected feasibility and compatibility with their setting.

Reflexivity and ethics

Two authors, NM and MN, identify as minority females. While their unique backgrounds enhance sensitivity towards minority peers’ experiences, a potential challenge arises where these experiences resonating with them might be more salient. To mitigate this, we organized reflective debriefing sessions addressing diverse viewpoints and emphasizing the researchers’ roles as instruments in data collection and analysis. During these sessions, we engaged in candid discussions probing our experiences, expectations, preoccupations, and opinions that could have influenced our approach to data collection and analysis.

Also, the roles of participating stakeholders may have influenced views they shared in this research process. They spanned all organizational positions, ranging from department heads to trainees, in-faculty teachers, and in-clinic supervisors, representing both, majority and minority backgrounds. While deliberately seeking these varied insights, we remained mindful of potential power dynamics influenced by different positions or ethnic backgrounds. To foster a safe space and address these dynamics, facilitators employed PLA-techniques, such as ice-breakers. Also, they established clear agreements with all stakeholder group members regarding privacy, openness to differing views, and ensuring safety. Should any commitments be breached, facilitators were trained to address them promptly. In fact, stakeholders demonstrated remarkable respect and curiosity towards understanding each other’s perspectives throughout the process.

Participant characteristics

Table  1 presents participant characteristics for the stakeholder, co-reader, and management groups. In total, 31 stakeholders participated, aged 24 to 60, including eight males, 24 staff members from diverse organizational positions, seven trainees, and 12 ethnic minority participants.

The stakeholder group sessions had an attendance rate of 97%. All co-readers responded to the request for comments. During the hybrid management group session, 40% of participants attended in-person, while 60% joined online.

Stakeholder group sessions

In line with the topic list, we organized the results into two sections: [ 1 ]Exploring and [ 2 ] Strategy developing and preparing for implementation. In Sect. 2, the stakeholders aligned their results with the BET framework and structured them according to the Wensing & Grol framework.

The initial educational context

Stakeholders defined inclusiveness in the GP-specialty training as collective curiosity and support for trainees’ unique professional identities, regardless of their characteristics or backgrounds. As preconditions for in-faculty teachers, in-clinic supervisors, and staff, participants mentioned [ 1 ] willingness to encounter emotional discomfort [ 2 ], embracing failures in order to learn, and [ 3 ] acknowledgement of unconscious bias.

‘… we will not always succeed to be without prejudice, that is allowed as long as we will put the effort in gaining awareness’ (participant 2, group 1).

Participants emphasized creating a safe learning environment where all voices, including minority voices, can be heard. They suggested reflective questions starting with:

‘ Could you imagine that…’.

Participants highlighted parallel processes whereby educators foster trainees’ personal and professional development, and GPs support patients’ individual coping styles. Such an inclusive and safe learning environment would act as a flywheel, enhancing the institute’s inclusive image and attracting prospective minority trainees, teachers, and in-clinic supervisors.

Co-readers confirmed these view points and they added their concerns regarding prioritization by some staff members:

‘I have nothing to add. I think it is essential that diversity is given a priority, that we as staff all agree that this is important. The pitfall is that some of them might not see the importance’. (co-reader 2)

Potential learning climate-related disparities

Stakeholders from ethnic minority groups expressed distress experiences in a dominant white world:

‘The GP-specialty training population is predominantly white and female; trainees, in-faculty teachers, and in-clinic supervisors even seem to resemble one another. Without them saying or acting, I continuously feel the stress of having to adapt to them, which I will never be able to’ (participant 2, group 1).

Stakeholders discussed the majority’s naivety in understanding the experience of belonging to a minority and expressed concerns about some DEI programs potentially leading to paradoxical stigmatization. They noted instances where in-faculty teachers appointed minority trainees as representatives for their cultural groups, ignoring the vast diversity within these groups. Also, participants reported stereotyping case reports:

‘They always use the example of the non-Dutch speaking overweight Moroccan mother of seven children, not engaged in any sports, who favors sweet and fatty food, and suffers from diabetes’ (participant 3, group 2).

Co-readers added that this one-sided picture made minority trainees uneasy, feeling discussed rather than equal partners in GP-training. Additionally, they emphasized that presenting DEI programs as non-mandatory, implied that diversity and inclusiveness were not necessarily integral to GP-skills requirements.

‘Mandatory inclusive training for mentors, staff, and teachers holds significant importance, signifying our commitment. Participation in these courses should be integrated into evaluations and annual interviews’. (co-reader 4)

Out-of-the-box wishes and key elements for an inclusive learning climate

Upon the invitation to make a wish:

‘Wouldn’t it be wonderful if….‘ ,

stakeholders wished for diverse staff as role models, willing to learn from each other, normalizing various meaningful insights, and embracing diverse worldviews:

‘By using these differences, we keep each other awake and open-minded in exploring possibilities; thus, we allow ourselves to grow without assuming that our paved path is always the best way at the time’ (participant 3, group 1).

Stakeholders indicated the institute’s responsibility to educate GP-trainees for a diverse patient population as an essential component of an inclusive learning environment. Key elements related to such inclusiveness were:

The GP-specialty training should represent society in all its diversity:

‘It’s been a few years since I started GP-specialty training, of course, but… I’m just digging whether I had a feeling of: “I fit in there” or: “I recognize my roots there”. These are important feelings to me to feel safe at my work- and study place’ (participant 7, group 1);

A diverse GP workforce meets patients’ appreciation for GPs they can identify with:

‘Regarding this cultural background or ethnicity, I have the impression that patients from ethnic minorities often liked that I obviously am not Dutch, they said, “oh, you are not Dutch, are you?“, it led to recognition, a little laugh, and connected us. Having a doctor just like them helped my patients to share their concerns.’ (participant 2, group 2);

GP-trainees need identifiable and diverse educational role models:

‘The moment you sit down together and see that diversity, …brings different working styles, learning styles, or communication styles… that you realize we have to do it together, the greater the diversity, the more we learn from one another, the higher we rise, the more fun and creative ideas…’ (participant 4, group 2);

Diverse GP-trainee cohorts improve mutual learning processes:

‘To me, utilizing diversity means that there’s always someone in the classroom who says, “Okay, so what if we look at it from that perspective or through those glasses?’ (participant 1, group 1).

Co-readers agreed and added that GP-specialty training already utilized diversity among in-faculty teachers to some extent:

‘Great idea! Diversity among teachers is already being leveraged to some extent. Trainees can synthesize a blend of styles and insights from different teachers and mentors. Expanding on this concept could help cultivate a more inclusive learning environment’ . (co-reader 1)

Strategy developing and preparing for implementation

Recommendations for an inclusive gp-specialty training.

Participants (stakeholders in collaboration with co-readers) made six fundamental recommendations and mapped these onto the BET framework levels to ensure all aspects of inclusive education would be covered [ 24 ] (Table  2 ).

Actionable strategies

From these recommendations, participants derived seven actionable strategies for promoting inclusive GP-specialty training (Table  3 ).

Provide a clear message of inclusiveness in all internal and external communications .

Participants explored various means and media platforms for promoting the GP-specialty training’s DEI core values (websites, ads, social media, podcasts), focusing on design, content, and appeal to the target group. They recommended involving trainees with media experience rather than exclusively hiring specialized communication consultants.

Appoint DEI ambassadors in all layers of the organization .

Participants suggested involving employees as DEI ambassadors to effectively spread DEI core values in the organization. Ambassadors would undergo comprehensive training in DEI, reflective skills, leadership, and change management. They would also attend conferences, masterclasses, join knowledge networks, and contribute to think tank initiatives as part of their preparation.

Facilitate procedures for secure incident reporting .

Participants highlighting the significant impact of unintentional discriminatory behavior, often resulting in experiencing barriers to reporting such incidents. They proposed implementing low-threshold and secure reporting procedures with targeted questions on DEI and (micro)aggression. Regular team sessions would enable open discussions based on anonymous reports, fostering inclusive education, uncovering organizational trends, and providing support for trainees who faced discrimination, microaggression, or exclusion. Confidential advisors would receive training in DEI, reflective skills, and relevant legislation.

Give a significant voice to minority trainees in ongoing program development .

Participants advised inviting minority trainees to round table discussions, fostering insider perspective exchange with mutual respect, critical reflection, and empathy. Including these diverse voices would promote resilience and professional growth and attract eligible trainees and staff from diverse backgrounds.

Assign more than one in-faculty teacher per group / in-clinic training .

GP-trainees - like all individuals - naturally mirror the behavior of significant others, such as teachers, in-clinic supervisors, or peers. Participants believed that trainees with multiple role models would outperform those with single role models. They suggested introducing dual in-faculty teachers and dual in-clinic supervisors as additional role models and an extra pair of eyes during education. To ensure success, participants recommended training programs for optimum role model utilization.

Offer ‘just-in-time’ learning .

Participants agreed that effective learning is closely related to immediate learning needs. For GP-trainees, such learning needs often arise from societal encounters in the consultation room, e.g., guiding Muslims during Ramadan while simultaneously managing diabetes or comprehending increasing PTSD symptoms around Keti Koti (Afro-Surinamese Emancipation Day). Timely incorporating these contextual factors into training programs could provide directly applicable knowledge.

Provide mandatory DEI relevant training programs for professional development .

Participants emphasized the necessity of new knowledge, skills, and attitudes. They considered within-group differences valuable learning tools for diverse personal and professional development paths. Well-trained staff and trainees could drive inclusive knowledge networks, empower the organization, and positively influence external perceptions. Thus, they recommended mandatory and tailored training programs aligned with the anticipated learning needs from the suggested strategies. Where applicable, they advised considering outsourcing.

Conditions and requirements for implementation

Participants indicated the importance of in-faculty teachers, in-clinic supervisors, and staff having the courage to be vulnerable. They emphasized the essence of transparent norms and values and a welcoming learning environment, and they highlighted an attitude of:

‘… genuinely enjoying to support a diverse population in their growth towards their professional identities’ (participant 6, group 2).

‘Implementing these ideas demands courage and vulnerability, particularly as their execution could inadvertently carry stigmatizing effects’. (co-reader 6)

In this context, they mentioned the risk of unconscious bias, which could require external expert trainers at certain stages:

‘Well, you know, I had a trainee of Moroccan descent, and it shocked me that, while I always thought to be very open, diversity-minded, and curious for everything and everyone, I found it way more difficult to connect than I’d admit. I wonder what would have helped me unveil this blind spot in an earlier stage…’ (participant 5, group 1).

‘… allow and embrace the differences, see them as opportunities that actually add learning qualities, and not take them away? So, professionalism will become more colorful, and it can be viewed from different points of view, not just the traditional, established perspectives and routes’ (participant 1, group 1).

Ultimately, we provided the management group with implementation guidance for these seven strategies, along with an analysis of the target group and context, and summaries of relevant literature on DEI best practices in educational settings (Appendix). The management team agreed that enhancing DEI should have priority in Dutch GP-specialty training:

‘We should acknowledge that we are trailing behind societal advancements in diversity. Therefore, maintaining a strong focus on this topic must stay a priority’ (participant 5, management group).

Based on these comprehensive data, the management group prioritized strategies that covered the overarching recommendations and BET-levels (detailed in Table  3 ; Fig.  1 ), which aided in selecting strategies with anticipated effectiveness. To enhance alignment with the organizational requirements and feasibility, they considered implementation requirements, staff feedback from our presentations during periodic meetings, and opportunities for synergy with existing projects in other Amsterdam UMC departments.

‘We can see that literature describes these strategies as effective and we assume that stakeholders meticulously aligned them with the institute’s needs. Let us not repeat that process but rather look into strategies that can be implemented effectively in our setting’ (participant 1, management group).

‘For each suggested strategy, this guidance envisions its coverage and practical implications. Now, it is up to us to consider how far we are willing to commit. This process prompts pertinent questions on specific effective actions’ (participant 2, management group).

The management group prioritized three strategies:

Appoint DEI ambassadors in all organizational levels,

Give a significant voice to minority trainees in ongoing program development,

Provide mandatory DEI-relevant training programs for professional development to all involved in GP-specialty training.

Summary of findings

In twelve PLA-based stakeholder sessions, participants explored perspectives on potential disparities, underlying causes, and aspirations for an inclusive learning climate in the Dutch GP-specialty training. They suggested seven strategies based on six overarching recommendations, which they presented embedded in an implementation guidance to the management group:

Provide a clear message of inclusiveness in all internal and external communications.

Appoint DEI ambassadors Footnote 1 in all layers of the organization

Facilitate procedures for secure incident reporting.

Give a significant voice to minority trainees in ongoing program development.

Assign more than one in-faculty teacher per group / in-clinic supervisor per trainee.

Offer ‘just-in-time’ learning.

Provide mandatory DEI relevant training programs for professional development.

The management team selected strategies 2, 4, and 7, deeming them most effective, feasible, and aligned with the organization’s requirements.

Comparison to existing literature

Worldwide attention to inclusive learning climates in postgraduate medical education revealed the complexity and multidimensionality of educational constructs and institutes [ 29 , 43 ]. Interpretations of formal and informal learning contexts within these environments depend on the perspectives of various stakeholders [ 15 ]. Consequently, unconsciously normalized rules and codes across all layers may implicitly exclude ethnic minority professionals and -trainees in many ways throughout their careers [ 44 ].

This paper extends the literature on inclusive GP-specialty training [ 15 , 43 , 44 ], detailing the efforts to design- and create broad support for inclusive training strategies. Like most organizational changes, implementing inclusive strategies in GP-specialty training posed challenges and demanded a focus on building confidence and trust in novel approaches [ 45 ]. Hence, understanding the values and expectations of target groups and tailoring strategies to meet their needs and aspirations was crucial. Our study involved representatives from all key stakeholders, including ethnic minority trainees, aiming to address critical research gaps and enhance knowledge quality, relevance, and impact [ 46 ]. Collaborative decisions, rooted in an equal and reciprocal partnership, empowered stakeholders, raised management team awareness and inspired the research team [ 47 ]. These effects mirror findings in previous PAR studies on inclusive primary healthcare [ 48 ] and highlight PAR’s role as a catalyst for transformative change in GP-specialty training [ 33 ].

Stakeholder insights, combined with DEI-strategy literature, underscored the need for a gradual, committed cultural shift towards inclusivity in the learning environment. Based on these insights, the management group recognized that this transformation would necessitate a set of strategies addressing inclusiveness at various levels rather than relying on one single intervention [ 26 , 28 , 49 , 50 ]. They employed our Wensing and Grol-based implementation guidance to select the following feasible strategies aligned with the GP-specialty training context as a first step in an ongoing process:

Providing mandatory DEI-relevant training programs to all stakeholders supports cultural responsiveness within all strategies to be implemented. It facilitates understanding how cultural backgrounds and experiences influence teaching and learning [ 49 ]. Ultimately, it fosters engagement and motivation to create collaborative learning environments and accommodate learners’ needs based on their diverse backgrounds [ 26 ].

Appointing DEI ambassadors in all layers of the organization has in other contexts proven to enhance the effectiveness of DEI-related strategic initiatives [ 51 ]. DEI ambassadors engage change agents within their teams, foster collaboration and effective communication, facilitate diversity goals, and involve key stakeholders in sustainable, inclusive changes [ 50 ].

Giving a significant voice to minority trainees empowers and amplifies their agency. Including their experiences and perspectives in staff meetings and brainstorming sessions is a crucial first step toward an open and innovative culture. Prior research indicated that promoting minority trainees’ participation requires supportive supervision, encouraging them to share transformative ideas [ 28 ].

Strengths and limitations

Our participatory approach fostered broad support across all organizational levels. PLA-based stakeholder discussions facilitated open dialogue, refined ideas, and sparked valuable insights. Co-reader feedback prompted stakeholder group participants to reevaluate their interpretation of specific experiences. This approach allowed diverse perspectives and theoretical idea saturation, aiding participants in identifying seven actionable strategies with high potential for effective implementation. In turn, these results allowed the management group to leverage their organizational expertise and prioritize three strategies they considered feasible and compatible with the organization’s requirements.

While most post-graduate medical education settings share similarities, contextual variations, such as educational emphasis and cultural factors, may exist, leading to potential limitations in the transferability of our findings. Nonetheless, the dynamics between informal and formal in-classroom learning remain pertinent across various postgraduate medical contexts, where in-clinic learning, shaped by day-to-day experiences and supervisor-trainee dynamics, inevitably influences formal learning objectives and settings. Also, our study’s confinement to two Dutch GP-specialty training institutes and its relatively modest participant count may require caution in the transferability of our findings to other similar settings. In light of this, it is noteworthy that statistics from a previous quantitative study suggest that by 2023, our pilot institutes closely mirrored Dutch GP-specialty training in terms of minority trainee [ 7 ]. Moreover, we provided meticulous descriptions of our setting to enhance contextual understanding, aiding in assessing transferability to similar settings. Additionally, the explicit commitment to inclusiveness by the participating GP-specialty training institutions, which could be instrumental in promoting successful implementation, could pose challenges when transferring the results to less DEI-focused settings.

Still, employing multiple sources by connecting stakeholder perspectives to relevant literature and two frameworks enabled participants to structure their thoughts and opinions on the organization’s DEI strengths and limitations, along with the opportunities and challenges for implementation. For future researchers, this approach may prove valuable in identifying overarching concepts and theories that transcend specific individuals or contexts and facilitate the assessment of the transferability of our findings to similar educational settings [ 52 , 53 , 54 , 55 ].

Implications for further research and practice

Fostering a DEI-minded culture in post-graduate medical training calls for a multifaceted strategy. As training institutes diversify and curricula address nuanced topics, skills for adeptly navigating complex conversations become increasingly critical for educational staff. The ongoing process of promoting inclusive teaching, assessment, and curriculum design abilities will necessitate the inclusion of a wide range of perspectives. Consequently, we recommend involving stakeholders from the most diverse backgrounds possible. Also, the explicit commitment to inclusiveness by the participating GP-specialty training institutions may pose challenges when transferring the results to less DEI-focused settings. Therefore, we suggest further investigation in such contexts to better understand the transferability of our results.

Ensuring high-quality, inclusive learning environments in postgraduate medical education is crucial for educational opportunities and the overall quality of healthcare [ 56 ]. However, this inclusiveness is not solely shaped by the beliefs and values of teachers; it is also intricately influenced by the complex social and cultural dynamics within educational institutions [ 29 ]. Inclusiveness strategies are catalysts for enduring cultural transformation, demanding the consistent integration of multiple strategies through incremental steps over an extended period [ 43 ]. The three strategies identified in our study, which were prioritized for implementation, represent initial strides toward instigating this cultural transformation. Subsequent phases involving evaluation, adaptation, and implementation of additional strategies are imperative for sustaining engagement in a culture of inclusive postgraduate medical education. All Dutch GP-specialty training institutes closely monitor our findings and have committed to implementing mandatory DEI-relevant training programs for their staff and trainees.

Additional research on the impact of the implemented strategies and the level of stakeholder engagement throughout the implementation phase is needed. This follow-up research should encompass inclusive teaching methods, assessment strategies, curriculum design, attitudes, and the ethnic minority trainees’ experienced inclusion aligned with the BET framework.

Engaging stakeholders in PLA-based sessions at two Dutch GP-specialty training institutes proved instrumental in identifying recommendations for an inclusive learning climate. Stakeholders identified seven tangible DEI-strategies, addressing all five BET aspects:

Provide a clear message of inclusiveness in all internal and external communications: enhances inclusive accessibility and a diverse learning community;

Appoint DEI ambassadors in all layers of the organization: promotes knowledge exchange, reflection on potential biases, and active engagement in DEI networks;

Facilitate secure DEI-incident reporting procedures;

Give a significant voice to minority trainees in ongoing program development: empowers them and creates reciprocal learning;

Assign more than one teacher per group / in-clinic training: creates multiple role models and perspectives;

Offer ‘just-in-time’ learning: fosters social and educational engagement;

Provide mandatory DEI-relevant training programs for professional development: promotes DEI-expertise and awareness among all involved.

Based on anticipated feasibility and effectiveness, the management group prioritized strategy numbers 2, 4, and 7 for implementation.

Our integrative approach supported collaborative, context-specific strategy development and prioritization, effectively balancing anticipated effectiveness and compatibility. As such, this approach will prove valuable in identifying widely supported DEI strategies within varying and complex post-graduate medical educational contexts.

Data availability

The datasets used and/or analysed during the current study available from the corresponding author on reasonable request.

Ambassadors actively promote DEI values within teams by exchanging knowledge and experiences, enhancing expertise through training and literature, and addressing diversity cases through consultancy roles.

Abel GJ, Cohen JE. Bilateral international migration flow estimates for 200 countries. Sci Data. 2019;6(1):82.

Article   Google Scholar  

Netherlands S. How many people immigrate to the Netherlands? Web page. The Hague, the Netherlands: Statistics Netherlands; 2022. 25/03/2022.

Google Scholar  

Jabbarpour Y, Westfall J. Diversity in the Family Medicine workforce. Fam Med. 2021;53(7):640–3.

Wusu MH, Tepperberg S, Weinberg JM, Saper RB. Matching our mission: a Strategic Plan to create a Diverse Family Medicine Residency. Fam Med. 2019;51(1):31–6.

Dinos S, Ascoli M, Owiti JA, Bhui K. Assessing explanatory models and health beliefs: an essential but overlooked competency for clinicians. BJPsych Adv. 2017;23(2):106–14.

Stegers-Jager K, Steyerberg E, Cohen-Schotanus J, Themmen A. Ethnic disparities in undergraduate pre-clinical and clinical performance. Med Educ. 2012;46:575–85.

van Moppes NM, Willems S, Nasori M, Bont J, Akkermans R, van Dijk N et al. Ethnic minority GP trainees at risk for underperformance assessments: a quantitative cohort study. BJGP Open. 2023;7(1).

Esmail A, Roberts C. Academic performance of ethnic minority candidates and discrimination in the MRCGP examinations between 2010 and 2012: analysis of data. BMJ (Clinical Res ed). 2013;347:f5662.

Jones AC, Nichols AC, McNicholas CM, Stanford FC. Admissions is not enough: the racial achievement gap in Medical Education. Acad Med. 2021;96(2):176–81.

Bullock JL, Lockspeiser T, Del Pino-Jones A, Richards R, Teherani A, Hauer KE. They Don’t See a Lot of People My Color: A Mixed Methods Study of Racial/Ethnic Stereotype Threat Among Medical Students on Core Clerkships. Acad Med. 2020;95(11S Association of American Medical Colleges Learn Serve Lead: Proceedings of the 59th Annual Research in Medical Education Presentations):S58-s66.

Diaz T, Navarro JR, Chen EH. An Institutional Approach to fostering inclusion and addressing racial Bias: implications for diversity in Academic Medicine. Teach Learn Med. 2020;32(1):110–6.

Roberts L, Belonging R, Inclusion, Diversity in Medical Education. Acad Medicine: J Association Am Med Colleges. 2020;95:661–4.

Karnieli-Miller O, Vu TR, Holtman MC, Clyman SG, Inui TS. Medical students’ professionalism narratives: a window on the informal and hidden curriculum. Acad Med. 2010;85(1):124–33.

Sagasser M, Kramer A, Fluit L, Weel C, Van der Vleuten C. Self-entrustment: how trainees’ self-regulated learning supports participation in the workplace. Adv Health Sci Education: Theory Pract. 2017;22.

Schönrock-Adema J, Bouwkamp-Timmer T, Hell E, Cohen-Schotanus J. Key elements in assessing the educational environment: Where is the theory? Advances in health sciences education: theory and practice. 2012;17.

Goodyear P, Carvalho L. The Analysis of Complex Learning Environments. 2019. pp. 49–65.

Kumagai AK, Lypson ML. Beyond cultural competence: critical consciousness, social justice, and multicultural education. Acad Med. 2009;84(6):782–7.

Brydon-Miller M, Maguire P. Participatory action research: contributions to the development of practitioner inquiry in education. Educational Action Res. 2009;17(1):79–93.

Chevalier J, Buckles D. Participatory Action Research: Theory and Methods for Engaged Inquiry2019.

Bozalek V, Biersteker L. Exploring power and privilege using participatory learning and action techniques. Social Work Educ. 2010;29(5):551–72.

de Brún T, Brún M, Weel-Baumgarten E, Burns N, Dowrick C, Lionis C et al. Using Participatory Learning & Action (PLA) research techniques for inter-stakeholder dialogue in primary healthcare: an analysis of stakeholders’ experiences. Res Involv Engagem. 2017;3.

Allaham S, Kumar A, Morriss F, Lakhanpaul M, Wilson E, Sikorski C, et al. Participatory learning and action (PLA) to improve health outcomes in high-income settings: a systematic review protocol. BMJ Open. 2022;12:e050784.

Ukowitz M. Who defines innovation in education? Participatory action research and organisational learning. Educational Action Res. 2023;31(2):366–83.

Smith D, Frey N, Pumpian I, Fisher D. Building equity: policies and practices to empower all learners. ASCD; 2017.

Maslow AH. Motivation and personality. Harper & Row; 1970.

Carter P, Darling-Hammond L. Teaching diverse learners. Handbook of research on teaching. 2016:593–638.

Severiens S, Wolff R. Study success of students from ethnic minority backgrounds: an overview of explanations for differences in study careers. Routledge Int Handb High Educ. 2009:61–72.

Voogt J, Taris T, van Rensen E, Schneider M, Noordegraaf M, Schaaf MF. Speaking up, support, control and work engagement of medical residents. A structural equation modelling analysis. Med Educ. 2019;53.

Gay G. Culturally responsive teaching: theory, research, and practice. teachers college; 2018.

Wensing M, Grol R. Knowledge translation in health: how implementation science could contribute more. BMC Med. 2019;17(1):88.

Fernandez ME, Ten Hoor GA, van Lieshout S, Rodriguez SA, Beidas RS, Parcel G, et al. Implementation mapping: using intervention mapping to develop implementation strategies. Front Public Health. 2019;7:158.

Mays N, Pope C. Qualitative research in health care. Assessing quality in qualitative research. BMJ. 2000;320(7226):50–2.

Cordeiro L, Rittenmeyer L, Soares CB. Action research methodology in the health care field: a scoping review protocol. JBI Database Syst Rev Implement Rep. 2015;13(8):70–8.

Akman H, Plewa C, Conduit J. Co-creating Value in Online Innovation communities. Eur J Mark. 2018;53.

Fernandez A, Shaw G. Academic Leadership in a time of Crisis: the coronavirus and COVID-19: academic leadership in a time of crisis. J Leadersh Stud. 2020;14.

Taylor B, Henshall C, Kenyon S, Litchfield I, Greenfield S. Can rapid approaches to qualitative analysis deliver timely, valid findings to clinical leaders? A mixed methods study comparing rapid and thematic analysis. BMJ Open. 2018;8:e019993.

Brown-Johnson C, Safaeinili N, Zionts D, Holdsworth LM, Shaw JG, Asch SM, et al. The Stanford Lightning Report Method: a comparison of rapid qualitative synthesis results across four implementation evaluations. Learn Health Syst. 2020;4(2):e10210.

Lewinski AA, Crowley MJ, Miller C, Bosworth HB, Jackson GL, Steinhauser K, et al. Applied Rapid Qualitative Analysis to develop a contextually appropriate intervention and increase the likelihood of Uptake. Med Care. 2021;59:S242–51.

Nevedal A, Reardon C, Opra Widerquist M, Jackson G, Cutrona S, White B, Damschroder L. Rapid versus traditional qualitative analysis using the Consolidated Framework for Implementation Research (CFIR). Implement Sci. 2021;16.

Vicsek L. A Scheme for analyzing the results of Focus Groups. Int J Qualitative Methods. 2007;6(4):20–34.

Vindrola-Padros C, Johnson G. Rapid Techniques in Qualitative Research: a critical review of the literature. Qual Health Res. 2020;30:1596–604.

Gale R, Wu J, Erhardt T, Bounthavong M, Reardon C, Damschroder L, Midboe A. Comparison of rapid vs in-depth qualitative analytic methods from a process evaluation of academic detailing in the Veterans Health Administration. Implement Sci. 2019;14.

Miles S, Ahuja A. Learning from difference: Sharing international experiences of developments in inclusive education. The SAGE handbook of special education. 2007:131 – 45.

Waldring IE. The fine art of boundary sensitivity: second-generation professionals engaging with social boundaries in the workplace. University of Antwerp; 2018.

May CR, Mair F, Finch T, MacFarlane A, Dowrick C, Treweek S, et al. Development of a theory of implementation and integration: normalization process theory. Implement Sci. 2009;4(1):29.

O’Reilly-de Brún M, de Brún T, Okonkwo E, Bonsenge-Bokanga J-S, De Almeida Silva MM, Ogbebor F, et al. Using Participatory Learning & Action research to access and engage with ‘hard to reach’ migrants in primary healthcare research. BMC Health Serv Res. 2016;16(1):25.

Weis L, Fine M. Working Method: Research and Social Justice. Working Method: Research and Social Justice. 2004:1-186.

O’Reilly-de Brún M, De Brún T. The use of Participatory Learning & Action (PLA) research in intercultural health: some examples and some questions. Engaging service-users in co-designing primary health care. 2010:27.

Lupton KL, O’Sullivan PS. How Medical Educators Can Foster Equity and Inclusion in Their Teaching: A Faculty Development Workshop Series. Acad Med. 2020;95(12S Addressing Harmful Bias and Eliminating Discrimination in Health Professions Learning Environments):S71-s6.

Kolluru S, Wanat MA, Ficzere CH, Akiyode O, Haber SL, Hayatshahi A, et al. Review of Best practices for Diversity, Equity, and inclusion committees within Colleges of Pharmacy. Am J Pharm Educ. 2023;87(4):ajpe9043.

Miguel A, Sloughter JM, Dorsey JM, Hartley R, Shih FJ, Crevier J, editors. What’s Next? From Analysis to Action. 2021 CoNECD; 2021.

Carminati L. Generalizability in qualitative research: a tale of two traditions. Qual Health Res. 2018;28(13):2094–101.

Shenton A. Strategies for ensuring trustworthiness in qualitative Research projects. Educ Inform. 2004;22:63–75.

Maxwell J. Why qualitative methods are necessary for generalization. Qualitative Psychol. 2020;8.

Polit D, Beck C. Generalization in quantitative and qualitative research: myths and strategies. Int J Nurs Stud. 2010;47:1451–8.

Brown J, Chapman T, Graham D. Becoming a new doctor: a learning or survival exercise? Med Educ. 2007;41:653–60.

Download references

Acknowledgements

We affirm that no individuals other than the listed authors provided professional writing or analysis services. Still, we thank all anonymous participants whose contributions enriched this study.

We declare that the authors have no external funding to report.

Author information

Authors and affiliations.

Amsterdam UMC location University of Amsterdam, Department of General Practice and Public Health Research Institute, Meibergdreef 9, Amsterdam, 1105 AZ, The Netherlands

N.M. van Moppes, M. Nasori, J. Bont, J.M. van Es & M.R.M. Visser

Department of General Practice, Radboud University Medical Center, Nijmegen, The Netherlands

M.E.T.C. van den Muijsenbergh

Pharos, centre of expertise on health disparities, Utrecht, The Netherlands

You can also search for this author in PubMed   Google Scholar

Contributions

All authors have made substantial contributions to the conception OR design of the work; OR the acquisition, analysis, OR interpretation of data; OR have drafted the work or substantively revised it. All authors have approved the submitted version (and any substantially modified version that involves the author’s contribution to the study); and have agreed both to be personally accountable for the author’s own contributions and to ensure that questions related to the accuracy or integrity of any part of the work, even ones in which the author was not personally involved, are appropriately investigated, resolved, and the resolution documented in the literature. Contributions per author: N.M. van Moppes: Conception, Design of the work, Acquisition, Analysis and interpretation of data, Drafted the work and integrated all revisions; M. Nasori: Conception, Design of the work, Acquisition, Analysis and interpretation of data, Substantively revised the work; J. Bont: Substantively revised the work; J.M. van Es: Substantively revised the work; M.R.M. Visser: Conception, Design of the work, Substantively revised the work; M.E.T.C. van den Muijsenbergh: Conception, Design of the work, Substantively revised the work.

Corresponding author

Correspondence to N.M. van Moppes .

Ethics declarations

Ethics approval and consent for participate.

Participation in the study was voluntary, with informed consent for participation and publication obtained from all participants. The Dutch Association for Medical Training’s Ethical Review Board approved the study (file number 2020.6.1). Data handling adhered to the General Data Protection Regulation and Amsterdam UMC Clinical Research Unit procedures, and we shared the study results with all participants.

Competing interests

We declare that the authors have no competing interests as defined by BMC Medical Education or any other interests that might be perceived to influence the results and/or discussion reported in this paper.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

van Moppes, N., Nasori, M., Bont, J. et al. Towards inclusive learning environments in post-graduate medical education: stakeholder-driven strategies in Dutch GP-specialty training. BMC Med Educ 24 , 550 (2024). https://doi.org/10.1186/s12909-024-05521-z

Download citation

Received : 12 December 2023

Accepted : 06 May 2024

Published : 17 May 2024

DOI : https://doi.org/10.1186/s12909-024-05521-z

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Inclusive Medical Education
• Stakeholder-driven Strategies
• GP-specialty Training

BMC Medical Education

ISSN: 1472-6920

• Submission enquiries: [email protected]
• General enquiries: [email protected]

Log in using your username and password

• Search More Search for this keyword Advanced search
• Latest content
• Current issue
• BMJ Journals More You are viewing from: Google Indexer

You are here

• Volume 83, Issue 6
• EULAR recommendations for the non-pharmacological core management of hip and knee osteoarthritis: 2023 update
• Article Text
• Article info
• Citation Tools
• Rapid Responses
• Article metrics

• http://orcid.org/0000-0003-0920-888X Tuva Moseng 1 ,
• http://orcid.org/0000-0001-6322-3859 Theodora P M Vliet Vlieland 2 ,
• Simone Battista 3 ,
• David Beckwée 4 ,
• Vladimira Boyadzhieva 5 ,
• http://orcid.org/0000-0002-3478-5665 Philip G Conaghan 6 ,
• Daniela Costa 7 ,
• Michael Doherty 8 ,
• Andrew G Finney 9 , 10 ,
• Tsvetoslav Georgiev 11 ,
• Milena Gobbo 12 ,
• Norelee Kennedy 13 ,
• Ingvild Kjeken 1 ,
• http://orcid.org/0000-0002-8940-0582 Féline P B Kroon 14 , 15 ,
• http://orcid.org/0000-0002-5424-9448 L Stefan Lohmander 16 ,
• Hans Lund 17 ,
• Christian D Mallen 18 ,
• Karel Pavelka 19 ,
• Irene A Pitsillidou 20 ,
• Margaret P Rayman 21 ,
• Anne Therese Tveter 1 ,
• http://orcid.org/0000-0003-4783-8663 Johanna E Vriezekolk 22 ,
• Dieter Wiek 23 ,
• Gustavo Zanoli 24 ,
• http://orcid.org/0000-0001-8602-342X Nina Østerås 1
• 1 Center for treatment of Rheumatic and Musculoskeletal Diseases (REMEDY) , Diakonhjemmet Hospital , Oslo , Norway
• 2 Department of Orthopaedics, Rehabilitation and Physical Therapy , Leiden University Medical Center (LUMC) , Leiden , Netherlands
• 3 University of Genoa Department of Neuroscience Ophthalmological Rehabilitation Genetics and Mother and Child Health , Genova , Italy
• 4 Rehabilitation Research Department , Vrije Universiteit Brussel , Brussel , Belgium
• 5 UMHAT “St. Iv. Rilski” Clinic of Rheumatology, Medical University Sofia , Sofia , Bulgaria
• 6 Leeds Institute of Rheumatic and Musculoskeletal Medicine , University of Leeds and NIHR Leeds Biomechanical Reserch Centre , Leeds , UK
• 7 Comprehensive Health Research Center (CHRC) , Universidade Nova de Lisboa , Lisboa , Portugal
• 8 Department of Academic Rheumatology , University of Nottingham , Nottingham , UK
• 9 Research Institute for Primary Care and Health Sciences , Keele University School of Medicine , Keele , UK
• 10 School of Nursing and Midwifery , Keele University , Keele , UK
• 11 Clinic of Rheumatology, University Hospital St. Marina, First Department of Internal Medicine , Medical University Varna , Varna , Bulgaria
• 12 Positivamente Centro de Psicología , Madrid , Spain
• 13 School of Allied Health, Faculty of Education and Health Sciences and Health Research Institute , University of Limerick , Limerick , Ireland
• 14 Department of Rheumatology , Leiden University Medical Center (LUMC) , Leiden , The Netherlands
• 15 Department of Rheumatology , Zuyderland Medical Centre Heerlen , Heerlen , The Netherlands
• 16 Department of Clinical Sciences Lund, Orthopaedics , Lund University , Lund , Sweden
• 17 Centre for Evidence-Based Practice , Western Norway University of Applied Sciences , Bergen , Norway
• 18 Keele University School of Medicine , Keele , UK
• 19 Institute of Rheumatology, Department of Rheumatology , Charles University First Faculty of Medicine , Praha , Czech Republic
• 20 EULAR Patient Research Partner , Cyprus League Against Rheumatism , Nicosia , Cyprus
• 21 Department of Nutritional Sciences , University of Surrey Faculty of Health and Medical Sciences , Guildford , UK
• 22 Research & Innovation , Sint Maartenskliniek , Nijmegen , The Netherlands
• 23 EULAR Patient Research Partner , Deutsche Rheuma-Liga , Bonn , Germany
• 24 Orthopaedic Ward , Casa di Cura Santa Maria Maddalena , Novara , Italy
• Correspondence to Dr Tuva Moseng, Center for treatment of Rheumatic and Musculoskeletal Diseases (REMEDY), Diakonhjemmet Hospital, Oslo, Norway; tuva.moseng{at}diakonsyk.no

Introduction Hip and knee osteoarthritis (OA) are increasingly common with a significant impact on individuals and society. Non-pharmacological treatments are considered essential to reduce pain and improve function and quality of life. EULAR recommendations for the non-pharmacological core management of hip and knee OA were published in 2013. Given the large number of subsequent studies, an update is needed.

Methods The Standardised Operating Procedures for EULAR recommendations were followed. A multidisciplinary Task Force with 25 members representing 14 European countries was established. The Task Force agreed on an updated search strategy of 11 research questions. The systematic literature review encompassed dates from 1 January 2012 to 27 May 2022. Retrieved evidence was discussed, updated recommendations were formulated, and research and educational agendas were developed.

Results The revised recommendations include two overarching principles and eight evidence-based recommendations including (1) an individualised, multicomponent management plan; (2) information, education and self-management; (3) exercise with adequate tailoring of dosage and progression; (4) mode of exercise delivery; (5) maintenance of healthy weight and weight loss; (6) footwear, walking aids and assistive devices; (7) work-related advice and (8) behaviour change techniques to improve lifestyle. The mean level of agreement on the recommendations ranged between 9.2 and 9.8 (0–10 scale, 10=total agreement). The research agenda highlighted areas related to these interventions including adherence, uptake and impact on work.

Conclusions The 2023 updated recommendations were formulated based on research evidence and expert opinion to guide the optimal management of hip and knee OA.

• Osteoarthritis, Knee
• Osteoarthritis
• Rehabilitation
• Physical Therapy Modalities
• Therapeutics

This is an open access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0). See: http://creativecommons.org/licenses/by-nc/4.0/ .

https://doi.org/10.1136/ard-2023-225041

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Introduction

Osteoarthritis (OA) is the most common joint disease worldwide, 1 with an increasing global burden of disability and healthcare utilisation. 2 The number of people with OA globally rose by 28% from 2010 to 2019, affecting over 500 million people, and about 6%, worldwide. 3 Due to an ageing population, increasing obesity and sport-related joint injuries, the disease will become even more prevalent in the forthcoming years. 2 In 2019, OA was the 15th highest-ranked cause of years lived with disability (YLDs) worldwide and was responsible for 2% of the total global YLDs. 3 OA is regarded as a severe disease, and serious condition and people with OA commonly experience pain, stiffness and associated functional loss. 4 Optimal management of hip and knee OA has important implications for the individual and society through the potential for improving individual health, work participation and utilisation of healthcare services. However, most people with OA do not receive optimal management. 5 6 In order to reduce the evidence-to-practice gap and the future burden 7 of this disease, the healthcare services’, policy-makers’ and the population awareness of the importance and benefits of evidence-based management of OA must be improved.

EULAR recommendations, including priorities for implementation and future research, can play a role in increasing awareness and uptake of best evidence care. In 2013, an EULAR Task Force (TF) developed recommendations for the non-pharmacological core management of hip and knee OA. 8 Since then there remains no cure in sight for OA, and effective disease-modifying drugs are lacking. 2 Therefore, non-pharmacological approaches are still considered a core treatment for people with hip and knee OA, aiming to alleviate symptoms and improve or maintain physical function. Since the publication of the 2013 recommendations, a large number of studies on the effectiveness of core non-pharmacological treatment modalities and new methods for delivery and follow-up of such treatments have been published. An update of these recommendations would potentially have implications for the level of evidence (LoE) categories and could lead to revisions of the recommendations and formulation of new recommendations with important implications for OA management.

The main aim of this TF process was to update the 2013 evidence-based recommendations for non-pharmacological core management, provide additional details on effectiveness, safety and cost-effectiveness, and formulate research and educational agendas and priorities for implementation activities. The target groups for the updated recommendations are people with hip or knee OA, all healthcare providers involved in the delivery of non-pharmacological interventions in OA care, researchers in the field of OA, officials in healthcare governance and reimbursement agencies and policy-makers.

The Standardised Operating Procedures for EULAR-endorsed Recommendations 9 were used as a framework for this project. The structure of the manuscript is guided by the Appraisal of Guidelines, Research and Evaluation instrument. 10

To pursue the task of updating the 2013 recommendations, a multidisciplinary TF with in-depth knowledge of non-pharmacological OA care was established. The TF consisted of 25 members from 14 European countries and included 9 physiotherapists, 6 rheumatologists, 2 orthopaedic surgeons, 2 psychologists, 2 patient research partners, 1 occupational therapist, 1 nurse, 1 general practitioner and 1 nutrition expert. A steering group, including a convenor (NØ), a methodologist (TPMVV) and a research fellow (TM), managed the process.

During the first digital TF meeting, the rationale for the update of the recommendations was presented, and the definition of core non-pharmacological management was clarified. The TF agreed on 11 research questions based on the research propositions from the 2013 recommendation. For the subsequent systematic literature review (SLR), the research questions were organised according to the population, intervention, control and outcome (PICO) format with associated search terms ( online supplemental file 1 ). The new search terms added to the previous search strategy were related to the following topics: remote care, shared decision-making, psychological interventions/cognitive behaviour therapy (CBT)-based interventions and specific exercise modalities (eg, strength training and aerobic exercise). Due to the expected large body of published literature since the previous literature review from 2012, combined with the available resources and strict timeline for this update, it was decided that this SLR should primarily focus on evidence from systematic reviews (SRs) and meta-analyses of randomised controlled trials (RCTs) and secondarily on evidence from single RCTs. As this SLR was an update of a previously unpublished SLR, along with its pragmatic approach, it was decided that the details were best presented as online supplemental file 1 rather than a publication of its own.

Supplemental material

The SLR was conducted by the fellow and convenor in close collaboration with an experienced librarian (HIF) and with support from the methodologist. Three main literature searches were conducted in the databases Medline (Ovid), Embase (Ovid), AMED (Ovid), Cochrane Library (Cochrane TRIALS), CINAHL (Ebsco) and Epistemonikos (SR search only).

The primary literature search aimed at identifying relevant SRs of RCTs investigating the effectiveness of core non-pharmacological management strategies as specified in the PICOs. The search was conducted from 2012 (the end year of the previous search) until 17 February 2022 and later updated until 27 May 2022 ( online supplemental file 1 ). Based on the PICOs, two authors (TM and NØ) independently screened titles and abstracts. Potentially relevant studies were read and evaluated in full text. Studies were included if they were SRs, including a meta-analysis of two or more RCTs on people diagnosed with hip or knee OA or with persisting knee pain in people 45 years or older and investigating non-pharmacological core management strategies. Relevant comparisons were no intervention, usual care or any other intervention. Relevant outcomes were pain, physical function, quality of life (QoL), patient global assessment of target joint, adverse effects or cost-effectiveness. The included studies were categorised under the 11 research questions. If relevant, one study could inform multiple research questions. The quality of the included SRs was evaluated with A MeaSurement Tool to Assess systematic Reviews (AMSTAR II). 11 The assessments were conducted independently by three assessors (GS, EAB and IS), working in pairs independent of the TF, with experience in quality assessment of SRs and RCTs. Disagreements between the assessors were resolved through discussion.

A second literature search with a comparable search strategy was conducted to identify newer RCTs not included in the latest published SR on the same topic, or relevant RCTs not included in any SRs, or RCTs on research questions for which no relevant SRs were identified. To identify such RCTs published in the past four to 5 years, the search was conducted from 1 January 2018 to 27 May 2022.

A third literature search was conducted with a similar search strategy from 1 January 2012 to 31 December 2017, aiming to identify relevant RCTs specifically on the research questions for which no relevant SRs had been identified. The two last searches were screened independently by the same two authors, and relevant studies were read and evaluated in full text. Studies were included if they were RCTs relevant to the PICOs. The quality of the included RCTs was assessed with the Cochrane Risk of Bias tool 2 (RoB2) 12 independently by two researchers (EAB and IS) independent of the TF. Disagreements between the assessors were resolved through discussion.

In the period before the second TF meeting, five digital subgroup meetings were arranged. Groups of 4–5 TF members and the steering group participated in each meeting. The purpose of the subgroup meetings was to go through the relevant results from the SLR and to discuss and prepare preliminary suggestions for revisions and updates of the recommendations to guide the discussion at the second TF meeting. The group discussed between 1 and 3 of the previous 11 recommendations in each subgroup meeting. This method was implemented to allow all TF members to express their opinions in smaller forums and potentially to reduce the workload of the second TF meeting.

During the second digital TF meeting, the results from the SLR, along with the proposed updates from the subgroups, were presented to the whole TF. The previous recommendations and the proposed updates were then discussed in light of the SLR and the expertise of the group. After the discussions and revisions, the TF members voted for consensus on each revised overarching principle and recommendation (defined as 75% or more in favour of the suggested updates). After the meeting, the updated list of recommendations was collated and emailed to the TF members in a digital survey to rate the level of agreement (LoA) on a 0–10 point scale (0=totally disagree, 10=totally agree). Further, the TF voted on the prioritised order of the recommendations for implementation activities. The TF also formulated a research agenda based on identified gaps in the evidence. The steering group defined the LoE and strength of each recommendation in accordance with the Oxford Levels of Evidence. 13 The steering group also formulated the educational agenda on behalf of the TF.

The three systematic literature searches yielded a total of 6816 references after the removal of duplicates ( figure 1 ). From these, 67 SRs and 31 RCTs were initially considered relevant for the SLR. However, we chose to extract data from 36 of the SRs due to reasons elaborated in online supplemental file 1, p.49 ,. The most frequent reason was that the interventions under study were not considered relevant for this review. The quality of the included SRs was generally poor, with 35 of 36 studies being rated with an overall low or critically low quality by the AMSTAR II tool ( online supplemental file 1 ). The critical items that most often contributed to the overall low quality of the studies were: the lack of an explicit statement that the review methods were established prior to the conduct of the review; the lack of the use of a comprehensive literature search strategy; and lack of a list of excluded studies with reasons for exclusion. There was large variation in the overall quality of the included RCTs as assessed by the RoB2 tool ( online supplemental file 1 ). Most studies with a low risk of bias were on exercise interventions and delivery, whereas there were higher concerns related to the studies on, for example, lifestyle-related interventions. Most commonly, these concerns were related to the elements of measurement of the outcome (eg, the lack of a blinded outcome assessor).

• Download figure
• Open in new tab
• Download powerpoint

PRISMA flow diagram. PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses.

The main updates to the recommendations are summarised in box 1 . The TF agreed to rephrase and change two previous recommendations into overarching principles. These were the recommendations on: (1) the use of a biopsychosocial approach in the initial assessment and (2) the recommendation on individualisation of treatment. It was decided that these were generic statements used to inform the basis for management rather than specific treatment recommendations. Inherent to the nature of these statements, relevant studies were absent from the SLR. 14

What is new?

The updated recommendations have been reorganised into two overarching principles and eight treatment recommendations.

The wording of each recommendation is condensed.

The level of agreement is above 9 for all recommendations.

The level of evidence is 1a/1b for seven of the eight recommendations.

It was further decided to revise the nine previous recommendations into eight updated recommendations by merging the recommendations on footwear and walking aids, other assistive devices and adaptations. Moreover, to improve readability the previous recommendations were shortened, and subsections were rewritten and moved to the explanatory text. In addition, the TF also discussed the order for the presentation of the recommendations and decided to change this into a more logical sequence.

High LoAs were achieved for all eight recommendations, and seven recommendations were graded with LoE 1a/1b and strength level A. Recommendation 2—on delivery of information, patient education and self-management—was ranked by the TF as having the highest priority for implementation. Table 1 summarises the updated overarching principles, recommendations, LoA, LoE, strength of recommendation and priority for implementation.

• View inline

Overarching principles and specific recommendations for the non-pharmacological core management of hip and knee osteoarthritis

Recommendation 1

People with hip or knee OA should be offered an individualised, multicomponent management plan that includes the recommended core non-pharmacological approaches.

This recommendation deals with the provision of an integrated package of care rather than single treatments alone or in succession. The majority of new, relevant SRs and RCTs informing this recommendation investigated the effectiveness of the combination of patient education and exercise or the combination of patient education, exercise and diet or the combination of behaviour change techniques/pain-coping skills training and exercise, compared with information or one of the treatments alone. 15–18 The updated evidence shows that combining treatments leads to larger effects on pain and function compared with providing the treatments separately, thereby providing a rationale for combining different treatment modalities. The combination of education, exercise and dietary weight management was also considered cost-effective compared with physician-delivered usual care investigated in five healthcare systems. 19 The TF discussed that, although not all potential combinations of treatments are investigated in meta-analyses or newer RCTs, the results of available studies are likely to be generalisable to different combinations. Thus, the TF agreed on the general consideration of multicomponent treatments from a broader spectrum of potential combinations based on an assessment of a patient’s individual needs and preferences.

Through the SLR, no specific evidence was retrieved with regard to the effects of pacing and maintenance of activity. This specific element was therefore removed from the recommendation.

Recommendation 2

People with hip or knee OA should be offered information, education and advice on self-management strategies (considering available modes of delivery) and these should be included and reinforced at subsequent clinical encounters.

Recommendation 2 concerns the delivery of information, education and advice on self-management strategies. New evidence from the SLR showed zero to small significant effects on pain and function from patient education as a single intervention in the short term, which is in line with the previous recommendation. 15 20 In 2013, this recommendation focused on how education and information should be delivered in terms of being individualised, being included in every aspect of management, and specifically addressing the nature, causes, consequences and prognosis of OA. Moreover, it was stated that this should be reinforced and developed, supported by written or other types of material, including partners or carers of the individual, if relevant. The current TF acknowledged the importance of these aspects to ensure the effective delivery of information and education for people with hip and knee OA. However, none of the studies from the SLR could provide specific evidence for any of these aspects, except with regard to delivery method. One SR reported the effects of patient education delivered through telephone when compared with usual care, but the results were not significant for pain or disability. 20 The TF further chose to add self-management to the updated recommendation. Evidence from two SRs, including seven RCTs, compared structured self-management programmes against a large range of control interventions. Zero to small favourable effects were found for self-management, delivered face to face or digitally, compared with routine/usual care. 21 22 Despite the limited effects reported in the literature, the TF agreed that self-management is a concept closely related both to the delivery of information and education in a clinical setting and to the uptake of other relevant treatment modalities.

Recommendation 3

All people with hip or knee OA should be offered an exercise programme (eg, strength, aerobic, flexibility or neuromotor) of adequate dosage with progression tailored to their physical function, preferences and available services.

The body of literature investigating the effects of different types of exercise regimes was already large when the 2013 recommendations were published. Aiming to progress the knowledge on the effects of exercise for hip and knee OA, the current SLR did not focus on studies investigating the effects of general exercise on hip and knee OA as these effects were well established previously. 23 24 The aim was rather to identify studies investigating the effects of well-defined exercise modalities, as well as studies looking more specifically into exercise dosage.

For hip OA, one SR summarised the effects of supervised, progressive resistance training, which reported beneficial effects on pain, function and QoL. The effect sizes, however, were small with large CIs. 25

For knee OA, four SRs and five additional RCTs were identified on the exercise 26–28 modalities Tai Chi, yoga, stationary cycling, proprioceptive training, weight-bearing and non-weight bearing exercise, and neuromuscular exercise combined with strength training. 29–33 Overall, the results showed small to moderate positive effects on pain and function for all these exercise modalities compared with no-exercise control (no intervention, waiting list or non-exercise interventions). Still, the results were less clear in head-to-head comparisons of different exercise types, modalities or doses.

In summary, results showed that a variety of exercise modalities might lead to improved pain and function for people with hip or knee OA, making it difficult to recommend one type of exercise over another. The optimal exercise dosage is also difficult to establish, with evidence from 1 SR on hip OA (including 12 RCTs) and 1 SR on knee OA (including 45 RCTs) providing some evidence that exercise in line with dose recommendations from the American College of Sports Medicine provided larger improvements in pain compared with non-compliant exercise programmes. 34–36 The differences, however, were small, and the clinical relevance is debatable. Two newer RCTs on knee OA, comparing high-intensity to low-intensity resistance training or no-exercise control, found no or only small between-group differences with regard to pain and function, 37 38 thus making it difficult to make explicit recommendations on exercise dosage.

With respect to safety, adverse events in exercise studies for hip and knee OA were investigated in two SRs. 39 40 The two studies concluded that, although the report of adverse events in exercise studies was inconsistent and some patient drop-outs were potentially misclassified, adverse events were generally uncommon and non-serious, and that exercise seemed to be associated with minimal risk of harm. Concerning the economic aspects of exercise, one SR on cost-effectiveness found that in the majority of the 12 included studies, exercise for hip and knee OA showed cost-effectiveness at conventional willingness-to-pay thresholds. 19

The TF chose to update this recommendation, highlighting that the choice of exercise should be based on individual function, patient preferences and available services. 41 Overall, exercise is by far the most studied and strongly recognised non-pharmacological core management treatment option and this recommendation has the strongest evidence base. The TF also expressed the importance of maintaining exercise over time for the positive effects to persist.

Recommendation 4

The mode of delivery of exercises (eg, individual or group sessions, supervised or unsupervised, face to face or by using digital technology, land-based or aquatic exercise) should be selected according to local availability and patient preferences. The exercises preferably should be embedded in an individual plan for physical activity.

As established in the description of recommendation 3, there is convincing evidence for the effectiveness of various exercise modalities on pain and function in hip and knee OA. However, the delivery method of exercise programmes varies largely across studies and may influence study outcomes.

One SR found superior effects from technology-supported exercise compared with control with non-technological or no care services on pain, function and QoL, 42 whereas another SR found superior effects from telehealth-based exercise compared with no-telehealth exercise control for pain but not for function or QoL. 43 The reported effect sizes were small. One additional RCT found a small, significant effect on function at 6 months follow-up for an education combined with strengthening exercise follow-up through telephone calls compared with education alone, but no other between-group differences in pain and function were detected after 6 and 12 months. 44 Another RCT comparing access to an educational website combined with exercise supported by automated behaviour change text messages to access to the educational website alone found significant superior effects of the combined first intervention on pain and function after 24 weeks. 45 For aquatic exercise, one SR reported small short-term beneficial effects for pain and function compared with no intervention or usual care. However, another SR comparing aquatic exercise to land-based exercise did not find any of these modes superior to the other. 46 47 One RCT of a three-stage stepped care exercise programme compared with educational materials found beneficial, although not clinically relevant, effects of the stepped care programme on pain and function at 3 and 9 months, but not at 6 months. 48 Analyses of the cost-effectiveness of the same stepped-care intervention concluded that there is a high probability of short-term cost-effectiveness. 49

The new evidence adds information on technology-supported delivery of exercise, aquatic exercise and a stepped care strategy for exercise delivery. The results from these studies show a wide variety of potentially effective delivery methods for exercise, which in clinical practice should be aligned with patient preferences and the availability of local services. The TF also underlined the importance of the exercise programme being embedded in an individual plan for physical activity. Such plans should be set up in accordance with well-recognised recommendations for physical activity, such as from the WHO or EULAR. 41 50 General physical activity has multiple health benefits and is also important for the management of common comorbidities associated with OA, such as cardiovascular disease and diabetes. 51 52

Recommendation 5

People with hip or knee OA should be offered education on the importance of maintaining a healthy weight. Those overweight or obese should be offered support to achieve and maintain weight loss.

In the updated SLR, three SRs were identified, including one network meta-analysis investigating the effects of weight loss interventions. Two were on studies of knee OA, 19 53 whereas the third included studies of both hip and knee OA, although only 2 of the 19 trials included in that study were conducted on a mixed hip and knee OA population. 54 The results from this SR showed beneficial effects, compared with minimal care, of both diet and multifocused weight-loss interventions (combining diets, telephone coaching, psychological pain-coping interventions/CBT, specialist referral education and exercise) on pain and disability, with the largest effect size on pain for multifocused interventions. Further, it was reported that when comparing weight-loss-focused interventions (diets) to exercise, no between-group differences were detected for pain or disability. When comparing combined interventions of dietary weight loss and exercise to dietary weight loss or exercise alone, small effects were found in favour of the combined intervention.

In the network meta-analysis, bariatric surgery was the most effective pain-reducing intervention, followed by a low-calorie diet combined with exercise intervention. 53 The last SR on knee OA used cost-effectiveness as an outcome and reported that an intensive 18-month diet and exercise intervention with the goal of 5% weigth loss was likely to be an efficient use of healthcare resources compared with a healthy lifestyle control. 19

The above-mentioned studies made it clear that there is increasing evidence supporting multifocused weight loss interventions as beneficial for OA pain and disability. Therefore, the TF recommended that people with overweight or obesity and OA should be offered support to achieve and maintain weight loss. The TF notes that the amount of evidence mainly stems from studies on knee OA. As overweight and obesity are strong risk factors for the development and progression of OA, and in particular knee OA, 2 the TF also wanted to add to the recommendation the importance of education on the benefits of maintaining a healthy weight.

Recommendation 6

For people with hip or knee OA, consider walking aids, appropriate footwear, assistive devices and adaptations at home and at work to reduce pain and increase participation.

Through the SLR, four SRs investigating the effects on knee OA of lateral wedge insoles compared with other types of insoles, including flat/neutral insoles or knee braces, were retrieved. These studies did not report any between-group differences for any comparisons on pain or function. 55–58 On the other hand, one RCT reported a small between-group difference in favour of lateral wedge insoles compared with neutral insoles on a single pain scale in people prescreened to knee adduction moment improvements (but not on other pain scales, function or QoL). 59 For footwear, one RCT found positive effects of biomechanical footwear with individually adjustable external convex pods attached to the outsole compared with control footwear. 60 Another RCT found small effects after 6 months on pain, but not on function, from wearing stable, supportive shoes over flat flexible shoes for at least 6 hours per day. 61

Summarised, most evidence did not support the use of any lateral wedged or other insoles to affect pain or function in knee OA. The results from one RCT provided some support for the use of stable, supportive shoes. The TF wanted to add that from a clinical perspective, the use of comfortable shoes, big enough to give ample space for the toes when weight-bearing, is still a general recommendation for people with hip and knee OA.

For other types of assistive aids and devices, two RCTs comparing the use of canes to the non-use of auxiliary gait devices were identified. The results were contradictory, and conclusions on the effect of cane were difficult to draw from the available evidence. 62 63 No studies were retrieved for other types of assistive devices or home adaptations. Based on the expert knowledge of the group, it was argued that such devices could still be useful to some people with hip or knee OA in terms of reducing pain, undertaking daily activities and improving participation. The TF wanted to emphasise that improving participation is an important aspect underpinning this specific recommendation. Assistive devices may serve as means to reduce pain and improve participation both at home and at work and should, therefore, be considered in that context. Examples of such devices might be devices to aid dressing, height-adjustable chairs, raised toilet seats, handrails in staircases or the use of appropriate walking aids.

Recommendation 7

People with hip or knee OA with or at risk of work disability should be offered timely advice on modifiable work-related factors and, where appropriate, referral for expert advice.

OA is one of the leading causes of reduced work participation, and the disease may critically affect the number of sick days and, ultimately, the extension of a person’s work career. 64 Although there are well-known occupational risk factors, such as heavy lifting and knee straining activities associated with the development of knee OA, 65 it was noted that there is a lack of studies on vocational rehabilitation for people with hip or knee OA. In the current update, only one relevant RCT was retrieved. This study used workability as an outcome, whereas the study intervention in both groups focused on self-management with the addition of an activity tracker in the intervention group. In this study, no between-group differences were reported for workability. 66

Although little research has been conducted, the TF considered that appropriate interventions to increase work participation for people with hip and knee OA are highly relevant. A proper assessment of the individual work situation may have a large impact and should receive attention during consultations. 67 Health professionals, in cooperation with the employer, should be able to offer timely advice on modifiable work-related factors such as working from home, the use of height-adjustable desks and office chairs, the possibility of changing work tasks, commuting to/from work, use of assistive technology, and receiving support from management, colleagues and family towards employment. The TF also noted that adaptations to improve workability might be considered and applied not only at the workplace but also in the home.

Recommendation 8

Consider employing elements of behaviour change techniques when lifestyle modifications are needed (eg, physical activity, weight loss) for people with hip or knee OA.

This recommendation concerns the potential need for lifestyle change in people with hip and knee OA. It focuses specifically on physical activity and weight loss as part of a healthy lifestyle since these aspects are specifically relevant for people with hip or knee OA. One SR and eight additional RCTs were identified on various interventions to enhance a healthy lifestyle, mainly through maintaining physical activity over time. The SR reported small to moderate effects of adding booster sessions to exercise programmes to improve mid-term to long-term adherence to exercise. 68 Furthermore, one RCT reported statistically significant improvements in pain and function from a combined programme of pain coping skills training and lifestyle behavioural weight management lasting 24 weeks compared with these interventions alone or standard care. 69 Interventions from the other RCTs aiming to support people with OA to improve their lifestyle and sustain such changes over time, included interventions of behaviour-graded activity, improving exercise adherence with telephone counselling, an app to enhance a healthy lifestyle, physical activity with telephone follow-up and a self-management lifestyle intervention. 70–72 However, when the effects on pain and function of these interventions were compared with standard care or other minimal interventions, none to very small between-group differences were observed for the comparisons. The TF wanted to enhance the importance of long-term follow-up on health behaviour change and not just recommend lifestyle change as a single intervention. The TF also discussed that the EULAR recommendation on core competencies for health professionals in rheumatology underlines that health professionals should be able to provide the principles of behaviour change techniques in the management of people with rheumatic and musculoskeletal disorders. 73

Research and educational agendas

The proposed research agenda ( table 2 ) was based on gaps identified in the literature and on topics which emerged during discussions among the TF members.

Research agenda for the non-pharmacological core management of people with hip and knee osteoarthritis

The education agenda ( table 3 ) highlights activities relevant to promote appropriate management of people with hip and knee OA.

Educational agenda for the non-pharmacological core management of people with hip and knee OA

Through this update, the recommendations for the non-pharmacological core management of hip and knee OA have been revised into two overarching principles and eight treatment recommendations. The revisions are based on research evidence, expert discussions and consensus. Since the publication of the 2013 recommendations, a number of new studies have been published on non-pharmacological treatment modalities and their methods of delivery. The updates to the recommendations are thus well anchored in evidence from research and the perspectives of the TF members, representing different professional, cultural and personal backgrounds, including the perspective of people with OA. The process led to a broad consensus within the TF on the updated principles and recommendations, reflected by the high LoA for all the revised recommendations. Such strong consensus gives reason to believe that the recommendations are suitable for use and implementation across European healthcare systems. These recommendations are also in line with recently published treatment recommendations for hip and knee OA by other societies. 74–76

The number of relevant SRs and RCTs retrieved through the SLR was high, especially for the research questions concerning exercise and delivery of exercise, with data drawn from a total of 15 SRs and 11 additional RCTs. The number of new studies led to an upgrade of the LoE for most of the recommendations, and seven of eight recommendations are now supported by level 1a or 1b evidence. However, it should be noted that the stated LoE does not necessarily involve all aspects of every recommendation and does not distinguish between hip and knee OA. The number of studies on hip OA was markedly lower than those on knee OA for all the treatment modalities. Therefore, the recommendations are generally weaker for hip OA than knee OA. There is an increasing recognition of differences between hip and knee OA, which heightens the need for more hip OA-specific studies to improve outcomes for this group specifically. 77 This is also highlighted in the proposed research agenda ( table 2 ). Further, as the aim was to address relevant non-pharmacological core management strategies, the recommendations do not specifically advise the management of subgroups of the OA population, for instance, younger adults or adults with a high burden of comorbidities. The authors are also aware of a number of ongoing studies addressing a range of innovative digital programmes in OA care. Such approaches will likely receive further attention in future updates of these recommendations. 78–81

With regard to outcomes, most of the included studies reported effects primarily on pain and physical function. To follow the recommendations on prioritised outcomes in OA research, 82 more studies investigating the effects of interventions on QoL and patients’ global assessment of the target joint may have provided additional relevant information. Workability and cost-effectiveness are two other outcomes of increasing interest when investigating the effect of interventions from a broader perspective. This SLR identified some studies including these outcomes, thus adding new and important knowledge to the recommendations. Nevertheless, additional studies with a focus on interventions to prevent the decline in workability and studies examining cost-effectiveness are still needed as such knowledge is important for healthcare governance and policy-makers when planning and prioritising effective OA care. Another relevant aspect of this update is the inclusion of studies investigating potential harm or adverse events from the interventions under study. Only two SRs specifically looking into this subject were identified. Still, the results add new knowledge to this important, although understudied, aspect of non-pharmacological interventions. 83

The challenges of implementing recommended care for people with hip and knee OA are well documented. 84 It is also apparent that developing recommendations is not sufficient on its own to influence practice. 85 Therefore, efforts have been made to address the impact and to develop strategies for the implementation of treatment recommendations. For future implementation, collaboration with other organisations focusing on OA care, such as The Osteoarthritis Research Society International, must be considered. EULAR highlights that implementing all recommendations at once is probably not feasible in practice. 86 The TF voted that the recommendation on information, education and self-management was ranked as the recommendation with the highest priority for implementation. This recommendation may play an important role as a basis for all other management and may improve people’s ability to live a good life with OA, as well as being an enabler of, aspects such as physical activity. 87 The prioritisation of the recommendations for implementation activities is also important with respect to the effective utilisation of healthcare services. As the OA population is growing, the need for effective healthcare utilisation and sustainable management strategies to improve outcomes will be vital to minimising the burden of OA at an individual and a societal level. 88

To conclude, the TF reached a broad consensus on the updated recommendation for non-pharmacological core OA management as well as on a research agenda highlighting the current evidence gaps, on an educational agenda and on the priority of the recommendations to support implementation activities.

Ethics statements

Patient consent for publication.

Not applicable.

Acknowledgments

We thank the librarian Hilde Iren Flaatten, University of Oslo, Norway, for supporting the literature searches and Emilie Andrea Bakke, Ingrid Skaalvik and Geir Smedslund, Diakonhjemmet Hospital, Center for treatment of Rheumatic and Musculoskeletal Diseases (REMEDY), Norwegian National Advisory Unit on Rehabilitation in Rheumatology, Oslo, Norway, for their thorough work in the AMSTAR II and Cochrane risk of bias assessments of the included studies.

PGC is funded in part by the National Institute for Health and Care Research (NIHR) through the Leeds Biomedical Research Centre.

• EULAR RheumaMap
• Hunter DJ ,
• Bierma-Zeinstra S
• Global Burden of Disease Collaborative Network
• Basedow M ,
• Smedslund G ,
• Østerås N , et al
• Kolahi A-A ,
• Smith E , et al
• Fernandes L ,
• Bijlsma JW , et al
• van der Heijde D ,
• Aletaha D ,
• Carmona L , et al
• Brouwers MC ,
• Browman GP , et al
• Reeves BC ,
• Wells G , et al
• Higgins JPT SJ ,
• Elbers RG , et al
• OCEBM Levels of Evidence Working Group*
• de Rooij M ,
• van der Leeden M ,
• Cheung J , et al
• De Oliveira Silva D ,
• Merolli M , et al
• Alrushud AS ,
• Rushton AB ,
• Kanavaki AM , et al
• Castelein B ,
• Wittoek R , et al
• Pitsillides A ,
• Stasinopoulos D ,
• Giannakou K
• Mazzei DR ,
• Ademola A ,
• Abbott JH , et al
• O’Brien KM ,
• Hodder RK ,
• Wiggers J , et al
• Zhu Y , et al
• Jackson J ,
• Sheffield D
• Uthman OA ,
• van der Windt DA ,
• Jordan JL , et al
• Verhagen AP ,
• Ferreira M ,
• Reijneveld-van de Vendel EAE , et al
• Mikkelsen LR ,
• Overgaard S , et al
• Bennell KL ,
• Nelligan RK ,
• Kimp AJ , et al
• Yen H-Y , et al
• Schrøder HM ,
• Wernbom M , et al
• Liu X , et al
• Pranata A , et al
• El-Ansary D ,
• Adams R , et al
• Chen Z , et al
• Garber CE ,
• Blissmer B ,
• Deschenes MR , et al
• Dagfinrud H ,
• Smedslund G , et al
• Bartholdy C ,
• Christensen R , et al
• de Zwart AH ,
• Roorda LD , et al
• Messier SP ,
• Mihalko SL ,
• Beavers DP , et al
• von Heideken J ,
• Chowdhry S ,
• Borg J , et al
• Rausch Osthoff A-K ,
• Niedermann K ,
• Braun J , et al
• Cai Y , et al
• Hinman RS ,
• Campbell PK ,
• Lawford BJ , et al
• Kasza J , et al
• Xu S , et al
• Zhou P , et al
• Woolson S ,
• Hoenig HM , et al
• Kaufman BG ,
• Coffman CJ , et al
• World Health Organization
• Williams MF ,
• London DA ,
• Husni EM , et al
• Mamas MA , et al
• Panunzi S ,
• Maltese S ,
• De Gaetano A , et al
• Robson EK ,
• Kamper SJ , et al
• Khosravi M ,
• Daryabor A , et al
• Yang J , et al
• Liang L , et al
• Felson DT ,
• Carter S , et al
• Reichenbach S ,
• Hincapié CA , et al
• Paterson KL ,
• Campbell PK , et al
• Silva AC , et al
• Van Ginckel A ,
• Wrigley TV , et al
• Viikari-Juntura E ,
• Solovieva S
• McWilliams DF ,
• Muthuri SG , et al
• Östlind E ,
• Stigmar K , et al
• Gwinnutt JM ,
• Wieczorek M ,
• Balanescu A , et al
• Nicolson PJA ,
• Dobson FL , et al
• Somers TJ ,
• Blumenthal JA ,
• Guilak F , et al
• Bendrik R ,
• Kallings LV ,
• Bröms K , et al
• LaValley MP ,
• Brown C , et al
• Lombard C ,
• Hussain SM , et al
• Edelaar L ,
• Nikiphorou E ,
• Fragoulis GE , et al
• National Institute for Health and Care Excellence: Guidelines
• Kolasinski SL ,
• Hochberg MC , et al
• Bannuru RR ,
• Vaysbrot EE , et al
• van der Esch M ,
• Hinman RS , et al
• Knox G , et al
• de Almeida AC ,
• de Noronha M , et al
• Groves-Williams D ,
• McHugh GA ,
• Bennell KL , et al
• Hawker GA ,
• Hunter DJ , et al
• Boutron I ,
• Baron G , et al
• Golightly YM ,
• Dziedzic KS ,
• EULAR European alliance of Associatons for rheumatology
• Kanavaki AM ,
• Rushton A ,
• Efstathiou N , et al
• Turkiewicz A ,
• Petersson IF ,
• Björk J , et al

Supplementary materials

Supplementary data.

This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

• Data supplement 1

Handling editor Josef S Smolen

X @DrS_Battista

Contributors TM was the research fellow for the project, undertaking the SLR in cooperation with NØ. The fellow was supervised by the steering group consisting of NØ (convenor) and TPMVV (methodologist). NØ and TPMVV supervised the process of the SLR. NØ organised and chaired the TF meetings. TM drafted the manuscript with advice from NØ and TPMVV. All authors have contributed to the recommendations by participating in the TF meetings; during discussion and agreement on the recommendations; revising and approving the manuscript for publication.

Funding This study was funded by European League Against Rheumatism (HPR055).

Disclaimer The views expressed are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health and Social Care.

Competing interests TPMVV was the Vice president EULAR health professionals 2020–2022 and is part of the EULAR Advocacy Committee 2020–present. MG holds a leadership position in OpenReuma/Spanish Association of Health Professionals in Rheumatology (unpaid). CDM received Grants from Versus Arthritis, MRC, NIHR (paid to Keele University) and is the director of the NIHR School for Primary Care Research. SL received payment as scientific consultant from Arthro Therapeutics AB and received payment from AstraZeneca as a member of DSMB. DC received grants from Fundação para a Ciência e Tecnologia SFRH/BD/148420/2019 and Pfizer (ID 64165707). GZ received payment for expert testimony from Casa di Cura San Francesco, Verona and Support for attending meetings and/or travel from Orthotech and Jtech, payment for participation on a Data Safety Monitoring Board or Advisory Board from VIVENKO for Gruenenthal and Ethos for Angelini and holds other financial interests related to clinical practice as an orthopedic surgeon (performing total joint replacement, arthroscopies and other types of surgeries), either directly from private patients or indirectly from the health system or insurances acting as a private consultant. JEV has received payment or honoraria for lectures, presentations, speakers bureaus, manuscript writing or educational events from Lilly Netherlands BV. TG has received paid honoraria for lectures by Abbvie, Novartis, Boehringer Ingelheim, UCB, Berlin-Chemie/A. Menarini Bulgaria, Sandoz and received support for attending meetings by Abbvie, Pfizer and UCB. DW is an International Advisory Board Member of DRFZ (Germany) 2019–current and was the EULAR PARE Chair 2015–2017and an EULAR Vice President representing PARE 2017–2021.

Provenance and peer review Not commissioned; externally peer reviewed.

Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

Read the full text or download the PDF: