the educational research review

• Technical Support
• Find My Rep

You are here

Review of Educational Research

Preview this book.

• Description
• Aims and Scope
• Editorial Board
• Abstracting / Indexing
• Submission Guidelines

The Review of Educational Research ( RER , quarterly, begun in 1931; approximately 640 pp./volume year) publishes critical, integrative reviews of research literature bearing on education. Such reviews should include conceptualizations, interpretations, and syntheses of literature and scholarly work in a field broadly relevant to education and educational research. RER encourages the submission of research relevant to education from any discipline, such as reviews of research in psychology, sociology, history, philosophy, political science, economics, computer science, statistics, anthropology, and biology, provided that the review bears on educational issues. RER does not publish original empirical research unless it is incorporated in a broader integrative review. RER will occasionally publish solicited, but carefully refereed, analytic reviews of special topics, particularly from disciplines infrequently represented.

The Review of Educational Research publishes critical, integrative reviews of research literature bearing on education. Such reviews should include conceptualizations, interpretations, and syntheses of literature and scholarly work in a field broadly relevant to education and educational research. RER encourages the submission of research relevant to education from any discipline, such as reviews of research in psychology, sociology, history, philosophy, political science, economics, computer science, statistics, anthropology, and biology, provided that the review bears on educational issues. RER does not publish original empirical research, and all analyses should be incorporated in a broader integrative review. RER will occasionally publish solicited, but carefully refereed, analytic reviews of special topics, particularly from disciplines infrequently represented. The following types of manuscripts fall within the journal’s purview:

Integrative reviews pull together the existing work on an educational topic and work to understand trends in that body of scholarship. In such a review, the author describes how the issue is conceptualized within the literature, how research methods and theories have shaped the outcomes of scholarship, and what the strengths and weaknesses of the literature are. Meta-analyses are of particular interest when they are accompanied by an interpretive framework that takes the article beyond the reporting of effect sizes and the bibliographic outcome of a computer search.

Theoretical reviews should explore how theory shapes research. To the extent that research is cited and interpreted, it is in the service of the specification, explication, and illumination of a theory. Theoretical reviews and integrative reviews have many similarities, but the former are primarily about how a theory is employed to frame research and our understandings, and refer to the research as it relates to the theory.

Methodological reviews are descriptions of research design, methods, and procedures that can be employed in literature reviews or research in general. The articles should highlight the strengths and weaknesses of methodological tools and explore how methods constrain or open up opportunities for learning about educational problems. They should be written in a style that is accessible to researchers in education rather than methodologists.

Historical reviews provide analyses that situate literature in historical contexts. Within these reviews, explanations for educational phenomena are framed within the historical forces that shape language and understanding.

Commissioned reviews and thematic issues. The editors may commission and solicit authors to review areas of literature. In all other respects, commissioned reviews are subject to the same review process as submitted reviews. The editors also encourage readers to propose thematic topics for special issues and, as potential guest editors, to submit plans for such issues.

In addition to review articles, RER will occasionally publish notes and responses which are short pieces of no more than 1,200 words on any topic that would be of use to reviewers of research. Typically, they point out shortcomings and differences in interpretation in RER articles and policy.

The standards and criteria for review articles in RER are the following:

1. Quality of the Literature. Standards used to determine quality of literature in education vary greatly. Any review needs to take into account the quality of the literature and its impact on findings. Authors should attempt to review all relevant literature on a topic (e.g., international literature, cross-disciplinary work, etc.).

2. Quality of Analysis. The review should go beyond description to include analysis and critiques of theories, methods, and conclusions represented in the literature. This analysis should also examine the issue of access—which perspectives are included or excluded in a body of work? Finally, the analysis should be reflexive—how does the scholars’ framework constrain what can be known in this review?

3. Significance of the Topic. The review should seek to inform and/or illuminate questions important to the field of education. While these questions may be broad-based, they should have implications for the educational problems and issues affecting our national and global societies.

4. Impact of the Article. The review should be seen as an important contribution and tool for the many different educators dealing with the educational problems and issues confronting society.

5. Advancement of the Field. The review should validate or inform the knowledge of researchers and guide and improve the quality of their research and scholarship.

6. Style. The review must be well written and conform to style of the Publication Manual of the American Psychological Association (6th edition). Authors should avoid the use of unexplained jargon and parochialism.

7. Balance and Fairness. The review should be careful not to misrepresent the positions taken by others, or be disrespectful of contrary positions.

8. Purpose. Any review should be accessible to the broad readership of RER. The purpose of any article should be to connect the particular problem addressed by the researcher(s) to a larger context of education.

We also encourage all authors interested in submitting a manuscript to RER to read our Editorial Vision for more information on our publication aims.

• Academic Search - Premier
• Academic Search Alumni Edition
• Academic Search Elite
• Clarivate Analytics: Current Contents - Physical, Chemical & Earth Sciences
• EBSCO: MasterFILE Elite
• EBSCO: MasterFILE Premier
• EBSCO: Professional Development Collection
• EBSCO: Sales & Marketing Source
• EBSCOhost: Current Abstracts
• ERIC (Education Resources Information Center)
• Educational Research Abstracts Online (T&F)
• Higher Education Abstracts
• MasterFILE Select - EBSCO
• ProQuest Education Journals
• Social SciSearch
• Social Sciences Citation Index (Web of Science)
• Teacher Reference Center
• Wilson Education Index/Abstracts

1. Publication Standards 2. Submission Preparation Checklist 3. How to Get Help With the Quality of English in Your Submission 4. Copyright Information 5. For authors who use figures or other materials for which they do not own copyright 6. Right of Reply 7. Sage Choice and Open Access

The Review of Educational Research (RER) publishes comprehensive reviews of literature related to education and does not publish new empirical work, except in the context of meta-analytic reviews of an area. Please check the journal’s Aims and Scope to see if your manuscript is appropriate to submit to RER.

All manuscripts should be submitted electronically to the editorial team at http://mc.manuscriptcentral.com/rer . For questions or inquiries about manuscripts, email us at [email protected] . Manuscripts may not be submitted via e-mail.

Publication Standards

Researchers who intend to submit studies for publication should consult the Standards for Research Conduct adopted by the AERA Council. We also recommend consulting (a) the Guidelines for Reviewers , which outline the criteria under which manuscripts are reviewed for publication by AERA and (b) recent previous editions of the journal. Individuals submitting systematic reviews or meta-analyses should also consult The PRISMA Statement ( http://www.prisma-statement.org ) as well the article on “Reporting Standards for Research in Psychology” in American Psychologist, 63 , 839 – 851 (doi:10.1037/0003-066X.63.9.839).

Submission Preparation Checklist

When you upload your initial submission, upload (1) a separate title page that is not anonymized. Please format the title page as described by the 7th edition of the APA Manual and (2) the main manuscript, which includes an ANONYMIZED title page, an abstract with keywords at the bottom, and the rest of the document including tables and figures, and finally (c) Author Bios.

Please ensure that your manuscript complies with the “ RER Formatting Requirements and Common Formatting Errors ” (see PDF on the RER website). If your submission does not meet these requirements, it will be returned to you.

Additionally, your submission should meet the following guidelines:

1. The submission has not been previously published and is not under consideration for publication elsewhere; or an explanation has been provided in the Cover Letter. Authors should indicate in the Author Note on the separate title page if sections of the manuscript have been published in other venues.

2. THE MANUSCRIPT CONTAINS NO IDENTIFYING INFORMATION, EVEN ON THE ANONYMIZED TITLE PAGE. Please anonymize any work of limited circulation (e.g., in press papers, manuscripts under submission) that would point to the author, both in the body of the manuscript and the reference list. More information on anonymizing is described subsequently. Please double check that the author’s name has been removed from the document’s Properties, which in Microsoft Word is found in the File menu (select “File,” “Properties,” “Summary,” and remove the author’s name; select “OK” to save).

3. The text conforms to APA style (currently the 7th ed.). Consult the guidelines spelled out under “Manuscript Style, Length, and Format” on this webpage and in the RER Formatting Requirements PDF included on our website.

4. The submission must be in Microsoft Word format (.doc or .docx), which will be converted into a PDF file. Please do not upload PDF files, or they will be returned to you.

5. All URL addresses and DOIs in the manuscript (e.g., http://www.aera.net ) should be activated and ready to click.

6. An abstract of 150 words maximum is included (both separately and on the second page of the main document after the ANONYMIZED title page). Please also include three to five keywords—the terms that researchers will use to find your article in indexes and databases.

Manuscript Style, Length, and Format

The style guide for the Review of Educational Research and all AERA journals is the Publication Manual of the American Psychological Association, 7th ed., 2020. The manual is available for purchase here . Guidelines are also available on the APA website .

Manuscripts should NOT exceed 65 pages (or 15,000 words), including tables, figures, appendices, notes, and references, but excluding anonymized title page, abstract, and any supplementary files. Pages should be numbered consecutively in the top right-hand corner, with a fully capitalized running head in the top-left corner. All manuscripts should begin with the anonymized title page (p.1). Manuscripts should be typed for 8½” x 11” paper, in upper and lower case, with 1-inch margins on all sides. Manuscripts should be typed in 12-point Times New Roman font. Manuscripts that exceed 65 pages may be returned without review.

All text, from the title page to the end of the manuscript should be double-spaced , including the abstract, block quotations, bulleted text, and the reference list. Single-spacing is allowed in tables when it is useful in making the table clearer. Do not leave blank lines after paragraphs or before sub-headings. However, if a heading or subheading is the last line on a page, use a page break to move it to the top of the next page. The Abstract, Introduction (beginning with the title), the References, and all tables and figures begin on new pages.

Please use the five subheadings as appropriate based on the 7th edition of the APA style manual. In addition to being on the title page, the title should also be placed at the beginning of the Introduction (in lieu of the word, “Introduction,” which should not appear) and the title at the beginning of the Introduction should be a Level 1 heading.

Tables and figures are to be placed after the references—all tables precede all figures—and should not be included in the body of the text. Each figure and table should begin on a separate page. Do NOT use the “Place Table 5 here” or “Place Figure 1 here” convention. The tables and figures will be placed nearest to where they are mentioned as appropriate when copyediting is done.

Figures and tables should present data to the reader in a clear and unambiguous manner, and should be referred to in the text. If the illustration/table/figure and text are redundant, eliminate the illustration or reduce the amount of detail provided in text. The use of lines in tables is limited (please consult the APA style manual for formatting guidelines ). Figure captions should be placed at the bottom of the figure. One high-quality electronic version of each figure must be submitted with the manuscript. Tables will be typeset. Note that any figures and tables uploaded separately from the main manuscript will still count toward the total 65-page limit.

Italics can be used for emphasis or contrast in special situations but should be used sparingly. Ideally, sentence structure should be used for these issues. All words to be set in italics (e.g., book titles, journal names) should be typed in italics. There should be no underlined text . Abbreviations and acronyms should be spelled out the first time they are mentioned unless they are found as entries in their abbreviated form in Merriam-Webster’s Collegiate Dictionary , 11th ed., 2003 (e.g., “IQ” can be used without being spelled out). Mathematical symbols and symbols for vectors should be clearly formatted in italics and boldface, respectively.

You can use the footnote or endnote feature of Microsoft Word. However, notes are only for explanations or amplifications of textual material that cannot be incorporated into the regular text; they are not for reference information. Moreover, notes are distracting to readers and expensive to produce and should be used sparingly and avoided whenever possible.

The reference list should contain only references that are cited in the text. Its accuracy and completeness are the responsibility of the authors. Reference each publicly available dataset with its title, author, date, and a persistent Web identifier such as a digital object identifier (doi), a handle, or a uniform resource name (URN). If necessary, this last element may be replaced by a web address. Additionally, any references that were included in the analysis but not cited in-text in the main manuscript can be included in a separate reference list that is uploaded as a Supplementary File for Review (this may assist in meeting the page limit).

Authors should anonymize their manuscripts for review . Anonymizing does not mean removing all self-citations. Authors should only anonymize citations of limited circulation (e.g., forthcoming, in press, unpublished) that point to the author. Publications already in the extant literature (e.g., books, book chapters, journal articles) should be cited normally, but authors should include self-citations judiciously . When anonymizing, please use “Author” or “Authors” as in the examples below and place this alphabetically in the reference list and not where the author’s actual name would typically appear.

For examples of common types of references, consult the APA 7th edition manual, or visit the webpage here: https://apastyle.apa.org/style-grammar-guidelines/references

How to Get Help with the Quality of English in Your Submission

Authors who would like to refine the use of English in their manuscripts might consider using the services of a professional English-language editing company. We highlight some of these companies at  https://languageservices.sagepub.com/en/ .

Please be aware that Sage has no affiliation with these companies and makes no endorsement of them. An author's use of these services in no way guarantees that his or her submission will ultimately be accepted. Any arrangement an author enters into will be exclusively between the author and the particular company, and any costs incurred are the sole responsibility of the author.

Copyright Information Accepted authors will be asked to  assign copyright  to AERA, in return for which AERA grants several rights to authors.

Permission to reproduce your own published material

No written or oral permission is necessary to reproduce a table, a figure, or an excerpt of fewer than 500 words from this journal, or to make photocopies for classroom use. Authors are granted permission, without fee, to photocopy their own material or make printouts from the final pdf of their article. Copies must include a full and accurate bibliographic citation and the following credit line: “Copyright [year] by the American Educational Research Association; reproduced with permission from the publisher.” Written permission must be obtained to reproduce or reprint material in circumstances other than those just described. Please review Sage Publishing’s  Journal Permissions  for further information on policies and fees.

Permission to submit material for which you do not own copyright

Authors who wish to use material, such as figures or tables, for which they do not own the copyright must obtain written permission from the copyright holder (usually the publisher) and submit it along with their manuscript. However, no written or oral permission is necessary to reproduce a table, a figure, or an excerpt of fewer than 500 words from an AERA journal.

Copyright transfer agreements for accepted works with more than one author

This journal uses a transfer of copyright agreement that requires just one author (the corresponding author) to sign on behalf of all authors. Please identify the corresponding author for your work when submitting your manuscript for review. The corresponding author will be responsible for the following:

1. Ensuring that all authors are identified on the copyright agreement, and notifying the editorial office of any changes in the authorship.

2. Securing written permission (by letter or e-mail) from each co-author to sign the copyright agreement on the co-author’s behalf.

3. Warranting and indemnifying the journal owner and publisher on behalf of all co-authors. Although such instances are very rare, you should be aware that in the event that a co-author has included content in his or her portion of the article that infringes the copyright of another or is otherwise in violation of any other warranty listed in the agreement, you will be the sole author indemnifying the publisher and the editor of the journal against such violation.

Please contact the publications office at  AERA  if you have questions or if you prefer to use a copyright agreement for all coauthors to sign.

Right of Reply

The right of reply policy encourages comments on recently published articles in AERA publications. They are, of course, subject to the same editorial review and decision process as articles. If the comment is accepted for publication, the editor shall inform the author of the original article. If the author submits a reply to the comment, the reply is also subject to editorial review and decision. The editor may allot a specific amount of journal space for the comment (ordinarily about 1,500 words) and for the reply (ordinarily about 750 words). The reply may appear in the same issue as the comment or in a later issue (Council, June 1980).

If an article is accepted for publication in an AERA journal that, in the judgment of the editor, has as its main theme or thrust a critique of a specific piece of work or a specific line of work associated with an individual or program of research, then the individual or representative of the research program whose work is critiqued should be notified in advance about the upcoming publication and given the opportunity to reply, ideally in the same issue. The author of the original article should also be notified. Normal guidelines for length and review of the reply and publication of a rejoinder by the original article’s author(s) should be followed. Articles in the format “an open letter to …” may constitute prototypical exemplars of the category defined here, but other formats may well be used, and would be included under the qualifications for response prescribed here (Council, January 2002).

Sage Choice and Open Access

If you or your funder wish your article to be freely available online to nonsubscribers immediately upon publication (gold open access), you can opt for it to be included in Sage Choice, subject to payment of a publication fee. The manuscript submission and peer review procedure is unchanged. On acceptance of your article, you will be asked to let Sage know directly if you are choosing Sage Choice. To check journal eligibility and the publication fee, please visit  Sage Choice . For more information on open access options and compliance at Sage, including self author archiving deposits (green open access) visit  Sage Publishing Policies  on our Journal Author Gateway.

• Read Online
• Sample Issues
• Current Issue
• Email Alert
• Permissions
• Foreign rights
• Reprints and sponsorship
• Advertising

Individual Subscription, Combined (Print & E-access)

Institutional Subscription, E-access

Institutional Subscription & Backfile Lease, E-access Plus Backfile (All Online Content)

Institutional Subscription, Print Only

Institutional Subscription, Combined (Print & E-access)

Institutional Subscription & Backfile Lease, Combined Plus Backfile (Current Volume Print & All Online Content)

Institutional Backfile Purchase, E-access (Content through 1998)

Individual, Single Print Issue

Institutional, Single Print Issue

To order single issues of this journal, please contact SAGE Customer Services at 1-800-818-7243 / 1-805-583-9774 with details of the volume and issue you would like to purchase.

• AERA Open Editors
• AERJ Editors
• EEPA Editors
• ER Issues and Archives
• JEBS Editors
• JSTOR Online Archives
• RER Editors
• RRE Editors
• AERA Examination and Desk Copies
• Mail/Fax Book Order Form
• International Distribution
• Books & Publications
• Merchandise
• Search The Store
• Online Paper Repository
• Inaugural Presentations in the i-Presentation Gallery
• Research Points
• AERA Journal Advertising Rate Cards
• Publications Permissions
• Publications FAQs

Share 

The  Review of Educational Research  ( RER , bimonthly, begun in 1931) publishes critical, integrative reviews of research literature bearing on education. Such reviews should include conceptualizations, interpretations, and syntheses of literature and scholarly work in a field broadly relevant to education and educational research.  RER  encourages the submission of research relevant to education from any discipline, such as reviews of research in psychology, sociology, history, philosophy, political science, economics, computer science, statistics, anthropology, and biology, provided that the review bears on educational issues.  RER  does not publish original empirical research unless it is incorporated in a broader integrative review.  RER  will occasionally publish solicited, but carefully refereed, analytic reviews of special topics, particularly from disciplines infrequently represented.

Impact Factor : 11.2 5-Year Impact Factor : 16.6 Ranking : 1/263 in Education & Educational Research

Systematic Reviews in Educational Research: Methodology, Perspectives and Application

• Open Access
• First Online: 22 November 2019

Cite this chapter

You have full access to this open access chapter

• Mark Newman 6 &
• David Gough 6  

83k Accesses

135 Citations

2 Altmetric

This chapter explores the processes of reviewing literature as a research method. The logic of the family of research approaches called systematic review is analysed and the variation in techniques used in the different approaches explored using examples from existing reviews. The key distinctions between aggregative and configurative approaches are illustrated and the chapter signposts further reading on key issues in the systematic review process.

You have full access to this open access chapter,  Download chapter PDF

Similar content being viewed by others

Reviewing Literature for and as Research

Methodological Approaches to Literature Review

The Role of Meta-analysis in Educational Research

1 what are systematic reviews.

A literature review is a scholarly paper which provides an overview of current knowledge about a topic. It will typically include substantive findings, as well as theoretical and methodological contributions to a particular topic (Hart 2018 , p. xiii). Traditionally in education ‘reviewing the literature’ and ‘doing research’ have been viewed as distinct activities. Consider the standard format of research proposals, which usually have some kind of ‘review’ of existing knowledge presented distinctly from the methods of the proposed new primary research. However, both reviews and research are undertaken in order to find things out. Reviews to find out what is already known from pre-existing research about a phenomena, subject or topic; new primary research to provide answers to questions about which existing research does not provide clear and/or complete answers.

When we use the term research in an academic sense it is widely accepted that we mean a process of asking questions and generating knowledge to answer these questions using rigorous accountable methods. As we have noted, reviews also share the same purposes of generating knowledge but historically we have not paid as much attention to the methods used for reviewing existing literature as we have to the methods used for primary research. Literature reviews can be used for making claims about what we know and do not know about a phenomenon and also about what new research we need to undertake to address questions that are unanswered. Therefore, it seems reasonable to conclude that ‘how’ we conduct a review of research is important.

The increased focus on the use of research evidence to inform policy and practice decision-making in Evidence Informed Education (Hargreaves 1996 ; Nelson and Campbell 2017 ) has increased the attention given to contextual and methodological limitations of research evidence provided by single studies. Reviews of research may help address these concerns when carried on in a systematic, rigorous and transparent manner. Thus, again emphasizing the importance of ‘how’ reviews are completed.

The logic of systematic reviews is that reviews are a form of research and thus can be improved by using appropriate and explicit methods. As the methods of systematic review have been applied to different types of research questions, there has been an increasing plurality of types of systematic review. Thus, the term ‘systematic review’ is used in this chapter to refer to a family of research approaches that are a form of secondary level analysis (secondary research) that brings together the findings of primary research to answer a research question. Systematic reviews can therefore be defined as “a review of existing research using explicit, accountable rigorous research methods” (Gough et al. 2017 , p. 4).

2 Variation in Review Methods

Reviews can address a diverse range of research questions. Consequently, as with primary research, there are many different approaches and methods that can be applied. The choices should be dictated by the review questions. These are shaped by reviewers’ assumptions about the meaning of a particular research question, the approach and methods that are best used to investigate it. Attempts to classify review approaches and methods risk making hard distinctions between methods and thereby to distract from the common defining logics that these approaches often share. A useful broad distinction is between reviews that follow a broadly configurative synthesis logic and reviews that follow a broadly aggregative synthesis logic (Sandelowski et al. 2012 ). However, it is important to keep in mind that most reviews have elements of both (Gough et al. 2012 ).

Reviews that follow a broadly configurative synthesis logic approach usually investigate research questions about meaning and interpretation to explore and develop theory. They tend to use exploratory and iterative review methods that emerge throughout the process of the review. Studies included in the review are likely to have investigated the phenomena of interest using methods such as interviews and observations, with data in the form of text. Reviewers are usually interested in purposive variety in the identification and selection of studies. Study quality is typically considered in terms of authenticity. Synthesis consists of the deliberative configuring of data by reviewers into patterns to create a richer conceptual understanding of a phenomenon. For example, meta ethnography (Noblit and Hare 1988 ) uses ethnographic data analysis methods to explore and integrate the findings of previous ethnographies in order to create higher-level conceptual explanations of phenomena. There are many other review approaches that follow a broadly configurative logic (for an overview see Barnett-Page and Thomas 2009 ); reflecting the variety of methods used in primary research in this tradition.

Reviews that follow a broadly aggregative synthesis logic usually investigate research questions about impacts and effects. For example, systematic reviews that seek to measure the impact of an educational intervention test the hypothesis that an intervention has the impact that has been predicted. Reviews following an aggregative synthesis logic do not tend to develop theory directly; though they can contribute by testing, exploring and refining theory. Reviews following an aggregative synthesis logic tend to specify their methods in advance (a priori) and then apply them without any deviation from a protocol. Reviewers are usually concerned to identify the comprehensive set of studies that address the research question. Studies included in the review will usually seek to determine whether there is a quantitative difference in outcome between groups receiving and not receiving an intervention. Study quality assessment in reviews following an aggregative synthesis logic focusses on the minimisation of bias and thus selection pays particular attention to homogeneity between studies. Synthesis aggregates, i.e. counts and adds together, the outcomes from individual studies using, for example, statistical meta-analysis to provide a pooled summary of effect.

3 The Systematic Review Process

Different types of systematic review are discussed in more detail later in this chapter. The majority of systematic review types share a common set of processes. These processes can be divided into distinct but interconnected stages as illustrated in Fig.  1 . Systematic reviews need to specify a research question and the methods that will be used to investigate the question. This is often written as a ‘protocol’ prior to undertaking the review. Writing a protocol or plan of the methods at the beginning of a review can be a very useful activity. It helps the review team to gain a shared understanding of the scope of the review and the methods that they will use to answer the review’s questions. Different types of systematic reviews will have more or less developed protocols. For example, for systematic reviews investigating research questions about the impact of educational interventions it is argued that a detailed protocol should be fully specified prior to the commencement of the review to reduce the possibility of reviewer bias (Torgerson 2003 , p. 26). For other types of systematic review, in which the research question is more exploratory, the protocol may be more flexible and/or developmental in nature.

The systematic review process

3.1 Systematic Review Questions and the Conceptual Framework

The review question gives each review its particular structure and drives key decisions about what types of studies to include; where to look for them; how to assess their quality; and how to combine their findings. Although a research question may appear to be simple, it will include many assumptions. Whether implicit or explicit, these assumptions will include: epistemological frameworks about knowledge and how we obtain it, theoretical frameworks, whether tentative or firm, about the phenomenon that is the focus of study.

Taken together, these produce a conceptual framework that shapes the research questions, choices about appropriate systematic review approach and methods. The conceptual framework may be viewed as a working hypothesis that can be developed, refined or confirmed during the course of the research. Its purpose is to explain the key issues to be studied, the constructs or variables, and the presumed relationships between them. The framework is a research tool intended to assist a researcher to develop awareness and understanding of the phenomena under scrutiny and to communicate this (Smyth 2004 ).

A review to investigate the impact of an educational intervention will have a conceptual framework that includes a hypothesis about a causal link between; who the review is about (the people), what the review is about (an intervention and what it is being compared with), and the possible consequences of intervention on the educational outcomes of these people. Such a review would follow a broadly aggregative synthesis logic. This is the shape of reviews of educational interventions carried out for the What Works Clearing House in the USA Footnote 1 and the Education Endowment Foundation in England. Footnote 2

A review to investigate meaning or understanding of a phenomenon for the purpose of building or further developing theory will still have some prior assumptions. Thus, an initial conceptual framework will contain theoretical ideas about how the phenomena of interest can be understood and some ideas justifying why a particular population and/or context is of specific interest or relevance. Such a review is likely to follow a broadly configurative logic.

3.2 Selection Criteria

Reviewers have to make decisions about which research studies to include in their review. In order to do this systematically and transparently they develop rules about which studies can be selected into the review. Selection criteria (sometimes referred to as inclusion or exclusion criteria) create restrictions on the review. All reviews, whether systematic or not, limit in some way the studies that are considered by the review. Systematic reviews simply make these restrictions transparent and therefore consistent across studies. These selection criteria are shaped by the review question and conceptual framework. For example, a review question about the impact of homework on educational attainment would have selection criteria specifying who had to do the homework; the characteristics of the homework and the outcomes that needed to be measured. Other commonly used selection criteria include study participant characteristics; the country where the study has taken place and the language in which the study is reported. The type of research method(s) may also be used as a selection criterion but this can be controversial given the lack of consensus in education research (Newman 2008 ), and the inconsistent terminology used to describe education research methods.

3.3 Developing the Search Strategy

The search strategy is the plan for how relevant research studies will be identified. The review question and conceptual framework shape the selection criteria. The selection criteria specify the studies to be included in a review and thus are a key driver of the search strategy. A key consideration will be whether the search aims to be exhaustive i.e. aims to try and find all the primary research that has addressed the review question. Where reviews address questions about effectiveness or impact of educational interventions the issue of publication bias is a concern. Publication bias is the phenomena whereby smaller and/or studies with negative findings are less likely to be published and/or be harder to find. We may therefore inadvertently overestimate the positive effects of an educational intervention because we do not find studies with negative or smaller effects (Chow and Eckholm 2018 ). Where the review question is not of this type then a more specific or purposive search strategy, that may or may not evolve as the review progresses, may be appropriate. This is similar to sampling approaches in primary research. In primary research studies using aggregative approaches, such as quasi-experiments, analysis is based on the study of complete or representative samples. In primary research studies using configurative approaches, such as ethnography, analysis is based on examining a range of instances of the phenomena in similar or different contexts.

The search strategy will detail the sources to be searched and the way in which the sources will be searched. A list of search source types is given in Box 1 below. An exhaustive search strategy would usually include all of these sources using multiple bibliographic databases. Bibliographic databases usually index academic journals and thus are an important potential source. However, in most fields, including education, relevant research is published in a range of journals which may be indexed in different bibliographic databases and thus it may be important to search multiple bibliographic databases. Furthermore, some research is published in books and an increasing amount of research is not published in academic journals or at least may not be published there first. Thus, it is important to also consider how you will find relevant research in other sources including ‘unpublished’ or ‘grey’ literature. The Internet is a valuable resource for this purpose and should be included as a source in any search strategy.

Box 1: Search Sources

The World Wide Web/Internet

Google, Specialist Websites, Google Scholar, Microsoft Academic

Bibliographic Databases

Subject specific e.g. Education—ERIC: Education Resources Information Centre

Generic e.g. ASSIA: Applied Social Sciences Index and Abstracts

Handsearching of specialist journals or books

Contacts with Experts

Citation Checking

New, federated search engines are being developed, which search multiple sources at the same time, eliminating duplicates automatically (Tsafnat et al. 2013 ). Technologies, including text mining, are being used to help develop search strategies, by suggesting topics and terms on which to search—terms that reviewers may not have thought of using. Searching is also being aided by technology through the increased use (and automation) of ‘citation chasing’, where papers that cite, or are cited by, a relevant study are checked in case they too are relevant.

A search strategy will identify the search terms that will be used to search the bibliographic databases. Bibliographic databases usually index records according to their topic using ‘keywords’ or ‘controlled terms’ (categories used by the database to classify papers). A comprehensive search strategy usually involves searching both a freetext search using keywords determined by the reviewers and controlled terms. An example of a bibliographic database search is given in Box 2. This search was used in a review that aimed to find studies that investigated the impact of Youth Work on positive youth outcomes (Dickson et al. 2013 ). The search is built using terms for the population of interest (Youth), the intervention of interest (Youth Work) and the outcomes of Interest (Positive Development). It used both keywords and controlled terms, ‘wildcards’ (the *sign in this database) and the Boolean operators ‘OR’ and ‘AND’ to combine terms. This example illustrates the potential complexity of bibliographic database search strings, which will usually require a process of iterative development to finalise.

Box 2: Search string example To identify studies that address the question What is the empirical research evidence on the impact of youth work on the lives of children and young people aged 10-24 years?: CSA ERIC Database

((TI = (adolescen* or (“young man*”) or (“young men”)) or TI = ((“young woman*”) or (“young women”) or (Young adult*”)) or TI = ((“young person*”) or (“young people*”) or teen*) or AB = (adolescen* or (“young man*”) or (“young men”)) or AB = ((“young woman*”) or (“young women”) or (Young adult*”)) or AB = ((“young person*”) or (“young people*”) or teen*)) or (DE = (“youth” or “adolescents” or “early adolescents” or “late adolescents” or “preadolescents”))) and(((TI = ((“positive youth development “) or (“youth development”) or (“youth program*”)) or TI = ((“youth club*”) or (“youth work”) or (“youth opportunit*”)) or TI = ((“extended school*”) or (“civic engagement”) or (“positive peer culture”)) or TI = ((“informal learning”) or multicomponent or (“multi-component “)) or TI = ((“multi component”) or multidimensional or (“multi-dimensional “)) or TI = ((“multi dimensional”) or empower* or asset*) or TI = (thriv* or (“positive development”) or resilienc*) or TI = ((“positive activity”) or (“positive activities”) or experiential) or TI = ((“community based”) or “community-based”)) or(AB = ((“positive youth development “) or (“youth development”) or (“youth program*”)) or AB = ((“youth club*”) or (“youth work”) or (“youth opportunit*”)) or AB = ((“extended school*”) or (“civic engagement”) or (“positive peer culture”)) or AB = ((“informal learning”) or multicomponent or (“multi-component “)) or AB = ((“multi component”) or multidimensional or (“multi-dimensional “)) or AB = ((“multi dimensional”) or empower* or asset*) or AB = (thriv* or (“positive development”) or resilienc*) or AB = ((“positive activity”) or (“positive activities”) or experiential) or AB = ((“community based”) or “community-based”))) or (DE=”community education”))

Detailed guidance for finding effectiveness studies is available from the Campbell Collaboration (Kugley et al. 2015 ). Guidance for finding a broader range of studies has been produced by the EPPI-Centre (Brunton et al. 2017a ).

3.4 The Study Selection Process

Studies identified by the search are subject to a process of checking (sometimes referred to as screening) to ensure they meet the selection criteria. This is usually done in two stages whereby titles and abstracts are checked first to determine whether the study is likely to be relevant and then a full copy of the paper is acquired to complete the screening exercise. The process of finding studies is not efficient. Searching bibliographic databases, for example, leads to many irrelevant studies being found which then have to be checked manually one by one to find the few relevant studies. There is increasing use of specialised software to support and in some cases, automate the selection process. Text mining, for example, can assist in selecting studies for a review (Brunton et al. 2017b ). A typical text mining or machine learning process might involve humans undertaking some screening, the results of which are used to train the computer software to learn the difference between included and excluded studies and thus be able to indicate which of the remaining studies are more likely to be relevant. Such automated support may result in some errors in selection, but this may be less than the human error in manual selection (O’Mara-Eves et al. 2015 ).

3.5 Coding Studies

Once relevant studies have been selected, reviewers need to systematically identify and record the information from the study that will be used to answer the review question. This information includes the characteristics of the studies, including details of the participants and contexts. The coding describes: (i) details of the studies to enable mapping of what research has been undertaken; (ii) how the research was undertaken to allow assessment of the quality and relevance of the studies in addressing the review question; (iii) the results of each study so that these can be synthesised to answer the review question.

The information is usually coded into a data collection system using some kind of technology that facilitates information storage and analysis (Brunton et al. 2017b ) such as the EPPI-Centre’s bespoke systematic review software EPPI Reviewer. Footnote 3 Decisions about which information to record will be made by the review team based on the review question and conceptual framework. For example, a systematic review about the relationship between school size and student outcomes collected data from the primary studies about each schools funding, students, teachers and school organisational structure as well as about the research methods used in the study (Newman et al. 2006 ). The information coded about the methods used in the research will vary depending on the type of research included and the approach that will be used to assess the quality and relevance of the studies (see the next section for further discussion of this point).

Similarly, the information recorded as ‘results’ of the individual studies will vary depending on the type of research that has been included and the approach to synthesis that will be used. Studies investigating the impact of educational interventions using statistical meta-analysis as a synthesis technique will require all of the data necessary to calculate effect sizes to be recorded from each study (see the section on synthesis below for further detail on this point). However, even in this type of study there will be multiple data that can be considered to be ‘results’ and so which data needs to be recorded from studies will need to be carefully specified so that recording is consistent across studies

3.6 Appraising the Quality of Studies

Methods are reinvented every time they are used to accommodate the real world of research practice (Sandelowski et al. 2012 ). The researcher undertaking a primary research study has attempted to design and execute a study that addresses the research question as rigorously as possible within the parameters of their resources, understanding, and context. Given the complexity of this task, the contested views about research methods and the inconsistency of research terminology, reviewers will need to make their own judgements about the quality of the any individual piece of research included in their review. From this perspective, it is evident that using a simple criteria, such as ‘published in a peer reviewed journal’ as a sole indicator of quality, is not likely to be an adequate basis for considering the quality and relevance of a study for a particular systematic review.

In the context of systematic reviews this assessment of quality is often referred to as Critical Appraisal (Petticrew and Roberts 2005 ). There is considerable variation in what is done during critical appraisal: which dimensions of study design and methods are considered; the particular issues that are considered under each dimension; the criteria used to make judgements about these issues and the cut off points used for these criteria (Oancea and Furlong 2007 ). There is also variation in whether the quality assessment judgement is used for excluding studies or weighting them in analysis and when in the process judgements are made.

There are broadly three elements that are considered in critical appraisal: the appropriateness of the study design in the context of the review question, the quality of the execution of the study methods and the study’s relevance to the review question (Gough 2007 ). Distinguishing study design from execution recognises that whilst a particular design may be viewed as more appropriate for a study it also needs to be well executed to achieve the rigour or trustworthiness attributed to the design. Study relevance is achieved by the review selection criteria but assessing the degree of relevance recognises that some studies may be less relevant than others due to differences in, for example, the characteristics of the settings or the ways that variables are measured.

The assessment of study quality is a contested and much debated issue in all research fields. Many published scales are available for assessing study quality. Each incorporates criteria relevant to the research design being evaluated. Quality scales for studies investigating the impact of interventions using (quasi) experimental research designs tend to emphasis establishing descriptive causality through minimising the effects of bias (for detailed discussion of issues associated with assessing study quality in this tradition see Waddington et al. 2017 ). Quality scales for appraising qualitative research tend to focus on the extent to which the study is authentic in reflecting on the meaning of the data (for detailed discussion of the issues associated with assessing study quality in this tradition see Carroll and Booth 2015 ).

3.7 Synthesis

A synthesis is more than a list of findings from the included studies. It is an attempt to integrate the information from the individual studies to produce a ‘better’ answer to the review question than is provided by the individual studies. Each stage of the review contributes toward the synthesis and so decisions made in earlier stages of the review shape the possibilities for synthesis. All types of synthesis involve some kind of data transformation that is achieved through common analytic steps: searching for patterns in data; Checking the quality of the synthesis; Integrating data to answer the review question (Thomas et al. 2012 ). The techniques used to achieve these vary for different types of synthesis and may appear more or less evident as distinct steps.

Statistical meta-analysis is an aggregative synthesis approach in which the outcome results from individual studies are transformed into a standardized, scale free, common metric and combined to produce a single pooled weighted estimate of effect size and direction. There are a number of different metrics of effect size, selection of which is principally determined by the structure of outcome data in the primary studies as either continuous or dichotomous. Outcome data with a dichotomous structure can be transformed into Odds Ratios (OR), Absolute Risk Ratios (ARR) or Relative Risk Ratios (RRR) (for detailed discussion of dichotomous outcome effect sizes see Altman 1991 ). More commonly seen in education research, outcome data with a continuous structure can be translated into Standardised Mean Differences (SMD) (Fitz-Gibbon 1984 ). At its most straightforward effect size calculation is simple arithmetic. However given the variety of analysis methods used and the inconsistency of reporting in primary studies it is also possible to calculate effect sizes using more complex transformation formulae (for detailed instructions on calculating effect sizes from a wide variety of data presentations see Lipsey and Wilson 2000 ).

The combination of individual effect sizes uses statistical procedures in which weighting is given to the effect sizes from the individual studies based on different assumptions about the causes of variance and this requires the use of statistical software. Statistical measures of heterogeneity produced as part of the meta-analysis are used to both explore patterns in the data and to assess the quality of the synthesis (Thomas et al. 2017a ).

In configurative synthesis the different kinds of text about individual studies and their results are meshed and linked to produce patterns in the data, explore different configurations of the data and to produce new synthetic accounts of the phenomena under investigation. The results from the individual studies are translated into and across each other, searching for areas of commonality and refutation. The specific techniques used are derived from the techniques used in primary research in this tradition. They include reading and re-reading, descriptive and analytical coding, the development of themes, constant comparison, negative case analysis and iteration with theory (Thomas et al. 2017b ).

4 Variation in Review Structures

All research requires time and resources and systematic reviews are no exception. There is always concern to use resources as efficiently as possible. For these reasons there is a continuing interest in how reviews can be carried out more quickly using fewer resources. A key issue is the basis for considering a review to be systematic. Any definitions are clearly open to interpretation. Any review can be argued to be insufficiently rigorous and explicit in method in any part of the review process. To assist reviewers in being rigorous, reporting standards and appraisal tools are being developed to assess what is required in different types of review (Lockwood and Geum Oh 2017 ) but these are also the subject of debate and disagreement.

In addition to the term ‘systematic review’ other terms are used to denote the outputs of systematic review processes. Some use the term ‘scoping review’ for a quick review that does not follow a fully systematic process. This term is also used by others (for example, Arksey and O’Malley 2005 ) to denote ‘systematic maps’ that describe the nature of a research field rather than synthesise findings. A ‘quick review’ type of scoping review may also be used as preliminary work to inform a fuller systematic review. Another term used is ‘rapid evidence assessment’. This term is usually used when systematic review needs to be undertaken quickly and in order to do this the methods of review are employed in a more minimal than usual way. For example, by more limited searching. Where such ‘shortcuts’ are taken there may be some loss of rigour, breadth and/or depth (Abrami et al. 2010 ; Thomas et al. 2013 ).

Another development has seen the emergence of the concept of ‘living reviews’, which do not have a fixed end point but are updated as new relevant primary studies are produced. Many review teams hope that their review will be updated over time, but what is different about living reviews is that it is built into the system from the start as an on-going developmental process. This means that the distribution of review effort is quite different to a standard systematic review, being a continuous lower-level effort spread over a longer time period, rather than the shorter bursts of intensive effort that characterise a review with periodic updates (Elliott et al. 2014 ).

4.1 Systematic Maps and Syntheses

One potentially useful aspect of reviewing the literature systematically is that it is possible to gain an understanding of the breadth, purpose and extent of research activity about a phenomenon. Reviewers can be more informed about how research on the phenomenon has been constructed and focused. This type of reviewing is known as ‘mapping’ (see for example, Peersman 1996 ; Gough et al. 2003 ). The aspects of the studies that are described in a map will depend on what is of most interest to those undertaking the review. This might include information such as topic focus, conceptual approach, method, aims, authors, location and context. The boundaries and purposes of a map are determined by decisions made regarding the breadth and depth of the review, which are informed by and reflected in the review question and selection criteria.

Maps can also be a useful stage in a systematic review where study findings are synthesised as well. Most synthesis reviews implicitly or explicitly include some sort of map in that they describe the nature of the relevant studies that they have identified. An explicit map is likely to be more detailed and can be used to inform the synthesis stage of a review. It can provide more information on the individual and grouped studies and thus also provide insights to help inform choices about the focus and strategy to be used in a subsequent synthesis.

4.2 Mixed Methods, Mixed Research Synthesis Reviews

Where studies included in a review consist of more than one type of study design, there may also be different types of data. These different types of studies and data can be analysed together in an integrated design or segregated and analysed separately (Sandelowski et al. 2012 ). In a segregated design, two or more separate sub-reviews are undertaken simultaneously to address different aspects of the same review question and are then compared with one another.

Such ‘mixed methods’ and ‘multiple component’ reviews are usually necessary when there are multiple layers of review question or when one study design alone would be insufficient to answer the question(s) adequately. The reviews are usually required, to have both breadth and depth. In doing so they can investigate a greater extent of the research problem than would be the case in a more focussed single method review. As they are major undertakings, containing what would normally be considered the work of multiple systematic reviews, they are demanding of time and resources and cannot be conducted quickly.

4.3 Reviews of Reviews

Systematic reviews of primary research are secondary levels of research analysis. A review of reviews (sometimes called ‘overviews’ or ‘umbrella’ reviews) is a tertiary level of analysis. It is a systematic map and/or synthesis of previous reviews. The ‘data’ for reviews of reviews are previous reviews rather than primary research studies (see for example Newman et al. ( 2018 ). Some review of reviews use previous reviews to combine both primary research data and synthesis data. It is also possible to have hybrid review models consisting of a review of reviews and then new systematic reviews of primary studies to fill in gaps in coverage where there is not an existing review (Caird et al. 2015 ). Reviews of reviews can be an efficient method for examining previous research. However, this approach is still comparatively novel and questions remain about the appropriate methodology. For example, care is required when assessing the way in which the source systematic reviews identified and selected data for inclusion, assessed study quality and to assess the overlap between the individual reviews (Aromataris et al. 2015 ).

5 Other Types of Research Based Review Structures

This chapter so far has presented a process or method that is shared by many different approaches within the family of systematic review approaches, notwithstanding differences in review question and types of study that are included as evidence. This is a helpful heuristic device for designing and reading systematic reviews. However, it is the case that there are some review approaches that also claim to use a research based review approach but that do not claim to be systematic reviews and or do not conform with the description of processes that we have given above at all or in part at least.

5.1 Realist Synthesis Reviews

Realist synthesis is a member of the theory-based school of evaluation (Pawson 2002 ). This means that it is underpinned by a ‘generative’ understanding of causation, which holds that, to infer a causal outcome/relationship between an intervention (e.g. a training programme) and an outcome (O) of interest (e.g. unemployment), one needs to understand the underlying mechanisms (M) that connect them and the context (C) in which the relationship occurs (e.g. the characteristics of both the subjects and the programme locality). The interest of this approach (and also of other theory driven reviews) is not simply which interventions work, but which mechanisms work in which context. Rather than identifying replications of the same intervention, the reviews adopt an investigative stance and identify different contexts within which the same underlying mechanism is operating.

Realist synthesis is concerned with hypothesising, testing and refining such context-mechanism-outcome (CMO) configurations. Based on the premise that programmes work in limited circumstances, the discovery of these conditions becomes the main task of realist synthesis. The overall intention is to first create an abstract model (based on the CMO configurations) of how and why programmes work and then to test this empirically against the research evidence. Thus, the unit of analysis in a realist synthesis is the programme mechanism, and this mechanism is the basis of the search. This means that a realist synthesis aims to identify different situations in which the same programme mechanism has been attempted. Integrative Reviewing, which is aligned to the Critical Realist tradition, follows a similar approach and methods (Jones-Devitt et al. 2017 ).

5.2 Critical Interpretive Synthesis (CIS)

Critical Interpretive Synthesis (CIS) (Dixon-Woods et al. 2006 ) takes a position that there is an explicit role for the ‘authorial’ (reviewer’s) voice in the review. The approach is derived from a distinctive tradition within qualitative enquiry and draws on some of the tenets of grounded theory in order to support explicitly the process of theory generation. In practice, this is operationalised in its inductive approach to searching and to developing the review question as part of the review process, its rejection of a ‘staged’ approach to reviewing and embracing the concept of theoretical sampling in order to select studies for inclusion. When assessing the quality of studies CIS prioritises relevance and theoretical contribution over research methods. In particular, a critical approach to reading the literature is fundamental in terms of contextualising findings within an analysis of the research traditions or theoretical assumptions of the studies included.

5.3 Meta-Narrative Reviews

Meta-narrative reviews, like critical interpretative synthesis, place centre-stage the importance of understanding the literature critically and understanding differences between research studies as possibly being due to differences between their underlying research traditions (Greenhalgh et al. 2005 ). This means that each piece of research is located (and, when appropriate, aggregated) within its own research tradition and the development of knowledge is traced (configured) through time and across paradigms. Rather than the individual study, the ‘unit of analysis’ is the unfolding ‘storyline’ of a research tradition over time’ (Greenhalgh et al. 2005 ).

6 Conclusions

This chapter has briefly described the methods, application and different perspectives in the family of systematic review approaches. We have emphasized the many ways in which systematic reviews can vary. This variation links to different research aims and review questions. But also to the different assumptions made by reviewers. These assumptions derive from different understandings of research paradigms and methods and from the personal, political perspectives they bring to their research practice. Although there are a variety of possible types of systematic reviews, a distinction in the extent that reviews follow an aggregative or configuring synthesis logic is useful for understanding variations in review approaches and methods. It can help clarify the ways in which reviews vary in the nature of their questions, concepts, procedures, inference and impact. Systematic review approaches continue to evolve alongside critical debate about the merits of various review approaches (systematic or otherwise). So there are many ways in which educational researchers can use and engage with systematic review methods to increase knowledge and understanding in the field of education.

https://ies.ed.gov/ncee/wwc/

https://educationendowmentfoundation.org.uk/evidence-summaries/teaching-learning-toolkit

https://eppi.ioe.ac.uk/cms/Default.aspx?tabid=2914

Abrami, P. C. Borokhovski, E. Bernard, R. M. Wade, CA. Tamim, R. Persson, T. Bethel, E. C. Hanz, K. & Surkes, M. A. (2010). Issues in conducting and disseminating brief reviews of evidence. Evidence & Policy , 6 (3), 371–389.

Google Scholar  

Altman, D.G. (1991) Practical statistics for medical research . London: Chapman and Hall.

Arksey, H. & O’Malley, L. (2005). Scoping studies: towards a methodological framework, International Journal of Social Research Methodology , 8 (1), 19–32,

Article   Google Scholar  

Aromataris, E. Fernandez, R. Godfrey, C. Holly, C. Khalil, H. Tungpunkom, P. (2015). Summarizing systematic reviews: methodological development, conduct and reporting of an umbrella review approach. International Journal of Evidence-Based Healthcare , 13 .

Barnett-Page, E. & Thomas, J. (2009). Methods for the synthesis of qualitative research: A critical review, BMC Medical Research Methodology, 9 (59), https://doi.org/10.1186/1471-2288-9-59 .

Brunton, G., Stansfield, C., Caird, J. & Thomas, J. (2017a). Finding relevant studies. In D. Gough, S. Oliver & J. Thomas (Eds.), An introduction to systematic reviews (2nd edition, pp. 93–132). London: Sage.

Brunton, J., Graziosi, S., & Thomas, J. (2017b). Tools and techniques for information management. In D. Gough, S. Oliver & J. Thomas (Eds.), An introduction to systematic reviews (2nd edition, pp. 154–180), London: Sage.

Carroll, C. & Booth, A. (2015). Quality assessment of qualitative evidence for systematic review and synthesis: Is it meaningful, and if so, how should it be performed? Research Synthesis Methods 6 (2), 149–154.

Caird, J. Sutcliffe, K. Kwan, I. Dickson, K. & Thomas, J. (2015). Mediating policy-relevant evidence at speed: are systematic reviews of systematic reviews a useful approach? Evidence & Policy, 11 (1), 81–97.

Chow, J. & Eckholm, E. (2018). Do published studies yield larger effect sizes than unpublished studies in education and special education? A meta-review. Educational Psychology Review 30 (3), 727–744.

Dickson, K., Vigurs, C. & Newman, M. (2013). Youth work a systematic map of the literature . Dublin: Dept of Government Affairs.

Dixon-Woods, M., Cavers, D., Agarwa, S., Annandale, E. Arthur, A., Harvey, J., Hsu, R., Katbamna, S., Olsen, R., Smith, L., Riley R., & Sutton, A. J. (2006). Conducting a critical interpretive synthesis of the literature on access to healthcare by vulnerable groups. BMC Medical Research Methodology 6:35 https://doi.org/10.1186/1471-2288-6-35 .

Elliott, J. H., Turner, T., Clavisi, O., Thomas, J., Higgins, J. P. T., Mavergames, C. & Gruen, R. L. (2014). Living systematic reviews: an emerging opportunity to narrow the evidence-practice gap. PLoS Medicine, 11 (2): e1001603. http://doi.org/10.1371/journal.pmed.1001603 .

Fitz-Gibbon, C.T. (1984) Meta-analysis: an explication. British Educational Research Journal , 10 (2), 135–144.

Gough, D. (2007). Weight of evidence: a framework for the appraisal of the quality and relevance of evidence. Research Papers in Education , 22 (2), 213–228.

Gough, D., Thomas, J. & Oliver, S. (2012). Clarifying differences between review designs and methods. Systematic Reviews, 1( 28).

Gough, D., Kiwan, D., Sutcliffe, K., Simpson, D. & Houghton, N. (2003). A systematic map and synthesis review of the effectiveness of personal development planning for improving student learning. In: Research Evidence in Education Library. London: EPPI-Centre, Social Science Research Unit, Institute of Education, University of London.

Gough, D., Oliver, S. & Thomas, J. (2017). Introducing systematic reviews. In D. Gough, S. Oliver & J. Thomas (Eds.), An introduction to systematic reviews (2nd edition, pp. 1–18). London: Sage.

Greenhalgh, T., Robert, G., Macfarlane, F., Bate, P., Kyriakidou, O. & Peacock, R. (2005). Storylines of research in diffusion of innovation: A meta-narrative approach to systematic review. Social Science & Medicine, 61 (2), 417–430.

Hargreaves, D. (1996). Teaching as a research based profession: possibilities and prospects . Teacher Training Agency Annual Lecture. Retrieved from https://eppi.ioe.ac.uk/cms/Default.aspx?tabid=2082 .

Hart, C. (2018). Doing a literature review: releasing the research imagination . London. SAGE.

Jones-Devitt, S. Austen, L. & Parkin H. J. (2017). Integrative reviewing for exploring complex phenomena. Social Research Update . Issue 66.

Kugley, S., Wade, A., Thomas, J., Mahood, Q., Klint Jørgensen, A. M., Hammerstrøm, K., & Sathe, N. (2015). Searching for studies: A guide to information retrieval for Campbell Systematic Reviews . Campbell Method Guides 2016:1 ( http://www.campbellcollaboration.org/images/Campbell_Methods_Guides_Information_Retrieval.pdf ).

Lipsey, M.W., & Wilson, D. B. (2000). Practical meta-analysis. Thousand Oaks: Sage.

Lockwood, C. & Geum Oh, E. (2017). Systematic reviews: guidelines, tools and checklists for authors. Nursing & Health Sciences, 19, 273–277.

Nelson, J. & Campbell, C. (2017). Evidence-informed practice in education: meanings and applications. Educational Research, 59 (2), 127–135.

Newman, M., Garrett, Z., Elbourne, D., Bradley, S., Nodenc, P., Taylor, J. & West, A. (2006). Does secondary school size make a difference? A systematic review. Educational Research Review, 1 (1), 41–60.

Newman, M. (2008). High quality randomized experimental research evidence: Necessary but not sufficient for effective education policy. Psychology of Education Review, 32 (2), 14–16.

Newman, M., Reeves, S. & Fletcher, S. (2018). A critical analysis of evidence about the impacts of faculty development in systematic reviews: a systematic rapid evidence assessment. Journal of Continuing Education in the Health Professions, 38 (2), 137–144.

Noblit, G. W. & Hare, R. D. (1988). Meta-ethnography: Synthesizing qualitative studies. Newbury Park, CA: Sage.

Book   Google Scholar  

Oancea, A. & Furlong, J. (2007). Expressions of excellence and the assessment of applied and practice‐based research. Research Papers in Education 22 .

O’Mara-Eves, A., Thomas, J., McNaught, J., Miwa, M., & Ananiadou, S. (2015). Using text mining for study identification in systematic reviews: a systematic review of current approaches. Systematic Reviews 4 (1): 5. https://doi.org/10.1186/2046-4053-4-5 .

Pawson, R. (2002). Evidence-based policy: the promise of “Realist Synthesis”. Evaluation, 8 (3), 340–358.

Peersman, G. (1996). A descriptive mapping of health promotion studies in young people, EPPI Research Report. London: EPPI-Centre, Social Science Research Unit, Institute of Education, University of London.

Petticrew, M. & Roberts, H. (2005). Systematic reviews in the social sciences: a practical guide . London: Wiley.

Sandelowski, M., Voils, C. I., Leeman, J. & Crandell, J. L. (2012). Mapping the mixed methods-mixed research synthesis terrain. Journal of Mixed Methods Research, 6 (4), 317–331.

Smyth, R. (2004). Exploring the usefulness of a conceptual framework as a research tool: A researcher’s reflections. Issues in Educational Research, 14 .

Thomas, J., Harden, A. & Newman, M. (2012). Synthesis: combining results systematically and appropriately. In D. Gough, S. Oliver & J. Thomas (Eds.), An introduction to systematic reviews (pp. 66–82). London: Sage.

Thomas, J., Newman, M. & Oliver, S. (2013). Rapid evidence assessments of research to inform social policy: taking stock and moving forward. Evidence and Policy, 9 (1), 5–27.

Thomas, J., O’Mara-Eves, A., Kneale, D. & Shemilt, I. (2017a). Synthesis methods for combining and configuring quantitative data. In D. Gough, S. Oliver & J. Thomas (Eds.), An Introduction to Systematic Reviews (2nd edition, pp. 211–250). London: Sage.

Thomas, J., O’Mara-Eves, A., Harden, A. & Newman, M. (2017b). Synthesis methods for combining and configuring textual or mixed methods data. In D. Gough, S. Oliver & J. Thomas (Eds.), An introduction to systematic reviews (2nd edition, pp. 181–211), London: Sage.

Tsafnat, G., Dunn, A. G., Glasziou, P. & Coiera, E. (2013). The automation of systematic reviews. British Medical Journal, 345 (7891), doi.org/10.1136/bmj.f139 .

Torgerson, C. (2003). Systematic reviews . London. Continuum.

Waddington, H., Aloe, A. M., Becker, B. J., Djimeu, E. W., Hombrados, J. G., Tugwell, P., Wells, G. & Reeves, B. (2017). Quasi-experimental study designs series—paper 6: risk of bias assessment. Journal of Clinical Epidemiology, 89 , 43–52.

Download references

Author information

Authors and affiliations.

Institute of Education, University College London, England, UK

Mark Newman & David Gough

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Mark Newman .

Editor information

Editors and affiliations.

Oldenburg, Germany

Olaf Zawacki-Richter

Essen, Germany

Michael Kerres

Svenja Bedenlier

Melissa Bond

Katja Buntins

Rights and permissions

Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/ ), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

Reprints and permissions

Copyright information

© 2020 The Author(s)

About this chapter

Newman, M., Gough, D. (2020). Systematic Reviews in Educational Research: Methodology, Perspectives and Application. In: Zawacki-Richter, O., Kerres, M., Bedenlier, S., Bond, M., Buntins, K. (eds) Systematic Reviews in Educational Research. Springer VS, Wiesbaden. https://doi.org/10.1007/978-3-658-27602-7_1

Download citation

DOI : https://doi.org/10.1007/978-3-658-27602-7_1

Published : 22 November 2019

Publisher Name : Springer VS, Wiesbaden

Print ISBN : 978-3-658-27601-0

Online ISBN : 978-3-658-27602-7

eBook Packages : Education Education (R0)

Share this chapter

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

Educational Research Review

Subject Area and Category

Elsevier Ltd

Publication type

Information.

How to publish in this journal

The set of journals have been ranked according to their SJR and divided into four equal groups, four quartiles. Q1 (green) comprises the quarter of the journals with the highest values, Q2 (yellow) the second highest values, Q3 (orange) the third highest values and Q4 (red) the lowest values.

The SJR is a size-independent prestige indicator that ranks journals by their 'average prestige per article'. It is based on the idea that 'all citations are not created equal'. SJR is a measure of scientific influence of journals that accounts for both the number of citations received by a journal and the importance or prestige of the journals where such citations come from It measures the scientific influence of the average article in a journal, it expresses how central to the global scientific discussion an average article of the journal is.

Evolution of the number of published documents. All types of documents are considered, including citable and non citable documents.

This indicator counts the number of citations received by documents from a journal and divides them by the total number of documents published in that journal. The chart shows the evolution of the average number of times documents published in a journal in the past two, three and four years have been cited in the current year. The two years line is equivalent to journal impact factor ™ (Thomson Reuters) metric.

Evolution of the total number of citations and journal's self-citations received by a journal's published documents during the three previous years. Journal Self-citation is defined as the number of citation from a journal citing article to articles published by the same journal.

Evolution of the number of total citation per document and external citation per document (i.e. journal self-citations removed) received by a journal's published documents during the three previous years. External citations are calculated by subtracting the number of self-citations from the total number of citations received by the journal’s documents.

International Collaboration accounts for the articles that have been produced by researchers from several countries. The chart shows the ratio of a journal's documents signed by researchers from more than one country; that is including more than one country address.

Not every article in a journal is considered primary research and therefore "citable", this chart shows the ratio of a journal's articles including substantial research (research articles, conference papers and reviews) in three year windows vs. those documents other than research articles, reviews and conference papers.

Ratio of a journal's items, grouped in three years windows, that have been cited at least once vs. those not cited during the following year.

Evolution of the percentage of female authors.

Evolution of the number of documents cited by public policy documents according to Overton database.

Evoution of the number of documents related to Sustainable Development Goals defined by United Nations. Available from 2018 onwards.

Leave a comment

Name * Required

Email (will not be published) * Required

* Required Cancel

The users of Scimago Journal & Country Rank have the possibility to dialogue through comments linked to a specific journal. The purpose is to have a forum in which general doubts about the processes of publication in the journal, experiences and other issues derived from the publication of papers are resolved. For topics on particular articles, maintain the dialogue through the usual channels with your editor.

Follow us on @ScimagoJR Scimago Lab , Copyright 2007-2024. Data Source: Scopus®

Cookie settings

Cookie Policy

Legal Notice

Privacy Policy

• New! Member Benefit New! Member Benefit
• Featured Analytics Hub
• Resources Resources
• Member Directory
• Networking Communities
• Advertise, Exhibit, Sponsor
• Find or Post Jobs

• Learn and Engage Learn and Engage
• Bridge Program

• Compare AACSB-Accredited Schools
• Explore Programs

• Advocacy Advocacy
• Featured AACSB Announces 2024 Class of Influential Leaders
• Diversity, Equity, Inclusion, and Belonging
• Influential Leaders
• Innovations That Inspire
• Connect With Us Connect With Us
• Accredited School Search
• Accreditation
• Learning and Events
• Advertise, Sponsor, Exhibit
• Tips and Advice
• Is Business School Right for Me?

Recognizing the Value of Educational Research

• A recent survey shows that research on teaching and learning is not valued at many AACSB-accredited schools across the U.S. and Canada.
• One reason that business schools might not recognize research on teaching and learning is that the journal quality lists they commonly use to assess faculty intellectual contributions focus primarily on discipline-based scholarship.
• STEM fields already place equal value on research on teaching and learning within individual disciplines. By following their lead, two Canadian scholars argue, business schools will enrich their students’ learning experiences.    

If business educators were asked to define the purpose of business schools, they likely would emphasize the need to “prepare the next generation of leaders.” But if this is the case, why do so few business schools prioritize research that advances teaching and curricular design?

Researcher Sanobar Siddiqui first explored this question as the subject of her doctoral dissertation. “One of my thesis findings was that the tenure system’s lack of rewards impedes business academics from pursuing research in teaching and learning,” she explains.

Now an assistant professor of accounting at the University of Regina’s Faculty of Business Administration in Canada, Siddiqui wanted to learn why so many business schools do not value research on teaching and learning (RoTL). This response is puzzling, she says, given that Standard 7 of the  AACSB Business Accreditation Standards  accepts “scholarship of teaching and learning” as documentation to indicate a business school’s teaching effectiveness and impact.

She and Camillo Lento, a professor with the Faculty of Business Administration at Lakehead University in Thunder Bay, Ontario, published a  paper  on the status of RoTL in the April 2022 edition of the International Journal of Educational Management . The paper’s findings are based on a survey in which Siddiqui and Lento asked educators two questions:

• How do AACSB-accredited business schools in the U.S. and Canada define “teaching effectiveness,” according to AACSB’s Standard 7?
• Do these schools consider research on teaching and learning in their promotion and tenure decisions?

This topic is particularly important, says Siddiqui, because business schools serve such diverse student audiences. Moreover, learner success is integral to every business school’s mission. Many of the instructional strategies “that we use in class are not research-informed or evidence-based. Hence, we are shortchanging our students,” she says. “Our teaching needs to catch up with the changes we see in our classroom.”

‘A Last Priority’

Siddiqui and Lento received 78 responses to their survey; in the second phase of their study, they conducted semi-structured interviews with 11 educators in the U.S. and Canada.

Among survey respondents, 42 percent noted that they were “unaware of an explicit teaching effectiveness definition” at their schools, but 58 percent said the policies in place at their schools communicated “an implied definition.” Only one respondent could quote a definition of teaching effectiveness from the school’s website.

Respondents noted a lack of “perceived respect and value” for RoTL, describing this line of scholarship as “a last priority” at their schools. As one educator put it, “Our department does not really care about teaching as long as you are cranking out strong scholarship.”

Schools that consider educational research for tenure and faculty qualification tend to focus on journal quality alone, not on whether published articles are discipline-based.

The good news is that 55 percent of respondents noted that their schools did take RoTL into account when making tenure decisions. Siddiqui and Lento found that these schools have two things in common. First, they focus on journal quality alone for the purposes of tenure and faculty qualification, not on whether faculty’s published articles are discipline-based.

Second, these schools are more likely to consider RoTL when faculty include this work “as part of a larger research plan that includes discipline-based research.” Only faculty following teaching tracks are likely to receive tenure based solely on publications in education-focused journals. 

Additionally, teaching-oriented schools are more likely than research-oriented schools to recognize RoTL. While this makes outward sense, Siddiqui wonders why prolific faculty who produce innovative scholarship on pedagogical issues that are critical to business education cannot “be hired, promoted, and awarded just like discipline-based researchers” at research-oriented institutions.

What Perpetuates the Stigma?

Siddiqui and Lento point to several factors that could be driving the lack of recognition of RoTL among AACSB-accredited schools:

No consensus about teaching quality.  Although many individual educational institutions have defined teaching effectiveness based on existing research, business schools have not yet established a shared definition of what constitutes effective teaching. However, the co-authors emphasize, more dedicated research could produce findings that inspire a common language around teaching and learning.

The complex nature of determining teaching quality. Schools often evaluate the quality of faculty’s research by whether the work appears in academic journals that are rated highly by certain  journal quality lists . However, they find they cannot use a similar approach to evaluate the quality of faculty’s teaching, says Lento. “The evaluation of teaching effectiveness is much more complex and requires many more sources of information, possibly compiled into a teaching dossier that is unique to an educator.”

A lack of attention in business doctoral programs. Most doctoral programs train young researchers to study topics related to their disciplines of choice. As a result of this early training, RoTL “may come with a stigma as it is outside of traditional discipline-specific research,” Lento says.

Lento admits that the reasons listed above are speculative. He and Siddiqui would like to see other researchers conduct follow-up studies that take deeper dives into the broader stigma surrounding RoTL.

Changing Mindsets, Taking Action

In the meantime, Siddiqui and Lento call on business school administrators and faculty to work together to create a “shared and precise definition of teaching effectiveness.” Educators can start by defining teaching quality within their own institutions.

From there, Siddiqui and Lento say that schools can take any or all the following actions to change mindsets about RoTL:

• Set appropriate objectives, incentives, and evaluation mechanisms.
• Create and nurture communities of practice that help like-minded faculty pursue research focused on solving issues they face in their classrooms.
• Consider weighing education research in peer-reviewed articles more heavily, particularly for faculty in teaching-focused roles.
• Recognize RoTL for accreditation and tenure and normalize it as a legitimate form of scholarship.
• Make seed funds available to faculty who pursue RoTL.
• Give awards and incentives to faculty who use research-informed teaching in their classrooms.
• Consider hiring tenure-track academics who also are expert educators with an expressed interest in pursuing RoTL. These scholars can investigate and develop “research-informed teaching tools ready to be put into practice in almost any business classroom,” says Siddiqui. This outcome, she emphasizes, is an indication of how RoTL contributes to the advancement of business disciplines.
• Encourage and teach RoTL in doctoral programs, with the aim of improving and advancing the quality of teaching at business schools.

Siddiqui points out that information on the websites of AACSB-accredited schools “are replete with research centers, research chairs and scholars, core research focus areas, research awards, annual research celebration reports, intellectual contributions, and grant-funding awards.”

There is no reason, she says, that schools could not also highlight information about their teaching philosophies, teaching awards, student feedback, educational leadership and professional development, and faculty research on teaching and learning.

Two B-School Perspectives

So far, Siddiqui and Lento’s paper has captured the attention of other like-minded educators in the business school community. This includes Nicola Charwat, associate dean of teaching and learning and senior lecturer of business law and taxation at Monash University’s Monash Business School (MBS) in Caulfield East, Australia.

MBS prioritizes scholarship on teaching and learning (SoTL) where appropriate, she says, through efforts that include identifying quality education-oriented journals and valuing publication in those journals equally to publication in discipline-based journals. The school uses “a consultative process” to identify journals specializing in teaching and learning that are equivalent to discipline-based journals rated as A*, A, B, and C on the quality list compiled by the Australian Business Deans Council.

“We have also instituted a Business Education and Research Group, which has been awarding both practice- and research-output-focused grants to staff for three years,” Charwat says. “Alongside these efforts, of course, there are moves in the university in line with the broader trend of raising the profile of teaching and ensuring its status is on par with other work of the university.”

Educators in STEM disciplines have long recognized educational research in tenure decisions and regularly reward academics who pursue RoTL in their disciplines.

Despite these changes, Charwat notes that the perception remains that accomplishments related to educational research are “somehow lesser” than those related to discipline-related scholarship. Additionally, many faculty remain uncertain about how to approach educational research. In response, MBS has built communities of practice dedicated to teaching and is now working “to increase awareness of and opportunities to undertake SoTL and education research,” Charwat says.

Charwat says that the questions raised in Siddiqui and Lento’s paper are “essential” to business education, and that their article “has prompted us to start exploring the patterns of our own SoTL and education research.” MBS faculty, she adds, might also pursue a similar study focused on AACSB-accredited schools in Australia. 

Another educator who read the article with interest is Martin Lockett, former dean and professor of strategic management at Nottingham University Business School China (NUBS China) in Zhejiang. Lockett explains that NUBS China uses the Academic Journal Guide , which is produced by the Chartered Association of Business Schools (CABS), to support tenure decisions and to classify faculty under AACSB accreditation standards.

But in the CABS guide, only four journals focused on teaching and learning are rated as 3, 4, or 4*, which are the targets that NUBS China uses to qualify faculty as Scholarly Academics under AACSB accreditation or for internal recognition of quality research, Lockett says.

This has led to worry among the school’s teaching-oriented faculty that if they focus on RoTL, they risk being classified as “additional faculty,” unless they can consistently publish in the few education-focused journals listed by CABS. That concern, Lockett says, deters most faculty from pursuing RoTL in any substantial way.

While this scenario is all too common at institutions with research-focused missions, it is not mandated by AACSB accreditation standards, emphasizes Stephanie Bryant, AACSB’s chief accreditation officer. She clarifies that whether a business school considers educational scholarship for the purpose of accreditation or tenure is its choice, based on the parameters it has set for its individual mission. “The standards do not say anywhere, or imply, that educational research is not valued,” Bryant stresses. The devaluation of RoTL, she adds, “is a school perspective.”

Time to ‘Balance the Scales’

The stigma surrounding RoTL at AACSB-accredited business schools could be lifted, say Siddiqui and Lento, if administrators acknowledge the benefits that fostering cultures of teaching and learning bring to all business school stakeholders. These advantages include a wider scope of scholarship and more evidence-based pedagogical tools for faculty, richer learning experiences and better learning outcomes for students, and more well-rounded job candidates for employers.

Educators in science, technology, engineering, and mathematics (STEM) disciplines already know this, says Siddiqui. STEM departments have long recognized educational research in tenure decisions and regularly reward academics who pursue RoTL in their disciplines.

As one example, Siddiqui points to Carl Edwin Wieman, winner of the 2001 Nobel Peace Prize in Physics. Wieman established the  Carl Wieman Science Education Initiative  at the University of British Columbia in Canada to encourage evidence-based teaching methods focused on improving undergraduate science education. Since its inception, the initiative has hired fellows who are interested in conducting education research, particularly based in the disciplines in which they have earned their doctorates. It also has inspired the creation of teaching materials in science education, a dedicated website, and a sister initiative at the University of Colorado Boulder in the United States.

Business schools, says Siddiqui, could achieve comparable results by raising awareness of the importance of RoTL, disseminating RoTL findings beyond peer-reviewed journals, and driving research-informed teaching methods that advance business education.

This year, the co-authors published a second paper that finds that scholarly and practice academics who developed rigorous research skills in their doctoral programs and who publish discipline-based research are more likely to pursue RoTL research. Here, Siddiqui and Lento more directly call on business school deans to reward and incentivize this line of research by creating communities of practice and expanding their journal ranking frameworks to include relevant peer-reviewed publications.

It is imperative, Siddiqui and Lento argue, that business schools place studies based on classroom settings on equal footing with studies based on corporate settings. “Research on teaching and learning balances the scales,” Siddiqui says, “by utilizing evidence-based, efficient, and effective teaching to foster deep learning amongst diverse student audiences.”

• accreditation
• administration
• faculty engagement

Professional teacher training in the PELC and PST social programs in Brazil: A systematic review

This research aimed to analyze the scientific production on public sports and leisure policies related to professional training of two programs in Brazil: City Sport and Leisure Program (CSLP) and Second Time Program (STP). As a methodological procedure, a systematic review with a qualitative approach was carried out between the years 2003 and 2022. The research sample was composed of 21 articles, published between 2012 and 2022. The results showed that researchers from the South and Southeast region boosted publications on professional training in CSLP and STP. Paper publications were concentrated in journals in the fields of physical education, education, sport and leisure. Professional training corresponds to a central element for structuring sports social programs, as a way to minimize gaps in initial training, but the format of sporadic courses and distance from the reality of professionals was the main limiting factor.

Author Biographies

Dirceu santos silva, universidade federal de mato grosso do sul (ufms).

Doutor em Educação Física pela Unicamp. Professor do Programa de Pós-graduação em Educação da Universidade Federal de Mato Grosso do Sul.

Luana Carla André, Universidade Federal de Mato Grosso do Sul (UFMS)

Mestre em educação pelo Programa de Pós-graduação em Educação da Universidade Federal de Mato Grosso do Sul.

Mariana Brasiliano Salerno, Universidade Federal de Mato Grosso do Sul (UFMS)

Doutora em Educação Física pela Unicamp. Professor do Programa de Pós-graduação em Educação da Universidade Federal de Mato Grosso do Sul.

• PDF (Português)

How to Cite

• Endnote/Zotero/Mendeley (RIS)

Copyright (c) 2024 Dirceu Santos Silva, Luana Carla André, Mariana Brasiliano Salerno

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License .

Education Policy Analysis Archives/Archivos Analíticos de Políticas Educativas/Arquivos Analíticos de Políticas Educativas (EPAA/AAPE;  ISSN 1068-2341 ) is a peer-reviewed, open-access, international, multilingual, and multidisciplinary journal designed for researchers, practitioners, policy makers, and development analysts concerned with education policies.

• Português (Brasil)
• Español (España)

Make a Gift

Submit a manuscript or revision, recent special issues.

Youth and Adult Education, Literacies and Decoloniality

Global Policy Mobilities in Federal Education Systems

Teacher Subjectivities in Latin America

Mortarboard Review

Montessori-related dissertations, 2023.

• Jennifer Moss Emporia State University
• Katie Keller Wood CMStep: Cincinnati Montessori Secondary Teacher Education Program

This is the second article in an ongoing series, published annually, highlighting a selection of English-language dissertations from the previous calendar year related to Montessori philosophy and education. Thirteen doctoral dissertations completed and approved during the 2023 calendar year were identified. The authors selected three dissertations to spotlight because they represent high-quality research in an area that is relevant to the current educational landscape: antibias and anti-racist (ABAR) educational practices.

Author Biography

Joel Parham is an Affiliate Researcher at the University of Kansas and owner of JRP Consulting & Research. He has a Bachelor of Science in Sociology and a Master of Library and Information Science. His research interests include the historical diffusion of the Montessori method of education, and the American Montessori movement. Joel is also the creator of the Montessori Bibliography Online – https://montessoribib.ku.edu – which he regularly maintains.

Bass-Barlow, K. (2023). Examination of Montessori training: Experiences of People of Color in public and charter Montessori schools [Doctoral dissertation, Arkansas State University]. https://www.proquest.com/docview/2856660597

Canzoneri-Golden, L., & King, J. (2020). An examination of culturally relevant pedagogy and antibias-antiracist curriculum in a Montessori setting [Doctoral dissertation, Lynn University]. https://spiral.lynn.edu/etds/360

Canzoneri-Golden, L., & King, J. (2023). Montessori education and critical race theory in the United States. In A. K. Murray, E.-M. Tebano Ahlquist, M. K. McKenna, & M. Debs (Eds.), The Bloomsbury handbook of Montessori education (pp. 503–511). Bloomsbury Academic.

Cooper, J. (2022). Searching for equity in education: A qualitative study examining the experiences of African American families in accessing and financing Montessori education [Doctoral dissertation, Saint Joseph’s University]. https://www.proquest.com/docview/2806396666

Cossentino, J. (2009). Culture, craft, & coherence: The unexpected vitality of Montessori teacher training. Journal of Teacher Education, 60(5), 520–527. https://doi.org/10.1177/00224871093445

D’Cruz Ramos, G. O. (2023). Critical Montessori education: Centering BIPOC Montessori educators and their anti-racist teaching practices [Doctoral dissertation, University of Maryland]. https://doi.org/10.13016/dspace/twpw-zy3r

Debs, M. (2019). Diverse families, desirable schools: Public Montessori in the era of school choice. Harvard Education Press.

Freire, P. (1968). Pedagogy of the oppressed. Seabury Press.

Giroux, H. A. (1988). Schooling and the struggle for public life: Critical pedagogy in the modern age. Minnesota University Press.

Giroux, H. A. (2011). On critical pedagogy. Bloomsbury.

Hammond, Z. (2015). Culturally responsive teaching and the brain: Promoting authentic engagement and rigor among culturally and linguistically diverse students. SAGE.

Hammons, M. S. (2023). Antiracist pedagogy in White spaces: An exploration of antiracist White teachers and their commitment to create antiracist classrooms [Doctoral dissertation, San Francisco State University]. https://doi.org/10.46569/8p58pm94q

hooks, b. (1994). Teaching to transgress: Education as the practice of freedom. Routledge.

hooks, b. (2003). Teaching community: A pedagogy of hope. Routledge.

McLaren, P. (1989). Life in schools: An introduction to critical pedagogy in the foundations of education. Irwin Publishing.

McLaren, P. (2016). Pedagogy of insurrection: From resurrection to revolution. Peter Lang.

Moquino, T. (2023). The need for Indigenous Montessori teacher education. In Equity examined: How to design schools and teacher education programs where everyone thrives (pp. 42–44). American Montessori Society.

Moquino, T., Walker, N., & Kitchens, K. (2023). Beyond authenticity: Indigenizing Montessori education in settler colonial United States. In A. K. Murray, E.-M. Tebano Ahlquist, M. K. McKenna, & M. Debs (Eds.), The Bloomsbury handbook of Montessori education (pp. 513–524). Bloomsbury Academic.

Schaeffer, K. (2021). U.S. public school students often go to schools where at least half of their peers are the same race or ethnicity. https://policycommons.net/artifacts/2047148/us/2800212

Welch, A. M. (2023). Racial identity development in the early years: A Montessori student/educator’s autoethnographic study [Master’s thesis, California State University, Los Angeles]. https://www.proquest.com/docview/2849816249

Copyright (c) 2024 Joel Parham, Jennifer Moss, Katie Keller Wood

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License .

Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution Non-Commercial License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal. Authors can view article download statistics for published articles within their accounts.

Journal of Montessori Research Author Agreement The following is an agreement between the Author (the “Corresponding Author”) acting on behalf of all authors of the work (“Authors”) and the Journal of Montessori Research (the “Journal”) regarding your article (the “Work”) that is being submitted for consideration.   Whereas the parties desire to promote effective scholarly communication that promotes local control of intellectual assets, the parties for valuable consideration agree as follows. A. CORRESPONDING AUTHOR’S GRANT OF RIGHTS After being accepted for publication, the Corresponding Author grants to the Journal, during the full term of copyright and any extensions or renewals of that term, the following: 1.    An irrevocable non-exclusive right to reproduce, republish, transmit, sell, distribute, and otherwise use the Work in electronic and print editions of the Journal and in derivative works throughout the world, in all languages, and in all media now known or later developed. 2.    An irrevocable non-exclusive right to create and store electronic archival copies of theWork, including the right to deposit the Work in open access digital repositories. 3.    An irrevocable non-exclusive right to license others to reproduce, republish, transmit,and distribute the Work under the condition that the Authors are attributed. (Currently this is carried out by publishing the content under a Creative Commons Attribution Non-Commercial 4.0 license (CC BY-NC.) 4.    Copyright in the Work remains with the Authors. B. CORRESPONDING AUTHOR’S DUTIES 1.    When distributing or re-publishing the Work, the Corresponding Author agrees to credit the Journal as the place of first publication. 2.    The Corresponding Author agrees to inform the Journal of any changes in contact information. C. CORRESPONDING AUTHOR’S WARRANTY The Corresponding Author represents and warrants that the Work is the Authors’ original work and that it does not violate or infringe the law or the rights of any third party and, specifically, that the Work contains no matter that is defamatory or that infringes literary or proprietary rights, intellectual property rights, or any rights of privacy. The Corresponding Author also warrants that he or she has the full power to make this agreement, and if the Work was prepared jointly, the Corresponding Author agrees to inform the Authors of the terms of this Agreement and to obtain their written permission to sign on their behalf. The Corresponding Author agrees to hold the Journal harmless from any breach of the aforestated representations. D.  JOURNAL’S DUTIES In consideration of the Author’s grant of rights, the Journal agrees to publish the Work, attributing the Work to the Authors. E. ENTIRE AGREEMENT This agreement reflects the entire understanding of the parties. This agreement may be amended only in writing by an addendum signed by the parties. Amendments are incorporated by reference to this agreement. ACCEPTED AND AGREED BY THE CORRESPONDING AUTHOR ON BEHALF OF ALL AUTHORS CONTRIBUTING TO THIS WORK

How to Cite

• Endnote/Zotero/Mendeley (RIS)

Make a Submission

• Norwegian Bokmål

Developed By

This electronic publication is supported by the

University of Kansas Libraries .

ISSN:  2378-3923

State of U.S. Science & Engineering

Talent is the Treasure

Two new reports show the growing role of R&D in global economy

Science & Engineering Indicators 2022

The State of U.S. S&E

The U.S. is a Keystone of Global Science & Engineering

January 20, 2022

NSF's Merit Review Process

Fiscal Year 2020 Digest

NSB Vision 2030 Report

Roadmap for a Decade of Discovery

CALL FOR BOARD NOMINATIONS

Deadline May 31, 2021

2020 NSB AWARD NOMINATIONS

Now open through October 4, 2019

Publications

The State of U.S. Science & Engineering Report

THE SKILLED TECHNICAL WORKFORCE: Crafting America's Science & Engineering Enterprise

Report to the NSB on the NSF's Merit Review Process Fiscal Year 2020

Science and Engineering Indicators 2022

Science and Engineering Indicators Thematic Reports 2022

July 24, 2024  - july 25, 2024.

National Science Board Meeting

NSB Meeting

December 4, 2024  - December 5, 2024

• National Science Foundation Updates Transparency and Accountability Practices

National Science Board elects first industry leader in 30 years

• The National Science Board will hold a hybrid meeting May 1 - 2, 2024
• New report shows U.S. and China are world’s top producers of Knowledge-and tech-intensive industries

NSB Activities

• Major Actions & Approvals
• Resolutions
• Skilled Technical Workforce Resources
• Vision 2030 Resources
• Multimedia Gallery
• OIG Semiannual Reports

IBM’s Darío Gil elected chair and University of the District of Columbia’s Victor McCrary re-elected vice-chair.

Left to right: NSB Chair Darío Gil, Vice Chair Victor McCrary ( Credit and Larger Version )

May 14, 2024

Darío Gil and Victor McCrary will lead the National Science Board (NSB) for the next two years as its respective chair and vice chair. The National Science Foundation Act of 1950 charged the NSB with two roles: governing board of the NSF and advisor to Congress and the President on policy matters related to science, technology, engineering, and mathematics (STEM) research and STEM education. 

Gil, Senior Vice President and Director of IBM Research, replaces University of Utah Computer Science Professor and former Microsoft Executive Dan Reed, whose six-year term on the Board ended on May 10. The Board re-elected McCrary, Vice President for Research and Professor of Chemistry at the University of the District of Columbia, to serve a third term as NSB’s vice chair. 

It has been more than 30 years since the NSB had a Chair who was working in industry at the time of his or her election. Prior NSB Chairs working in industry at the time of their election were Mary Good (AlliedSignal), Roland Schmitt (GE), and Lewis Branscomb (IBM). Gil also has the distinction of being one of only three NSB Chairs under 50 at time of election.

“I look forward to working with my colleagues to build bridges across the Federal, business, academia, and philanthropy worlds to renew our commitments to STEM education, workforce development, and to tackling ambitious grand challenges to outpace our greatest strategic competitors,” says Gil. “It is time to join forces across sectors to ensure the long-term success of the United States.” 

“For our country to stay competitive – which is crucial to our economic and national security – we need to do much more to develop STEM workers at all levels, from skilled technical workers to advanced degree holders,” says Victor McCrary. “We need plumbers, we need Ph.D.s; we need electricians to electrical engineers to advanced degree holders in every STEM field, and we need an ‘all hands-on deck’ philosophy to create this reality for our nation. NSF is key in developing talent for our national security and for our national security sector.” 

About Darío Gil

Gil is a member of the NSB class of 2020 - 2026. Over the last two years, Gil chaired the Board’s Committee on External Engagement, leading the Board’s engagement strategy and initiatives. Gil has also served on NSB’s Committee on Strategy and chaired the sub-committee on Technology, Innovation and Partnerships (TIP), which worked with the NSF to launch the agency’s first new directorate in 30 years.  

Gil is Senior Vice President and Director of IBM Research, responsible for one of the world’s largest and most influential corporate research labs, with over 3,000 researchers. He leads the technical community of IBM, directing innovation strategies in hybrid cloud, AI, semiconductors, quantum computing, and exploratory science. 

Gil, who received his Ph.D. in Electrical Engineering and Computer Science from MIT, was recently elected to the National Academy of Engineering for his contributions to the “advancement and practical use of artificial intelligence and quantum computing in industry and society.” An advocate of collaborative research models, Gil co-chairs the Executive Board of the International Science Reserve, a global network of open scientific communities that provides specialized resources to prepare for and help mitigate urgent, complex global challenges. Gil has served on the President’s Council of Science and Technology Advisors (PCAST) and serves on the boards of the Semiconductor Industry Association (SIA), the Center for Strategic and International Studies (CSIS), the New York Academy of Sciences, the New York Hall of Science, and Rensselaer Polytechnic Institute (RPI). 

About Victor McCrary

In May 2022, the Biden-Harris Administration re-appointed McCrary to serve a second six-year term on the NSB. As Vice Chair, McCrary regularly engages with a multitude of science and engineering communities, congressional offices, and Federal agency leaders. During his first six years on the Board, McCrary led the effort to recognize the non-degreed STEM workforce, resulting in the Board’s report,  The Skilled Technical Workforce: Crafting America’s Science and Engineering Enterprise . 

McCrary is Vice President for Research and Professor of Chemistry at the University of the District of Columbia (UDC) where he guides the development and sustainability of UDC’s research ecosystem through acquired grants and partnerships. He started his career as a research scientist at AT&T Bell Laboratories-Murray Hill and then joined the Advanced Technology Program at the National Institute of Standards and Technology (NIST), eventually becoming the inaugural Chief of the Convergent Information Systems Division. At NIST McCrary led the effort for electronic book standards and prototypes by employing high school and college students. McCrary’s executive research leadership positions include the Vice Chancellor for Research at University of Tennessee, Knoxville; Vice President for Research and Economic Development at Morgan State University; and Business Area Executive for Science & Technology at the Johns Hopkins University Applied Physics Laboratory. He is a former national president of the National Organization for the Professional Advancement of Black Chemists and Chemical Engineers and a Fellow of the American Chemical Society. McCrary was elected in 2023 to the Executive Committee of Council on Research (COR) of the Association of Public & Land- Grant Universities (APLU).

About the National Science Board

The National Science Foundation Act of 1950 charged the NSB with two roles: governing board of the NSF and advisor to Congress and the President on policy matters related to STEM research and STEM education. Selected for their distinguished service and accomplishments in academia, government, and the private sector,  the Board’s 24 presidentially appointed members  are leaders in STEM research and STEM education.

Media Contact  Nadine Lymn, National Science Board, (703) 292-2490,  [email protected]

Useful NSB Web Sites:

Home Page: http://www.nsf.gov/nsb Media Contact: http://www.nsf.gov/staff/staff_bio.jsp?lan=nlymn&org=NSF News: http://www.nsf.gov/nsb/news Meetings: http://www.nsf.gov/nsb/meetings Publications: http://www.nsf.gov/nsb/publications Facebook: https://www.facebook.com/NationalScienceBoard Twitter: Twitter: https://twitter.com/intent/user?screen_name=NSF_NSB YouTube: https://www.youtube.com/channel/UCkrHRzuGSrPp2haQs0T_Pww

To view PDF documents, please download Adobe Acrobat Reader .

National Science Board

National Science Foundation

School of Dentistry

Continuing dental education, ohsu continuing dental education.

Welcome to the OHSU School of Dentistry Continuing Dental Education Department.  Receive your year-round courses taught by world-renowned researchers and scholars who present on topics that will make an immediate positive impact on your professional practices.

Explore our offerings, including pre-recorded online, live webinars, live lectures, and hands-on participation options. Stay ahead in your field with us. Feel free to reach out if you have any questions. Welcome to OHSU dental education excellence.

Course Listings

General information, privacy statement.

The OHSU School of Dentistry is committed to protecting your privacy. The information you provide when you register for a course on this website is collected for the purpose of correctly processing your payment. The OHSU School of Dentistry does not share, sell or trade information collected on our web site with outside parties.

Security Statement

The OHSU School of Dentistry understands the importance of the security of your personal information. We are committed to protect the confidentiality and integrity of any personal information that is shared with us on the web or offline. The OHSU School of Dentistry may collect your personal information ONLY if you elect to give us that information such as name, address, phone number, and e-mail address. However, the OHSU School of Dentistry will never store any credit card information when you make a gift online. An audited secure third party, such as Authorize.net, is used to process any online credit card transactions.

Registration Information

Upon registration for any activity with the Continuing Dental Education Department, the participant agrees that Oregon Health &Science University School of Dentistry Continuing Dental Education Department, its affiliates and all personnel associated with the courses are not responsible or liable for any injuries or damages sustained by the participant in connection with the scheduled activity and the participant hereby releases each of them from any claims against them arising directly or indirectly from any such injury or damage.

The registered participant grants permission of the Continuing Dental Education Department (and its designees and agents) to use the participant's image, likeness, actions and/or statements in any live recorded audio, video or photographic display or other transmission, exhibition, publications, or reproductions made of, about, or at, the activity without further authorization or compensation.

Participants may not use video or audio recording devices during the course or schedule activities.

Cancellation & Refund Policy

Cancellation Policy:

If you wish to cancel a lecture course, you must notify the CDE office prior to the date of the course for the appropriate refund to be issued. Failure to do so will result in a "No Show". 

For cancellations:

When you need to cancel, you have two options:

1. Receive a full refund, minus a $35 processing fee. 

2. Transfer the full course fee to another CDE course(s) within the same CDE calendar year (Sept-June). 

If the CDE Department needs to cancel the course, you have two options:

1. Receive a full refund*

2. Transfer the full course fee to another CDE course(s) or when the course gets rescheduled*

(*Please keep in mind that the CDE office is not responsible for any travel expenses or penalties incurred by course registrants in case of course cancellation or revision.)

For "No Shows"

No refunds or fund transfers are available for "No Shows"

Participant Courses:

Refunds for CDE participation courses must be requested at least 14 days before the first meeting of the course. These courses have a non-refundable $100, which covers the cost of non-refundable materials. If you cancel less than 14 days before the course, you will receive a monetary refund if the CDE office can fill your spot with another participant. In such cases, funds, minus the $100 deposit, may be transferred to another course within the same CDE calendar year if the request is made 14 days before the course.

Pre-Recorded Online Courses: 

Each course may be a little different as to how long you have to access the course material. Typically course materials need to be viewed within 3 months from the date of registration. No refunds will be given for online courses. 

Refund Exceptions:

No refund penalty will be charged for last-minute emergencies. However, the CDE office must be contacted by noon on the course date to report an emergency. You can receive a full refund or transfer the money to another CDE course within the same course year (September – June).

Acts of Nature:

In the event of snow, earthquakes, or other acts of nature, please check the CDE homepage or call the CDE office for instructions.

Programs Offered in Partnership with Other Organizations:

Cancellation policies for programs offered in partnership with other organizations may vary. Refer to specific course information for details.

Controversial Materials

Dental education institutions have an obligation to disseminate new knowledge related to dental practice. In so doing, some presentations may include controversial materials or commercial references. Sponsorship of a continuing education activity by Oregon Health & Science University School of Dentistry does not necessarily imply endorsement of a particular philosophy, procedure or product by this institution.

Rental Information & Study Clubs

If you are interested in renting space from the SOD, please send an email to [email protected] 

Looking to offer CE for your event?

We would be happy to see if your event qualifies for ADA CERP CE credits. There is a $400 application fee and a $35 fee for every CE verification needed. Please click on the link here to be transferred to our application page. 

Continuing Dental Education Department 2730  S. Moody Ave Portland, OR. 97201 Phone: 503-494-8857 Email: [email protected]

Credentials

OHSU is an ADA CERP Recognized Provider. ADA CERP is a service of the American Dental Association to assist dental professionals in identifying quality providers of continuing dental education. ADA CERP does not approve or endorse individual courses or instructors, nor does it imply acceptance of credit hours by boards of dentistry.

OHSU is a member of the Association for Continuing Dental Education. 

• Find My Rep

You are here

Review of Educational Research

Preview this book.

• Description
• Aims and Scope
• Editorial Board
• Abstracting / Indexing
• Submission Guidelines

The Review of Educational Research ( RER , quarterly, begun in 1931; approximately 640 pp./volume year) publishes critical, integrative reviews of research literature bearing on education. Such reviews should include conceptualizations, interpretations, and syntheses of literature and scholarly work in a field broadly relevant to education and educational research. RER encourages the submission of research relevant to education from any discipline, such as reviews of research in psychology, sociology, history, philosophy, political science, economics, computer science, statistics, anthropology, and biology, provided that the review bears on educational issues. RER does not publish original empirical research unless it is incorporated in a broader integrative review. RER will occasionally publish solicited, but carefully refereed, analytic reviews of special topics, particularly from disciplines infrequently represented.

The Review of Educational Research publishes critical, integrative reviews of research literature bearing on education. Such reviews should include conceptualizations, interpretations, and syntheses of literature and scholarly work in a field broadly relevant to education and educational research. RER encourages the submission of research relevant to education from any discipline, such as reviews of research in psychology, sociology, history, philosophy, political science, economics, computer science, statistics, anthropology, and biology, provided that the review bears on educational issues. RER does not publish original empirical research, and all analyses should be incorporated in a broader integrative review. RER will occasionally publish solicited, but carefully refereed, analytic reviews of special topics, particularly from disciplines infrequently represented. The following types of manuscripts fall within the journal’s purview:

Integrative reviews pull together the existing work on an educational topic and work to understand trends in that body of scholarship. In such a review, the author describes how the issue is conceptualized within the literature, how research methods and theories have shaped the outcomes of scholarship, and what the strengths and weaknesses of the literature are. Meta-analyses are of particular interest when they are accompanied by an interpretive framework that takes the article beyond the reporting of effect sizes and the bibliographic outcome of a computer search.

Theoretical reviews should explore how theory shapes research. To the extent that research is cited and interpreted, it is in the service of the specification, explication, and illumination of a theory. Theoretical reviews and integrative reviews have many similarities, but the former are primarily about how a theory is employed to frame research and our understandings, and refer to the research as it relates to the theory.

Methodological reviews are descriptions of research design, methods, and procedures that can be employed in literature reviews or research in general. The articles should highlight the strengths and weaknesses of methodological tools and explore how methods constrain or open up opportunities for learning about educational problems. They should be written in a style that is accessible to researchers in education rather than methodologists.

Historical reviews provide analyses that situate literature in historical contexts. Within these reviews, explanations for educational phenomena are framed within the historical forces that shape language and understanding.

Commissioned reviews and thematic issues. The editors may commission and solicit authors to review areas of literature. In all other respects, commissioned reviews are subject to the same review process as submitted reviews. The editors also encourage readers to propose thematic topics for special issues and, as potential guest editors, to submit plans for such issues.

In addition to review articles, RER will occasionally publish notes and responses which are short pieces of no more than 1,200 words on any topic that would be of use to reviewers of research. Typically, they point out shortcomings and differences in interpretation in RER articles and policy.

The standards and criteria for review articles in RER are the following:

1. Quality of the Literature. Standards used to determine quality of literature in education vary greatly. Any review needs to take into account the quality of the literature and its impact on findings. Authors should attempt to review all relevant literature on a topic (e.g., international literature, cross-disciplinary work, etc.).

2. Quality of Analysis. The review should go beyond description to include analysis and critiques of theories, methods, and conclusions represented in the literature. This analysis should also examine the issue of access—which perspectives are included or excluded in a body of work? Finally, the analysis should be reflexive—how does the scholars’ framework constrain what can be known in this review?

3. Significance of the Topic. The review should seek to inform and/or illuminate questions important to the field of education. While these questions may be broad-based, they should have implications for the educational problems and issues affecting our national and global societies.

4. Impact of the Article. The review should be seen as an important contribution and tool for the many different educators dealing with the educational problems and issues confronting society.

5. Advancement of the Field. The review should validate or inform the knowledge of researchers and guide and improve the quality of their research and scholarship.

6. Style. The review must be well written and conform to style of the Publication Manual of the American Psychological Association (6th edition). Authors should avoid the use of unexplained jargon and parochialism.

7. Balance and Fairness. The review should be careful not to misrepresent the positions taken by others, or be disrespectful of contrary positions.

8. Purpose. Any review should be accessible to the broad readership of RER. The purpose of any article should be to connect the particular problem addressed by the researcher(s) to a larger context of education.

We also encourage all authors interested in submitting a manuscript to RER to read our Editorial Vision for more information on our publication aims.

• Academic Search - Premier
• Academic Search Alumni Edition
• Academic Search Elite
• Clarivate Analytics: Current Contents - Physical, Chemical & Earth Sciences
• EBSCO: MasterFILE Elite
• EBSCO: MasterFILE Premier
• EBSCO: Professional Development Collection
• EBSCO: Sales & Marketing Source
• EBSCOhost: Current Abstracts
• ERIC (Education Resources Information Center)
• Educational Research Abstracts Online (T&F)
• Higher Education Abstracts
• MasterFILE Select - EBSCO
• ProQuest Education Journals
• Social SciSearch
• Social Sciences Citation Index (Web of Science)
• Teacher Reference Center
• Wilson Education Index/Abstracts

1. Publication Standards 2. Submission Preparation Checklist 3. How to Get Help With the Quality of English in Your Submission 4. Copyright Information 5. For authors who use figures or other materials for which they do not own copyright 6. Right of Reply 7. Sage Choice and Open Access

The Review of Educational Research (RER) publishes comprehensive reviews of literature related to education and does not publish new empirical work, except in the context of meta-analytic reviews of an area. Please check the journal’s Aims and Scope to see if your manuscript is appropriate to submit to RER.

All manuscripts should be submitted electronically to the editorial team at http://mc.manuscriptcentral.com/rer . For questions or inquiries about manuscripts, email us at [email protected] . Manuscripts may not be submitted via e-mail.

Publication Standards

Researchers who intend to submit studies for publication should consult the Standards for Research Conduct adopted by the AERA Council. We also recommend consulting (a) the Guidelines for Reviewers , which outline the criteria under which manuscripts are reviewed for publication by AERA and (b) recent previous editions of the journal. Individuals submitting systematic reviews or meta-analyses should also consult The PRISMA Statement ( http://www.prisma-statement.org ) as well the article on “Reporting Standards for Research in Psychology” in American Psychologist, 63 , 839 – 851 (doi:10.1037/0003-066X.63.9.839).

Submission Preparation Checklist

When you upload your initial submission, upload (1) a separate title page that is not anonymized. Please format the title page as described by the 7th edition of the APA Manual and (2) the main manuscript, which includes an ANONYMIZED title page, an abstract with keywords at the bottom, and the rest of the document including tables and figures, and finally (c) Author Bios.

Please ensure that your manuscript complies with the “ RER Formatting Requirements and Common Formatting Errors ” (see PDF on the RER website). If your submission does not meet these requirements, it will be returned to you.

Additionally, your submission should meet the following guidelines:

1. The submission has not been previously published and is not under consideration for publication elsewhere; or an explanation has been provided in the Cover Letter. Authors should indicate in the Author Note on the separate title page if sections of the manuscript have been published in other venues.

2. THE MANUSCRIPT CONTAINS NO IDENTIFYING INFORMATION, EVEN ON THE ANONYMIZED TITLE PAGE. Please anonymize any work of limited circulation (e.g., in press papers, manuscripts under submission) that would point to the author, both in the body of the manuscript and the reference list. More information on anonymizing is described subsequently. Please double check that the author’s name has been removed from the document’s Properties, which in Microsoft Word is found in the File menu (select “File,” “Properties,” “Summary,” and remove the author’s name; select “OK” to save).

3. The text conforms to APA style (currently the 7th ed.). Consult the guidelines spelled out under “Manuscript Style, Length, and Format” on this webpage and in the RER Formatting Requirements PDF included on our website.

4. The submission must be in Microsoft Word format (.doc or .docx), which will be converted into a PDF file. Please do not upload PDF files, or they will be returned to you.

5. All URL addresses and DOIs in the manuscript (e.g., http://www.aera.net ) should be activated and ready to click.

6. An abstract of 150 words maximum is included (both separately and on the second page of the main document after the ANONYMIZED title page). Please also include three to five keywords—the terms that researchers will use to find your article in indexes and databases.

Manuscript Style, Length, and Format

The style guide for the Review of Educational Research and all AERA journals is the Publication Manual of the American Psychological Association, 7th ed., 2020. The manual is available for purchase here . Guidelines are also available on the APA website .

Manuscripts should NOT exceed 65 pages (or 15,000 words), including tables, figures, appendices, notes, and references, but excluding anonymized title page, abstract, and any supplementary files. Pages should be numbered consecutively in the top right-hand corner, with a fully capitalized running head in the top-left corner. All manuscripts should begin with the anonymized title page (p.1). Manuscripts should be typed for 8½” x 11” paper, in upper and lower case, with 1-inch margins on all sides. Manuscripts should be typed in 12-point Times New Roman font. Manuscripts that exceed 65 pages may be returned without review.

All text, from the title page to the end of the manuscript should be double-spaced , including the abstract, block quotations, bulleted text, and the reference list. Single-spacing is allowed in tables when it is useful in making the table clearer. Do not leave blank lines after paragraphs or before sub-headings. However, if a heading or subheading is the last line on a page, use a page break to move it to the top of the next page. The Abstract, Introduction (beginning with the title), the References, and all tables and figures begin on new pages.

Please use the five subheadings as appropriate based on the 7th edition of the APA style manual. In addition to being on the title page, the title should also be placed at the beginning of the Introduction (in lieu of the word, “Introduction,” which should not appear) and the title at the beginning of the Introduction should be a Level 1 heading.

Tables and figures are to be placed after the references—all tables precede all figures—and should not be included in the body of the text. Each figure and table should begin on a separate page. Do NOT use the “Place Table 5 here” or “Place Figure 1 here” convention. The tables and figures will be placed nearest to where they are mentioned as appropriate when copyediting is done.

Figures and tables should present data to the reader in a clear and unambiguous manner, and should be referred to in the text. If the illustration/table/figure and text are redundant, eliminate the illustration or reduce the amount of detail provided in text. The use of lines in tables is limited (please consult the APA style manual for formatting guidelines ). Figure captions should be placed at the bottom of the figure. One high-quality electronic version of each figure must be submitted with the manuscript. Tables will be typeset. Note that any figures and tables uploaded separately from the main manuscript will still count toward the total 65-page limit.

Italics can be used for emphasis or contrast in special situations but should be used sparingly. Ideally, sentence structure should be used for these issues. All words to be set in italics (e.g., book titles, journal names) should be typed in italics. There should be no underlined text . Abbreviations and acronyms should be spelled out the first time they are mentioned unless they are found as entries in their abbreviated form in Merriam-Webster’s Collegiate Dictionary , 11th ed., 2003 (e.g., “IQ” can be used without being spelled out). Mathematical symbols and symbols for vectors should be clearly formatted in italics and boldface, respectively.

You can use the footnote or endnote feature of Microsoft Word. However, notes are only for explanations or amplifications of textual material that cannot be incorporated into the regular text; they are not for reference information. Moreover, notes are distracting to readers and expensive to produce and should be used sparingly and avoided whenever possible.

The reference list should contain only references that are cited in the text. Its accuracy and completeness are the responsibility of the authors. Reference each publicly available dataset with its title, author, date, and a persistent Web identifier such as a digital object identifier (doi), a handle, or a uniform resource name (URN). If necessary, this last element may be replaced by a web address. Additionally, any references that were included in the analysis but not cited in-text in the main manuscript can be included in a separate reference list that is uploaded as a Supplementary File for Review (this may assist in meeting the page limit).

Authors should anonymize their manuscripts for review . Anonymizing does not mean removing all self-citations. Authors should only anonymize citations of limited circulation (e.g., forthcoming, in press, unpublished) that point to the author. Publications already in the extant literature (e.g., books, book chapters, journal articles) should be cited normally, but authors should include self-citations judiciously . When anonymizing, please use “Author” or “Authors” as in the examples below and place this alphabetically in the reference list and not where the author’s actual name would typically appear.

For examples of common types of references, consult the APA 7th edition manual, or visit the webpage here: https://apastyle.apa.org/style-grammar-guidelines/references

How to Get Help with the Quality of English in Your Submission

Authors who would like to refine the use of English in their manuscripts might consider using the services of a professional English-language editing company. We highlight some of these companies at  https://languageservices.sagepub.com/en/ .

Please be aware that Sage has no affiliation with these companies and makes no endorsement of them. An author's use of these services in no way guarantees that his or her submission will ultimately be accepted. Any arrangement an author enters into will be exclusively between the author and the particular company, and any costs incurred are the sole responsibility of the author.

Copyright Information Accepted authors will be asked to  assign copyright  to AERA, in return for which AERA grants several rights to authors.

Permission to reproduce your own published material

No written or oral permission is necessary to reproduce a table, a figure, or an excerpt of fewer than 500 words from this journal, or to make photocopies for classroom use. Authors are granted permission, without fee, to photocopy their own material or make printouts from the final pdf of their article. Copies must include a full and accurate bibliographic citation and the following credit line: “Copyright [year] by the American Educational Research Association; reproduced with permission from the publisher.” Written permission must be obtained to reproduce or reprint material in circumstances other than those just described. Please review Sage Publishing’s  Journal Permissions  for further information on policies and fees.

Permission to submit material for which you do not own copyright

Authors who wish to use material, such as figures or tables, for which they do not own the copyright must obtain written permission from the copyright holder (usually the publisher) and submit it along with their manuscript. However, no written or oral permission is necessary to reproduce a table, a figure, or an excerpt of fewer than 500 words from an AERA journal.

Copyright transfer agreements for accepted works with more than one author

This journal uses a transfer of copyright agreement that requires just one author (the corresponding author) to sign on behalf of all authors. Please identify the corresponding author for your work when submitting your manuscript for review. The corresponding author will be responsible for the following:

1. Ensuring that all authors are identified on the copyright agreement, and notifying the editorial office of any changes in the authorship.

2. Securing written permission (by letter or e-mail) from each co-author to sign the copyright agreement on the co-author’s behalf.

3. Warranting and indemnifying the journal owner and publisher on behalf of all co-authors. Although such instances are very rare, you should be aware that in the event that a co-author has included content in his or her portion of the article that infringes the copyright of another or is otherwise in violation of any other warranty listed in the agreement, you will be the sole author indemnifying the publisher and the editor of the journal against such violation.

Please contact the publications office at  AERA  if you have questions or if you prefer to use a copyright agreement for all coauthors to sign.

Right of Reply

The right of reply policy encourages comments on recently published articles in AERA publications. They are, of course, subject to the same editorial review and decision process as articles. If the comment is accepted for publication, the editor shall inform the author of the original article. If the author submits a reply to the comment, the reply is also subject to editorial review and decision. The editor may allot a specific amount of journal space for the comment (ordinarily about 1,500 words) and for the reply (ordinarily about 750 words). The reply may appear in the same issue as the comment or in a later issue (Council, June 1980).

If an article is accepted for publication in an AERA journal that, in the judgment of the editor, has as its main theme or thrust a critique of a specific piece of work or a specific line of work associated with an individual or program of research, then the individual or representative of the research program whose work is critiqued should be notified in advance about the upcoming publication and given the opportunity to reply, ideally in the same issue. The author of the original article should also be notified. Normal guidelines for length and review of the reply and publication of a rejoinder by the original article’s author(s) should be followed. Articles in the format “an open letter to …” may constitute prototypical exemplars of the category defined here, but other formats may well be used, and would be included under the qualifications for response prescribed here (Council, January 2002).

Sage Choice and Open Access

If you or your funder wish your article to be freely available online to nonsubscribers immediately upon publication (gold open access), you can opt for it to be included in Sage Choice, subject to payment of a publication fee. The manuscript submission and peer review procedure is unchanged. On acceptance of your article, you will be asked to let Sage know directly if you are choosing Sage Choice. To check journal eligibility and the publication fee, please visit  Sage Choice . For more information on open access options and compliance at Sage, including self author archiving deposits (green open access) visit  Sage Publishing Policies  on our Journal Author Gateway.

• Read Online
• Sample Issues
• Current Issue
• Email Alert
• Permissions
• Foreign rights
• Reprints and sponsorship
• Advertising

Individual Subscription, Combined (Print & E-access)

Institutional Subscription, E-access

Institutional Subscription & Backfile Lease, E-access Plus Backfile (All Online Content)

Institutional Subscription, Print Only

Institutional Subscription, Combined (Print & E-access)

Institutional Subscription & Backfile Lease, Combined Plus Backfile (Current Volume Print & All Online Content)

Institutional Backfile Purchase, E-access (Content through 1998)

Individual, Single Print Issue

Institutional, Single Print Issue

Subscription Information

To purchase a non-standard subscription or a back issue, please contact SAGE Customer Services for availability.

[email protected]  +44 (0) 20 7324 8701

Understanding Artificial Intelligence with the IRB: Recommendations from SACHRP

Understanding artificial intelligence with the irb: recommendations from the secretary's advisory committee on human research protections, as ai is used in research, the united states department of health and human services (hhs) shares considerations to ensure transparency of information between the irb and primary investigators (pis)..

The Secretary's Advisory Committee on Human Research Protections (SACHRP) provides expert advice and recommendations to the Secretary of HHS on issues about the protection of human subjects in research. SACHRP provided several resources, listed below. TC IRB will expand upon, where applicable with specific guidance for TC IRB researchers. 

Under what conditions would a collection of data for AI or AI validation activities meet the Common Rule definition of research that is “designed to develop or contribute to generalizable knowledge”?

• TC IRB Reviewers will ask that ​​researchers identify the technology (AI, bot, etc.) they plan to use for research purposes and what type of engagement that technology will have with human subjects (e.g., recruitment, data collection, data analysis, etc.). For any technology not already approved by TC IT, researchers will have to contact TC IT for data security before TC IRB approval. Researchers may also be asked to include the privacy and confidentiality statement from the technology’s website. IRB Reviewers may ask the PI to list any data or privacy statements on the informed consent form.
• Researchers may also be asked to provide an information sheet to supplement their IRB application and elaborate on the technology in order for TC IRB Reviewers to thoroughly understand and assess any potential risk factors in making human subjects research determinations. A TC IRB Reviewer may also ask the ​​researcher to provide a participant-centered information sheet if the technology is new or nuanced. 

When AI involves research involving private identifiable information (PII), when are those persons human subjects? Does the research capture the “about whom” part of the HS definition? Are there other ethical considerations for these persons?

• TC researchers can review the guide on identifiers and confidentiality considerations .
• TC IRB Reviewers may ask the researcher to present the AI or technology mission statement, confidentiality or privacy statement, or any data use statements that would apply to subject protection. Such language may be reviewed by TC IT or General Counsel to ensure that the technology does not violate any institutional, city, state, or federal policies in place for human subjects research protection. 

When would the collection of data for AI or AI validation activities typically be exempt under the Common Rule?

• For information on exempt review categories, researchers can review this series on the topic .  
• TC IRB Reviewers may ask researchers to elaborate on the use of AI in their studies and to differentiate between instances when the technology would be used closely and indirectly with human subjects (e.g., direct engagement with the AI or sharing of subject information with the AI) or for researcher-led initiatives (e.g., manuscript support)

For studies requiring review under the Common Rule, what human protection action considerations are most prominent for the humans whose information is included in datasets used and shared for AI development? Do those considerations differ where the research is focused on the testing or validating of AI? Are other ethical considerations relevant for those who are not human subjects?

• For information on expedited review categories, researchers can review this series on the topic .
• TC IRB Reviewers aim to mitigate risk in human subjects research. If the researcher uses AI or other technology, but that use/engagement does not involve a human or the human’s data it may not need IRB oversight. TC IRB Reviewers need to understand the parameters of the study when technology is/is not used and how that use may impact a study subject. Depending on the scenario IRB may or may not be needed. Describing these nuances will aid the TC IRB Reviewer in their determinations. 

Are there existing frameworks or tools that funding agencies, investigators, Human Research Protection Program (HRPP) staff, and IRBs can use to illuminate and mitigate ethical concerns with human-focused AI research and development?

• Researchers should consult TC IRB Reviewers and any grantors who may be involved in the research project to discuss the parameters of the study and risk mitigation strategies.

Are there considerations specific to AI that impact the adequacy of disclosure of research activities in the research informed consent form.

• Visit these resources for a guide on informed consent forms .

What is “unique” about research that includes AI that would require the IRB to think about and determine the applicability of the Common Rule that isn’t already considered for all human subject’s research?

• Researchers may be asked to provide an information sheet or meet with TC IRB Reviewers to clarify the proposed AI use and how it weaves into the study protocol. 

What specific sections of 45 CFR 46.111 would need special attention in research with AI; i.e., privacy and confidentiality; informed and consent; risks?

• Researchers can review our guide to understand risks in research . 

What are the specific considerations regarding AI that are pertinent to institutional /HRPP responsibilities, versus responsibilities for other studies under the purview of the IRB?

• Researchers should continually uphold the highest ethical standards in research with human subjects and report any protocol deviations or adverse events promptly. 

Is there a larger potential for bias and/or flaws in the use of AI in research and how should IRBs think about this potential in their review? (i.e. facial recognition algorithms could be heavily based on dominant populations, but the researchers “using the algorithm” might not be aware of this.)

• Researchers should consider the ethical ramifications of the use of AI and include any potential risks in the informed consent form or information sheets presented to the participants. 

AI strives to be a reliable tool for researchers but there are still many facets that have not been fully analyzed. To protect the reputation and integrity of research, AI in its current state must be used sparingly and with extreme caution. For further information about the ethics of using AI in research, please refer to the next part of the series: Ethics and Risks Involved Using Artificial Intelligence (AI) in Research . The IRB’s most significant priority is the protection of human subjects, which includes securing their data and reducing risks in all research studies. As AI continues to develop and improve, PIs may soon encounter sturdy and trustworthy technologies that ensure the safety of their participants. 

Office for Human Research Protections (2022, August 26) . Considerations for IRB review of  research involving artificial intelligence. United States Department of Health and Human Services. https://www.hhs.gov/ohrp/sachrp-committee/recommendations/attachment-e-july-25-2022-letter/index.html

— Diana Bae, B.A. & Myra Luna-Lucero, Ed.D.

Published Tuesday, Apr 30, 2024

Institutional Review Board

Address: Russell Hall, Room 13

* Phone: 212-678-4105 * Email:   [email protected]

Appointments are available by request . Make sure to have your IRB protocol number (e.g., 19-011) available.  If you are unable to access any of the downloadable resources, please contact  OASID via email [email protected] .