research on impact of technology in education

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• My Account Login
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Review Article
• Open access
• Published: 12 February 2024

Education reform and change driven by digital technology: a bibliometric study from a global perspective

• Chengliang Wang 1 ,
• Xiaojiao Chen 1 ,
• Teng Yu   ORCID: orcid.org/0000-0001-5198-7261 2 , 3 ,
• Yidan Liu 1 , 4 &
• Yuhui Jing 1  

Humanities and Social Sciences Communications volume  11 , Article number:  256 ( 2024 ) Cite this article

6235 Accesses

1 Citations

1 Altmetric

Metrics details

• Development studies
• Science, technology and society

Amidst the global digital transformation of educational institutions, digital technology has emerged as a significant area of interest among scholars. Such technologies have played an instrumental role in enhancing learner performance and improving the effectiveness of teaching and learning. These digital technologies also ensure the sustainability and stability of education during the epidemic. Despite this, a dearth of systematic reviews exists regarding the current state of digital technology application in education. To address this gap, this study utilized the Web of Science Core Collection as a data source (specifically selecting the high-quality SSCI and SCIE) and implemented a topic search by setting keywords, yielding 1849 initial publications. Furthermore, following the PRISMA guidelines, we refined the selection to 588 high-quality articles. Using software tools such as CiteSpace, VOSviewer, and Charticulator, we reviewed these 588 publications to identify core authors (such as Selwyn, Henderson, Edwards), highly productive countries/regions (England, Australia, USA), key institutions (Monash University, Australian Catholic University), and crucial journals in the field ( Education and Information Technologies , Computers & Education , British Journal of Educational Technology ). Evolutionary analysis reveals four developmental periods in the research field of digital technology education application: the embryonic period, the preliminary development period, the key exploration, and the acceleration period of change. The study highlights the dual influence of technological factors and historical context on the research topic. Technology is a key factor in enabling education to transform and upgrade, and the context of the times is an important driving force in promoting the adoption of new technologies in the education system and the transformation and upgrading of education. Additionally, the study identifies three frontier hotspots in the field: physical education, digital transformation, and professional development under the promotion of digital technology. This study presents a clear framework for digital technology application in education, which can serve as a valuable reference for researchers and educational practitioners concerned with digital technology education application in theory and practice.

Similar content being viewed by others

A bibliometric analysis of knowledge mapping in Chinese education digitalization research from 2012 to 2022

Digital transformation and digital literacy in the context of complexity within higher education institutions: a systematic literature review

Education big data and learning analytics: a bibliometric analysis

Introduction.

Digital technology has become an essential component of modern education, facilitating the extension of temporal and spatial boundaries and enriching the pedagogical contexts (Selwyn and Facer, 2014 ). The advent of mobile communication technology has enabled learning through social media platforms (Szeto et al. 2015 ; Pires et al. 2022 ), while the advancement of augmented reality technology has disrupted traditional conceptions of learning environments and spaces (Perez-Sanagustin et al., 2014 ; Kyza and Georgiou, 2018 ). A wide range of digital technologies has enabled learning to become a norm in various settings, including the workplace (Sjöberg and Holmgren, 2021 ), home (Nazare et al. 2022 ), and online communities (Tang and Lam, 2014 ). Education is no longer limited to fixed locations and schedules, but has permeated all aspects of life, allowing learning to continue at any time and any place (Camilleri and Camilleri, 2016 ; Selwyn and Facer, 2014 ).

The advent of digital technology has led to the creation of several informal learning environments (Greenhow and Lewin, 2015 ) that exhibit divergent form, function, features, and patterns in comparison to conventional learning environments (Nygren et al. 2019 ). Consequently, the associated teaching and learning processes, as well as the strategies for the creation, dissemination, and acquisition of learning resources, have undergone a complete overhaul. The ensuing transformations have posed a myriad of novel issues, such as the optimal structuring of teaching methods by instructors and the adoption of appropriate learning strategies by students in the new digital technology environment. Consequently, an examination of the principles that underpin effective teaching and learning in this environment is a topic of significant interest to numerous scholars engaged in digital technology education research.

Over the course of the last two decades, digital technology has made significant strides in the field of education, notably in extending education time and space and creating novel educational contexts with sustainability. Despite research attempts to consolidate the application of digital technology in education, previous studies have only focused on specific aspects of digital technology, such as Pinto and Leite’s ( 2020 ) investigation into digital technology in higher education and Mustapha et al.’s ( 2021 ) examination of the role and value of digital technology in education during the pandemic. While these studies have provided valuable insights into the practical applications of digital technology in particular educational domains, they have not comprehensively explored the macro-mechanisms and internal logic of digital technology implementation in education. Additionally, these studies were conducted over a relatively brief period, making it challenging to gain a comprehensive understanding of the macro-dynamics and evolutionary process of digital technology in education. Some studies have provided an overview of digital education from an educational perspective but lack a precise understanding of technological advancement and change (Yang et al. 2022 ). Therefore, this study seeks to employ a systematic scientific approach to collate relevant research from 2000 to 2022, comprehend the internal logic and development trends of digital technology in education, and grasp the outstanding contribution of digital technology in promoting the sustainability of education in time and space. In summary, this study aims to address the following questions:

RQ1: Since the turn of the century, what is the productivity distribution of the field of digital technology education application research in terms of authorship, country/region, institutional and journal level?

RQ2: What is the development trend of research on the application of digital technology in education in the past two decades?

RQ3: What are the current frontiers of research on the application of digital technology in education?

Literature review

Although the term “digital technology” has become ubiquitous, a unified definition has yet to be agreed upon by scholars. Because the meaning of the word digital technology is closely related to the specific context. Within the educational research domain, Selwyn’s ( 2016 ) definition is widely favored by scholars (Pinto and Leite, 2020 ). Selwyn ( 2016 ) provides a comprehensive view of various concrete digital technologies and their applications in education through ten specific cases, such as immediate feedback in classes, orchestrating teaching, and community learning. Through these specific application scenarios, Selwyn ( 2016 ) argues that digital technology encompasses technologies associated with digital devices, including but not limited to tablets, smartphones, computers, and social media platforms (such as Facebook and YouTube). Furthermore, Further, the behavior of accessing the internet at any location through portable devices can be taken as an extension of the behavior of applying digital technology.

The evolving nature of digital technology has significant implications in the field of education. In the 1890s, the focus of digital technology in education was on comprehending the nuances of digital space, digital culture, and educational methodologies, with its connotations aligned more towards the idea of e-learning. The advent and subsequent widespread usage of mobile devices since the dawn of the new millennium have been instrumental in the rapid expansion of the concept of digital technology. Notably, mobile learning devices such as smartphones and tablets, along with social media platforms, have become integral components of digital technology (Conole and Alevizou, 2010 ; Batista et al. 2016 ). In recent times, the burgeoning application of AI technology in the education sector has played a vital role in enriching the digital technology lexicon (Banerjee et al. 2021 ). ChatGPT, for instance, is identified as a novel educational technology that has immense potential to revolutionize future education (Rospigliosi, 2023 ; Arif, Munaf and Ul-Haque, 2023 ).

Pinto and Leite ( 2020 ) conducted a comprehensive macroscopic survey of the use of digital technologies in the education sector and identified three distinct categories, namely technologies for assessment and feedback, mobile technologies, and Information Communication Technologies (ICT). This classification criterion is both macroscopic and highly condensed. In light of the established concept definitions of digital technology in the educational research literature, this study has adopted the characterizations of digital technology proposed by Selwyn ( 2016 ) and Pinto and Leite ( 2020 ) as crucial criteria for analysis and research inclusion. Specifically, this criterion encompasses several distinct types of digital technologies, including Information and Communication Technologies (ICT), Mobile tools, eXtended Reality (XR) Technologies, Assessment and Feedback systems, Learning Management Systems (LMS), Publish and Share tools, Collaborative systems, Social media, Interpersonal Communication tools, and Content Aggregation tools.

Methodology and materials

Research method: bibliometric.

The research on econometric properties has been present in various aspects of human production and life, yet systematic scientific theoretical guidance has been lacking, resulting in disorganization. In 1969, British scholar Pritchard ( 1969 ) proposed “bibliometrics,” which subsequently emerged as an independent discipline in scientific quantification research. Initially, Pritchard defined bibliometrics as “the application of mathematical and statistical methods to books and other media of communication,” however, the definition was not entirely rigorous. To remedy this, Hawkins ( 2001 ) expanded Pritchard’s definition to “the quantitative analysis of the bibliographic features of a body of literature.” De Bellis further clarified the objectives of bibliometrics, stating that it aims to analyze and identify patterns in literature, such as the most productive authors, institutions, countries, and journals in scientific disciplines, trends in literary production over time, and collaboration networks (De Bellis, 2009 ). According to Garfield ( 2006 ), bibliometric research enables the examination of the history and structure of a field, the flow of information within the field, the impact of journals, and the citation status of publications over a longer time scale. All of these definitions illustrate the unique role of bibliometrics as a research method for evaluating specific research fields.

This study uses CiteSpace, VOSviewer, and Charticulator to analyze data and create visualizations. Each of these three tools has its own strengths and can complement each other. CiteSpace and VOSviewer use set theory and probability theory to provide various visualization views in fields such as keywords, co-occurrence, and co-authors. They are easy to use and produce visually appealing graphics (Chen, 2006 ; van Eck and Waltman, 2009 ) and are currently the two most widely used bibliometric tools in the field of visualization (Pan et al. 2018 ). In this study, VOSviewer provided the data necessary for the Performance Analysis; Charticulator was then used to redraw using the tabular data exported from VOSviewer (for creating the chord diagram of country collaboration); this was to complement the mapping process, while CiteSpace was primarily utilized to generate keyword maps and conduct burst word analysis.

Data retrieval

This study selected documents from the Science Citation Index Expanded (SCIE) and Social Science Citation Index (SSCI) in the Web of Science Core Collection as the data source, for the following reasons:

(1) The Web of Science Core Collection, as a high-quality digital literature resource database, has been widely accepted by many researchers and is currently considered the most suitable database for bibliometric analysis (Jing et al. 2023a ). Compared to other databases, Web of Science provides more comprehensive data information (Chen et al. 2022a ), and also provides data formats suitable for analysis using VOSviewer and CiteSpace (Gaviria-Marin et al. 2019 ).

(2) The application of digital technology in the field of education is an interdisciplinary research topic, involving technical knowledge literature belonging to the natural sciences and education-related literature belonging to the social sciences. Therefore, it is necessary to select Science Citation Index Expanded (SCIE) and Social Science Citation Index (SSCI) as the sources of research data, ensuring the comprehensiveness of data while ensuring the reliability and persuasiveness of bibliometric research (Hwang and Tsai, 2011 ; Wang et al. 2022 ).

After establishing the source of research data, it is necessary to determine a retrieval strategy (Jing et al. 2023b ). The choice of a retrieval strategy should consider a balance between the breadth and precision of the search formula. That is to say, it should encompass all the literature pertaining to the research topic while excluding irrelevant documents as much as possible. In light of this, this study has set a retrieval strategy informed by multiple related papers (Mustapha et al. 2021 ; Luo et al. 2021 ). The research by Mustapha et al. ( 2021 ) guided us in selecting keywords (“digital” AND “technolog*”) to target digital technology, while Luo et al. ( 2021 ) informed the selection of terms (such as “instruct*,” “teach*,” and “education”) to establish links with the field of education. Then, based on the current application of digital technology in the educational domain and the scope of selection criteria, we constructed the final retrieval strategy. Following the general patterns of past research (Jing et al. 2023a , 2023b ), we conducted a specific screening using the topic search (Topics, TS) function in Web of Science. For the specific criteria used in the screening for this study, please refer to Table 1 .

Literature screening

Literature acquired through keyword searches may contain ostensibly related yet actually unrelated works. Therefore, to ensure the close relevance of literature included in the analysis to the research topic, it is often necessary to perform a manual screening process to identify the final literature to be analyzed, subsequent to completing the initial literature search.

The manual screening process consists of two steps. Initially, irrelevant literature is weeded out based on the title and abstract, with two members of the research team involved in this phase. This stage lasted about one week, resulting in 1106 articles being retained. Subsequently, a comprehensive review of the full text is conducted to accurately identify the literature required for the study. To carry out the second phase of manual screening effectively and scientifically, and to minimize the potential for researcher bias, the research team established the inclusion criteria presented in Table 2 . Three members were engaged in this phase, which took approximately 2 weeks, culminating in the retention of 588 articles after meticulous screening. The entire screening process is depicted in Fig. 1 , adhering to the PRISMA guidelines (Page et al. 2021 ).

The process of obtaining and filtering the necessary literature data for research.

Data standardization

Nguyen and Hallinger ( 2020 ) pointed out that raw data extracted from scientific databases often contains multiple expressions of the same term, and not addressing these synonymous expressions could affect research results in bibliometric analysis. For instance, in the original data, the author list may include “Tsai, C. C.” and “Tsai, C.-C.”, while the keyword list may include “professional-development” and “professional development,” which often require merging. Therefore, before analyzing the selected literature, a data disambiguation process is necessary to standardize the data (Strotmann and Zhao, 2012 ; Van Eck and Waltman, 2019 ). This study adopted the data standardization process proposed by Taskin and Al ( 2019 ), mainly including the following standardization operations:

Firstly, the author and source fields in the data are corrected and standardized to differentiate authors with similar names.

Secondly, the study checks whether the journals to which the literature belongs have been renamed in the past over 20 years, so as to avoid the influence of periodical name change on the analysis results.

Finally, the keyword field is standardized by unifying parts of speech and singular/plural forms of keywords, which can help eliminate redundant entries in the knowledge graph.

Performance analysis (RQ1)

This section offers a thorough and detailed analysis of the state of research in the field of digital technology education. By utilizing descriptive statistics and visual maps, it provides a comprehensive overview of the development trends, authors, countries, institutions, and journal distribution within the field. The insights presented in this section are of great significance in advancing our understanding of the current state of research in this field and identifying areas for further investigation. The use of visual aids to display inter-country cooperation and the evolution of the field adds to the clarity and coherence of the analysis.

Time trend of the publications

To understand a research field, it is first necessary to understand the most basic quantitative information, among which the change in the number of publications per year best reflects the development trend of a research field. Figure 2 shows the distribution of publication dates.

Time trend of the publications on application of digital technology in education.

From the Fig. 2 , it can be seen that the development of this field over the past over 20 years can be roughly divided into three stages. The first stage was from 2000 to 2007, during which the number of publications was relatively low. Due to various factors such as technological maturity, the academic community did not pay widespread attention to the role of digital technology in expanding the scope of teaching and learning. The second stage was from 2008 to 2019, during which the overall number of publications showed an upward trend, and the development of the field entered an accelerated period, attracting more and more scholars’ attention. The third stage was from 2020 to 2022, during which the number of publications stabilized at around 100. During this period, the impact of the pandemic led to a large number of scholars focusing on the role of digital technology in education during the pandemic, and research on the application of digital technology in education became a core topic in social science research.

Analysis of authors

An analysis of the author’s publication volume provides information about the representative scholars and core research strengths of a research area. Table 3 presents information on the core authors in adaptive learning research, including name, publication number, and average number of citations per article (based on the analysis and statistics from VOSviewer).

Variations in research foci among scholars abound. Within the field of digital technology education application research over the past two decades, Neil Selwyn stands as the most productive author, having published 15 papers garnering a total of 1027 citations, resulting in an average of 68.47 citations per paper. As a Professor at the Faculty of Education at Monash University, Selwyn concentrates on exploring the application of digital technology in higher education contexts (Selwyn et al. 2021 ), as well as related products in higher education such as Coursera, edX, and Udacity MOOC platforms (Bulfin et al. 2014 ). Selwyn’s contributions to the educational sociology perspective include extensive research on the impact of digital technology on education, highlighting the spatiotemporal extension of educational processes and practices through technological means as the greatest value of educational technology (Selwyn, 2012 ; Selwyn and Facer, 2014 ). In addition, he provides a blueprint for the development of future schools in 2030 based on the present impact of digital technology on education (Selwyn et al. 2019 ). The second most productive author in this field, Henderson, also offers significant contributions to the understanding of the important value of digital technology in education, specifically in the higher education setting, with a focus on the impact of the pandemic (Henderson et al. 2015 ; Cohen et al. 2022 ). In contrast, Edwards’ research interests focus on early childhood education, particularly the application of digital technology in this context (Edwards, 2013 ; Bird and Edwards, 2015 ). Additionally, on the technical level, Edwards also mainly prefers digital game technology, because it is a digital technology that children are relatively easy to accept (Edwards, 2015 ).

Analysis of countries/regions and organization

The present study aimed to ascertain the leading countries in digital technology education application research by analyzing 75 countries related to 558 works of literature. Table 4 depicts the top ten countries that have contributed significantly to this field in terms of publication count (based on the analysis and statistics from VOSviewer). Our analysis of Table 4 data shows that England emerged as the most influential country/region, with 92 published papers and 2401 citations. Australia and the United States secured the second and third ranks, respectively, with 90 papers (2187 citations) and 70 papers (1331 citations) published. Geographically, most of the countries featured in the top ten publication volumes are situated in Australia, North America, and Europe, with China being the only exception. Notably, all these countries, except China, belong to the group of developed nations, suggesting that economic strength is a prerequisite for fostering research in the digital technology education application field.

This study presents a visual representation of the publication output and cooperation relationships among different countries in the field of digital technology education application research. Specifically, a chord diagram is employed to display the top 30 countries in terms of publication output, as depicted in Fig. 3 . The chord diagram is composed of nodes and chords, where the nodes are positioned as scattered points along the circumference, and the length of each node corresponds to the publication output, with longer lengths indicating higher publication output. The chords, on the other hand, represent the cooperation relationships between any two countries, and are weighted based on the degree of closeness of the cooperation, with wider chords indicating closer cooperation. Through the analysis of the cooperation relationships, the findings suggest that the main publishing countries in this field are engaged in cooperative relationships with each other, indicating a relatively high level of international academic exchange and research internationalization.

In the diagram, nodes are scattered along the circumference of a circle, with the length of each node representing the volume of publications. The weighted arcs connecting any two points on the circle are known as chords, representing the collaborative relationship between the two, with the width of the arc indicating the closeness of the collaboration.

Further analyzing Fig. 3 , we can extract more valuable information, enabling a deeper understanding of the connections between countries in the research field of digital technology in educational applications. It is evident that certain countries, such as the United States, China, and England, display thicker connections, indicating robust collaborative relationships in terms of productivity. These thicker lines signify substantial mutual contributions and shared objectives in certain sectors or fields, highlighting the interconnectedness and global integration in these areas. By delving deeper, we can also explore potential future collaboration opportunities through the chord diagram, identifying possible partners to propel research and development in this field. In essence, the chord diagram successfully encapsulates and conveys the multi-dimensionality of global productivity and cooperation, allowing for a comprehensive understanding of the intricate inter-country relationships and networks in a global context, providing valuable guidance and insights for future research and collaborations.

An in-depth examination of the publishing institutions is provided in Table 5 , showcasing the foremost 10 institutions ranked by their publication volume. Notably, Monash University and Australian Catholic University, situated in Australia, have recorded the most prolific publications within the digital technology education application realm, with 22 and 10 publications respectively. Moreover, the University of Oslo from Norway is featured among the top 10 publishing institutions, with an impressive average citation count of 64 per publication. It is worth highlighting that six institutions based in the United Kingdom were also ranked within the top 10 publishing institutions, signifying their leading position in this area of research.

Analysis of journals

Journals are the main carriers for publishing high-quality papers. Some scholars point out that the two key factors to measure the influence of journals in the specified field are the number of articles published and the number of citations. The more papers published in a magazine and the more citations, the greater its influence (Dzikowski, 2018 ). Therefore, this study utilized VOSviewer to statistically analyze the top 10 journals with the most publications in the field of digital technology in education and calculated the average citations per article (see Table 6 ).

Based on Table 6 , it is apparent that the highest number of articles in the domain of digital technology in education research were published in Education and Information Technologies (47 articles), Computers & Education (34 articles), and British Journal of Educational Technology (32 articles), indicating a higher article output compared to other journals. This underscores the fact that these three journals concentrate more on the application of digital technology in education. Furthermore, several other journals, such as Technology Pedagogy and Education and Sustainability, have published more than 15 articles in this domain. Sustainability represents the open access movement, which has notably facilitated research progress in this field, indicating that the development of open access journals in recent years has had a significant impact. Although there is still considerable disagreement among scholars on the optimal approach to achieve open access, the notion that research outcomes should be accessible to all is widely recognized (Huang et al. 2020 ). On further analysis of the research fields to which these journals belong, except for Sustainability, it is evident that they all pertain to educational technology, thus providing a qualitative definition of the research area of digital technology education from the perspective of journals.

Temporal keyword analysis: thematic evolution (RQ2)

The evolution of research themes is a dynamic process, and previous studies have attempted to present the developmental trajectory of fields by drawing keyword networks in phases (Kumar et al. 2021 ; Chen et al. 2022b ). To understand the shifts in research topics across different periods, this study follows past research and, based on the significant changes in the research field and corresponding technological advancements during the outlined periods, divides the timeline into four stages (the first stage from January 2000 to December 2005, the second stage from January 2006 to December 2011, the third stage from January 2012 to December 2017; and the fourth stage from January 2018 to December 2022). The division into these four stages was determined through a combination of bibliometric analysis and literature review, which presented a clear trajectory of the field’s development. The research analyzes the keyword networks for each time period (as there are only three articles in the first stage, it was not possible to generate an appropriate keyword co-occurrence map, hence only the keyword co-occurrence maps from the second to the fourth stages are provided), to understand the evolutionary track of the digital technology education application research field over time.

2000.1–2005.12: germination period

From January 2000 to December 2005, digital technology education application research was in its infancy. Only three studies focused on digital technology, all of which were related to computers. Due to the popularity of computers, the home became a new learning environment, highlighting the important role of digital technology in expanding the scope of learning spaces (Sutherland et al. 2000 ). In specific disciplines and contexts, digital technology was first favored in medical clinical practice, becoming an important tool for supporting the learning of clinical knowledge and practice (Tegtmeyer et al. 2001 ; Durfee et al. 2003 ).

2006.1–2011.12: initial development period

Between January 2006 and December 2011, it was the initial development period of digital technology education research. Significant growth was observed in research related to digital technology, and discussions and theoretical analyses about “digital natives” emerged. During this phase, scholars focused on the debate about “how to use digital technology reasonably” and “whether current educational models and school curriculum design need to be adjusted on a large scale” (Bennett and Maton, 2010 ; Selwyn, 2009 ; Margaryan et al. 2011 ). These theoretical and speculative arguments provided a unique perspective on the impact of cognitive digital technology on education and teaching. As can be seen from the vocabulary such as “rethinking”, “disruptive pedagogy”, and “attitude” in Fig. 4 , many scholars joined the calm reflection and analysis under the trend of digital technology (Laurillard, 2008 ; Vratulis et al. 2011 ). During this phase, technology was still undergoing dramatic changes. The development of mobile technology had already caught the attention of many scholars (Wong et al. 2011 ), but digital technology represented by computers was still very active (Selwyn et al. 2011 ). The change in technological form would inevitably lead to educational transformation. Collins and Halverson ( 2010 ) summarized the prospects and challenges of using digital technology for learning and educational practices, believing that digital technology would bring a disruptive revolution to the education field and bring about a new educational system. In addition, the term “teacher education” in Fig. 4 reflects the impact of digital technology development on teachers. The rapid development of technology has widened the generation gap between teachers and students. To ensure smooth communication between teachers and students, teachers must keep up with the trend of technological development and establish a lifelong learning concept (Donnison, 2009 ).

In the diagram, each node represents a keyword, with the size of the node indicating the frequency of occurrence of the keyword. The connections represent the co-occurrence relationships between keywords, with a higher frequency of co-occurrence resulting in tighter connections.

2012.1–2017.12: critical exploration period

During the period spanning January 2012 to December 2017, the application of digital technology in education research underwent a significant exploration phase. As can be seen from Fig. 5 , different from the previous stage, the specific elements of specific digital technology have started to increase significantly, including the enrichment of technological contexts, the greater variety of research methods, and the diversification of learning modes. Moreover, the temporal and spatial dimensions of the learning environment were further de-emphasized, as noted in previous literature (Za et al. 2014 ). Given the rapidly accelerating pace of technological development, the education system in the digital era is in urgent need of collaborative evolution and reconstruction, as argued by Davis, Eickelmann, and Zaka ( 2013 ).

In the domain of digital technology, social media has garnered substantial scholarly attention as a promising avenue for learning, as noted by Pasquini and Evangelopoulos ( 2016 ). The implementation of social media in education presents several benefits, including the liberation of education from the restrictions of physical distance and time, as well as the erasure of conventional educational boundaries. The user-generated content (UGC) model in social media has emerged as a crucial source for knowledge creation and distribution, with the widespread adoption of mobile devices. Moreover, social networks have become an integral component of ubiquitous learning environments (Hwang et al. 2013 ). The utilization of social media allows individuals to function as both knowledge producers and recipients, which leads to a blurring of the conventional roles of learners and teachers. On mobile platforms, the roles of learners and teachers are not fixed, but instead interchangeable.

In terms of research methodology, the prevalence of empirical studies with survey designs in the field of educational technology during this period is evident from the vocabulary used, such as “achievement,” “acceptance,” “attitude,” and “ict.” in Fig. 5 . These studies aim to understand learners’ willingness to adopt and attitudes towards new technologies, and some seek to investigate the impact of digital technologies on learning outcomes through quasi-experimental designs (Domínguez et al. 2013 ). Among these empirical studies, mobile learning emerged as a hot topic, and this is not surprising. First, the advantages of mobile learning environments over traditional ones have been empirically demonstrated (Hwang et al. 2013 ). Second, learners born around the turn of the century have been heavily influenced by digital technologies and have developed their own learning styles that are more open to mobile devices as a means of learning. Consequently, analyzing mobile learning as a relatively novel mode of learning has become an important issue for scholars in the field of educational technology.

The intervention of technology has led to the emergence of several novel learning modes, with the blended learning model being the most representative one in the current phase. Blended learning, a novel concept introduced in the information age, emphasizes the integration of the benefits of traditional learning methods and online learning. This learning mode not only highlights the prominent role of teachers in guiding, inspiring, and monitoring the learning process but also underlines the importance of learners’ initiative, enthusiasm, and creativity in the learning process. Despite being an early conceptualization, blended learning’s meaning has been expanded by the widespread use of mobile technology and social media in education. The implementation of new technologies, particularly mobile devices, has resulted in the transformation of curriculum design and increased flexibility and autonomy in students’ learning processes (Trujillo Maza et al. 2016 ), rekindling scholarly attention to this learning mode. However, some scholars have raised concerns about the potential drawbacks of the blended learning model, such as its significant impact on the traditional teaching system, the lack of systematic coping strategies and relevant policies in several schools and regions (Moskal et al. 2013 ).

2018.1–2022.12: accelerated transformation period

The period spanning from January 2018 to December 2022 witnessed a rapid transformation in the application of digital technology in education research. The field of digital technology education research reached a peak period of publication, largely influenced by factors such as the COVID-19 pandemic (Yu et al. 2023 ). Research during this period was built upon the achievements, attitudes, and social media of the previous phase, and included more elements that reflect the characteristics of this research field, such as digital literacy, digital competence, and professional development, as depicted in Fig. 6 . Alongside this, scholars’ expectations for the value of digital technology have expanded, and the pursuit of improving learning efficiency and performance is no longer the sole focus. Some research now aims to cultivate learners’ motivation and enhance their self-efficacy by applying digital technology in a reasonable manner, as demonstrated by recent studies (Beardsley et al. 2021 ; Creely et al. 2021 ).

The COVID-19 pandemic has emerged as a crucial backdrop for the digital technology’s role in sustaining global education, as highlighted by recent scholarly research (Zhou et al. 2022 ; Pan and Zhang, 2020 ; Mo et al. 2022 ). The online learning environment, which is supported by digital technology, has become the primary battleground for global education (Yu, 2022 ). This social context has led to various studies being conducted, with some scholars positing that the pandemic has impacted the traditional teaching order while also expanding learning possibilities in terms of patterns and forms (Alabdulaziz, 2021 ). Furthermore, the pandemic has acted as a catalyst for teacher teaching and technological innovation, and this viewpoint has been empirically substantiated (Moorhouse and Wong, 2021 ). Additionally, some scholars believe that the pandemic’s push is a crucial driving force for the digital transformation of the education system, serving as an essential mechanism for overcoming the system’s inertia (Romero et al. 2021 ).

The rapid outbreak of the pandemic posed a challenge to the large-scale implementation of digital technologies, which was influenced by a complex interplay of subjective and objective factors. Objective constraints included the lack of infrastructure in some regions to support digital technologies, while subjective obstacles included psychological resistance among certain students and teachers (Moorhouse, 2021 ). These factors greatly impacted the progress of online learning during the pandemic. Additionally, Timotheou et al. ( 2023 ) conducted a comprehensive systematic review of existing research on digital technology use during the pandemic, highlighting the critical role played by various factors such as learners’ and teachers’ digital skills, teachers’ personal attributes and professional development, school leadership and management, and administration in facilitating the digitalization and transformation of schools.

The current stage of research is characterized by the pivotal term “digital literacy,” denoting a growing interest in learners’ attitudes and adoption of emerging technologies. Initially, the term “literacy” was restricted to fundamental abilities and knowledge associated with books and print materials (McMillan, 1996 ). However, with the swift advancement of computers and digital technology, there have been various attempts to broaden the scope of literacy beyond its traditional meaning, including game literacy (Buckingham and Burn, 2007 ), information literacy (Eisenberg, 2008 ), and media literacy (Turin and Friesem, 2020 ). Similarly, digital literacy has emerged as a crucial concept, and Gilster and Glister ( 1997 ) were the first to introduce this concept, referring to the proficiency in utilizing technology and processing digital information in academic, professional, and daily life settings. In practical educational settings, learners who possess higher digital literacy often exhibit an aptitude for quickly mastering digital devices and applying them intelligently to education and teaching (Yu, 2022 ).

The utilization of digital technology in education has undergone significant changes over the past two decades, and has been a crucial driver of educational reform with each new technological revolution. The impact of these changes on the underlying logic of digital technology education applications has been noticeable. From computer technology to more recent developments such as virtual reality (VR), augmented reality (AR), and artificial intelligence (AI), the acceleration in digital technology development has been ongoing. Educational reforms spurred by digital technology development continue to be dynamic, as each new digital innovation presents new possibilities and models for teaching practice. This is especially relevant in the post-pandemic era, where the importance of technological progress in supporting teaching cannot be overstated (Mughal et al. 2022 ). Existing digital technologies have already greatly expanded the dimensions of education in both time and space, while future digital technologies aim to expand learners’ perceptions. Researchers have highlighted the potential of integrated technology and immersive technology in the development of the educational metaverse, which is highly anticipated to create a new dimension for the teaching and learning environment, foster a new value system for the discipline of educational technology, and more effectively and efficiently achieve the grand educational blueprint of the United Nations’ Sustainable Development Goals (Zhang et al. 2022 ; Li and Yu, 2023 ).

Hotspot evolution analysis (RQ3)

The examination of keyword evolution reveals a consistent trend in the advancement of digital technology education application research. The emergence and transformation of keywords serve as indicators of the varying research interests in this field. Thus, the utilization of the burst detection function available in CiteSpace allowed for the identification of the top 10 burst words that exhibited a high level of burst strength. This outcome is illustrated in Table 7 .

According to the results presented in Table 7 , the explosive terminology within the realm of digital technology education research has exhibited a concentration mainly between the years 2018 and 2022. Prior to this time frame, the emerging keywords were limited to “information technology” and “computer”. Notably, among them, computer, as an emergent keyword, has always had a high explosive intensity from 2008 to 2018, which reflects the important position of computer in digital technology and is the main carrier of many digital technologies such as Learning Management Systems (LMS) and Assessment and Feedback systems (Barlovits et al. 2022 ).

Since 2018, an increasing number of research studies have focused on evaluating the capabilities of learners to accept, apply, and comprehend digital technologies. As indicated by the use of terms such as “digital literacy” and “digital skill,” the assessment of learners’ digital literacy has become a critical task. Scholarly efforts have been directed towards the development of literacy assessment tools and the implementation of empirical assessments. Furthermore, enhancing the digital literacy of both learners and educators has garnered significant attention. (Nagle, 2018 ; Yu, 2022 ). Simultaneously, given the widespread use of various digital technologies in different formal and informal learning settings, promoting learners’ digital skills has become a crucial objective for contemporary schools (Nygren et al. 2019 ; Forde and OBrien, 2022 ).

Since 2020, the field of applied research on digital technology education has witnessed the emergence of three new hotspots, all of which have been affected to some extent by the pandemic. Firstly, digital technology has been widely applied in physical education, which is one of the subjects that has been severely affected by the pandemic (Parris et al. 2022 ; Jiang and Ning, 2022 ). Secondly, digital transformation has become an important measure for most schools, especially higher education institutions, to cope with the impact of the pandemic globally (García-Morales et al. 2021 ). Although the concept of digital transformation was proposed earlier, the COVID-19 pandemic has greatly accelerated this transformation process. Educational institutions must carefully redesign their educational products to face this new situation, providing timely digital learning methods, environments, tools, and support systems that have far-reaching impacts on modern society (Krishnamurthy, 2020 ; Salas-Pilco et al. 2022 ). Moreover, the professional development of teachers has become a key mission of educational institutions in the post-pandemic era. Teachers need to have a certain level of digital literacy and be familiar with the tools and online teaching resources used in online teaching, which has become a research hotspot today. Organizing digital skills training for teachers to cope with the application of emerging technologies in education is an important issue for teacher professional development and lifelong learning (Garzón-Artacho et al. 2021 ). As the main organizers and practitioners of emergency remote teaching (ERT) during the pandemic, teachers must put cognitive effort into their professional development to ensure effective implementation of ERT (Romero-Hall and Jaramillo Cherrez, 2022 ).

The burst word “digital transformation” reveals that we are in the midst of an ongoing digital technology revolution. With the emergence of innovative digital technologies such as ChatGPT and Microsoft 365 Copilot, technology trends will continue to evolve, albeit unpredictably. While the impact of these advancements on school education remains uncertain, it is anticipated that the widespread integration of technology will significantly affect the current education system. Rejecting emerging technologies without careful consideration is unwise. Like any revolution, the technological revolution in the education field has both positive and negative aspects. Detractors argue that digital technology disrupts learning and memory (Baron, 2021 ) or causes learners to become addicted and distracted from learning (Selwyn and Aagaard, 2020 ). On the other hand, the prudent use of digital technology in education offers a glimpse of a golden age of open learning. Educational leaders and practitioners have the opportunity to leverage cutting-edge digital technologies to address current educational challenges and develop a rational path for the sustainable and healthy growth of education.

Discussion on performance analysis (RQ1)

The field of digital technology education application research has experienced substantial growth since the turn of the century, a phenomenon that is quantifiably apparent through an analysis of authorship, country/region contributions, and institutional engagement. This expansion reflects the increased integration of digital technologies in educational settings and the heightened scholarly interest in understanding and optimizing their use.

Discussion on authorship productivity in digital technology education research

The authorship distribution within digital technology education research is indicative of the field’s intellectual structure and depth. A primary figure in this domain is Neil Selwyn, whose substantial citation rate underscores the profound impact of his work. His focus on the implications of digital technology in higher education and educational sociology has proven to be seminal. Selwyn’s research trajectory, especially the exploration of spatiotemporal extensions of education through technology, provides valuable insights into the multifaceted role of digital tools in learning processes (Selwyn et al. 2019 ).

Other notable contributors, like Henderson and Edwards, present diversified research interests, such as the impact of digital technologies during the pandemic and their application in early childhood education, respectively. Their varied focuses highlight the breadth of digital technology education research, encompassing pedagogical innovation, technological adaptation, and policy development.

Discussion on country/region-level productivity and collaboration

At the country/region level, the United Kingdom, specifically England, emerges as a leading contributor with 92 published papers and a significant citation count. This is closely followed by Australia and the United States, indicating a strong English-speaking research axis. Such geographical concentration of scholarly output often correlates with investment in research and development, technological infrastructure, and the prevalence of higher education institutions engaging in cutting-edge research.

China’s notable inclusion as the only non-Western country among the top contributors to the field suggests a growing research capacity and interest in digital technology in education. However, the lower average citation per paper for China could reflect emerging engagement or different research focuses that may not yet have achieved the same international recognition as Western counterparts.

The chord diagram analysis furthers this understanding, revealing dense interconnections between countries like the United States, China, and England, which indicates robust collaborations. Such collaborations are fundamental in addressing global educational challenges and shaping international research agendas.

Discussion on institutional-level contributions to digital technology education

Institutional productivity in digital technology education research reveals a constellation of universities driving the field forward. Monash University and the Australian Catholic University have the highest publication output, signaling Australia’s significant role in advancing digital education research. The University of Oslo’s remarkable average citation count per publication indicates influential research contributions, potentially reflecting high-quality studies that resonate with the broader academic community.

The strong showing of UK institutions, including the University of London, The Open University, and the University of Cambridge, reinforces the UK’s prominence in this research field. Such institutions are often at the forefront of pedagogical innovation, benefiting from established research cultures and funding mechanisms that support sustained inquiry into digital education.

Discussion on journal publication analysis

An examination of journal outputs offers a lens into the communicative channels of the field’s knowledge base. Journals such as Education and Information Technologies , Computers & Education , and the British Journal of Educational Technology not only serve as the primary disseminators of research findings but also as indicators of research quality and relevance. The impact factor (IF) serves as a proxy for the quality and influence of these journals within the academic community.

The high citation counts for articles published in Computers & Education suggest that research disseminated through this medium has a wide-reaching impact and is of particular interest to the field. This is further evidenced by its significant IF of 11.182, indicating that the journal is a pivotal platform for seminal work in the application of digital technology in education.

The authorship, regional, and institutional productivity in the field of digital technology education application research collectively narrate the evolution of this domain since the turn of the century. The prominence of certain authors and countries underscores the importance of socioeconomic factors and existing academic infrastructure in fostering research productivity. Meanwhile, the centrality of specific journals as outlets for high-impact research emphasizes the role of academic publishing in shaping the research landscape.

As the field continues to grow, future research may benefit from leveraging the collaborative networks that have been elucidated through this analysis, perhaps focusing on underrepresented regions to broaden the scope and diversity of research. Furthermore, the stabilization of publication numbers in recent years invites a deeper exploration into potential plateaus in research trends or saturation in certain sub-fields, signaling an opportunity for novel inquiries and methodological innovations.

Discussion on the evolutionary trends (RQ2)

The evolution of the research field concerning the application of digital technology in education over the past two decades is a story of convergence, diversification, and transformation, shaped by rapid technological advancements and shifting educational paradigms.

At the turn of the century, the inception of digital technology in education was largely exploratory, with a focus on how emerging computer technologies could be harnessed to enhance traditional learning environments. Research from this early period was primarily descriptive, reflecting on the potential and challenges of incorporating digital tools into the educational setting. This phase was critical in establishing the fundamental discourse that would guide subsequent research, as it set the stage for understanding the scope and impact of digital technology in learning spaces (Wang et al. 2023 ).

As the first decade progressed, the narrative expanded to encompass the pedagogical implications of digital technologies. This was a period of conceptual debates, where terms like “digital natives” and “disruptive pedagogy” entered the academic lexicon, underscoring the growing acknowledgment of digital technology as a transformative force within education (Bennett and Maton, 2010 ). During this time, the research began to reflect a more nuanced understanding of the integration of technology, considering not only its potential to change where and how learning occurred but also its implications for educational equity and access.

In the second decade, with the maturation of internet connectivity and mobile technology, the focus of research shifted from theoretical speculations to empirical investigations. The proliferation of digital devices and the ubiquity of social media influenced how learners interacted with information and each other, prompting a surge in studies that sought to measure the impact of these tools on learning outcomes. The digital divide and issues related to digital literacy became central concerns, as scholars explored the varying capacities of students and educators to engage with technology effectively.

Throughout this period, there was an increasing emphasis on the individualization of learning experiences, facilitated by adaptive technologies that could cater to the unique needs and pacing of learners (Jing et al. 2023a ). This individualization was coupled with a growing recognition of the importance of collaborative learning, both online and offline, and the role of digital tools in supporting these processes. Blended learning models, which combined face-to-face instruction with online resources, emerged as a significant trend, advocating for a balance between traditional pedagogies and innovative digital strategies.

The later years, particularly marked by the COVID-19 pandemic, accelerated the necessity for digital technology in education, transforming it from a supplementary tool to an essential platform for delivering education globally (Mo et al. 2022 ; Mustapha et al. 2021 ). This era brought about an unprecedented focus on online learning environments, distance education, and virtual classrooms. Research became more granular, examining not just the pedagogical effectiveness of digital tools, but also their role in maintaining continuity of education during crises, their impact on teacher and student well-being, and their implications for the future of educational policy and infrastructure.

Across these two decades, the research field has seen a shift from examining digital technology as an external addition to the educational process, to viewing it as an integral component of curriculum design, instructional strategies, and even assessment methods. The emergent themes have broadened from a narrow focus on specific tools or platforms to include wider considerations such as data privacy, ethical use of technology, and the environmental impact of digital tools.

Moreover, the field has moved from considering the application of digital technology in education as a primarily cognitive endeavor to recognizing its role in facilitating socio-emotional learning, digital citizenship, and global competencies. Researchers have increasingly turned their attention to the ways in which technology can support collaborative skills, cultural understanding, and ethical reasoning within diverse student populations.

In summary, the past over twenty years in the research field of digital technology applications in education have been characterized by a progression from foundational inquiries to complex analyses of digital integration. This evolution has mirrored the trajectory of technology itself, from a facilitative tool to a pervasive ecosystem defining contemporary educational experiences. As we look to the future, the field is poised to delve into the implications of emerging technologies like AI, AR, and VR, and their potential to redefine the educational landscape even further. This ongoing metamorphosis suggests that the application of digital technology in education will continue to be a rich area of inquiry, demanding continual adaptation and forward-thinking from educators and researchers alike.

Discussion on the study of research hotspots (RQ3)

The analysis of keyword evolution in digital technology education application research elucidates the current frontiers in the field, reflecting a trajectory that is in tandem with the rapidly advancing digital age. This landscape is sculpted by emergent technological innovations and shaped by the demands of an increasingly digital society.

Interdisciplinary integration and pedagogical transformation

One of the frontiers identified from recent keyword bursts includes the integration of digital technology into diverse educational contexts, particularly noted with the keyword “physical education.” The digitalization of disciplines traditionally characterized by physical presence illustrates the pervasive reach of technology and signifies a push towards interdisciplinary integration where technology is not only a facilitator but also a transformative agent. This integration challenges educators to reconceptualize curriculum delivery to accommodate digital tools that can enhance or simulate the physical aspects of learning.

Digital literacy and skills acquisition

Another pivotal frontier is the focus on “digital literacy” and “digital skill”, which has intensified in recent years. This suggests a shift from mere access to technology towards a comprehensive understanding and utilization of digital tools. In this realm, the emphasis is not only on the ability to use technology but also on critical thinking, problem-solving, and the ethical use of digital resources (Yu, 2022 ). The acquisition of digital literacy is no longer an additive skill but a fundamental aspect of modern education, essential for navigating and contributing to the digital world.

Educational digital transformation

The keyword “digital transformation” marks a significant research frontier, emphasizing the systemic changes that education institutions must undergo to align with the digital era (Romero et al. 2021 ). This transformation includes the redesigning of learning environments, pedagogical strategies, and assessment methods to harness digital technology’s full potential. Research in this area explores the complexity of institutional change, addressing the infrastructural, cultural, and policy adjustments needed for a seamless digital transition.

Engagement and participation

Further exploration into “engagement” and “participation” underscores the importance of student-centered learning environments that are mediated by technology. The current frontiers examine how digital platforms can foster collaboration, inclusivity, and active learning, potentially leading to more meaningful and personalized educational experiences. Here, the use of technology seeks to support the emotional and cognitive aspects of learning, moving beyond the transactional view of education to one that is relational and interactive.

Professional development and teacher readiness

As the field evolves, “professional development” emerges as a crucial area, particularly in light of the pandemic which necessitated emergency remote teaching. The need for teacher readiness in a digital age is a pressing frontier, with research focusing on the competencies required for educators to effectively integrate technology into their teaching practices. This includes familiarity with digital tools, pedagogical innovation, and an ongoing commitment to personal and professional growth in the digital domain.

Pandemic as a catalyst

The recent pandemic has acted as a catalyst for accelerated research and application in this field, particularly in the domains of “digital transformation,” “professional development,” and “physical education.” This period has been a litmus test for the resilience and adaptability of educational systems to continue their operations in an emergency. Research has thus been directed at understanding how digital technologies can support not only continuity but also enhance the quality and reach of education in such contexts.

Ethical and societal considerations

The frontier of digital technology in education is also expanding to consider broader ethical and societal implications. This includes issues of digital equity, data privacy, and the sociocultural impact of technology on learning communities. The research explores how educational technology can be leveraged to address inequities and create more equitable learning opportunities for all students, regardless of their socioeconomic background.

Innovation and emerging technologies

Looking forward, the frontiers are set to be influenced by ongoing and future technological innovations, such as artificial intelligence (AI) (Wu and Yu, 2023 ; Chen et al. 2022a ). The exploration into how these technologies can be integrated into educational practices to create immersive and adaptive learning experiences represents a bold new chapter for the field.

In conclusion, the current frontiers of research on the application of digital technology in education are multifaceted and dynamic. They reflect an overarching movement towards deeper integration of technology in educational systems and pedagogical practices, where the goals are not only to facilitate learning but to redefine it. As these frontiers continue to expand and evolve, they will shape the educational landscape, requiring a concerted effort from researchers, educators, policymakers, and technologists to navigate the challenges and harness the opportunities presented by the digital revolution in education.

Conclusions and future research

Conclusions.

The utilization of digital technology in education is a research area that cuts across multiple technical and educational domains and continues to experience dynamic growth due to the continuous progress of technology. In this study, a systematic review of this field was conducted through bibliometric techniques to examine its development trajectory. The primary focus of the review was to investigate the leading contributors, productive national institutions, significant publications, and evolving development patterns. The study’s quantitative analysis resulted in several key conclusions that shed light on this research field’s current state and future prospects.

(1) The research field of digital technology education applications has entered a stage of rapid development, particularly in recent years due to the impact of the pandemic, resulting in a peak of publications. Within this field, several key authors (Selwyn, Henderson, Edwards, etc.) and countries/regions (England, Australia, USA, etc.) have emerged, who have made significant contributions. International exchanges in this field have become frequent, with a high degree of internationalization in academic research. Higher education institutions in the UK and Australia are the core productive forces in this field at the institutional level.

(2) Education and Information Technologies , Computers & Education , and the British Journal of Educational Technology are notable journals that publish research related to digital technology education applications. These journals are affiliated with the research field of educational technology and provide effective communication platforms for sharing digital technology education applications.

(3) Over the past two decades, research on digital technology education applications has progressed from its early stages of budding, initial development, and critical exploration to accelerated transformation, and it is currently approaching maturity. Technological progress and changes in the times have been key driving forces for educational transformation and innovation, and both have played important roles in promoting the continuous development of education.

(4) Influenced by the pandemic, three emerging frontiers have emerged in current research on digital technology education applications, which are physical education, digital transformation, and professional development under the promotion of digital technology. These frontier research hotspots reflect the core issues that the education system faces when encountering new technologies. The evolution of research hotspots shows that technology breakthroughs in education’s original boundaries of time and space create new challenges. The continuous self-renewal of education is achieved by solving one hotspot problem after another.

The present study offers significant practical implications for scholars and practitioners in the field of digital technology education applications. Firstly, it presents a well-defined framework of the existing research in this area, serving as a comprehensive guide for new entrants to the field and shedding light on the developmental trajectory of this research domain. Secondly, the study identifies several contemporary research hotspots, thus offering a valuable decision-making resource for scholars aiming to explore potential research directions. Thirdly, the study undertakes an exhaustive analysis of published literature to identify core journals in the field of digital technology education applications, with Sustainability being identified as a promising open access journal that publishes extensively on this topic. This finding can potentially facilitate scholars in selecting appropriate journals for their research outputs.

Limitation and future research

Influenced by some objective factors, this study also has some limitations. First of all, the bibliometrics analysis software has high standards for data. In order to ensure the quality and integrity of the collected data, the research only selects the periodical papers in SCIE and SSCI indexes, which are the core collection of Web of Science database, and excludes other databases, conference papers, editorials and other publications, which may ignore some scientific research and original opinions in the field of digital technology education and application research. In addition, although this study used professional software to carry out bibliometric analysis and obtained more objective quantitative data, the analysis and interpretation of data will inevitably have a certain subjective color, and the influence of subjectivity on data analysis cannot be completely avoided. As such, future research endeavors will broaden the scope of literature screening and proactively engage scholars in the field to gain objective and state-of-the-art insights, while minimizing the adverse impact of personal subjectivity on research analysis.

Data availability

The datasets analyzed during the current study are available in the Dataverse repository: https://doi.org/10.7910/DVN/F9QMHY

Alabdulaziz MS (2021) COVID-19 and the use of digital technology in mathematics education. Educ Inf Technol 26(6):7609–7633. https://doi.org/10.1007/s10639-021-10602-3

Arif TB, Munaf U, Ul-Haque I (2023) The future of medical education and research: is ChatGPT a blessing or blight in disguise? Med Educ Online 28. https://doi.org/10.1080/10872981.2023.2181052

Banerjee M, Chiew D, Patel KT, Johns I, Chappell D, Linton N, Cole GD, Francis DP, Szram J, Ross J, Zaman S (2021) The impact of artificial intelligence on clinical education: perceptions of postgraduate trainee doctors in London (UK) and recommendations for trainers. BMC Med Educ 21. https://doi.org/10.1186/s12909-021-02870-x

Barlovits S, Caldeira A, Fesakis G, Jablonski S, Koutsomanoli Filippaki D, Lázaro C, Ludwig M, Mammana MF, Moura A, Oehler DXK, Recio T, Taranto E, Volika S(2022) Adaptive, synchronous, and mobile online education: developing the ASYMPTOTE learning environment. Mathematics 10:1628. https://doi.org/10.3390/math10101628

Article   Google Scholar  

Baron NS(2021) Know what? How digital technologies undermine learning and remembering J Pragmat 175:27–37. https://doi.org/10.1016/j.pragma.2021.01.011

Batista J, Morais NS, Ramos F (2016) Researching the use of communication technologies in higher education institutions in Portugal. https://doi.org/10.4018/978-1-5225-0571-6.ch057

Beardsley M, Albó L, Aragón P, Hernández-Leo D (2021) Emergency education effects on teacher abilities and motivation to use digital technologies. Br J Educ Technol 52. https://doi.org/10.1111/bjet.13101

Bennett S, Maton K(2010) Beyond the “digital natives” debate: towards a more nuanced understanding of students’ technology experiences J Comput Assist Learn 26:321–331. https://doi.org/10.1111/j.1365-2729.2010.00360.x

Buckingham D, Burn A (2007) Game literacy in theory and practice 16:323–349

Google Scholar  

Bulfin S, Pangrazio L, Selwyn N (2014) Making “MOOCs”: the construction of a new digital higher education within news media discourse. In: The International Review of Research in Open and Distributed Learning 15. https://doi.org/10.19173/irrodl.v15i5.1856

Camilleri MA, Camilleri AC(2016) Digital learning resources and ubiquitous technologies in education Technol Knowl Learn 22:65–82. https://doi.org/10.1007/s10758-016-9287-7

Chen C(2006) CiteSpace II: detecting and visualizing emerging trends and transient patterns in scientific literature J Am Soc Inf Sci Technol 57:359–377. https://doi.org/10.1002/asi.20317

Chen J, Dai J, Zhu K, Xu L(2022) Effects of extended reality on language learning: a meta-analysis Front Psychol 13:1016519. https://doi.org/10.3389/fpsyg.2022.1016519

Article   PubMed   PubMed Central   Google Scholar  

Chen J, Wang CL, Tang Y (2022b) Knowledge mapping of volunteer motivation: a bibliometric analysis and cross-cultural comparative study. Front Psychol 13. https://doi.org/10.3389/fpsyg.2022.883150

Cohen A, Soffer T, Henderson M(2022) Students’ use of technology and their perceptions of its usefulness in higher education: International comparison J Comput Assist Learn 38(5):1321–1331. https://doi.org/10.1111/jcal.12678

Collins A, Halverson R(2010) The second educational revolution: rethinking education in the age of technology J Comput Assist Learn 26:18–27. https://doi.org/10.1111/j.1365-2729.2009.00339.x

Conole G, Alevizou P (2010) A literature review of the use of Web 2.0 tools in higher education. Walton Hall, Milton Keynes, UK: the Open University, retrieved 17 February

Creely E, Henriksen D, Crawford R, Henderson M(2021) Exploring creative risk-taking and productive failure in classroom practice. A case study of the perceived self-efficacy and agency of teachers at one school Think Ski Creat 42:100951. https://doi.org/10.1016/j.tsc.2021.100951

Davis N, Eickelmann B, Zaka P(2013) Restructuring of educational systems in the digital age from a co-evolutionary perspective J Comput Assist Learn 29:438–450. https://doi.org/10.1111/jcal.12032

De Belli N (2009) Bibliometrics and citation analysis: from the science citation index to cybermetrics, Scarecrow Press. https://doi.org/10.1111/jcal.12032

Domínguez A, Saenz-de-Navarrete J, de-Marcos L, Fernández-Sanz L, Pagés C, Martínez-Herráiz JJ(2013) Gamifying learning experiences: practical implications and outcomes Comput Educ 63:380–392. https://doi.org/10.1016/j.compedu.2012.12.020

Donnison S (2009) Discourses in conflict: the relationship between Gen Y pre-service teachers, digital technologies and lifelong learning. Australasian J Educ Technol 25. https://doi.org/10.14742/ajet.1138

Durfee SM, Jain S, Shaffer K (2003) Incorporating electronic media into medical student education. Acad Radiol 10:205–210. https://doi.org/10.1016/s1076-6332(03)80046-6

Dzikowski P(2018) A bibliometric analysis of born global firms J Bus Res 85:281–294. https://doi.org/10.1016/j.jbusres.2017.12.054

van Eck NJ, Waltman L(2009) Software survey: VOSviewer, a computer program for bibliometric mapping Scientometrics 84:523–538 https://doi.org/10.1007/s11192-009-0146-3

Edwards S(2013) Digital play in the early years: a contextual response to the problem of integrating technologies and play-based pedagogies in the early childhood curriculum Eur Early Child Educ Res J 21:199–212. https://doi.org/10.1080/1350293x.2013.789190

Edwards S(2015) New concepts of play and the problem of technology, digital media and popular-culture integration with play-based learning in early childhood education Technol Pedagogy Educ 25:513–532 https://doi.org/10.1080/1475939x.2015.1108929

Article   MathSciNet   Google Scholar  

Eisenberg MB(2008) Information literacy: essential skills for the information age DESIDOC J Libr Inf Technol 28:39–47. https://doi.org/10.14429/djlit.28.2.166

Forde C, OBrien A (2022) A literature review of barriers and opportunities presented by digitally enhanced practical skill teaching and learning in health science education. Med Educ Online 27. https://doi.org/10.1080/10872981.2022.2068210

García-Morales VJ, Garrido-Moreno A, Martín-Rojas R (2021) The transformation of higher education after the COVID disruption: emerging challenges in an online learning scenario. Front Psychol 12. https://doi.org/10.3389/fpsyg.2021.616059

Garfield E(2006) The history and meaning of the journal impact factor JAMA 295:90. https://doi.org/10.1001/jama.295.1.90

Article   PubMed   Google Scholar  

Garzón-Artacho E, Sola-Martínez T, Romero-Rodríguez JM, Gómez-García G(2021) Teachers’ perceptions of digital competence at the lifelong learning stage Heliyon 7:e07513. https://doi.org/10.1016/j.heliyon.2021.e07513

Gaviria-Marin M, Merigó JM, Baier-Fuentes H(2019) Knowledge management: a global examination based on bibliometric analysis Technol Forecast Soc Change 140:194–220. https://doi.org/10.1016/j.techfore.2018.07.006

Gilster P, Glister P (1997) Digital literacy. Wiley Computer Pub, New York

Greenhow C, Lewin C(2015) Social media and education: reconceptualizing the boundaries of formal and informal learning Learn Media Technol 41:6–30. https://doi.org/10.1080/17439884.2015.1064954

Hawkins DT(2001) Bibliometrics of electronic journals in information science Infor Res 7(1):7–1. http://informationr.net/ir/7-1/paper120.html

Henderson M, Selwyn N, Finger G, Aston R(2015) Students’ everyday engagement with digital technology in university: exploring patterns of use and “usefulness J High Educ Policy Manag 37:308–319 https://doi.org/10.1080/1360080x.2015.1034424

Huang CK, Neylon C, Hosking R, Montgomery L, Wilson KS, Ozaygen A, Brookes-Kenworthy C (2020) Evaluating the impact of open access policies on research institutions. eLife 9. https://doi.org/10.7554/elife.57067

Hwang GJ, Tsai CC(2011) Research trends in mobile and ubiquitous learning: a review of publications in selected journals from 2001 to 2010 Br J Educ Technol 42:E65–E70. https://doi.org/10.1111/j.1467-8535.2011.01183.x

Hwang GJ, Wu PH, Zhuang YY, Huang YM(2013) Effects of the inquiry-based mobile learning model on the cognitive load and learning achievement of students Interact Learn Environ 21:338–354. https://doi.org/10.1080/10494820.2011.575789

Jiang S, Ning CF (2022) Interactive communication in the process of physical education: are social media contributing to the improvement of physical training performance. Universal Access Inf Soc, 1–10. https://doi.org/10.1007/s10209-022-00911-w

Jing Y, Zhao L, Zhu KK, Wang H, Wang CL, Xia Q(2023) Research landscape of adaptive learning in education: a bibliometric study on research publications from 2000 to 2022 Sustainability 15:3115–3115. https://doi.org/10.3390/su15043115

Jing Y, Wang CL, Chen Y, Wang H, Yu T, Shadiev R (2023b) Bibliometric mapping techniques in educational technology research: a systematic literature review. Educ Inf Technol 1–29. https://doi.org/10.1007/s10639-023-12178-6

Krishnamurthy S (2020) The future of business education: a commentary in the shadow of the Covid-19 pandemic. J Bus Res. https://doi.org/10.1016/j.jbusres.2020.05.034

Kumar S, Lim WM, Pandey N, Christopher Westland J (2021) 20 years of electronic commerce research. Electron Commer Res 21:1–40

Kyza EA, Georgiou Y(2018) Scaffolding augmented reality inquiry learning: the design and investigation of the TraceReaders location-based, augmented reality platform Interact Learn Environ 27:211–225. https://doi.org/10.1080/10494820.2018.1458039

Laurillard D(2008) Technology enhanced learning as a tool for pedagogical innovation J Philos Educ 42:521–533. https://doi.org/10.1111/j.1467-9752.2008.00658.x

Li M, Yu Z (2023) A systematic review on the metaverse-based blended English learning. Front Psychol 13. https://doi.org/10.3389/fpsyg.2022.1087508

Luo H, Li G, Feng Q, Yang Y, Zuo M (2021) Virtual reality in K-12 and higher education: a systematic review of the literature from 2000 to 2019. J Comput Assist Learn. https://doi.org/10.1111/jcal.12538

Margaryan A, Littlejohn A, Vojt G(2011) Are digital natives a myth or reality? University students’ use of digital technologies Comput Educ 56:429–440. https://doi.org/10.1016/j.compedu.2010.09.004

McMillan S(1996) Literacy and computer literacy: definitions and comparisons Comput Educ 27:161–170. https://doi.org/10.1016/s0360-1315(96)00026-7

Mo CY, Wang CL, Dai J, Jin P (2022) Video playback speed influence on learning effect from the perspective of personalized adaptive learning: a study based on cognitive load theory. Front Psychology 13. https://doi.org/10.3389/fpsyg.2022.839982

Moorhouse BL (2021) Beginning teaching during COVID-19: newly qualified Hong Kong teachers’ preparedness for online teaching. Educ Stud 1–17. https://doi.org/10.1080/03055698.2021.1964939

Moorhouse BL, Wong KM (2021) The COVID-19 Pandemic as a catalyst for teacher pedagogical and technological innovation and development: teachers’ perspectives. Asia Pac J Educ 1–16. https://doi.org/10.1080/02188791.2021.1988511

Moskal P, Dziuban C, Hartman J (2013) Blended learning: a dangerous idea? Internet High Educ 18:15–23

Mughal MY, Andleeb N, Khurram AFA, Ali MY, Aslam MS, Saleem MN (2022) Perceptions of teaching-learning force about Metaverse for education: a qualitative study. J. Positive School Psychol 6:1738–1745

Mustapha I, Thuy Van N, Shahverdi M, Qureshi MI, Khan N (2021) Effectiveness of digital technology in education during COVID-19 pandemic. a bibliometric analysis. Int J Interact Mob Technol 15:136

Nagle J (2018) Twitter, cyber-violence, and the need for a critical social media literacy in teacher education: a review of the literature. Teach Teach Education 76:86–94

Nazare J, Woolf A, Sysoev I, Ballinger S, Saveski M, Walker M, Roy D (2022) Technology-assisted coaching can increase engagement with learning technology at home and caregivers’ awareness of it. Comput Educ 188:104565

Nguyen UP, Hallinger P (2020) Assessing the distinctive contributions of simulation & gaming to the literature, 1970-2019: a bibliometric review. Simul Gaming 104687812094156. https://doi.org/10.1177/1046878120941569

Nygren H, Nissinen K, Hämäläinen R, Wever B(2019) Lifelong learning: formal, non-formal and informal learning in the context of the use of problem-solving skills in technology-rich environments Br J Educ Technol 50:1759–1770. https://doi.org/10.1111/bjet.12807

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, Moher D (2021) The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. Int J Surg 88:105906

Pan SL, Zhang S(2020) From fighting COVID-19 pandemic to tackling sustainable development goals: an opportunity for responsible information systems research Int J Inf Manage 55:102196. https://doi.org/10.1016/j.ijinfomgt.2020.102196

Pan X, Yan E, Cui M, Hua W(2018) Examining the usage, citation, and diffusion patterns of bibliometric mapping software: a comparative study of three tools J Informetr 12:481–493. https://doi.org/10.1016/j.joi.2018.03.005

Parris Z, Cale L, Harris J, Casey A (2022) Physical activity for health, covid-19 and social media: what, where and why?. Movimento, 28. https://doi.org/10.22456/1982-8918.122533

Pasquini LA, Evangelopoulos N (2016) Sociotechnical stewardship in higher education: a field study of social media policy documents. J Comput High Educ 29:218–239

Pérez-Sanagustín M, Hernández-Leo D, Santos P, Delgado Kloos C, Blat J(2014) Augmenting reality and formality of informal and non-formal settings to enhance blended learning IEEE Trans Learn Technol 7:118–131. https://doi.org/10.1109/TLT.2014.2312719

Pinto M, Leite C (2020) Digital technologies in support of students learning in Higher Education: literature review. Digital Education Review 343–360. https://doi.org/10.1344/der.2020.37.343-360

Pires F, Masanet MJ, Tomasena JM, Scolari CA(2022) Learning with YouTube: beyond formal and informal through new actors, strategies and affordances Convergence 28(3):838–853. https://doi.org/10.1177/1354856521102054

Pritchard A (1969) Statistical bibliography or bibliometrics 25:348

Romero M, Romeu T, Guitert M, Baztán P (2021) Digital transformation in higher education: the UOC case. In ICERI2021 Proceedings (pp. 6695–6703). IATED https://doi.org/10.21125/iceri.2021.1512

Romero-Hall E, Jaramillo Cherrez N (2022) Teaching in times of disruption: faculty digital literacy in higher education during the COVID-19 pandemic. Innovations in Education and Teaching International 1–11. https://doi.org/10.1080/14703297.2022.2030782

Rospigliosi PA(2023) Artificial intelligence in teaching and learning: what questions should we ask of ChatGPT? Interactive Learning Environments 31:1–3. https://doi.org/10.1080/10494820.2023.2180191

Salas-Pilco SZ, Yang Y, Zhang Z(2022) Student engagement in online learning in Latin American higher education during the COVID-19 pandemic: a systematic review. Br J Educ Technol 53(3):593–619. https://doi.org/10.1111/bjet.13190

Selwyn N(2009) The digital native-myth and reality In Aslib proceedings 61(4):364–379. https://doi.org/10.1108/00012530910973776

Selwyn N(2012) Making sense of young people, education and digital technology: the role of sociological theory Oxford Review of Education 38:81–96. https://doi.org/10.1080/03054985.2011.577949

Selwyn N, Facer K(2014) The sociology of education and digital technology: past, present and future Oxford Rev Educ 40:482–496. https://doi.org/10.1080/03054985.2014.933005

Selwyn N, Banaji S, Hadjithoma-Garstka C, Clark W(2011) Providing a platform for parents? Exploring the nature of parental engagement with school Learning Platforms J Comput Assist Learn 27:314–323. https://doi.org/10.1111/j.1365-2729.2011.00428.x

Selwyn N, Aagaard J (2020) Banning mobile phones from classrooms-an opportunity to advance understandings of technology addiction, distraction and cyberbullying. Br J Educ Technol 52. https://doi.org/10.1111/bjet.12943

Selwyn N, O’Neill C, Smith G, Andrejevic M, Gu X (2021) A necessary evil? The rise of online exam proctoring in Australian universities. Media Int Austr 1329878X2110058. https://doi.org/10.1177/1329878x211005862

Selwyn N, Pangrazio L, Nemorin S, Perrotta C (2019) What might the school of 2030 be like? An exercise in social science fiction. Learn, Media Technol 1–17. https://doi.org/10.1080/17439884.2020.1694944

Selwyn, N (2016) What works and why?* Understanding successful technology enabled learning within institutional contexts 2016 Final report Appendices (Part B). Monash University Griffith University

Sjöberg D, Holmgren R (2021) Informal workplace learning in swedish police education-a teacher perspective. Vocations and Learning. https://doi.org/10.1007/s12186-021-09267-3

Strotmann A, Zhao D (2012) Author name disambiguation: what difference does it make in author-based citation analysis? J Am Soc Inf Sci Technol 63:1820–1833

Article   CAS   Google Scholar  

Sutherland R, Facer K, Furlong R, Furlong J(2000) A new environment for education? The computer in the home. Comput Educ 34:195–212. https://doi.org/10.1016/s0360-1315(99)00045-7

Szeto E, Cheng AY-N, Hong J-C(2015) Learning with social media: how do preservice teachers integrate YouTube and Social Media in teaching? Asia-Pac Educ Res 25:35–44. https://doi.org/10.1007/s40299-015-0230-9

Tang E, Lam C(2014) Building an effective online learning community (OLC) in blog-based teaching portfolios Int High Educ 20:79–85. https://doi.org/10.1016/j.iheduc.2012.12.002

Taskin Z, Al U(2019) Natural language processing applications in library and information science Online Inf Rev 43:676–690. https://doi.org/10.1108/oir-07-2018-0217

Tegtmeyer K, Ibsen L, Goldstein B(2001) Computer-assisted learning in critical care: from ENIAC to HAL Crit Care Med 29:N177–N182. https://doi.org/10.1097/00003246-200108001-00006

Article   CAS   PubMed   Google Scholar  

Timotheou S, Miliou O, Dimitriadis Y, Sobrino SV, Giannoutsou N, Cachia R, Moné AM, Ioannou A(2023) Impacts of digital technologies on education and factors influencing schools' digital capacity and transformation: a literature review. Educ Inf Technol 28(6):6695–6726. https://doi.org/10.1007/s10639-022-11431-8

Trujillo Maza EM, Gómez Lozano MT, Cardozo Alarcón AC, Moreno Zuluaga L, Gamba Fadul M (2016) Blended learning supported by digital technology and competency-based medical education: a case study of the social medicine course at the Universidad de los Andes, Colombia. Int J Educ Technol High Educ 13. https://doi.org/10.1186/s41239-016-0027-9

Turin O, Friesem Y(2020) Is that media literacy?: Israeli and US media scholars’ perceptions of the field J Media Lit Educ 12:132–144. https://doi.org/10.1007/s11192-009-0146-3

Van Eck NJ, Waltman L (2019) VOSviewer manual. Universiteit Leiden

Vratulis V, Clarke T, Hoban G, Erickson G(2011) Additive and disruptive pedagogies: the use of slowmation as an example of digital technology implementation Teach Teach Educ 27:1179–1188. https://doi.org/10.1016/j.tate.2011.06.004

Wang CL, Dai J, Xu LJ (2022) Big data and data mining in education: a bibliometrics study from 2010 to 2022. In 2022 7th International Conference on Cloud Computing and Big Data Analytics ( ICCCBDA ) (pp. 507-512). IEEE. https://doi.org/10.1109/icccbda55098.2022.9778874

Wang CL, Dai J, Zhu KK, Yu T, Gu XQ (2023) Understanding the continuance intention of college students toward new E-learning spaces based on an integrated model of the TAM and TTF. Int J Hum-Comput Int 1–14. https://doi.org/10.1080/10447318.2023.2291609

Wong L-H, Boticki I, Sun J, Looi C-K(2011) Improving the scaffolds of a mobile-assisted Chinese character forming game via a design-based research cycle Comput Hum Behav 27:1783–1793. https://doi.org/10.1016/j.chb.2011.03.005

Wu R, Yu Z (2023) Do AI chatbots improve students learning outcomes? Evidence from a meta-analysis. Br J Educ Technol. https://doi.org/10.1111/bjet.13334

Yang D, Zhou J, Shi D, Pan Q, Wang D, Chen X, Liu J (2022) Research status, hotspots, and evolutionary trends of global digital education via knowledge graph analysis. Sustainability 14:15157–15157. https://doi.org/10.3390/su142215157

Yu T, Dai J, Wang CL (2023) Adoption of blended learning: Chinese university students’ perspectives. Humanit Soc Sci Commun 10:390. https://doi.org/10.3390/su142215157

Yu Z (2022) Sustaining student roles, digital literacy, learning achievements, and motivation in online learning environments during the COVID-19 pandemic. Sustainability 14:4388. https://doi.org/10.3390/su14084388

Za S, Spagnoletti P, North-Samardzic A(2014) Organisational learning as an emerging process: the generative role of digital tools in informal learning practices Br J Educ Technol 45:1023–1035. https://doi.org/10.1111/bjet.12211

Zhang X, Chen Y, Hu L, Wang Y (2022) The metaverse in education: definition, framework, features, potential applications, challenges, and future research topics. Front Psychol 13:1016300. https://doi.org/10.3389/fpsyg.2022.1016300

Zhou M, Dzingirai C, Hove K, Chitata T, Mugandani R (2022) Adoption, use and enhancement of virtual learning during COVID-19. Education and Information Technologies. https://doi.org/10.1007/s10639-022-10985-x

Download references

Acknowledgements

This research was supported by the Zhejiang Provincial Social Science Planning Project, “Mechanisms and Pathways for Empowering Classroom Teaching through Learning Spaces under the Strategy of High-Quality Education Development”, the 2022 National Social Science Foundation Education Youth Project “Research on the Strategy of Creating Learning Space Value and Empowering Classroom Teaching under the background of ‘Double Reduction’” (Grant No. CCA220319) and the National College Student Innovation and Entrepreneurship Training Program of China (Grant No. 202310337023).

Author information

Authors and affiliations.

College of Educational Science and Technology, Zhejiang University of Technology, Zhejiang, China

Chengliang Wang, Xiaojiao Chen, Yidan Liu & Yuhui Jing

Graduate School of Business, Universiti Sains Malaysia, Minden, Malaysia

Department of Management, The Chinese University of Hong Kong, Hong Kong, China

College of Humanities and Social Sciences, Beihang University, Beijing, China

You can also search for this author in PubMed   Google Scholar

Contributions

Conceptualization: Y.J., C.W.; methodology, C.W.; software, C.W., Y.L.; writing-original draft preparation, C.W., Y.L.; writing-review and editing, T.Y., Y.L., C.W.; supervision, X.C., T.Y.; project administration, Y.J.; funding acquisition, X.C., Y.L. All authors read and approved the final manuscript. All authors have read and approved the re-submission of the manuscript.

Corresponding author

Correspondence to Yuhui Jing .

Ethics declarations

Ethical approval.

Ethical approval was not required as the study did not involve human participants.

Informed consent

Informed consent was not required as the study did not involve human participants.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Wang, C., Chen, X., Yu, T. et al. Education reform and change driven by digital technology: a bibliometric study from a global perspective. Humanit Soc Sci Commun 11 , 256 (2024). https://doi.org/10.1057/s41599-024-02717-y

Download citation

Received : 11 July 2023

Accepted : 17 January 2024

Published : 12 February 2024

DOI : https://doi.org/10.1057/s41599-024-02717-y

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

This article is cited by

A meta-analysis of learners’ continuance intention toward online education platforms.

• Chengliang Wang

Education and Information Technologies (2024)

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Suggestions or feedback?

MIT News | Massachusetts Institute of Technology

• Machine learning
• Social justice
• Black holes
• Classes and programs

Departments

• Aeronautics and Astronautics
• Brain and Cognitive Sciences
• Architecture
• Political Science
• Mechanical Engineering

Centers, Labs, & Programs

• Abdul Latif Jameel Poverty Action Lab (J-PAL)
• Picower Institute for Learning and Memory
• Lincoln Laboratory
• School of Architecture + Planning
• School of Engineering
• School of Humanities, Arts, and Social Sciences
• Sloan School of Management
• School of Science
• MIT Schwarzman College of Computing

What 126 studies say about education technology

Press contact :.

Previous image Next image

In recent years, there has been widespread excitement around the transformative potential of technology in education. In the United States alone, spending on education technology has now exceeded $13 billion . Programs and policies to promote the use of education technology may expand access to quality education, support students’ learning in innovative ways, and help families navigate complex school systems.

However, the rapid development of education technology in the United States is occurring in a context of deep and persistent inequality . Depending on how programs are designed, how they are used, and who can access them, education technologies could alleviate or aggravate existing disparities. To harness education technology’s full potential, education decision-makers, product developers, and funders need to understand the ways in which technology can help — or in some cases hurt — student learning.

To address this need, J-PAL North America recently released a new publication summarizing 126 rigorous evaluations of different uses of education technology. Drawing primarily from research in developed countries, the publication looks at randomized evaluations and regression discontinuity designs across four broad categories: (1) access to technology, (2) computer-assisted learning or educational software, (3) technology-enabled nudges in education, and (4) online learning.

This growing body of evidence suggests some areas of promise and points to four key lessons on education technology.

First, supplying computers and internet alone generally do not improve students’ academic outcomes from kindergarten to 12th grade, but do increase computer usage and improve computer proficiency. Disparities in access to information and communication technologies can exacerbate existing educational inequalities. Students without access at school or at home may struggle to complete web-based assignments and may have a hard time developing digital literacy skills.

Broadly, programs to expand access to technology have been effective at increasing use of computers and improving computer skills. However, computer distribution and internet subsidy programs generally did not improve grades and test scores and in some cases led to adverse impacts on academic achievement. The limited rigorous evidence suggests that distributing computers may have a more direct impact on learning outcomes at the postsecondary level.

Second, educational software (often called “computer-assisted learning”) programs designed to help students develop particular skills have shown enormous promise in improving learning outcomes, particularly in math. Targeting instruction to meet students’ learning levels has been found to be effective in improving student learning, but large class sizes with a wide range of learning levels can make it hard for teachers to personalize instruction. Software has the potential to overcome traditional classroom constraints by customizing activities for each student. Educational software programs range from light-touch homework support tools to more intensive interventions that re-orient the classroom around the use of software.

Most educational software that have been rigorously evaluated help students practice particular skills through personalized tutoring approaches. Computer-assisted learning programs have shown enormous promise in improving academic achievement, especially in math. Of all 30 studies of computer-assisted learning programs, 20 reported statistically significant positive effects, 15 of which were focused on improving math outcomes.

Third, technology-based nudges — such as text message reminders — can have meaningful, if modest, impacts on a variety of education-related outcomes, often at extremely low costs. Low-cost interventions like text message reminders can successfully support students and families at each stage of schooling. Text messages with reminders, tips, goal-setting tools, and encouragement can increase parental engagement in learning activities, such as reading with their elementary-aged children.

Middle and high schools, meanwhile, can help parents support their children by providing families with information about how well their children are doing in school. Colleges can increase application and enrollment rates by leveraging technology to suggest specific action items, streamline financial aid procedures, and/or provide personalized support to high school students.

Online courses are developing a growing presence in education, but the limited experimental evidence suggests that online-only courses lower student academic achievement compared to in-person courses. In four of six studies that directly compared the impact of taking a course online versus in-person only, student performance was lower in the online courses. However, students performed similarly in courses with both in-person and online components compared to traditional face-to-face classes.

The new publication is meant to be a resource for decision-makers interested in learning which uses of education technology go beyond the hype to truly help students learn. At the same time, the publication outlines key open questions about the impacts of education technology, including questions relating to the long-term impacts of education technology and the impacts of education technology on different types of learners.

To help answer these questions, J-PAL North America’s Education, Technology, and Opportunity Initiative is working to build the evidence base on promising uses of education technology by partnering directly with education leaders.

Education leaders are invited to submit letters of interest to partner with J-PAL North America through its  Innovation Competition . Anyone interested in learning more about how to apply is encouraged to contact initiative manager Vincent Quan .

Share this news article on:

Related links.

• J-PAL Education, Technology, and Opportunity Initiative
• Education, Technology, and Opportunity Innovation Competition
• Article: "Will Technology Transform Education for the Better?"
• Abdul Latif Jameel Poverty Action Lab
• Department of Economics

Related Topics

• School of Humanities Arts and Social Sciences
• Education, teaching, academics
• Technology and society
• Computer science and technology

Related Articles

J-PAL North America calls for proposals from education leaders

J-PAL North America’s Education, Technology, and Opportunity Innovation Competition announces inaugural partners

New learning opportunities for displaced persons

J-PAL North America announces new partnerships with three state and local governments

A new way to measure women’s and girls’ empowerment in impact evaluations

Previous item Next item

More MIT News

An expansive approach to making new compounds

Read full story →

Q&A: A graduating student looks back on his MIT experience

Eleven from MIT awarded 2024 Fulbright fellowships

Robotic palm mimics human touch

Trying to make the grade

Janabel Xia: Algorithms, dance rhythms, and the drive to succeed

• More news on MIT News homepage →

Massachusetts Institute of Technology 77 Massachusetts Avenue, Cambridge, MA, USA

• Map (opens in new window)
• Events (opens in new window)
• People (opens in new window)
• Careers (opens in new window)
• Accessibility
• Social Media Hub
• MIT on Facebook
• MIT on YouTube
• MIT on Instagram

Advertisement

A Comprehensive Review of Educational Technology on Objective Learning Outcomes in Academic Contexts

• Review Article
• Published: 05 April 2021
• Volume 33 , pages 1583–1630, ( 2021 )

Cite this article

• Kam Leung Yeung 1 ,
• Shana K. Carpenter 1 &
• Daniel Corral 2  

3518 Accesses

21 Citations

39 Altmetric

Explore all metrics

Rapid advances in technology during the last few decades have provided a multitude of new options for teaching and learning. Although technology is being widely adopted in education, there is a shortage of research on the effects that this technology might have on student learning, and why those effects occur. We conducted a comprehensive review of the literature on various uses of digital technology in educational settings, and the effects of that technology on students’ objective learning outcomes. We interpret these effects within the context of empirical research on effective principles of learning, and the extent to which the affordances of technology permit opportunities for increased engagement with the material, retrieval practice, and spacing. Results revealed that technology is neither beneficial nor harmful for learning when used primarily as a means of presenting information (e.g., information viewed on a computer screen vs. on paper), but can be beneficial when it involves unique affordances that leverage effective learning principles. We discues these findings in light of the ever-increasing availability of technology in education, and the importance of evidence-guided criteria in decisions about adoption and implementation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Similar content being viewed by others

Educational Technology and Response to Intervention: Affordances and Considerations

Technology Integration in Schools

Technology Education: History of Research

Even with the same instructor across all conditions, there is a possibility that some instructor-related factors could change across conditions or across time (e.g., instructors could improve their teaching effectiveness from one term to the next, or have difficulty implementing a new technology). Notwithstanding these possibilities, instructor-related factors that could influence student learning are likely to be greater when there are different instructors across the conditions (e.g., bringing differences in teaching style, personality, grading practices, or experience), such that the potential influence of these factors was minimized by ensuring that the same instructor taught all students.

In these studies it cannot be determined whether the immediacy of the feedback per se was responsible for the learning gains. Some studies have directly explored the timing of feedback and have found that feedback can be more beneficial for learning some types of materials—particularly those involving non-overlapping materials—when it is delayed rather than provided immediately (Carpenter and Vul 2011 ; Corral et al. in press ). In the studies reviewed here, however, the answer to any one item (such as a math problem or grammatical rule) could have informed students’ answers to subsequent problems of the same type. Beyond the timing of feedback per se, therefore, the immediacy of the correct answers could have changed the way that students approached subsequent questions of the same type, increasing the likelihood that they would apply the correct answer.

A third group was included that used 3-D printers but did not receive the same type of lecture-based guidance from the instructor. Due to the difference in instructional procedures, this “experiential learning” group is not included in the comparisons.

* indicates articles included in the review.

* Anderson, G. R., & Vander Meer, A. W. (1954). A comparative study on the effectiveness of lessons on the slide rule presented via television and in person. The Mathematics Teacher, 47, 323–327.

* Anderson, H. G., Frazier, L., Anderson, S. L., Stanton, R., Gillette, C., Broedel-Zaugg, K., & Yingling, K. (2017). Comparison of pharmaceutical calculations learning outcomes achieved within a traditional lecture or flipped classroom andragogy. American Journal of Pharmaceutical Education, 81, 1-9.

* Arias, J. J., Swinton, J., & Anderson, K. (2018). Online vs. face-to-face: A comparison of student outcomes with random assignment. e-Journal of Business Education & Scholarship of Teaching, 12, 1-23.

* Arús, N. A., da Silva, A. M., Duarte, R., da Silveira, P. F., Vizzotto, M. B., da Silveira, H. L. D., & da Silveira, H. E. D. (2017). Teaching dental students to understand the temporomandibular joint using MRI: Comparison of conventional and digital learning methods. Journal of Dental Education, 81, 752-758.

* Baumann-Birkbeck, L., Karaksha, A., Anoopkumar-Dukie, S., Grant, G., Davey, A., Nirthanan, S., & Owen, S. (2015). Benefits of e-learning in chemotherapy pharmacology education. Currents in Pharmacy Teaching & Learning, 7, 106-111.

Benjamin, L. T. (1988). A history of teaching machines. American Psychologist, 43 , 703–712.

* Blázquez, B. O., Masluk, B., Gascon, S., Díaz, R. F., Aguilar-Latorre, A., Magallón, I. A., & Botaya, R. M. (2019). The use of flipped classroom as an active learning approach improves academic performance in social work: A randomized trial in a university. PLOS ONE, 14, e0214623.

* Boblick, J. M. (1972). Writing chemical formulas: A comparison of computer assisted instruction with traditional teaching techniques. Science Education, 56, 221-225.

* Bortnik, B., Stozhko, N., Pervukhina, I., Tchernysheva, A., & Belysheva, G. (2017). Effect of virtual analytical chemistry laboratory on enhancing student research skills and practices. Research in Learning Technology, 25, 1-20.

* Botezatu, M., Hult, H., Tessma, M. K., & Fors, U. G. H. (2010). Virtual patient simulation for learning and assessment: Superior results in comparison with regular course exams. Medical Teacher, 32, 845-850.

* Bryner, B. S., Saddawi-Konefka, D., Gest, T. R. (2008). The impact of interactive, computerized educational modules on preclinical medical education. Anatomical Sciences Education, 1, 247-251.

* Cakir, O., & Simsek, N. (2010). A comparative analysis of the effects of computer and paper-based personalization on student achievement. Computers & Education, 55, 1524-1531.

* Campbell, D. L., Peck, D. L., Horn, C. J., & Leigh, R. K. (1987). Comparison of computer-assisted instruction and print drill performance: A research note. Educational Communication & Technology, 35, 95-103.

Carpenter, S. K. (2009). Cue strength as a moderator of the testing effect: The benefits of elaborative retrieval. Journal of Experimental Psychology: Learning, Memory, & Cognition, 35 , 1563–1569.

Carpenter, S. K. (2011). Semantic information activated during retrieval contributes to later retention: Support for the mediator effectiveness hypothesis of the testing effect. Journal of Experimental Psychology: Learning, Memory, & Cognition, 37 , 1547–1552.

Carpenter, S. K. (2014). Spacing and interleaving of study and practice. In V. A. Benassi, C. E. Overson, & C. M. Hakala (Eds.), Applying the science of learning in education: Infusing psychological science into the curriculum (pp. 131-141) . American Psychological Association.

Carpenter, S. K. (2017). Spacing effects in learning and memory. In J. T. Wixted (Ed.), Cognitive psychology of memory, Vol. 2, Learning & memory: A comprehensive reference, 2 nd edition, J. H. Byrne (Ed.), pp. 465-485. Oxford: Academic Press.

Carpenter, S. K. (2020). Distributed practice/spacing effect. In L.-f. Zhang (Ed.), Oxford Research Encyclopedia of Education . Oxford University Press.

Carpenter, S. K., & Vul, E. (2011). Delaying feedback by three seconds benefits retention of face-name pairs: The role of active anticipatory processing. Memory & Cognition, 39 , 1211–1221.

Carpenter, S. K., Cepeda, N. J., Rohrer, D., Kang, S. H. K., & Pashler, H. (2012). Using spacing to enhance diverse forms of learning: Review of recent research and implications for instruction. Educational Psychology Review, 24 , 369–378.

Carpenter, S. K., Rahman, S., & Perkins, K. (2018). The effects of prequestions on classroom learning. Journal of Experimental Psychology: Applied, 24 , 34–42.

Cepeda, N. J., Pasher, H., Vul, E., Wixted, J. T., & Rohrer, D. (2006). Distributed practice in verbal recall tasks: A review and quantitative synthesis. Psychological Bulletin, 132 , 354–380.

* Cerra, P. P., González, J. M. S., Parra, B. B., Ortiz, D. R., & Peñín, P. I. A. (2014). Can interactive web-based CAD tools improve the learning of engineering drawing? A case study. Journal of Science Education Technology, 23, 398-411.

* Chang, C.-Y. (2000). Enhancing tenth graders’ earth-science learning through computer-assisted instruction. Journal of Geoscience Education, 48, 636-640.

* Chang, R-C., & Yu, Z-S. (2018). Using augmented reality technologies to enhance students’ engagement and achievement in science laboratories. International Journal of Distance Education Technologies, 16, 54-72.

* Chang, K.-E., Wu, L.-J., Lai, S.-C., & Sung, Y.-T. (2016). Using mobile devices to enhance the interactive learning for spatial geometry. Interactive Learning Environments, 24, 916-934.

* Chen, J. C., Kadlowec, J. A., & Whittinghill, D. C. (2008). Using handheld computers for instantaneous feedback to enhance student learning and promote interaction. International Journal of Engineering Education, 24, 616-624.

Clunie, L., Morris, N. P., Joynes, V. C. T., & Pickering, J. D. (2018). How comprehensive are research studies investigating the efficacy of technology-enhanced learning resources in anatomy education? A systematic review. Anatomical Sciences Education, 11 , 303–319.

Corral, D., Carpenter, S. K., Perkins, K., & Gentile, D. A. (2020). Assessing students’ use of optional online lecture reviews. Applied Cognitive Psychology, 34 , 318–329.

Corral, D., Carpenter, S. K., & Clingan-Siverly, S. (in press). The effects of immediate versus delayed explanatory feedback on complex concept learning. Quarterly Journal of Experimental Psychology.

Cuban, L. (1986). Teachers and machines: The classroom use of technology since 1920 . New York: Teachers College Press.

* Daly, C. J., Bulloch, J. M., Ma, M., & Aidulis, D. (2016). A comparison of animated versus static images in an instructional multimedia presentation. Advances in Physiology Education, 40, 201-205.

* Debevc, M., Weiss, J., Šorgo, A., & Kožuh, I. (2020). Solfeggio learning and the influence of a mobile application based on visual, auditory and tactile modalities. British Journal of Educational Technology, 51, 177-193.

* Delafuente, J. C., Araujo, O. E., & Legg, S. M. (1998). Traditional lecture format compared to computer-assisted instruction in pharmacy calculations. American Journal of Pharmaceutical Education, 62, 62-66.

Delaney, P. F., Verkoeijen, P. P. J. L., & Spirgel, A. (2010). Spacing and testing effects: A deeply critical, lengthy, and at times discursive review of the literature. In B. H. Ross (Ed.), The psychology of learning & motivation: Advances in research & theory (Vol. 53 , pp. 63–147). New York: Academic Press.

Deslauriers, L., McCarty, L. S., Miller, K., Callaghan, K., & Kestin, G. (2019). Measuring actual learning versus feeling of learning in response to being actively engaged in the classroom. Proceedings of the National Academy of Sciences, 116 , 19251–19257.

* Dewhurst, D. G., Hardcastle, J., Hardcastle, P. T., & Stuart, E. (1994). Comparison of a computer simulation program and a traditional laboratory practical class for teaching the principles of intestinal absorption. Educational Experiments, 12, 95-104.

* Diliberto-Macaluso, K., & Hughes, A. (2016). The use of mobile apps to enhance student learning in introduction to psychology. Teaching of Psychology, 43, 48-52.

* Dorji, U., Panjaburee, P., & Srisawasdi, N. (2015). A learning cycle approach to developing educational computer game for improving students’ learning and awareness in electric energy consumption and conservation. Educational Technology & Society, 18, 91-105.

* Du, C. (2011). A comparison of traditional and blended learning in introductory principles of accounting course. American Journal of Business Education, 4, 1-10.

* Ebadi, S., & Ghuchi, K. D. (2018). Investigating the effects of blended learning approach on vocabulary enhancement from EFL learners’ perspective. i-Manager’s Journal on English Language Teaching, 8, 57-68.

* Ebadi, S., & Rahimi, M. (2017). Exploring the impact of online peer-editing using google docs on EFL learners’ academic writing skills: A mixed methods study. Computer Assisted Language Learning, 30, 787-815.

* Ebadi, S., & Rahimi, M. (2018). An exploration into the impact of WebQuest-based classroom on EFL learners’ critical thinking and academic writing skills: A mixed methods study. Computer Assisted Language Learning, 31, 617-651.

Ebbinghaus, H. (1885/1913). Memory (H. A. Ruger, C. E. Bussenius, Transl.). Teachers College, Columbia University, New York.

* Edwards, C. M., Rule, A. C., & Boody, R. M. (2013). Comparison of face-to-face and online mathematics learning of sixth graders. Journal of Computers in Mathematics & Science Teaching, 32, 25-47.

* Ellinger, R. S., & Frankland, P. (1976). Computer-assisted and lecture instruction: A comparative experiment. Journal of Geography, 75, 109-120.

* Englert, C. S., Zhao, Y., Collings, N., & Romig, N. (2005). Learning to read words: The effects of internet-based software on the improvement of reading performance. Remedial & Special Education, 26, 357-371.

* Fajardo-Lira, C., & Heiss, C. (2006). Comparing the effectiveness of a supplemental computer-based food safety tutorial to traditional education in an introductory food science course. Journal of Food Science Education, 5, 31-33.

Fernandez, J., & Jamet, E. (2017). Extending the testing effect to self-regulated learning. Metacognition & Learning, 12 , 131–156.

* Francescucci, A., & Foster, M. (2013). The VIRI (virtual, interactive, real-time, instructor-led) classroom: The impact of blended Synchronous online courses on student performance, engagement, and satisfaction. Canadian Journal of Higher Education, 43, 78-91.

* Francescucci, A., & Rohani, L. (2019). Exclusively synchronous online (VIRI) learning: The impact on student performance and engagement outcomes. Journal of Marketing Education, 41, 60-69.

Geller, J., Carpenter, S. K., Lamm, M. H., Rahman, S., Armstrong, P. I., & Coffman, C. R. (2017). Prequestions do not enhance the benefits of retrieval in a STEM classroom. Cognitive Research: Principles & Implications, 2 , 42.

Gerbier, E., & Toppino, T. C. (2015). The effect of distributed practice: Neuroscience, cognition, and education. Trends in Neuroscience & Education, 4 , 49–59.

* Gibbons, N. J., Evans, C., Payne, A., Shah, K., & Griffin, D. K. (2004). Computer simulations improve university instructional laboratories. Cell Biology Education, 3, 263-269.

* Goh, C. F., & Ong, E. T. (2019). Flipped classroom as an effective approach in enhancing student learning of a pharmacy course with a historically low student pass rate. Currents in Pharmacy Teaching & Learning, 11, 621-629.

Golonka, E. M., Bowles, A. R., Frank, V. M., Richardson, D. L., & Freynik, S. (2014). Technologies for foreign language learning: A review of technology types and their effectiveness. Computer Assisted Language Learning, 27 , 70–105.

* González, J. A., Jover, L. Cobo, E., & Muñoz, P. (2010). A web-based learning tool improves student performance in statistics: A randomized masked trial. Computers & Education, 55, 704-713.

Gray, L., Thomas, N., & Lewis, L. (2010). Teachers’ use of educational technology in US public schools: 2009. First look. NCES 2010-040 . Washington, DC: National Center for Education Statistics, Institute of Education Sciences, US Department of Education.

Grgurović, M., Chapelle, C. A., & Shelley, M. C. (2013). A meta-analysis of effectiveness studies on computer technology-supported language learning. ReCALL, 25 , 165–198.

* Hahn, W., Fairchild, C., & Dowis, W. B. (2013). Online homework managers and intelligent tutoring systems: A study of their impact on student learning in the introductory financial accounting classroom. Issues in Accounting Education, 28, 513-535.

* Harrington, D. (1999). Teaching statistics: A comparison of traditional classroom and programmed instruction/distance learning approaches. Journal of Social Work Education, 35, 343-352.

* Hollerbach, K., & Mims, B. (2007). Choosing wisely: A comparison of online, televised, and face-to-face instructional methods on knowledge acquisition of broadcast audience concepts. Journalism & Mass Communication Educator, 62, 176-189.

* Hsiao, H-S., Chen, J-C., Lin, C-Y., Zhuo, P-W., & Lin, K-Y. (2019). Using 3D printing technology with experiential learning strategies to improve preengineering students’ comprehension of abstract scientific concepts and hands-on ability. Journal of Computer Assisted Learning, 35, 178-187.

* Huang, H.-C. (2014). Online versus paper-based instruction: Comparing two strategy training modules for improving reading comprehension. RELC Journal, 45, 165-180.

* Jeffries, P. R. (2001). Computer versus lecture: A comparison of two methods of teaching oral medication administration in a nursing skills laboratory. Journal of Nursing Education, 40, 323-329.

* Johnson, S. D., Aragon, S. R., Shaik, N., & Palma-Rivas, N. (2000). Comparative analysis of learner satisfaction and learning outcomes in online and face-to-face learning environments. Journal of Interactive Learning Research, 11, 29-49.

* Johnson, D., Burnett, M., & Rolling, P. (2002). Comparison of internet and traditional classroom instruction in a consumer economics course. Journal of Family & Consumer Sciences Education, 20, 20-28.

* Karaksha, A., Grant, G., Nirthanan, S. N., Davey, A. K., & Anoopkumar-Dukie, S. (2014). A comparative study to evaluate the educational impact of e-learning tools on Griffith University pharmacy students’ level of understanding using Bloom’s and SOLO taxonomies. Education Research International, 1-11.

Karpicke, J. D. (2017). Retrieval-based learning: A decade of progress. In J. T. Wixted (Ed.), Cognitive psychology of memory, Vol. 2. Learning and memory: A comprehensive reference (J. H. Byrne, Series Ed.), pp. 487-514. Oxford: Academic Press.

* Kiliçkaya, F. (2015). Computer-based grammar instruction in an EFL context: Improving the effectiveness of teaching adverbial clauses. Computer Assisted Language Learning, 28, 325-340.

Kirkwood, A., & Price, L. (2013). Missing: Evidence of a scholarly approach to teaching and learning with technology in higher education. Teaching in Higher Education, 18 , 327–337.

Kirkwood, A., & Price, L. (2014). Technology-enhanced learning and teaching in higher education: What is ‘enhanced’ and how do we know? A critical literature review. Learning, Media, & Technology, 39 , 6–36.

Kornell, N., & Vaughn, K. E. (2016). How retrieval attempts affect learning: A review and synthesis. Psychology of Learning & Motivation, 65 , 183–215.

Kuepper-Tetzel, C. E. (2014). Strong effects on weak theoretical grounds: Understanding the distributed practice effect. Zeitschrift für Psychologie, 222 , 71–81.

* Kühl, T., & Münzer, S. (2019). The moderating role of additional information when learning with animations compared to static pictures. Instructional Science, 47, 659-677.

* Kunnath, B., & Kriek, J. (2018). Exploring effective pedagogies using computer simulations to improve grade 12 learners’ understanding of the photoelectric effect. African Journal of Research in Mathematics, Science & Technology Education, 22, 329-339.

* Lancellotti, M., Thomas, S., & Kohli, C. (2016). Online video modules for improvement in student learning. Journal of Education for Business, 91, 19-22.

Lee, S. W.-Y., & Tsai, C.-C. (2013). Technology-supported learning in secondary and undergraduate biological education: Observations from literature review. Journal of Science Education & Technology, 22 , 226–233.

* Lee, C. S. C., Rutecki, G. W., Whittier, F. C., Clarett, M. R., & Jarjoura, D. (1997). A comparison of interactive computerized medical education software with a more traditional teaching format. Teaching & Learning in Medicine, 9, 111-115.

* Lents, N. H., & Cifuentes, O. E. (2009). Web-based learning enhancements: Video lectures through voice-over powerpoint in a majors-level biology course. Journal of College Science Teaching, 39, 38-46.

* Lewis, J. L. (2015). A comparison between two different activities for teaching learning principles: Virtual animal labs versus human demonstrations. Scholarship of Teaching & Learning in Psychology, 1, 182-188.

Li, Q., & Ma, X. (2010). A meta-analysis of the effects of computer technology on school students’ mathematics learning. Educational Psychology Review, 22 , 215–243.

* Li, J-T., & Tong, F. (2019). Multimedia-assisted self-learning materials: The benefits of E-flashcards for vocabulary learning in Chinese as a foreign language. Reading & Writing, 32, 1175-1195.

* Lin, Y-T. (2019). Impacts of a flipped classroom with a smart learning diagnosis system on students’ learning performance, perception, and problem solving ability in a software engineering course. Computers in Human Behavior, 95, 187-196.

Little, J. L., & McDaniel, M. A. (2015). Metamemory monitoring and control following retrieval practice for text. Memory & Cognition, 43 , 85–98.

* Liu, H.-C., & Su, I.-H. (2011). Learning residential electrical wiring through computer simulation: The impact of computer-based learning environments on student achievement and cognitive load. British Journal of Educational Technology, 42, 598-607.

* Liu, T.-C., Lin, Y.-C., & Kinshuk. (2010). The application of simulation-assisted learning statistics (SALS) for correcting misconceptions and improving understanding of correlation. Journal of Computer Assisted Learning, 26, 143-158.

* Liu, K-P, Tai, S-J. D., & Liu, C-C. (2018). Enhancing language learning through creation: The effect of digital storytelling on student learning motivation and performance in a school English course. Educational Technology Research & Development, 66, 913-935.

* Lucchetti, A. L. G., Ezequiel, O. D. S., de Oliveira, I. N., Moreira-Almeida, A., & Lucchetti, G. (2018). Using traditional or flipped classrooms to teach “Geriatrics and Gerontology?” Investigating the impact of active learning on medical students’ competencies. Medical Teacher, 40, 1248-1256.

Lui, A. K.-F., Poon, M. H. M., & Wong, R. M. H. (2019). Automated generators of examples and problems for studying computer algorithms. Interactive Technology & Smart Education, 16 , 204–218.

* MacLaughlin, E. J., Supernaw, R. B., & Howard, K. A. (2004). Impact of distance learning using videoconferencing technology on student performance. American Journal of Pharmaceutical Education, 68, 58.

* Mathiowetz, V., Yu, C.-H., & Quake-Rapp, C. (2016). Comparison of a gross anatomy laboratory to online anatomy software for teaching anatomy. Anatomical Sciences Education, 9 , 52–59.

Mayer, R. E., & Moreno, R. (2002). Animation as an aid to multimedia learning. Educational Psychology Review, 14 , 87–99.

* McClean, P., Johnson, C., Rogers, R., Daniels, L., Reber, J., Slator, B. M., Terpstra, J., & White, A. (2005). Molecular and cellular biology animations: Development and impact on student learning. Cell Biology Education, 4, 169-175.

McDaniel, M. A., Agarwal, P. K., Huelser, B. J., McDermott, K. B., & Roediger III, H. L. (2011). Test-enhanced learning in a middle school science classroom: The effects of quiz frequency and placement. Journal of Educational Psychology, 103 , 399–414.

* McDonough, M., & Marks, I. M. (2002). Teaching medical students exposure therapy for phobia/panic – randomized, controlled comparison of face-to-face tutorial in small groups vs. solo computer instruction. Medical Education, 36, 412-417.

* McLaughlin, J. E., & Rhoney, D. H. (2015). Comparison of an interactive e-learning preparatory tool and a conventional downloadable handout used within a flipped neurologic pharmacotherapy lecture. Currents in Pharmacy Teaching & Learning, 7, 12-19.

* Mešić, V., Dervić, D., Gazibegović-Busuladžić, A., & Salibašić, D. (2015). Comparing the impact of dynamic and static media on students’ learning of one-dimensional kinematics. Eurasia Journal of Mathematics, Science & Technology Education, 11, 1119-1140.

* Nguyen, D. M., & Kulm, G. (2005). Using web-based practice to enhance mathematics learning and achievement. Journal of Interactive Online Learning, 100 1-16.

* Nguyen, J., & Paschal, C. B. (2002). Development of online ultrasound instructional module and comparison to traditional teaching methods. Journal of Engineering Education, 91, 275-283.

* Nikou, S. A., & Economides, A. A. (2018). Mobile-based micro-learning and assessment: Impact on learning performance and motivation of high school students. Journal of Computer Assisted Learning, 34, 269-278.

Nora, A., & Snyder, B. P. (2008). Technology and higher education: The impact of e-learning approaches on student academic achievement, perceptions and persistence. Journal of College Student Retention: Research, Theory & Practice, 10 , 3–19.

* Nouri, J., Cerratto-Pargman, T., Rossitto, C., & Ramberg, R. (2014). Learning with or without mobile devices? A comparison of traditional school fieldtrips and inquiry-based mobile learning activities. Research & Practice in Technology Enhanced Education, 9, 241-262.

* Oglesbee, T. W., Bitner, L. N., & Wright, G. B. (1988). Measurement of incremental benefits in computer enhanced instruction. Issues in Accounting Education, 3, 365-377.

* Olkun, S. (2003). Comparing computer versus concrete manipulatives in learning 2D geometry. Journal of Computers in Mathematics & Science Teaching, 22, 43-56.

* Pei, X., Jin, Y., Zheng, T., & Zhao, J. (2020). Longitudinal effect of a technology-enhanced learning environment on sixth-grade students’ science learning: The role of reflection. International Journal of Science Education, 42, 271-289.

* Perry, J. L., Cunningham, L. D., Gamage, J. K., & Kuehn, D. P. (2011). Do 3D stereoscopic computer animations improve student learning of surgical procedures? International Journal of Instructional Media, 38, 369-378.

Pressey, S. L. (1926). A simple apparatus which gives tests and scores—and teaches. School & Society, 23 , 373–376.

Pressey, S. L. (1927). A machine for automatic teaching of drill material. School & Society, 25 , 549–552.

Price, L., & Kirkwood, A. (2014). Using technology for teaching and learning in higher education: A critical review of the role of evidence in informing practice. Higher Education Research & Development, 33 , 549–564.

Rawson, K. A., & Dunlosky, J. (2011). Optimizing schedules of retrieval practice for durable and efficient learning: How much is enough? Journal of Experimental Psychology: General, 140 , 283–302.

Roediger III, H. L., & Butler, A. C. (2011). The critical role of retrieval practice in long-term retention. Trends in Cognitive Sciences, 15 , 20–27.

Roediger III, H. L., & Karpicke, J. D. (2006). Test-enhanced learning: Taking memory tests improves long-term retention. Psychological Science, 17 , 249–255.

Rohrer, D. (2015). Student instruction should be distributed over long time periods. Educational Psychology Review, 27 , 635–643.

Rosen, Y., & Salomon, G. (2007). The differential learning achievements of constructivist technology-intensive learning environments as compared with traditional ones: A meta-analysis. Journal of Educational Computing Research, 36 , 1–14.

Schacter, J., & Fagnano, C. (1999). Does computer technology improve student learning and achievement? How, when, and under what conditions? Journal of Educational Computing Research, 20 , 329–343.

* Schoenfeld-Tacher, R., McConnell, S., & Graham, M. (2001). Do no harm—A comparison of the effects of on-line vs. traditional delivery media on a science course. Journal of Science Education & Technology, 10, 257-265.

* Shadiev, R., Hwang, W-Y., & Liu, T-Y. (2018). Investigating the effectiveness of a learning activity supported by a mobile multimedia learning system to enhance autonomous EFL learning in authentic contexts. Educational Technology Research & Development, 66, 893-912.

* Siciliano, P. C., Jenks, M. A., Dana, M. N., & Talbert, B. A. (2011). The impact of audio technology on undergraduate instruction in a study abroad course on English gardens. NACTA Journal, 55, 46-53.

Skinner, B. F. (1958). Teaching machines. Science, 128 , 969–977.

* Spichtig, A. N., Gehsmann, K. M., Pascoe, J. P., & Ferrara, J. D. (2019). The impact of adaptive, web-based, scaffolded silent reading instruction on the reading achievement of students in grades 4 and 5. The Elementary School Journal, 119, 443-467.

* Steinweg, S. B., Davis, M. L., & Thomson, W. S. (2005). A comparison of traditional and online instruction in an introduction to special education course. Teacher Education & Special Education, 28, 62-73.

* Su, C.-H., & Cheng, C.-H. (2014). A mobile gamification learning system for improving the learning motivation and achievements. Journal of Computer Assisted Learning, 31, 268-286.

Swenson, P. W., & Evans, M. (2003). Hybrid courses as learning communities. In S. Reisman (Ed.), Electronic learning communities issues and practices (pp. 27–72). Greenwich, CT: Information Age Publishing.

Thalheimer, W., & Cook, S. (2019). How to calculate effect sizes from published research articles: A simplified methodology. Retrieved September 3, 2019 from http://work-learning.com/effect_sizes.htm .

* Tilidetzke, R. (1992). A comparison of CAI and traditional instruction in a college algebra course. Journal of Computers in Mathematics & Science Teaching, 11, 53-62.

* Turan, Z., Meral, E., & Sahin, I. F. (2018). The impact of mobile augmented reality in geography education: Achievements, cognitive loads and views of university students. Journal of Geography in Higher Education, 42, 427-441.

* Verdugo, D. R., & Belmonte, I. A. (2007). Using digital stories to improve listening comprehension with Spanish young learners of English. Language Learning & Technology, 11, 87-101.

* Vichitvejpaisal, P., Sitthikongsak, S., Preechakoon, B., Kraiprasit, K., Parakkamodom, S., Manon, C., & Petcharatana, S. (2001). Does computer-assisted instruction really help to improve the learning process? Medical Education, 35, 983-989.

* Wang, S., & Sleeman, P. J. (1993). A comparison of the relative effectiveness of computer-assisted instruction and conventional methods for teaching an operations management course in a school of business. International Journal of Instructional Media, 20, 225-234.

* Wiebe, J. H., & Martin, N. J. (1994). The impact of a computer-based adventure game on achievement and attitudes in geography. Journal of Computing in Childhood Education, 5, 61-71.

* Wiesner, T. F., & Lan, W. (2004). Comparison of student learning in physical and simulated unit operations experiments. Journal of Engineering Education, 93, 195-204.

* William, A., Vidal, V. L., & John, P. (2016). Traditional instruction versus virtual reality simulation: A comparative study of phlebotomy training among nursing students in Kuwait. Journal of Education & Practice, 7, 18-25.

* Wu, T-T. (2018). Improving the effectiveness of English vocabulary review by integrating ARCS with mobile game-based learning. Journal of Computer Assisted Learning, 34, 315-323.

* Yarahmadzehi, N., & Goodarzi, M. (2020). Investigating the role of formative mobile based assessment in vocabulary learning of pre-intermediate EFL learners in comparison with paper based assessment. Turkish Online Journal of Distance Education, 21, 181-196.

* Yildirim, Z., Ozden, M. Y., & Aksu, M. (2001). Comparison of hypermedia learning and traditional instruction on knowledge acquisition and retention. The Journal of Educational Research, 94, 207-214.

* Zaini, A., & Mazdayasna, G. (2015). The impact of computer-based instruction on the development of EFL learners’ writing skills. Journal of Computer Assisted Learning, 31, 516-528.

* Zubas, P., Heiss, C., & Pedersen, M. (2006). Comparing the effectiveness of a supplemental online tutorial to traditional instruction with nutritional science students. Journal of Interactive Online Learning, 5, 75-81.

Download references

Author information

Authors and affiliations.

Department of Psychology, Iowa State University, W112 Lagomarcino Hall, 901 Stange Road, Ames, IA, 50011, USA

Kam Leung Yeung & Shana K. Carpenter

Department of Psychology, Syracuse University, Syracuse, NY, USA

Daniel Corral

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Shana K. Carpenter .

Ethics declarations

Conflict of interest.

Shana Carpenter has received grants from the National Science Foundation (DUE 1504480) and the James S. McDonnell Foundation (220020483).

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This material is based upon work supported by the James S. McDonnell Foundation 21 st Century Science Initiative in Understanding Human Cognition, Collaborative Grant No. 220020483. We thank Sierra Lauber, Luke Huber, and Kyle St. Hilaire for their help in locating articles.

Rights and permissions

Reprints and permissions

About this article

Yeung, K.L., Carpenter, S.K. & Corral, D. A Comprehensive Review of Educational Technology on Objective Learning Outcomes in Academic Contexts. Educ Psychol Rev 33 , 1583–1630 (2021). https://doi.org/10.1007/s10648-020-09592-4

Download citation

Accepted : 28 December 2020

Published : 05 April 2021

Issue Date : December 2021

DOI : https://doi.org/10.1007/s10648-020-09592-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Cognitive Science
• Effective Learning Principles
• Find a journal
• Publish with us
• Track your research

Along with Stanford news and stories, show me:

• Student information
• Faculty/Staff information

We want to provide announcements, events, leadership messages and resources that are relevant to you. Your selection is stored in a browser cookie which you can remove at any time using “Clear all personalization” below.

Image credit: Claire Scully

New advances in technology are upending education, from the recent debut of new artificial intelligence (AI) chatbots like ChatGPT to the growing accessibility of virtual-reality tools that expand the boundaries of the classroom. For educators, at the heart of it all is the hope that every learner gets an equal chance to develop the skills they need to succeed. But that promise is not without its pitfalls.

“Technology is a game-changer for education – it offers the prospect of universal access to high-quality learning experiences, and it creates fundamentally new ways of teaching,” said Dan Schwartz, dean of Stanford Graduate School of Education (GSE), who is also a professor of educational technology at the GSE and faculty director of the Stanford Accelerator for Learning . “But there are a lot of ways we teach that aren’t great, and a big fear with AI in particular is that we just get more efficient at teaching badly. This is a moment to pay attention, to do things differently.”

For K-12 schools, this year also marks the end of the Elementary and Secondary School Emergency Relief (ESSER) funding program, which has provided pandemic recovery funds that many districts used to invest in educational software and systems. With these funds running out in September 2024, schools are trying to determine their best use of technology as they face the prospect of diminishing resources.

Here, Schwartz and other Stanford education scholars weigh in on some of the technology trends taking center stage in the classroom this year.

AI in the classroom

In 2023, the big story in technology and education was generative AI, following the introduction of ChatGPT and other chatbots that produce text seemingly written by a human in response to a question or prompt. Educators immediately worried that students would use the chatbot to cheat by trying to pass its writing off as their own. As schools move to adopt policies around students’ use of the tool, many are also beginning to explore potential opportunities – for example, to generate reading assignments or coach students during the writing process.

AI can also help automate tasks like grading and lesson planning, freeing teachers to do the human work that drew them into the profession in the first place, said Victor Lee, an associate professor at the GSE and faculty lead for the AI + Education initiative at the Stanford Accelerator for Learning. “I’m heartened to see some movement toward creating AI tools that make teachers’ lives better – not to replace them, but to give them the time to do the work that only teachers are able to do,” he said. “I hope to see more on that front.”

He also emphasized the need to teach students now to begin questioning and critiquing the development and use of AI. “AI is not going away,” said Lee, who is also director of CRAFT (Classroom-Ready Resources about AI for Teaching), which provides free resources to help teach AI literacy to high school students across subject areas. “We need to teach students how to understand and think critically about this technology.”

Immersive environments

The use of immersive technologies like augmented reality, virtual reality, and mixed reality is also expected to surge in the classroom, especially as new high-profile devices integrating these realities hit the marketplace in 2024.

The educational possibilities now go beyond putting on a headset and experiencing life in a distant location. With new technologies, students can create their own local interactive 360-degree scenarios, using just a cell phone or inexpensive camera and simple online tools.

“This is an area that’s really going to explode over the next couple of years,” said Kristen Pilner Blair, director of research for the Digital Learning initiative at the Stanford Accelerator for Learning, which runs a program exploring the use of virtual field trips to promote learning. “Students can learn about the effects of climate change, say, by virtually experiencing the impact on a particular environment. But they can also become creators, documenting and sharing immersive media that shows the effects where they live.”

Integrating AI into virtual simulations could also soon take the experience to another level, Schwartz said. “If your VR experience brings me to a redwood tree, you could have a window pop up that allows me to ask questions about the tree, and AI can deliver the answers.”

Gamification

Another trend expected to intensify this year is the gamification of learning activities, often featuring dynamic videos with interactive elements to engage and hold students’ attention.

“Gamification is a good motivator, because one key aspect is reward, which is very powerful,” said Schwartz. The downside? Rewards are specific to the activity at hand, which may not extend to learning more generally. “If I get rewarded for doing math in a space-age video game, it doesn’t mean I’m going to be motivated to do math anywhere else.”

Gamification sometimes tries to make “chocolate-covered broccoli,” Schwartz said, by adding art and rewards to make speeded response tasks involving single-answer, factual questions more fun. He hopes to see more creative play patterns that give students points for rethinking an approach or adapting their strategy, rather than only rewarding them for quickly producing a correct response.

Data-gathering and analysis

The growing use of technology in schools is producing massive amounts of data on students’ activities in the classroom and online. “We’re now able to capture moment-to-moment data, every keystroke a kid makes,” said Schwartz – data that can reveal areas of struggle and different learning opportunities, from solving a math problem to approaching a writing assignment.

But outside of research settings, he said, that type of granular data – now owned by tech companies – is more likely used to refine the design of the software than to provide teachers with actionable information.

The promise of personalized learning is being able to generate content aligned with students’ interests and skill levels, and making lessons more accessible for multilingual learners and students with disabilities. Realizing that promise requires that educators can make sense of the data that’s being collected, said Schwartz – and while advances in AI are making it easier to identify patterns and findings, the data also needs to be in a system and form educators can access and analyze for decision-making. Developing a usable infrastructure for that data, Schwartz said, is an important next step.

With the accumulation of student data comes privacy concerns: How is the data being collected? Are there regulations or guidelines around its use in decision-making? What steps are being taken to prevent unauthorized access? In 2023 K-12 schools experienced a rise in cyberattacks, underscoring the need to implement strong systems to safeguard student data.

Technology is “requiring people to check their assumptions about education,” said Schwartz, noting that AI in particular is very efficient at replicating biases and automating the way things have been done in the past, including poor models of instruction. “But it’s also opening up new possibilities for students producing material, and for being able to identify children who are not average so we can customize toward them. It’s an opportunity to think of entirely new ways of teaching – this is the path I hope to see.”

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Springer Nature - PMC COVID-19 Collection

Impacts of digital technologies on education and factors influencing schools' digital capacity and transformation: A literature review

Stella timotheou.

1 CYENS Center of Excellence & Cyprus University of Technology (Cyprus Interaction Lab), Cyprus, CYENS Center of Excellence & Cyprus University of Technology, Nicosia-Limassol, Cyprus

Ourania Miliou

Yiannis dimitriadis.

2 Universidad de Valladolid (UVA), Spain, Valladolid, Spain

Sara Villagrá Sobrino

Nikoleta giannoutsou, romina cachia.

3 JRC - Joint Research Centre of the European Commission, Seville, Spain

Alejandra Martínez Monés

Andri ioannou, associated data.

Data sharing not applicable to this article as no datasets were generated or analysed during the current study.

Digital technologies have brought changes to the nature and scope of education and led education systems worldwide to adopt strategies and policies for ICT integration. The latter brought about issues regarding the quality of teaching and learning with ICTs, especially concerning the understanding, adaptation, and design of the education systems in accordance with current technological trends. These issues were emphasized during the recent COVID-19 pandemic that accelerated the use of digital technologies in education, generating questions regarding digitalization in schools. Specifically, many schools demonstrated a lack of experience and low digital capacity, which resulted in widening gaps, inequalities, and learning losses. Such results have engendered the need for schools to learn and build upon the experience to enhance their digital capacity and preparedness, increase their digitalization levels, and achieve a successful digital transformation. Given that the integration of digital technologies is a complex and continuous process that impacts different actors within the school ecosystem, there is a need to show how these impacts are interconnected and identify the factors that can encourage an effective and efficient change in the school environments. For this purpose, we conducted a non-systematic literature review. The results of the literature review were organized thematically based on the evidence presented about the impact of digital technology on education and the factors that affect the schools’ digital capacity and digital transformation. The findings suggest that ICT integration in schools impacts more than just students’ performance; it affects several other school-related aspects and stakeholders, too. Furthermore, various factors affect the impact of digital technologies on education. These factors are interconnected and play a vital role in the digital transformation process. The study results shed light on how ICTs can positively contribute to the digital transformation of schools and which factors should be considered for schools to achieve effective and efficient change.

Introduction

Digital technologies have brought changes to the nature and scope of education. Versatile and disruptive technological innovations, such as smart devices, the Internet of Things (IoT), artificial intelligence (AI), augmented reality (AR) and virtual reality (VR), blockchain, and software applications have opened up new opportunities for advancing teaching and learning (Gaol & Prasolova-Førland, 2021 ; OECD, 2021 ). Hence, in recent years, education systems worldwide have increased their investment in the integration of information and communication technology (ICT) (Fernández-Gutiérrez et al., 2020 ; Lawrence & Tar, 2018 ) and prioritized their educational agendas to adapt strategies or policies around ICT integration (European Commission, 2019 ). The latter brought about issues regarding the quality of teaching and learning with ICTs (Bates, 2015 ), especially concerning the understanding, adaptation, and design of education systems in accordance with current technological trends (Balyer & Öz, 2018 ). Studies have shown that despite the investment made in the integration of technology in schools, the results have not been promising, and the intended outcomes have not yet been achieved (Delgado et al., 2015 ; Lawrence & Tar, 2018 ). These issues were exacerbated during the COVID-19 pandemic, which forced teaching across education levels to move online (Daniel, 2020 ). Online teaching accelerated the use of digital technologies generating questions regarding the process, the nature, the extent, and the effectiveness of digitalization in schools (Cachia et al., 2021 ; König et al., 2020 ). Specifically, many schools demonstrated a lack of experience and low digital capacity, which resulted in widening gaps, inequalities, and learning losses (Blaskó et al., 2021 ; Di Pietro et al, 2020 ). Such results have engendered the need for schools to learn and build upon the experience in order to enhance their digital capacity (European Commission, 2020 ) and increase their digitalization levels (Costa et al., 2021 ). Digitalization offers possibilities for fundamental improvement in schools (OECD, 2021 ; Rott & Marouane, 2018 ) and touches many aspects of a school’s development (Delcker & Ifenthaler, 2021 ) . However, it is a complex process that requires large-scale transformative changes beyond the technical aspects of technology and infrastructure (Pettersson, 2021 ). Namely, digitalization refers to “ a series of deep and coordinated culture, workforce, and technology shifts and operating models ” (Brooks & McCormack, 2020 , p. 3) that brings cultural, organizational, and operational change through the integration of digital technologies (JISC, 2020 ). A successful digital transformation requires that schools increase their digital capacity levels, establishing the necessary “ culture, policies, infrastructure as well as digital competence of students and staff to support the effective integration of technology in teaching and learning practices ” (Costa et al, 2021 , p.163).

Given that the integration of digital technologies is a complex and continuous process that impacts different actors within the school ecosystem (Eng, 2005 ), there is a need to show how the different elements of the impact are interconnected and to identify the factors that can encourage an effective and efficient change in the school environment. To address the issues outlined above, we formulated the following research questions:

a) What is the impact of digital technologies on education?

b) Which factors might affect a school’s digital capacity and transformation?

In the present investigation, we conducted a non-systematic literature review of publications pertaining to the impact of digital technologies on education and the factors that affect a school’s digital capacity and transformation. The results of the literature review were organized thematically based on the evidence presented about the impact of digital technology on education and the factors which affect the schools’ digital capacity and digital transformation.

Methodology

The non-systematic literature review presented herein covers the main theories and research published over the past 17 years on the topic. It is based on meta-analyses and review papers found in scholarly, peer-reviewed content databases and other key studies and reports related to the concepts studied (e.g., digitalization, digital capacity) from professional and international bodies (e.g., the OECD). We searched the Scopus database, which indexes various online journals in the education sector with an international scope, to collect peer-reviewed academic papers. Furthermore, we used an all-inclusive Google Scholar search to include relevant key terms or to include studies found in the reference list of the peer-reviewed papers, and other key studies and reports related to the concepts studied by professional and international bodies. Lastly, we gathered sources from the Publications Office of the European Union ( https://op.europa.eu/en/home ); namely, documents that refer to policies related to digital transformation in education.

Regarding search terms, we first searched resources on the impact of digital technologies on education by performing the following search queries: “impact” OR “effects” AND “digital technologies” AND “education”, “impact” OR “effects” AND “ICT” AND “education”. We further refined our results by adding the terms “meta-analysis” and “review” or by adjusting the search options based on the features of each database to avoid collecting individual studies that would provide limited contributions to a particular domain. We relied on meta-analyses and review studies as these consider the findings of multiple studies to offer a more comprehensive view of the research in a given area (Schuele & Justice, 2006 ). Specifically, meta-analysis studies provided quantitative evidence based on statistically verifiable results regarding the impact of educational interventions that integrate digital technologies in school classrooms (Higgins et al., 2012 ; Tolani-Brown et al., 2011 ).

However, quantitative data does not offer explanations for the challenges or difficulties experienced during ICT integration in learning and teaching (Tolani-Brown et al., 2011 ). To fill this gap, we analyzed literature reviews and gathered in-depth qualitative evidence of the benefits and implications of technology integration in schools. In the analysis presented herein, we also included policy documents and reports from professional and international bodies and governmental reports, which offered useful explanations of the key concepts of this study and provided recent evidence on digital capacity and transformation in education along with policy recommendations. The inclusion and exclusion criteria that were considered in this study are presented in Table ​ Table1 1 .

Inclusion and exclusion criteria for the selection of resources on the impact of digital technologies on education

To ensure a reliable extraction of information from each study and assist the research synthesis we selected the study characteristics of interest (impact) and constructed coding forms. First, an overview of the synthesis was provided by the principal investigator who described the processes of coding, data entry, and data management. The coders followed the same set of instructions but worked independently. To ensure a common understanding of the process between coders, a sample of ten studies was tested. The results were compared, and the discrepancies were identified and resolved. Additionally, to ensure an efficient coding process, all coders participated in group meetings to discuss additions, deletions, and modifications (Stock, 1994 ). Due to the methodological diversity of the studied documents we began to synthesize the literature review findings based on similar study designs. Specifically, most of the meta-analysis studies were grouped in one category due to the quantitative nature of the measured impact. These studies tended to refer to student achievement (Hattie et al., 2014 ). Then, we organized the themes of the qualitative studies in several impact categories. Lastly, we synthesized both review and meta-analysis data across the categories. In order to establish a collective understanding of the concept of impact, we referred to a previous impact study by Balanskat ( 2009 ) which investigated the impact of technology in primary schools. In this context, the impact had a more specific ICT-related meaning and was described as “ a significant influence or effect of ICT on the measured or perceived quality of (parts of) education ” (Balanskat, 2009 , p. 9). In the study presented herein, the main impacts are in relation to learning and learners, teaching, and teachers, as well as other key stakeholders who are directly or indirectly connected to the school unit.

The study’s results identified multiple dimensions of the impact of digital technologies on students’ knowledge, skills, and attitudes; on equality, inclusion, and social integration; on teachers’ professional and teaching practices; and on other school-related aspects and stakeholders. The data analysis indicated various factors that might affect the schools’ digital capacity and transformation, such as digital competencies, the teachers’ personal characteristics and professional development, as well as the school’s leadership and management, administration, infrastructure, etc. The impacts and factors found in the literature review are presented below.

Impacts of digital technologies on students’ knowledge, skills, attitudes, and emotions

The impact of ICT use on students’ knowledge, skills, and attitudes has been investigated early in the literature. Eng ( 2005 ) found a small positive effect between ICT use and students' learning. Specifically, the author reported that access to computer-assisted instruction (CAI) programs in simulation or tutorial modes—used to supplement rather than substitute instruction – could enhance student learning. The author reported studies showing that teachers acknowledged the benefits of ICT on pupils with special educational needs; however, the impact of ICT on students' attainment was unclear. Balanskat et al. ( 2006 ) found a statistically significant positive association between ICT use and higher student achievement in primary and secondary education. The authors also reported improvements in the performance of low-achieving pupils. The use of ICT resulted in further positive gains for students, namely increased attention, engagement, motivation, communication and process skills, teamwork, and gains related to their behaviour towards learning. Evidence from qualitative studies showed that teachers, students, and parents recognized the positive impact of ICT on students' learning regardless of their competence level (strong/weak students). Punie et al. ( 2006 ) documented studies that showed positive results of ICT-based learning for supporting low-achieving pupils and young people with complex lives outside the education system. Liao et al. ( 2007 ) reported moderate positive effects of computer application instruction (CAI, computer simulations, and web-based learning) over traditional instruction on primary school student's achievement. Similarly, Tamim et al. ( 2011 ) reported small to moderate positive effects between the use of computer technology (CAI, ICT, simulations, computer-based instruction, digital and hypermedia) and student achievement in formal face-to-face classrooms compared to classrooms that did not use technology. Jewitt et al., ( 2011 ) found that the use of learning platforms (LPs) (virtual learning environments, management information systems, communication technologies, and information- and resource-sharing technologies) in schools allowed primary and secondary students to access a wider variety of quality learning resources, engage in independent and personalized learning, and conduct self- and peer-review; LPs also provide opportunities for teacher assessment and feedback. Similar findings were reported by Fu ( 2013 ), who documented a list of benefits and opportunities of ICT use. According to the author, the use of ICTs helps students access digital information and course content effectively and efficiently, supports student-centered and self-directed learning, as well as the development of a creative learning environment where more opportunities for critical thinking skills are offered, and promotes collaborative learning in a distance-learning environment. Higgins et al. ( 2012 ) found consistent but small positive associations between the use of technology and learning outcomes of school-age learners (5–18-year-olds) in studies linking the provision and use of technology with attainment. Additionally, Chauhan ( 2017 ) reported a medium positive effect of technology on the learning effectiveness of primary school students compared to students who followed traditional learning instruction.

The rise of mobile technologies and hardware devices instigated investigations into their impact on teaching and learning. Sung et al. ( 2016 ) reported a moderate effect on students' performance from the use of mobile devices in the classroom compared to the use of desktop computers or the non-use of mobile devices. Schmid et al. ( 2014 ) reported medium–low to low positive effects of technology integration (e.g., CAI, ICTs) in the classroom on students' achievement and attitude compared to not using technology or using technology to varying degrees. Tamim et al. ( 2015 ) found a low statistically significant effect of the use of tablets and other smart devices in educational contexts on students' achievement outcomes. The authors suggested that tablets offered additional advantages to students; namely, they reported improvements in students’ notetaking, organizational and communication skills, and creativity. Zheng et al. ( 2016 ) reported a small positive effect of one-to-one laptop programs on students’ academic achievement across subject areas. Additional reported benefits included student-centered, individualized, and project-based learning enhanced learner engagement and enthusiasm. Additionally, the authors found that students using one-to-one laptop programs tended to use technology more frequently than in non-laptop classrooms, and as a result, they developed a range of skills (e.g., information skills, media skills, technology skills, organizational skills). Haßler et al. ( 2016 ) found that most interventions that included the use of tablets across the curriculum reported positive learning outcomes. However, from 23 studies, five reported no differences, and two reported a negative effect on students' learning outcomes. Similar results were indicated by Kalati and Kim ( 2022 ) who investigated the effect of touchscreen technologies on young students’ learning. Specifically, from 53 studies, 34 advocated positive effects of touchscreen devices on children’s learning, 17 obtained mixed findings and two studies reported negative effects.

More recently, approaches that refer to the impact of gamification with the use of digital technologies on teaching and learning were also explored. A review by Pan et al. ( 2022 ) that examined the role of learning games in fostering mathematics education in K-12 settings, reported that gameplay improved students’ performance. Integration of digital games in teaching was also found as a promising pedagogical practice in STEM education that could lead to increased learning gains (Martinez et al., 2022 ; Wang et al., 2022 ). However, although Talan et al. ( 2020 ) reported a medium effect of the use of educational games (both digital and non-digital) on academic achievement, the effect of non-digital games was higher.

Over the last two years, the effects of more advanced technologies on teaching and learning were also investigated. Garzón and Acevedo ( 2019 ) found that AR applications had a medium effect on students' learning outcomes compared to traditional lectures. Similarly, Garzón et al. ( 2020 ) showed that AR had a medium impact on students' learning gains. VR applications integrated into various subjects were also found to have a moderate effect on students’ learning compared to control conditions (traditional classes, e.g., lectures, textbooks, and multimedia use, e.g., images, videos, animation, CAI) (Chen et al., 2022b ). Villena-Taranilla et al. ( 2022 ) noted the moderate effect of VR technologies on students’ learning when these were applied in STEM disciplines. In the same meta-analysis, Villena-Taranilla et al. ( 2022 ) highlighted the role of immersive VR, since its effect on students’ learning was greater (at a high level) across educational levels (K-6) compared to semi-immersive and non-immersive integrations. In another meta-analysis study, the effect size of the immersive VR was small and significantly differentiated across educational levels (Coban et al., 2022 ). The impact of AI on education was investigated by Su and Yang ( 2022 ) and Su et al. ( 2022 ), who showed that this technology significantly improved students’ understanding of AI computer science and machine learning concepts.

It is worth noting that the vast majority of studies referred to learning gains in specific subjects. Specifically, several studies examined the impact of digital technologies on students’ literacy skills and reported positive effects on language learning (Balanskat et al., 2006 ; Grgurović et al., 2013 ; Friedel et al., 2013 ; Zheng et al., 2016 ; Chen et al., 2022b ; Savva et al., 2022 ). Also, several studies documented positive effects on specific language learning areas, namely foreign language learning (Kao, 2014 ), writing (Higgins et al., 2012 ; Wen & Walters, 2022 ; Zheng et al., 2016 ), as well as reading and comprehension (Cheung & Slavin, 2011 ; Liao et al., 2007 ; Schwabe et al., 2022 ). ICTs were also found to have a positive impact on students' performance in STEM (science, technology, engineering, and mathematics) disciplines (Arztmann et al., 2022 ; Bado, 2022 ; Villena-Taranilla et al., 2022 ; Wang et al., 2022 ). Specifically, a number of studies reported positive impacts on students’ achievement in mathematics (Balanskat et al., 2006 ; Hillmayr et al., 2020 ; Li & Ma, 2010 ; Pan et al., 2022 ; Ran et al., 2022 ; Verschaffel et al., 2019 ; Zheng et al., 2016 ). Furthermore, studies documented positive effects of ICTs on science learning (Balanskat et al., 2006 ; Liao et al., 2007 ; Zheng et al., 2016 ; Hillmayr et al., 2020 ; Kalemkuş & Kalemkuş, 2022 ; Lei et al., 2022a ). Çelik ( 2022 ) also noted that computer simulations can help students understand learning concepts related to science. Furthermore, some studies documented that the use of ICTs had a positive impact on students’ achievement in other subjects, such as geography, history, music, and arts (Chauhan, 2017 ; Condie & Munro, 2007 ), and design and technology (Balanskat et al., 2006 ).

More specific positive learning gains were reported in a number of skills, e.g., problem-solving skills and pattern exploration skills (Higgins et al., 2012 ), metacognitive learning outcomes (Verschaffel et al., 2019 ), literacy skills, computational thinking skills, emotion control skills, and collaborative inquiry skills (Lu et al., 2022 ; Su & Yang, 2022 ; Su et al., 2022 ). Additionally, several investigations have reported benefits from the use of ICT on students’ creativity (Fielding & Murcia, 2022 ; Liu et al., 2022 ; Quah & Ng, 2022 ). Lastly, digital technologies were also found to be beneficial for enhancing students’ lifelong learning skills (Haleem et al., 2022 ).

Apart from gaining knowledge and skills, studies also reported improvement in motivation and interest in mathematics (Higgins et. al., 2019 ; Fadda et al., 2022 ) and increased positive achievement emotions towards several subjects during interventions using educational games (Lei et al., 2022a ). Chen et al. ( 2022a ) also reported a small but positive effect of digital health approaches in bullying and cyberbullying interventions with K-12 students, demonstrating that technology-based approaches can help reduce bullying and related consequences by providing emotional support, empowerment, and change of attitude. In their meta-review study, Su et al. ( 2022 ) also documented that AI technologies effectively strengthened students’ attitudes towards learning. In another meta-analysis, Arztmann et al. ( 2022 ) reported positive effects of digital games on motivation and behaviour towards STEM subjects.

Impacts of digital technologies on equality, inclusion and social integration

Although most of the reviewed studies focused on the impact of ICTs on students’ knowledge, skills, and attitudes, reports were also made on other aspects in the school context, such as equality, inclusion, and social integration. Condie and Munro ( 2007 ) documented research interventions investigating how ICT can support pupils with additional or special educational needs. While those interventions were relatively small scale and mostly based on qualitative data, their findings indicated that the use of ICTs enabled the development of communication, participation, and self-esteem. A recent meta-analysis (Baragash et al., 2022 ) with 119 participants with different disabilities, reported a significant overall effect size of AR on their functional skills acquisition. Koh’s meta-analysis ( 2022 ) also revealed that students with intellectual and developmental disabilities improved their competence and performance when they used digital games in the lessons.

Istenic Starcic and Bagon ( 2014 ) found that the role of ICT in inclusion and the design of pedagogical and technological interventions was not sufficiently explored in educational interventions with people with special needs; however, some benefits of ICT use were found in students’ social integration. The issue of gender and technology use was mentioned in a small number of studies. Zheng et al. ( 2016 ) reported a statistically significant positive interaction between one-to-one laptop programs and gender. Specifically, the results showed that girls and boys alike benefitted from the laptop program, but the effect on girls’ achievement was smaller than that on boys’. Along the same lines, Arztmann et al. ( 2022 ) reported no difference in the impact of game-based learning between boys and girls, arguing that boys and girls equally benefited from game-based interventions in STEM domains. However, results from a systematic review by Cussó-Calabuig et al. ( 2018 ) found limited and low-quality evidence on the effects of intensive use of computers on gender differences in computer anxiety, self-efficacy, and self-confidence. Based on their view, intensive use of computers can reduce gender differences in some areas and not in others, depending on contextual and implementation factors.

Impacts of digital technologies on teachers’ professional and teaching practices

Various research studies have explored the impact of ICT on teachers’ instructional practices and student assessment. Friedel et al. ( 2013 ) found that the use of mobile devices by students enabled teachers to successfully deliver content (e.g., mobile serious games), provide scaffolding, and facilitate synchronous collaborative learning. The integration of digital games in teaching and learning activities also gave teachers the opportunity to study and apply various pedagogical practices (Bado, 2022 ). Specifically, Bado ( 2022 ) found that teachers who implemented instructional activities in three stages (pre-game, game, and post-game) maximized students’ learning outcomes and engagement. For instance, during the pre-game stage, teachers focused on lectures and gameplay training, at the game stage teachers provided scaffolding on content, addressed technical issues, and managed the classroom activities. During the post-game stage, teachers organized activities for debriefing to ensure that the gameplay had indeed enhanced students’ learning outcomes.

Furthermore, ICT can increase efficiency in lesson planning and preparation by offering possibilities for a more collaborative approach among teachers. The sharing of curriculum plans and the analysis of students’ data led to clearer target settings and improvements in reporting to parents (Balanskat et al., 2006 ).

Additionally, the use and application of digital technologies in teaching and learning were found to enhance teachers’ digital competence. Balanskat et al. ( 2006 ) documented studies that revealed that the use of digital technologies in education had a positive effect on teachers’ basic ICT skills. The greatest impact was found on teachers with enough experience in integrating ICTs in their teaching and/or who had recently participated in development courses for the pedagogical use of technologies in teaching. Punie et al. ( 2006 ) reported that the provision of fully equipped multimedia portable computers and the development of online teacher communities had positive impacts on teachers’ confidence and competence in the use of ICTs.

Moreover, online assessment via ICTs benefits instruction. In particular, online assessments support the digitalization of students’ work and related logistics, allow teachers to gather immediate feedback and readjust to new objectives, and support the improvement of the technical quality of tests by providing more accurate results. Additionally, the capabilities of ICTs (e.g., interactive media, simulations) create new potential methods of testing specific skills, such as problem-solving and problem-processing skills, meta-cognitive skills, creativity and communication skills, and the ability to work productively in groups (Punie et al., 2006 ).

Impacts of digital technologies on other school-related aspects and stakeholders

There is evidence that the effective use of ICTs and the data transmission offered by broadband connections help improve administration (Balanskat et al., 2006 ). Specifically, ICTs have been found to provide better management systems to schools that have data gathering procedures in place. Condie and Munro ( 2007 ) reported impacts from the use of ICTs in schools in the following areas: attendance monitoring, assessment records, reporting to parents, financial management, creation of repositories for learning resources, and sharing of information amongst staff. Such data can be used strategically for self-evaluation and monitoring purposes which in turn can result in school improvements. Additionally, they reported that online access to other people with similar roles helped to reduce headteachers’ isolation by offering them opportunities to share insights into the use of ICT in learning and teaching and how it could be used to support school improvement. Furthermore, ICTs provided more efficient and successful examination management procedures, namely less time-consuming reporting processes compared to paper-based examinations and smooth communications between schools and examination authorities through electronic data exchange (Punie et al., 2006 ).

Zheng et al. ( 2016 ) reported that the use of ICTs improved home-school relationships. Additionally, Escueta et al. ( 2017 ) reported several ICT programs that had improved the flow of information from the school to parents. Particularly, they documented that the use of ICTs (learning management systems, emails, dedicated websites, mobile phones) allowed for personalized and customized information exchange between schools and parents, such as attendance records, upcoming class assignments, school events, and students’ grades, which generated positive results on students’ learning outcomes and attainment. Such information exchange between schools and families prompted parents to encourage their children to put more effort into their schoolwork.

The above findings suggest that the impact of ICT integration in schools goes beyond students’ performance in school subjects. Specifically, it affects a number of school-related aspects, such as equality and social integration, professional and teaching practices, and diverse stakeholders. In Table ​ Table2, 2 , we summarize the different impacts of digital technologies on school stakeholders based on the literature review, while in Table ​ Table3 3 we organized the tools/platforms and practices/policies addressed in the meta-analyses, literature reviews, EU reports, and international bodies included in the manuscript.

The impact of digital technologies on schools’ stakeholders based on the literature review

Tools/platforms and practices/policies addressed in the meta-analyses, literature reviews, EU reports, and international bodies included in the manuscript

Additionally, based on the results of the literature review, there are many types of digital technologies with different affordances (see, for example, studies on VR vs Immersive VR), which evolve over time (e.g. starting from CAIs in 2005 to Augmented and Virtual reality 2020). Furthermore, these technologies are linked to different pedagogies and policy initiatives, which are critical factors in the study of impact. Table ​ Table3 3 summarizes the different tools and practices that have been used to examine the impact of digital technologies on education since 2005 based on the review results.

Factors that affect the integration of digital technologies

Although the analysis of the literature review demonstrated different impacts of the use of digital technology on education, several authors highlighted the importance of various factors, besides the technology itself, that affect this impact. For example, Liao et al. ( 2007 ) suggested that future studies should carefully investigate which factors contribute to positive outcomes by clarifying the exact relationship between computer applications and learning. Additionally, Haßler et al., ( 2016 ) suggested that the neutral findings regarding the impact of tablets on students learning outcomes in some of the studies included in their review should encourage educators, school leaders, and school officials to further investigate the potential of such devices in teaching and learning. Several other researchers suggested that a number of variables play a significant role in the impact of ICTs on students’ learning that could be attributed to the school context, teaching practices and professional development, the curriculum, and learners’ characteristics (Underwood, 2009 ; Tamim et al., 2011 ; Higgins et al., 2012 ; Archer et al., 2014 ; Sung et al., 2016 ; Haßler et al., 2016 ; Chauhan, 2017 ; Lee et al., 2020 ; Tang et al., 2022 ).

Digital competencies

One of the most common challenges reported in studies that utilized digital tools in the classroom was the lack of students’ skills on how to use them. Fu ( 2013 ) found that students’ lack of technical skills is a barrier to the effective use of ICT in the classroom. Tamim et al. ( 2015 ) reported that students faced challenges when using tablets and smart mobile devices, associated with the technical issues or expertise needed for their use and the distracting nature of the devices and highlighted the need for teachers’ professional development. Higgins et al. ( 2012 ) reported that skills training about the use of digital technologies is essential for learners to fully exploit the benefits of instruction.

Delgado et al. ( 2015 ), meanwhile, reported studies that showed a strong positive association between teachers’ computer skills and students’ use of computers. Teachers’ lack of ICT skills and familiarization with technologies can become a constraint to the effective use of technology in the classroom (Balanskat et al., 2006 ; Delgado et al., 2015 ).

It is worth noting that the way teachers are introduced to ICTs affects the impact of digital technologies on education. Previous studies have shown that teachers may avoid using digital technologies due to limited digital skills (Balanskat, 2006 ), or they prefer applying “safe” technologies, namely technologies that their own teachers used and with which they are familiar (Condie & Munro, 2007 ). In this regard, the provision of digital skills training and exposure to new digital tools might encourage teachers to apply various technologies in their lessons (Condie & Munro, 2007 ). Apart from digital competence, technical support in the school setting has also been shown to affect teachers’ use of technology in their classrooms (Delgado et al., 2015 ). Ferrari et al. ( 2011 ) found that while teachers’ use of ICT is high, 75% stated that they needed more institutional support and a shift in the mindset of educational actors to achieve more innovative teaching practices. The provision of support can reduce time and effort as well as cognitive constraints, which could cause limited ICT integration in the school lessons by teachers (Escueta et al., 2017 ).

Teachers’ personal characteristics, training approaches, and professional development

Teachers’ personal characteristics and professional development affect the impact of digital technologies on education. Specifically, Cheok and Wong ( 2015 ) found that teachers’ personal characteristics (e.g., anxiety, self-efficacy) are associated with their satisfaction and engagement with technology. Bingimlas ( 2009 ) reported that lack of confidence, resistance to change, and negative attitudes in using new technologies in teaching are significant determinants of teachers’ levels of engagement in ICT. The same author reported that the provision of technical support, motivation support (e.g., awards, sufficient time for planning), and training on how technologies can benefit teaching and learning can eliminate the above barriers to ICT integration. Archer et al. ( 2014 ) found that comfort levels in using technology are an important predictor of technology integration and argued that it is essential to provide teachers with appropriate training and ongoing support until they are comfortable with using ICTs in the classroom. Hillmayr et al. ( 2020 ) documented that training teachers on ICT had an important effecton students’ learning.

According to Balanskat et al. ( 2006 ), the impact of ICTs on students’ learning is highly dependent on the teachers’ capacity to efficiently exploit their application for pedagogical purposes. Results obtained from the Teaching and Learning International Survey (TALIS) (OECD, 2021 ) revealed that although schools are open to innovative practices and have the capacity to adopt them, only 39% of teachers in the European Union reported that they are well or very well prepared to use digital technologies for teaching. Li and Ma ( 2010 ) and Hardman ( 2019 ) showed that the positive effect of technology on students’ achievement depends on the pedagogical practices used by teachers. Schmid et al. ( 2014 ) reported that learning was best supported when students were engaged in active, meaningful activities with the use of technological tools that provided cognitive support. Tamim et al. ( 2015 ) compared two different pedagogical uses of tablets and found a significant moderate effect when the devices were used in a student-centered context and approach rather than within teacher-led environments. Similarly, Garzón and Acevedo ( 2019 ) and Garzón et al. ( 2020 ) reported that the positive results from the integration of AR applications could be attributed to the existence of different variables which could influence AR interventions (e.g., pedagogical approach, learning environment, and duration of the intervention). Additionally, Garzón et al. ( 2020 ) suggested that the pedagogical resources that teachers used to complement their lectures and the pedagogical approaches they applied were crucial to the effective integration of AR on students’ learning gains. Garzón and Acevedo ( 2019 ) also emphasized that the success of a technology-enhanced intervention is based on both the technology per se and its characteristics and on the pedagogical strategies teachers choose to implement. For instance, their results indicated that the collaborative learning approach had the highest impact on students’ learning gains among other approaches (e.g., inquiry-based learning, situated learning, or project-based learning). Ran et al. ( 2022 ) also found that the use of technology to design collaborative and communicative environments showed the largest moderator effects among the other approaches.

Hattie ( 2008 ) reported that the effective use of computers is associated with training teachers in using computers as a teaching and learning tool. Zheng et al. ( 2016 ) noted that in addition to the strategies teachers adopt in teaching, ongoing professional development is also vital in ensuring the success of technology implementation programs. Sung et al. ( 2016 ) found that research on the use of mobile devices to support learning tends to report that the insufficient preparation of teachers is a major obstacle in implementing effective mobile learning programs in schools. Friedel et al. ( 2013 ) found that providing training and support to teachers increased the positive impact of the interventions on students’ learning gains. Trucano ( 2005 ) argued that positive impacts occur when digital technologies are used to enhance teachers’ existing pedagogical philosophies. Higgins et al. ( 2012 ) found that the types of technologies used and how they are used could also affect students’ learning. The authors suggested that training and professional development of teachers that focuses on the effective pedagogical use of technology to support teaching and learning is an important component of successful instructional approaches (Higgins et al., 2012 ). Archer et al. ( 2014 ) found that studies that reported ICT interventions during which teachers received training and support had moderate positive effects on students’ learning outcomes, which were significantly higher than studies where little or no detail about training and support was mentioned. Fu ( 2013 ) reported that the lack of teachers’ knowledge and skills on the technical and instructional aspects of ICT use in the classroom, in-service training, pedagogy support, technical and financial support, as well as the lack of teachers’ motivation and encouragement to integrate ICT on their teaching were significant barriers to the integration of ICT in education.

School leadership and management

Management and leadership are important cornerstones in the digital transformation process (Pihir et al., 2018 ). Zheng et al. ( 2016 ) documented leadership among the factors positively affecting the successful implementation of technology integration in schools. Strong leadership, strategic planning, and systematic integration of digital technologies are prerequisites for the digital transformation of education systems (Ređep, 2021 ). Management and leadership play a significant role in formulating policies that are translated into practice and ensure that developments in ICT become embedded into the life of the school and in the experiences of staff and pupils (Condie & Munro, 2007 ). Policy support and leadership must include the provision of an overall vision for the use of digital technologies in education, guidance for students and parents, logistical support, as well as teacher training (Conrads et al., 2017 ). Unless there is a commitment throughout the school, with accountability for progress at key points, it is unlikely for ICT integration to be sustained or become part of the culture (Condie & Munro, 2007 ). To achieve this, principals need to adopt and promote a whole-institution strategy and build a strong mutual support system that enables the school’s technological maturity (European Commission, 2019 ). In this context, school culture plays an essential role in shaping the mindsets and beliefs of school actors towards successful technology integration. Condie and Munro ( 2007 ) emphasized the importance of the principal’s enthusiasm and work as a source of inspiration for the school staff and the students to cultivate a culture of innovation and establish sustainable digital change. Specifically, school leaders need to create conditions in which the school staff is empowered to experiment and take risks with technology (Elkordy & Lovinelli, 2020 ).

In order for leaders to achieve the above, it is important to develop capacities for learning and leading, advocating professional learning, and creating support systems and structures (European Commission, 2019 ). Digital technology integration in education systems can be challenging and leadership needs guidance to achieve it. Such guidance can be introduced through the adoption of new methods and techniques in strategic planning for the integration of digital technologies (Ređep, 2021 ). Even though the role of leaders is vital, the relevant training offered to them has so far been inadequate. Specifically, only a third of the education systems in Europe have put in place national strategies that explicitly refer to the training of school principals (European Commission, 2019 , p. 16).

Connectivity, infrastructure, and government and other support

The effective integration of digital technologies across levels of education presupposes the development of infrastructure, the provision of digital content, and the selection of proper resources (Voogt et al., 2013 ). Particularly, a high-quality broadband connection in the school increases the quality and quantity of educational activities. There is evidence that ICT increases and formalizes cooperative planning between teachers and cooperation with managers, which in turn has a positive impact on teaching practices (Balanskat et al., 2006 ). Additionally, ICT resources, including software and hardware, increase the likelihood of teachers integrating technology into the curriculum to enhance their teaching practices (Delgado et al., 2015 ). For example, Zheng et al. ( 2016 ) found that the use of one-on-one laptop programs resulted in positive changes in teaching and learning, which would not have been accomplished without the infrastructure and technical support provided to teachers. Delgado et al. ( 2015 ) reported that limited access to technology (insufficient computers, peripherals, and software) and lack of technical support are important barriers to ICT integration. Access to infrastructure refers not only to the availability of technology in a school but also to the provision of a proper amount and the right types of technology in locations where teachers and students can use them. Effective technical support is a central element of the whole-school strategy for ICT (Underwood, 2009 ). Bingimlas ( 2009 ) reported that lack of technical support in the classroom and whole-school resources (e.g., failing to connect to the Internet, printers not printing, malfunctioning computers, and working on old computers) are significant barriers that discourage the use of ICT by teachers. Moreover, poor quality and inadequate hardware maintenance, and unsuitable educational software may discourage teachers from using ICTs (Balanskat et al., 2006 ; Bingimlas, 2009 ).

Government support can also impact the integration of ICTs in teaching. Specifically, Balanskat et al. ( 2006 ) reported that government interventions and training programs increased teachers’ enthusiasm and positive attitudes towards ICT and led to the routine use of embedded ICT.

Lastly, another important factor affecting digital transformation is the development and quality assurance of digital learning resources. Such resources can be support textbooks and related materials or resources that focus on specific subjects or parts of the curriculum. Policies on the provision of digital learning resources are essential for schools and can be achieved through various actions. For example, some countries are financing web portals that become repositories, enabling teachers to share resources or create their own. Additionally, they may offer e-learning opportunities or other services linked to digital education. In other cases, specific agencies of projects have also been set up to develop digital resources (Eurydice, 2019 ).

Administration and digital data management

The digital transformation of schools involves organizational improvements at the level of internal workflows, communication between the different stakeholders, and potential for collaboration. Vuorikari et al. ( 2020 ) presented evidence that digital technologies supported the automation of administrative practices in schools and reduced the administration’s workload. There is evidence that digital data affects the production of knowledge about schools and has the power to transform how schooling takes place. Specifically, Sellar ( 2015 ) reported that data infrastructure in education is developing due to the demand for “ information about student outcomes, teacher quality, school performance, and adult skills, associated with policy efforts to increase human capital and productivity practices ” (p. 771). In this regard, practices, such as datafication which refers to the “ translation of information about all kinds of things and processes into quantified formats” have become essential for decision-making based on accountability reports about the school’s quality. The data could be turned into deep insights about education or training incorporating ICTs. For example, measuring students’ online engagement with the learning material and drawing meaningful conclusions can allow teachers to improve their educational interventions (Vuorikari et al., 2020 ).

Students’ socioeconomic background and family support

Research show that the active engagement of parents in the school and their support for the school’s work can make a difference to their children’s attitudes towards learning and, as a result, their achievement (Hattie, 2008 ). In recent years, digital technologies have been used for more effective communication between school and family (Escueta et al., 2017 ). The European Commission ( 2020 ) presented data from a Eurostat survey regarding the use of computers by students during the pandemic. The data showed that younger pupils needed additional support and guidance from parents and the challenges were greater for families in which parents had lower levels of education and little to no digital skills.

In this regard, the socio-economic background of the learners and their socio-cultural environment also affect educational achievements (Punie et al., 2006 ). Trucano documented that the use of computers at home positively influenced students’ confidence and resulted in more frequent use at school, compared to students who had no home access (Trucano, 2005 ). In this sense, the socio-economic background affects the access to computers at home (OECD, 2015 ) which in turn influences the experience of ICT, an important factor for school achievement (Punie et al., 2006 ; Underwood, 2009 ). Furthermore, parents from different socio-economic backgrounds may have different abilities and availability to support their children in their learning process (Di Pietro et al., 2020 ).

Schools’ socioeconomic context and emergency situations

The socio-economic context of the school is closely related to a school’s digital transformation. For example, schools in disadvantaged, rural, or deprived areas are likely to lack the digital capacity and infrastructure required to adapt to the use of digital technologies during emergency periods, such as the COVID-19 pandemic (Di Pietro et al., 2020 ). Data collected from school principals confirmed that in several countries, there is a rural/urban divide in connectivity (OECD, 2015 ).

Emergency periods also affect the digitalization of schools. The COVID-19 pandemic led to the closure of schools and forced them to seek appropriate and connective ways to keep working on the curriculum (Di Pietro et al., 2020 ). The sudden large-scale shift to distance and online teaching and learning also presented challenges around quality and equity in education, such as the risk of increased inequalities in learning, digital, and social, as well as teachers facing difficulties coping with this demanding situation (European Commission, 2020 ).

Looking at the findings of the above studies, we can conclude that the impact of digital technologies on education is influenced by various actors and touches many aspects of the school ecosystem. Figure  1 summarizes the factors affecting the digital technologies’ impact on school stakeholders based on the findings from the literature review.

Factors that affect the impact of ICTs on education

The findings revealed that the use of digital technologies in education affects a variety of actors within a school’s ecosystem. First, we observed that as technologies evolve, so does the interest of the research community to apply them to school settings. Figure  2 summarizes the trends identified in current research around the impact of digital technologies on schools’ digital capacity and transformation as found in the present study. Starting as early as 2005, when computers, simulations, and interactive boards were the most commonly applied tools in school interventions (e.g., Eng, 2005 ; Liao et al., 2007 ; Moran et al., 2008 ; Tamim et al., 2011 ), moving towards the use of learning platforms (Jewitt et al., 2011 ), then to the use of mobile devices and digital games (e.g., Tamim et al., 2015 ; Sung et al., 2016 ; Talan et al., 2020 ), as well as e-books (e.g., Savva et al., 2022 ), to the more recent advanced technologies, such as AR and VR applications (e.g., Garzón & Acevedo, 2019 ; Garzón et al., 2020 ; Kalemkuş & Kalemkuş, 2022 ), or robotics and AI (e.g., Su & Yang, 2022 ; Su et al., 2022 ). As this evolution shows, digital technologies are a concept in flux with different affordances and characteristics. Additionally, from an instructional perspective, there has been a growing interest in different modes and models of content delivery such as online, blended, and hybrid modes (e.g., Cheok & Wong, 2015 ; Kazu & Yalçin, 2022 ; Ulum, 2022 ). This is an indication that the value of technologies to support teaching and learning as well as other school-related practices is increasingly recognized by the research and school community. The impact results from the literature review indicate that ICT integration on students’ learning outcomes has effects that are small (Coban et al., 2022 ; Eng, 2005 ; Higgins et al., 2012 ; Schmid et al., 2014 ; Tamim et al., 2015 ; Zheng et al., 2016 ) to moderate (Garzón & Acevedo, 2019 ; Garzón et al., 2020 ; Liao et al., 2007 ; Sung et al., 2016 ; Talan et al., 2020 ; Wen & Walters, 2022 ). That said, a number of recent studies have reported high effect sizes (e.g., Kazu & Yalçin, 2022 ).

Current work and trends in the study of the impact of digital technologies on schools’ digital capacity

Based on these findings, several authors have suggested that the impact of technology on education depends on several variables and not on the technology per se (Tamim et al., 2011 ; Higgins et al., 2012 ; Archer et al., 2014 ; Sung et al., 2016 ; Haßler et al., 2016 ; Chauhan, 2017 ; Lee et al., 2020 ; Lei et al., 2022a ). While the impact of ICTs on student achievement has been thoroughly investigated by researchers, other aspects related to school life that are also affected by ICTs, such as equality, inclusion, and social integration have received less attention. Further analysis of the literature review has revealed a greater investment in ICT interventions to support learning and teaching in the core subjects of literacy and STEM disciplines, especially mathematics, and science. These were the most common subjects studied in the reviewed papers often drawing on national testing results, while studies that investigated other subject areas, such as social studies, were limited (Chauhan, 2017 ; Condie & Munro, 2007 ). As such, research is still lacking impact studies that focus on the effects of ICTs on a range of curriculum subjects.

The qualitative research provided additional information about the impact of digital technologies on education, documenting positive effects and giving more details about implications, recommendations, and future research directions. Specifically, the findings regarding the role of ICTs in supporting learning highlight the importance of teachers’ instructional practice and the learning context in the use of technologies and consequently their impact on instruction (Çelik, 2022 ; Schmid et al., 2014 ; Tamim et al., 2015 ). The review also provided useful insights regarding the various factors that affect the impact of digital technologies on education. These factors are interconnected and play a vital role in the transformation process. Specifically, these factors include a) digital competencies; b) teachers’ personal characteristics and professional development; c) school leadership and management; d) connectivity, infrastructure, and government support; e) administration and data management practices; f) students’ socio-economic background and family support and g) the socioeconomic context of the school and emergency situations. It is worth noting that we observed factors that affect the integration of ICTs in education but may also be affected by it. For example, the frequent use of ICTs and the use of laptops by students for instructional purposes positively affect the development of digital competencies (Zheng et al., 2016 ) and at the same time, the digital competencies affect the use of ICTs (Fu, 2013 ; Higgins et al., 2012 ). As a result, the impact of digital technologies should be explored more as an enabler of desirable and new practices and not merely as a catalyst that improves the output of the education process i.e. namely student attainment.

Conclusions

Digital technologies offer immense potential for fundamental improvement in schools. However, investment in ICT infrastructure and professional development to improve school education are yet to provide fruitful results. Digital transformation is a complex process that requires large-scale transformative changes that presuppose digital capacity and preparedness. To achieve such changes, all actors within the school’s ecosystem need to share a common vision regarding the integration of ICTs in education and work towards achieving this goal. Our literature review, which synthesized quantitative and qualitative data from a list of meta-analyses and review studies, provided useful insights into the impact of ICTs on different school stakeholders and showed that the impact of digital technologies touches upon many different aspects of school life, which are often overlooked when the focus is on student achievement as the final output of education. Furthermore, the concept of digital technologies is a concept in flux as technologies are not only different among them calling for different uses in the educational practice but they also change through time. Additionally, we opened a forum for discussion regarding the factors that affect a school’s digital capacity and transformation. We hope that our study will inform policy, practice, and research and result in a paradigm shift towards more holistic approaches in impact and assessment studies.

Study limitations and future directions

We presented a review of the study of digital technologies' impact on education and factors influencing schools’ digital capacity and transformation. The study results were based on a non-systematic literature review grounded on the acquisition of documentation in specific databases. Future studies should investigate more databases to corroborate and enhance our results. Moreover, search queries could be enhanced with key terms that could provide additional insights about the integration of ICTs in education, such as “policies and strategies for ICT integration in education”. Also, the study drew information from meta-analyses and literature reviews to acquire evidence about the effects of ICT integration in schools. Such evidence was mostly based on the general conclusions of the studies. It is worth mentioning that, we located individual studies which showed different, such as negative or neutral results. Thus, further insights are needed about the impact of ICTs on education and the factors influencing the impact. Furthermore, the nature of the studies included in meta-analyses and reviews is different as they are based on different research methodologies and data gathering processes. For instance, in a meta-analysis, the impact among the studies investigated is measured in a particular way, depending on policy or research targets (e.g., results from national examinations, pre-/post-tests). Meanwhile, in literature reviews, qualitative studies offer additional insights and detail based on self-reports and research opinions on several different aspects and stakeholders who could affect and be affected by ICT integration. As a result, it was challenging to draw causal relationships between so many interrelating variables.

Despite the challenges mentioned above, this study envisaged examining school units as ecosystems that consist of several actors by bringing together several variables from different research epistemologies to provide an understanding of the integration of ICTs. However, the use of other tools and methodologies and models for evaluation of the impact of digital technologies on education could give more detailed data and more accurate results. For instance, self-reflection tools, like SELFIE—developed on the DigCompOrg framework- (Kampylis et al., 2015 ; Bocconi & Lightfoot, 2021 ) can help capture a school’s digital capacity and better assess the impact of ICTs on education. Furthermore, the development of a theory of change could be a good approach for documenting the impact of digital technologies on education. Specifically, theories of change are models used for the evaluation of interventions and their impact; they are developed to describe how interventions will work and give the desired outcomes (Mayne, 2015 ). Theory of change as a methodological approach has also been used by researchers to develop models for evaluation in the field of education (e.g., Aromatario et al., 2019 ; Chapman & Sammons, 2013 ; De Silva et al., 2014 ).

We also propose that future studies aim at similar investigations by applying more holistic approaches for impact assessment that can provide in-depth data about the impact of digital technologies on education. For instance, future studies could focus on different research questions about the technologies that are used during the interventions or the way the implementation takes place (e.g., What methodologies are used for documenting impact? How are experimental studies implemented? How can teachers be taken into account and trained on the technology and its functions? What are the elements of an appropriate and successful implementation? How is the whole intervention designed? On which learning theories is the technology implementation based?).

Future research could also focus on assessing the impact of digital technologies on various other subjects since there is a scarcity of research related to particular subjects, such as geography, history, arts, music, and design and technology. More research should also be done about the impact of ICTs on skills, emotions, and attitudes, and on equality, inclusion, social interaction, and special needs education. There is also a need for more research about the impact of ICTs on administration, management, digitalization, and home-school relationships. Additionally, although new forms of teaching and learning with the use of ICTs (e.g., blended, hybrid, and online learning) have initiated several investigations in mainstream classrooms, only a few studies have measured their impact on students’ learning. Additionally, our review did not document any study about the impact of flipped classrooms on K-12 education. Regarding teaching and learning approaches, it is worth noting that studies referred to STEM or STEAM did not investigate the impact of STEM/STEAM as an interdisciplinary approach to learning but only investigated the impact of ICTs on learning in each domain as a separate subject (science, technology, engineering, arts, mathematics). Hence, we propose future research to also investigate the impact of the STEM/STEAM approach on education. The impact of emerging technologies on education, such as AR, VR, robotics, and AI has also been investigated recently, but more work needs to be done.

Finally, we propose that future studies could focus on the way in which specific factors, e.g., infrastructure and government support, school leadership and management, students’ and teachers’ digital competencies, approaches teachers utilize in the teaching and learning (e.g., blended, online and hybrid learning, flipped classrooms, STEM/STEAM approach, project-based learning, inquiry-based learning), affect the impact of digital technologies on education. We hope that future studies will give detailed insights into the concept of schools’ digital transformation through further investigation of impacts and factors which influence digital capacity and transformation based on the results and the recommendations of the present study.

Acknowledgements

This project has received funding under Grant Agreement No Ref Ares (2021) 339036 7483039 as well as funding from the European Union’s Horizon 2020 Research and Innovation Program under Grant Agreement No 739578 and the Government of the Republic of Cyprus through the Deputy Ministry of Research, Innovation and Digital Policy. The UVa co-authors would like also to acknowledge funding from the European Regional Development Fund and the National Research Agency of the Spanish Ministry of Science and Innovation, under project grant PID2020-112584RB-C32.

Data availability statement

Declarations.

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

• Archer K, Savage R, Sanghera-Sidhu S, Wood E, Gottardo A, Chen V. Examining the effectiveness of technology use in classrooms: A tertiary meta-analysis. Computers & Education. 2014; 78 :140–149. doi: 10.1016/j.compedu.2014.06.001. [ CrossRef ] [ Google Scholar ]
• Aromatario O, Van Hoye A, Vuillemin A, Foucaut AM, Pommier J, Cambon L. Using theory of change to develop an intervention theory for designing and evaluating behavior change SDApps for healthy eating and physical exercise: The OCAPREV theory. BMC Public Health. 2019; 19 (1):1–12. doi: 10.1186/s12889-019-7828-4. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Arztmann, M., Hornstra, L., Jeuring, J., & Kester, L. (2022). Effects of games in STEM education: A meta-analysis on the moderating role of student background characteristics. Studies in Science Education , 1-37. 10.1080/03057267.2022.2057732
• Bado N. Game-based learning pedagogy: A review of the literature. Interactive Learning Environments. 2022; 30 (5):936–948. doi: 10.1080/10494820.2019.1683587. [ CrossRef ] [ Google Scholar ]
• Balanskat, A. (2009). Study of the impact of technology in primary schools – Synthesis Report. Empirica and European Schoolnet. Retrieved 30 June 2022 from: https://erte.dge.mec.pt/sites/default/files/Recursos/Estudos/synthesis_report_steps_en.pdf
• Balanskat, A. (2006). The ICT Impact Report: A review of studies of ICT impact on schools in Europe, European Schoolnet. Retrieved 30 June 2022 from:  https://en.unesco.org/icted/content/ict-impact-report-review-studies-ict-impact-schools-europe
• Balanskat, A., Blamire, R., & Kefala, S. (2006). The ICT impact report.  European Schoolnet . Retrieved from: http://colccti.colfinder.org/sites/default/files/ict_impact_report_0.pdf
• Balyer, A., & Öz, Ö. (2018). Academicians’ views on digital transformation in education. International Online Journal of Education and Teaching (IOJET), 5 (4), 809–830. Retrieved 30 June 2022 from  http://iojet.org/index.php/IOJET/article/view/441/295
• Baragash RS, Al-Samarraie H, Moody L, Zaqout F. Augmented reality and functional skills acquisition among individuals with special needs: A meta-analysis of group design studies. Journal of Special Education Technology. 2022; 37 (1):74–81. doi: 10.1177/0162643420910413. [ CrossRef ] [ Google Scholar ]
• Bates, A. W. (2015). Teaching in a digital age: Guidelines for designing teaching and learning . Open Educational Resources Collection . 6. Retrieved 30 June 2022 from: https://irl.umsl.edu/oer/6
• Bingimlas KA. Barriers to the successful integration of ICT in teaching and learning environments: A review of the literature. Eurasia Journal of Mathematics, Science and Technology Education. 2009; 5 (3):235–245. doi: 10.12973/ejmste/75275. [ CrossRef ] [ Google Scholar ]
• Blaskó Z, Costa PD, Schnepf SV. Learning losses and educational inequalities in Europe: Mapping the potential consequences of the COVID-19 crisis. Journal of European Social Policy. 2022; 32 (4):361–375. doi: 10.1177/09589287221091687. [ CrossRef ] [ Google Scholar ]
• Bocconi S, Lightfoot M. Scaling up and integrating the selfie tool for schools' digital capacity in education and training systems: Methodology and lessons learnt. European Training Foundation. 2021 doi: 10.2816/907029,JRC123936. [ CrossRef ] [ Google Scholar ]
• Brooks, D. C., & McCormack, M. (2020). Driving Digital Transformation in Higher Education . Retrieved 30 June 2022 from: https://library.educause.edu/-/media/files/library/2020/6/dx2020.pdf?la=en&hash=28FB8C377B59AFB1855C225BBA8E3CFBB0A271DA
• Cachia, R., Chaudron, S., Di Gioia, R., Velicu, A., & Vuorikari, R. (2021). Emergency remote schooling during COVID-19, a closer look at European families. Retrieved 30 June 2022 from  https://publications.jrc.ec.europa.eu/repository/handle/JRC125787
• Çelik B. The effects of computer simulations on students’ science process skills: Literature review. Canadian Journal of Educational and Social Studies. 2022; 2 (1):16–28. doi: 10.53103/cjess.v2i1.17. [ CrossRef ] [ Google Scholar ]
• Chapman, C., & Sammons, P. (2013). School Self-Evaluation for School Improvement: What Works and Why? . CfBT Education Trust. 60 Queens Road, Reading, RG1 4BS, England.
• Chauhan S. A meta-analysis of the impact of technology on learning effectiveness of elementary students. Computers & Education. 2017; 105 :14–30. doi: 10.1016/j.compedu.2016.11.005. [ CrossRef ] [ Google Scholar ]
• Chen, Q., Chan, K. L., Guo, S., Chen, M., Lo, C. K. M., & Ip, P. (2022a). Effectiveness of digital health interventions in reducing bullying and cyberbullying: a meta-analysis. Trauma, Violence, & Abuse , 15248380221082090. 10.1177/15248380221082090 [ PubMed ]
• Chen B, Wang Y, Wang L. The effects of virtual reality-assisted language learning: A meta-analysis. Sustainability. 2022; 14 (6):3147. doi: 10.3390/su14063147. [ CrossRef ] [ Google Scholar ]
• Cheok ML, Wong SL. Predictors of e-learning satisfaction in teaching and learning for school teachers: A literature review. International Journal of Instruction. 2015; 8 (1):75–90. doi: 10.12973/iji.2015.816a. [ CrossRef ] [ Google Scholar ]
• Cheung, A. C., & Slavin, R. E. (2011). The Effectiveness of Education Technology for Enhancing Reading Achievement: A Meta-Analysis. Center for Research and reform in Education .
• Coban, M., Bolat, Y. I., & Goksu, I. (2022). The potential of immersive virtual reality to enhance learning: A meta-analysis. Educational Research Review , 100452. 10.1016/j.edurev.2022.100452
• Condie, R., & Munro, R. K. (2007). The impact of ICT in schools-a landscape review. Retrieved 30 June 2022 from: https://oei.org.ar/ibertic/evaluacion/sites/default/files/biblioteca/33_impact_ict_in_schools.pdf
• Conrads, J., Rasmussen, M., Winters, N., Geniet, A., Langer, L., (2017). Digital Education Policies in Europe and Beyond: Key Design Principles for More Effective Policies. Redecker, C., P. Kampylis, M. Bacigalupo, Y. Punie (ed.), EUR 29000 EN, Publications Office of the European Union, Luxembourg, 10.2760/462941
• Costa P, Castaño-Muñoz J, Kampylis P. Capturing schools’ digital capacity: Psychometric analyses of the SELFIE self-reflection tool. Computers & Education. 2021; 162 :104080. doi: 10.1016/j.compedu.2020.104080. [ CrossRef ] [ Google Scholar ]
• Cussó-Calabuig R, Farran XC, Bosch-Capblanch X. Effects of intensive use of computers in secondary school on gender differences in attitudes towards ICT: A systematic review. Education and Information Technologies. 2018; 23 (5):2111–2139. doi: 10.1007/s10639-018-9706-6. [ CrossRef ] [ Google Scholar ]
• Daniel SJ. Education and the COVID-19 pandemic. Prospects. 2020; 49 (1):91–96. doi: 10.1007/s11125-020-09464-3. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Delcker J, Ifenthaler D. Teachers’ perspective on school development at German vocational schools during the Covid-19 pandemic. Technology, Pedagogy and Education. 2021; 30 (1):125–139. doi: 10.1080/1475939X.2020.1857826. [ CrossRef ] [ Google Scholar ]
• Delgado, A., Wardlow, L., O’Malley, K., & McKnight, K. (2015). Educational technology: A review of the integration, resources, and effectiveness of technology in K-12 classrooms. Journal of Information Technology Education Research , 14, 397. Retrieved 30 June 2022 from  http://www.jite.org/documents/Vol14/JITEv14ResearchP397-416Delgado1829.pdf
• De Silva MJ, Breuer E, Lee L, Asher L, Chowdhary N, Lund C, Patel V. Theory of change: A theory-driven approach to enhance the Medical Research Council's framework for complex interventions. Trials. 2014; 15 (1):1–13. doi: 10.1186/1745-6215-15-267. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Di Pietro G, Biagi F, Costa P, Karpiński Z, Mazza J. The likely impact of COVID-19 on education: Reflections based on the existing literature and recent international datasets. Publications Office of the European Union; 2020. [ Google Scholar ]
• Elkordy A, Lovinelli J. Competencies, Culture, and Change: A Model for Digital Transformation in K12 Educational Contexts. In: Ifenthaler D, Hofhues S, Egloffstein M, Helbig C, editors. Digital Transformation of Learning Organizations. Springer; 2020. pp. 203–219. [ Google Scholar ]
• Eng TS. The impact of ICT on learning: A review of research. International Education Journal. 2005; 6 (5):635–650. [ Google Scholar ]
• European Commission. (2020). Digital Education Action Plan 2021 – 2027. Resetting education and training for the digital age. Retrieved 30 June 2022 from  https://ec.europa.eu/education/sites/default/files/document-library-docs/deap-communication-sept2020_en.pdf
• European Commission. (2019). 2 nd survey of schools: ICT in education. Objective 1: Benchmark progress in ICT in schools . Retrieved 30 June 2022 from: https://data.europa.eu/euodp/data/storage/f/2019-03-19T084831/FinalreportObjective1-BenchmarkprogressinICTinschools.pdf
• Eurydice. (2019). Digital Education at School in Europe , Luxembourg: Publications Office of the European Union. Retrieved 30 June 2022 from: https://eacea.ec.europa.eu/national-policies/eurydice/content/digital-education-school-europe_en
• Escueta, M., Quan, V., Nickow, A. J., & Oreopoulos, P. (2017). Education technology: An evidence-based review. Retrieved 30 June 2022 from  https://ssrn.com/abstract=3031695
• Fadda D, Pellegrini M, Vivanet G, Zandonella Callegher C. Effects of digital games on student motivation in mathematics: A meta-analysis in K-12. Journal of Computer Assisted Learning. 2022; 38 (1):304–325. doi: 10.1111/jcal.12618. [ CrossRef ] [ Google Scholar ]
• Fernández-Gutiérrez M, Gimenez G, Calero J. Is the use of ICT in education leading to higher student outcomes? Analysis from the Spanish Autonomous Communities. Computers & Education. 2020; 157 :103969. doi: 10.1016/j.compedu.2020.103969. [ CrossRef ] [ Google Scholar ]
• Ferrari, A., Cachia, R., & Punie, Y. (2011). Educational change through technology: A challenge for obligatory schooling in Europe. Lecture Notes in Computer Science , 6964 , 97–110. Retrieved 30 June 2022  https://link.springer.com/content/pdf/10.1007/978-3-642-23985-4.pdf
• Fielding, K., & Murcia, K. (2022). Research linking digital technologies to young children’s creativity: An interpretive framework and systematic review. Issues in Educational Research , 32 (1), 105–125. Retrieved 30 June 2022 from  http://www.iier.org.au/iier32/fielding-abs.html
• Friedel, H., Bos, B., Lee, K., & Smith, S. (2013). The impact of mobile handheld digital devices on student learning: A literature review with meta-analysis. In Society for Information Technology & Teacher Education International Conference (pp. 3708–3717). Association for the Advancement of Computing in Education (AACE).
• Fu JS. ICT in education: A critical literature review and its implications. International Journal of Education and Development Using Information and Communication Technology (IJEDICT) 2013; 9 (1):112–125. [ Google Scholar ]
• Gaol FL, Prasolova-Førland E. Special section editorial: The frontiers of augmented and mixed reality in all levels of education. Education and Information Technologies. 2022; 27 (1):611–623. doi: 10.1007/s10639-021-10746-2. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Garzón J, Acevedo J. Meta-analysis of the impact of Augmented Reality on students’ learning gains. Educational Research Review. 2019; 27 :244–260. doi: 10.1016/j.edurev.2019.04.001. [ CrossRef ] [ Google Scholar ]
• Garzón, J., Baldiris, S., Gutiérrez, J., & Pavón, J. (2020). How do pedagogical approaches affect the impact of augmented reality on education? A meta-analysis and research synthesis. Educational Research Review , 100334. 10.1016/j.edurev.2020.100334
• Grgurović M, Chapelle CA, Shelley MC. A meta-analysis of effectiveness studies on computer technology-supported language learning. ReCALL. 2013; 25 (2):165–198. doi: 10.1017/S0958344013000013. [ CrossRef ] [ Google Scholar ]
• Haßler B, Major L, Hennessy S. Tablet use in schools: A critical review of the evidence for learning outcomes. Journal of Computer Assisted Learning. 2016; 32 (2):139–156. doi: 10.1111/jcal.12123. [ CrossRef ] [ Google Scholar ]
• Haleem A, Javaid M, Qadri MA, Suman R. Understanding the role of digital technologies in education: A review. Sustainable Operations and Computers. 2022; 3 :275–285. doi: 10.1016/j.susoc.2022.05.004. [ CrossRef ] [ Google Scholar ]
• Hardman J. Towards a pedagogical model of teaching with ICTs for mathematics attainment in primary school: A review of studies 2008–2018. Heliyon. 2019; 5 (5):e01726. doi: 10.1016/j.heliyon.2019.e01726. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Hattie J, Rogers HJ, Swaminathan H. The role of meta-analysis in educational research. In: Reid AD, Hart P, Peters MA, editors. A companion to research in education. Springer; 2014. pp. 197–207. [ Google Scholar ]
• Hattie J. Visible learning: A synthesis of over 800 meta-analyses relating to achievement. Routledge. 2008 doi: 10.4324/9780203887332. [ CrossRef ] [ Google Scholar ]
• Higgins S, Xiao Z, Katsipataki M. The impact of digital technology on learning: A summary for the education endowment foundation. Education Endowment Foundation and Durham University; 2012. [ Google Scholar ]
• Higgins, K., Huscroft-D’Angelo, J., & Crawford, L. (2019). Effects of technology in mathematics on achievement, motivation, and attitude: A meta-analysis. Journal of Educational Computing Research , 57(2), 283-319.
• Hillmayr D, Ziernwald L, Reinhold F, Hofer SI, Reiss KM. The potential of digital tools to enhance mathematics and science learning in secondary schools: A context-specific meta-analysis. Computers & Education. 2020; 153 (1038):97. doi: 10.1016/j.compedu.2020.103897. [ CrossRef ] [ Google Scholar ]
• Istenic Starcic A, Bagon S. ICT-supported learning for inclusion of people with special needs: Review of seven educational technology journals, 1970–2011. British Journal of Educational Technology. 2014; 45 (2):202–230. doi: 10.1111/bjet.12086. [ CrossRef ] [ Google Scholar ]
• Jewitt C, Clark W, Hadjithoma-Garstka C. The use of learning platforms to organise learning in English primary and secondary schools. Learning, Media and Technology. 2011; 36 (4):335–348. doi: 10.1080/17439884.2011.621955. [ CrossRef ] [ Google Scholar ]
• JISC. (2020). What is digital transformation?.  Retrieved 30 June 2022 from: https://www.jisc.ac.uk/guides/digital-strategy-framework-for-university-leaders/what-is-digital-transformation
• Kalati, A. T., & Kim, M. S. (2022). What is the effect of touchscreen technology on young children’s learning?: A systematic review. Education and Information Technologies , 1-19. 10.1007/s10639-021-10816-5
• Kalemkuş, J., & Kalemkuş, F. (2022). Effect of the use of augmented reality applications on academic achievement of student in science education: Meta-analysis review. Interactive Learning Environments , 1-18. 10.1080/10494820.2022.2027458
• Kao C-W. The effects of digital game-based learning task in English as a foreign language contexts: A meta-analysis. Education Journal. 2014; 42 (2):113–141. [ Google Scholar ]
• Kampylis P, Punie Y, Devine J. Promoting effective digital-age learning - a European framework for digitally competent educational organisations. JRC Technical Reports. 2015 doi: 10.2791/54070. [ CrossRef ] [ Google Scholar ]
• Kazu IY, Yalçin CK. Investigation of the effectiveness of hybrid learning on academic achievement: A meta-analysis study. International Journal of Progressive Education. 2022; 18 (1):249–265. doi: 10.29329/ijpe.2022.426.14. [ CrossRef ] [ Google Scholar ]
• Koh C. A qualitative meta-analysis on the use of serious games to support learners with intellectual and developmental disabilities: What we know, what we need to know and what we can do. International Journal of Disability, Development and Education. 2022; 69 (3):919–950. doi: 10.1080/1034912X.2020.1746245. [ CrossRef ] [ Google Scholar ]
• König J, Jäger-Biela DJ, Glutsch N. Adapting to online teaching during COVID-19 school closure: Teacher education and teacher competence effects among early career teachers in Germany. European Journal of Teacher Education. 2020; 43 (4):608–622. doi: 10.1080/02619768.2020.1809650. [ CrossRef ] [ Google Scholar ]
• Lawrence JE, Tar UA. Factors that influence teachers’ adoption and integration of ICT in teaching/learning process. Educational Media International. 2018; 55 (1):79–105. doi: 10.1080/09523987.2018.1439712. [ CrossRef ] [ Google Scholar ]
• Lee, S., Kuo, L. J., Xu, Z., & Hu, X. (2020). The effects of technology-integrated classroom instruction on K-12 English language learners’ literacy development: A meta-analysis. Computer Assisted Language Learning , 1-32. 10.1080/09588221.2020.1774612
• Lei, H., Chiu, M. M., Wang, D., Wang, C., & Xie, T. (2022a). Effects of game-based learning on students’ achievement in science: a meta-analysis. Journal of Educational Computing Research . 10.1177/07356331211064543
• Lei H, Wang C, Chiu MM, Chen S. Do educational games affect students' achievement emotions? Evidence from a meta-analysis. Journal of Computer Assisted Learning. 2022; 38 (4):946–959. doi: 10.1111/jcal.12664. [ CrossRef ] [ Google Scholar ]
• Liao YKC, Chang HW, Chen YW. Effects of computer application on elementary school student's achievement: A meta-analysis of students in Taiwan. Computers in the Schools. 2007; 24 (3–4):43–64. doi: 10.1300/J025v24n03_04. [ CrossRef ] [ Google Scholar ]
• Li Q, Ma X. A meta-analysis of the effects of computer technology on school students’ mathematics learning. Educational Psychology Review. 2010; 22 (3):215–243. doi: 10.1007/s10648-010-9125-8. [ CrossRef ] [ Google Scholar ]
• Liu, M., Pang, W., Guo, J., & Zhang, Y. (2022). A meta-analysis of the effect of multimedia technology on creative performance. Education and Information Technologies , 1-28. 10.1007/s10639-022-10981-1
• Lu Z, Chiu MM, Cui Y, Mao W, Lei H. Effects of game-based learning on students’ computational thinking: A meta-analysis. Journal of Educational Computing Research. 2022 doi: 10.1177/07356331221100740. [ CrossRef ] [ Google Scholar ]
• Martinez L, Gimenes M, Lambert E. Entertainment video games for academic learning: A systematic review. Journal of Educational Computing Research. 2022 doi: 10.1177/07356331211053848. [ CrossRef ] [ Google Scholar ]
• Mayne J. Useful theory of change models. Canadian Journal of Program Evaluation. 2015; 30 (2):119–142. doi: 10.3138/cjpe.230. [ CrossRef ] [ Google Scholar ]
• Moran J, Ferdig RE, Pearson PD, Wardrop J, Blomeyer RL., Jr Technology and reading performance in the middle-school grades: A meta-analysis with recommendations for policy and practice. Journal of Literacy Research. 2008; 40 (1):6–58. doi: 10.1080/10862960802070483. [ CrossRef ] [ Google Scholar ]
• OECD. (2015). Students, Computers and Learning: Making the Connection . PISA, OECD Publishing, Paris. Retrieved from: 10.1787/9789264239555-en
• OECD. (2021). OECD Digital Education Outlook 2021: Pushing the Frontiers with Artificial Intelligence, Blockchain and Robots. Retrieved from: https://www.oecd-ilibrary.org/education/oecd-digital-education-outlook-2021_589b283f-en
• Pan Y, Ke F, Xu X. A systematic review of the role of learning games in fostering mathematics education in K-12 settings. Educational Research Review. 2022; 36 :100448. doi: 10.1016/j.edurev.2022.100448. [ CrossRef ] [ Google Scholar ]
• Pettersson F. Understanding digitalization and educational change in school by means of activity theory and the levels of learning concept. Education and Information Technologies. 2021; 26 (1):187–204. doi: 10.1007/s10639-020-10239-8. [ CrossRef ] [ Google Scholar ]
• Pihir, I., Tomičić-Pupek, K., & Furjan, M. T. (2018). Digital transformation insights and trends. In Central European Conference on Information and Intelligent Systems (pp. 141–149). Faculty of Organization and Informatics Varazdin. Retrieved 30 June 2022 from https://www.proquest.com/conference-papers-proceedings/digital-transformation-insights-trends/docview/2125639934/se-2
• Punie, Y., Zinnbauer, D., & Cabrera, M. (2006). A review of the impact of ICT on learning. Working Paper prepared for DG EAC. Retrieved 30 June 2022 from: http://www.eurosfaire.prd.fr/7pc/doc/1224678677_jrc47246n.pdf
• Quah CY, Ng KH. A systematic literature review on digital storytelling authoring tool in education: January 2010 to January 2020. International Journal of Human-Computer Interaction. 2022; 38 (9):851–867. doi: 10.1080/10447318.2021.1972608. [ CrossRef ] [ Google Scholar ]
• Ran H, Kim NJ, Secada WG. A meta-analysis on the effects of technology's functions and roles on students' mathematics achievement in K-12 classrooms. Journal of computer assisted learning. 2022; 38 (1):258–284. doi: 10.1111/jcal.12611. [ CrossRef ] [ Google Scholar ]
• Ređep, N. B. (2021). Comparative overview of the digital preparedness of education systems in selected CEE countries. Center for Policy Studies. CEU Democracy Institute .
• Rott, B., & Marouane, C. (2018). Digitalization in schools–organization, collaboration and communication. In Digital Marketplaces Unleashed (pp. 113–124). Springer, Berlin, Heidelberg.
• Savva M, Higgins S, Beckmann N. Meta-analysis examining the effects of electronic storybooks on language and literacy outcomes for children in grades Pre-K to grade 2. Journal of Computer Assisted Learning. 2022; 38 (2):526–564. doi: 10.1111/jcal.12623. [ CrossRef ] [ Google Scholar ]
• Schmid RF, Bernard RM, Borokhovski E, Tamim RM, Abrami PC, Surkes MA, Wade CA, Woods J. The effects of technology use in postsecondary education: A meta-analysis of classroom applications. Computers & Education. 2014; 72 :271–291. doi: 10.1016/j.compedu.2013.11.002. [ CrossRef ] [ Google Scholar ]
• Schuele CM, Justice LM. The importance of effect sizes in the interpretation of research: Primer on research: Part 3. The ASHA Leader. 2006; 11 (10):14–27. doi: 10.1044/leader.FTR4.11102006.14. [ CrossRef ] [ Google Scholar ]
• Schwabe, A., Lind, F., Kosch, L., & Boomgaarden, H. G. (2022). No negative effects of reading on screen on comprehension of narrative texts compared to print: A meta-analysis. Media Psychology , 1-18. 10.1080/15213269.2022.2070216
• Sellar S. Data infrastructure: a review of expanding accountability systems and large-scale assessments in education. Discourse: Studies in the Cultural Politics of Education. 2015; 36 (5):765–777. doi: 10.1080/01596306.2014.931117. [ CrossRef ] [ Google Scholar ]
• Stock WA. Systematic coding for research synthesis. In: Cooper H, Hedges LV, editors. The handbook of research synthesis, 236. Russel Sage; 1994. pp. 125–138. [ Google Scholar ]
• Su, J., Zhong, Y., & Ng, D. T. K. (2022). A meta-review of literature on educational approaches for teaching AI at the K-12 levels in the Asia-Pacific region. Computers and Education: Artificial Intelligence , 100065. 10.1016/j.caeai.2022.100065
• Su J, Yang W. Artificial intelligence in early childhood education: A scoping review. Computers and Education: Artificial Intelligence. 2022; 3 :100049. doi: 10.1016/j.caeai.2022.100049. [ CrossRef ] [ Google Scholar ]
• Sung YT, Chang KE, Liu TC. The effects of integrating mobile devices with teaching and learning on students' learning performance: A meta-analysis and research synthesis. Computers & Education. 2016; 94 :252–275. doi: 10.1016/j.compedu.2015.11.008. [ CrossRef ] [ Google Scholar ]
• Talan T, Doğan Y, Batdı V. Efficiency of digital and non-digital educational games: A comparative meta-analysis and a meta-thematic analysis. Journal of Research on Technology in Education. 2020; 52 (4):474–514. doi: 10.1080/15391523.2020.1743798. [ CrossRef ] [ Google Scholar ]
• Tamim, R. M., Bernard, R. M., Borokhovski, E., Abrami, P. C., & Schmid, R. F. (2011). What forty years of research says about the impact of technology on learning: A second-order meta-analysis and validation study. Review of Educational research, 81 (1), 4–28. Retrieved 30 June 2022 from 10.3102/0034654310393361
• Tamim, R. M., Borokhovski, E., Pickup, D., Bernard, R. M., & El Saadi, L. (2015). Tablets for teaching and learning: A systematic review and meta-analysis. Commonwealth of Learning. Retrieved from: http://oasis.col.org/bitstream/handle/11599/1012/2015_Tamim-et-al_Tablets-for-Teaching-and-Learning.pdf
• Tang C, Mao S, Xing Z, Naumann S. Improving student creativity through digital technology products: A literature review. Thinking Skills and Creativity. 2022; 44 :101032. doi: 10.1016/j.tsc.2022.101032. [ CrossRef ] [ Google Scholar ]
• Tolani-Brown, N., McCormac, M., & Zimmermann, R. (2011). An analysis of the research and impact of ICT in education in developing country contexts. In ICTs and sustainable solutions for the digital divide: Theory and perspectives (pp. 218–242). IGI Global.
• Trucano, M. (2005). Knowledge Maps: ICTs in Education. Washington, DC: info Dev / World Bank. Retrieved 30 June 2022 from  https://files.eric.ed.gov/fulltext/ED496513.pdf
• Ulum H. The effects of online education on academic success: A meta-analysis study. Education and Information Technologies. 2022; 27 (1):429–450. doi: 10.1007/s10639-021-10740-8. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Underwood, J. D. (2009). The impact of digital technology: A review of the evidence of the impact of digital technologies on formal education. Retrieved 30 June 2022 from: http://dera.ioe.ac.uk/id/eprint/10491
• Verschaffel, L., Depaepe, F., & Mevarech, Z. (2019). Learning Mathematics in metacognitively oriented ICT-Based learning environments: A systematic review of the literature. Education Research International , 2019 . 10.1155/2019/3402035
• Villena-Taranilla R, Tirado-Olivares S, Cózar-Gutiérrez R, González-Calero JA. Effects of virtual reality on learning outcomes in K-6 education: A meta-analysis. Educational Research Review. 2022; 35 :100434. doi: 10.1016/j.edurev.2022.100434. [ CrossRef ] [ Google Scholar ]
• Voogt J, Knezek G, Cox M, Knezek D, ten Brummelhuis A. Under which conditions does ICT have a positive effect on teaching and learning? A call to action. Journal of Computer Assisted Learning. 2013; 29 (1):4–14. doi: 10.1111/j.1365-2729.2011.00453.x. [ CrossRef ] [ Google Scholar ]
• Vuorikari, R., Punie, Y., & Cabrera, M. (2020). Emerging technologies and the teaching profession: Ethical and pedagogical considerations based on near-future scenarios  (No. JRC120183). Joint Research Centre. Retrieved 30 June 2022 from: https://publications.jrc.ec.europa.eu/repository/handle/JRC120183
• Wang LH, Chen B, Hwang GJ, Guan JQ, Wang YQ. Effects of digital game-based STEM education on students’ learning achievement: A meta-analysis. International Journal of STEM Education. 2022; 9 (1):1–13. doi: 10.1186/s40594-022-00344-0. [ CrossRef ] [ Google Scholar ]
• Wen X, Walters SM. The impact of technology on students’ writing performances in elementary classrooms: A meta-analysis. Computers and Education Open. 2022; 3 :100082. doi: 10.1016/j.caeo.2022.100082. [ CrossRef ] [ Google Scholar ]
• Zheng B, Warschauer M, Lin CH, Chang C. Learning in one-to-one laptop environments: A meta-analysis and research synthesis. Review of Educational Research. 2016; 86 (4):1052–1084. doi: 10.3102/0034654316628645. [ CrossRef ] [ Google Scholar ]

Global Education Monitoring Report

• 2023 GEM REPORT

Technology in education

• Recommendations
• 2023 Webpage
• Press Release
• RELATED PUBLICATIONS
• Background papers
• 2021/2 GEM Report
• 2020 Report
• 2019 Report
• 2017/8 Report
• 2016 Report

A tool on whose terms?

Ismael Martínez Sánchez/ProFuturo

• Monitoring SDG 4
• 2023 webpage

Major advances in technology, especially digital technology, are rapidly transforming the world. Information and communication technology (ICT) has been applied for 100 years in education, ever since the popularization of radio in the 1920s. But it is the use of digital technology over the past 40 years that has the most significant potential to transform education. An education technology industry has emerged and focused, in turn, on the development and distribution of education content, learning management systems, language applications, augmented and virtual reality, personalized tutoring, and testing. Most recently, breakthroughs in artificial intelligence (AI), methods have increased the power of education technology tools, leading to speculation that technology could even supplant human interaction in education.

In the past 20 years, learners, educators and institutions have widely adopted digital technology tools. The number of students in MOOCs increased from 0 in 2012 to at least 220 million in 2021. The language learning application Duolingo had 20 million daily active users in 2023, and Wikipedia had 244 million page views per day in 2021. The 2018 PISA found that 65% of 15-year-old students in OECD countries were in schools whose principals agreed that teachers had the technical and pedagogical skills to integrate digital devices in instruction and 54% in schools where an effective online learning support platform was available; these shares are believed to have increased during the COVID-19 pandemic. Globally, the percentage of internet users rose from 16% in 2005 to 66% in 2022. About 50% of the world’s lower secondary schools were connected to the internet for pedagogical purposes in 2022.

The adoption of digital technology has resulted in many changes in education and learning. The set of basic skills that young people are expected to learn in school, at least in richer countries, has expanded to include a broad range of new ones to navigate the digital world. In many classrooms, paper has been replaced by screens and pens by keyboards. COVID-19 can be seen as a natural experiment where learning switched online for entire education systems virtually overnight. Higher education is the subsector with the highest rate of digital technology adoption, with online management platforms replacing campuses. The use of data analytics has grown in education management. Technology has made a wide range of informal learning opportunities accessible.

Yet the extent to which technology has transformed education needs to be debated. Change resulting from the use of digital technology is incremental, uneven and bigger in some contexts than others. The application of digital technology varies by community and socioeconomic level, by teacher willingness and preparedness, by education level, and by country income. Except in the most technologically advanced countries, computers and devices are not used in classrooms on a large scale. Technology use is not universal and will not become so any time soon. Moreover, evidence is mixed on its impact: Some types of technology seem to be effective in improving some kinds of learning. The short- and long-term costs of using digital technology appear to be significantly underestimated. The most disadvantaged are typically denied the opportunity to benefit from this technology.

Too much attention on technology in education usually comes at a high cost. Resources spent on technology, rather than on classrooms, teachers and textbooks for all children in low- and lower-middle-income countries lacking access to these resources are likely to lead to the world being further away from achieving the global education goal, SDG 4. Some of the world’s richest countries ensured universal secondary schooling and minimum learning competencies before the advent of digital technology. Children can learn without it.

However, their education is unlikely to be as relevant without digital technology. The Universal Declaration of Human Rights defines the purpose of education as promoting the ‘full development of the human personality’, strengthening ‘respect for … fundamental freedoms’ and promoting ‘understanding, tolerance and friendship’. This notion needs to move with the times. An expanded definition of the right to education could include effective support by technology for all learners to fulfil their potential, regardless of context or circumstance.

Clear objectives and principles are needed to ensure that technology use is of benefit and avoids harm. The negative and harmful aspects in the use of digital technology in education and society include risk of distraction and lack of human contact. Unregulated technology even poses threats to democracy and human rights, for instance through invasion of privacy and stoking of hatred. Education systems need to be better prepared to teach about and through digital technology, a tool that must serve the best interests of all learners, teachers and administrators. Impartial evidence showing that technology is being used in some places to improve education, and good examples of such use, need to be shared more widely so that the optimal mode of delivery can be assured for each context.

CAN TECHNOLOGY HELP SOLVE THE MOST IMPORTANT CHALLENGES IN EDUCATION?

Discussions about education technology are focused on technology rather than education. The first question should be: What are the most important challenges in education? As a basis for discussion, consider the following three challenges:

• Equity and inclusion: Is fulfilment of the right to choose the education one wants and to realize one’s full potential through education compatible with the goal of equality? If not, how can education become the great equalizer?
• Quality: Do education’s content and delivery support societies in achieving sustainable development objectives? If not, how can education help learners to not only acquire knowledge but also be agents of change?
• Efficiency: Does the current institutional arrangement of teaching learners in classrooms support the achievement of equity and quality? If not, how can education balance individualized instruction and socialization needs?

How best can digital technology be included in a strategy to tackle these challenges, and under what conditions? Digital technology packages and transmits information on an unprecedented scale at high speed and low cost. Information storage has revolutionized the volume of accessible knowledge. Information processing enables learners to receive immediate feedback and, through interaction with machines, adapt their learning pace and trajectory: Learners can organize the sequence of what they learn to suit their background and characteristics. Information sharing lowers the cost of interaction and communication. But while such technology has tremendous potential, many tools have not been designed for application to education. Not enough attention has been given to how they are applied in education and even less to how they should be applied in different education contexts.

On the question of equity and inclusion , ICT – and digital technology in particular – helps lower the education access cost for some disadvantaged groups: Those who live in remote areas are displaced, face learning difficulties, lack time or have missed out on past education opportunities. But while access to digital technology has expanded rapidly, there are deep divides in access. Disadvantaged groups own fewer devices, are less connected to the internet (Figure 1) and have fewer resources at home. The cost of much technology is falling rapidly but is still too high for some. Households that are better off can buy technology earlier, giving them more advantages and compounding disparity. Inequality in access to technology exacerbates existing inequality in access to education, a weakness exposed during the COVID-19 school closures.

Figure 1: Internet connectivity is highly unequal

Percentage of 3- to 17-year-olds with internet connection at home, by wealth quintile, selected countries, 2017–19 Source: UNICEF database.

Education quality is a multifaceted concept. It encompasses adequate inputs (e.g. availability of technology infrastructure), prepared teachers (e.g. teacher standards for technology use in classrooms), relevant content (e.g. integration of digital literacy in the curriculum) and individual learning outcomes (e.g. minimum levels of proficiency in reading and mathematics). But education quality should also encompass social outcomes. It is not enough for students to be vessels receiving knowledge; they need to be able to use it to help achieve sustainable development in social, economic and environmental terms.

There are a variety of views on the extent to which digital technologies can enhance education quality. Some argue that, in principle, digital technology creates engaging learning environments, enlivens student experiences, simulates situations, facilitates collaboration and expands connections. But others say digital technology tends to support an individualized approach to education, reducing learners’ opportunities to socialize and learn by observing each other in real-life settings. Moreover, just as new technology overcomes some constraints, it brings its own problems. Increased screen time has been associated with adverse impact on physical and mental health. Insufficient regulation has led to unauthorized use of personal data for commercial purposes. Digital technology has also helped spread misinformation and hate speech, including through education.

Improvements to efficiency may be the most promising way for digital technology to make a difference in education. Technology is touted as being able to reduce the time students and teachers spend on menial tasks, time that can be used in other, educationally more meaningful activities. However, there are conflicting views on what is meaningful. The way that education technology is used is more complex than just a substitution of resources. Technology may be one-to-many, one-to-one or peer-to-peer technology. It may require students to learn alone or with others, online or offline, independently or networked. It delivers content, creates learner communities and connects teachers with students. It provides access to information. It may be used for formal or informal learning and can assess what has been learned. It is used as a tool for productivity, creativity, communication, collaboration, design and data management. It may be professionally produced or have user-generated content. It may be specific to schools and place-based or transcend time and place. As in any complex system, each technology tool involves distinct infrastructure, design, content and pedagogy, and each may promote different types of learning.

Technology is evolving too fast to permit evaluation that could inform decisions on legislation, policy and regulation. Research on technology in education is as complex as technology itself. Studies evaluate experiences of learners of various ages using various methodologies applied in contexts as different as self-study, classrooms and schools of diverse sizes and features, non-school settings, and at system level. Findings that apply in some contexts are not always replicable elsewhere. Some conclusions can be drawn from long-term studies as technologies mature but there is an endless stream of new products. Meanwhile, not all impact can be easily measured, given technology’s ubiquity, complexity, utility and heterogeneity. In brief, while there is much general research on education technology, the amount of research for specific applications and contexts is insufficient, making it difficult to prove that a particular technology enhances a particular kind of learning.

Why is there often the perception nevertheless that technology can address major education challenges? To understand the discourse around education technology, it is necessary to look behind the language being used to promote it, and the interests it serves. Who frames the problems technology should address? What are the consequences of such framing for education? Who promotes education technology as a precondition for education transformation? How credible are such claims? What criteria and standards need to be set to evaluate digital technology’s current and potential future contribution to education so as to separate hype from substance? Can evaluation go beyond short-term assessments of impact on learning and capture potential far-reaching consequences of the generalized use of digital technology in education?

Exaggerated claims about technology go hand in hand with exaggerated estimates of its global market size. In 2022, business intelligence providers’ estimates ranged from USD 123 billion to USD 300 billion. These accounts are almost always projected forward, predicting optimistic expansion, yet they fail to give historic trends and verify whether past projections proved true. Such reporting routinely characterizes education technology as essential and technology companies as enablers and disruptors. If optimistic projections are not fulfilled, responsibility is implicitly placed on governments as a way of maintaining indirect pressure on them to increase procurement. Education is criticized as being slow to change, stuck in the past and a laggard when it comes to innovation. Such coverage plays on users’ fascination with novelty but also their fear of being left behind.

The sections below further explore the three challenges this report addresses: equity and inclusion (in terms of access to education for disadvantaged groups and access to content), quality (in terms of teaching through and about digital technology) and efficiency (in terms of education management). After identifying technology’s potential to tackle these challenges, it discusses three conditions that need to be met for that potential to be fulfilled: equitable access, appropriate governance and regulation, and sufficient teacher capacity.

EQUITY AND INCLUSION: ACCESS FOR DISADVANTAGED GROUPS

A wide range of technology brings education to hard-to-reach learners. Technology has historically opened up education to learners facing significant obstacles in access to schools or well-trained teachers. Interactive radio instruction is used in nearly 40 countries. In Nigeria, radio instruction combined with print and audiovisual materials has been used since the 1990s, reaching nearly 80% of nomads and increasing their literacy, numeracy and life skills. Television has helped educate marginalized groups, notably in Latin America and the Caribbean. The Telesecundaria programme in Mexico, combining televised lessons with in-class support and extensive teacher training, increased secondary school enrolment by 21%. Mobile learning devices, often the only type of device accessible to disadvantaged learners, have been used in hard-to-reach areas and emergencies to share educational materials; complement in-person or remote channels; and foster interactions between students, teachers and parents, notably during COVID-19. Adults have been the main target of online distance learning, with open universities having increased participation for both working and disadvantaged adults.

Inclusive technology supports accessibility and personalization for learners with disabilities. Assistive technology removes learning and communication barriers, with numerous studies reporting a significant positive impact on academic engagement, social participation and the well-being of learners with disabilities. However, such devices remain inaccessible and unaffordable in many countries, and teachers often lack specialized training to use them effectively in learning environments. While people with disabilities used to rely exclusively on specialized devices to gain access to education, technology platforms and devices are increasingly incorporating accessibility features, which support inclusive, personalized learning for all students.

Technology supports learning continuity in emergencies. Mapping of 101 distance education projects in crisis contexts in 2020 showed that 70% used radio, television and basic mobile phones. During the Boko Haram crisis in Nigeria, the Technology Enhanced Learning for All programme used mobile phones and radios to support the learning continuity of 22,000 disadvantaged children, with recorded improvement in literacy and numeracy skills. However, there are significant gaps in terms of rigorous evaluation of education technology in emergencies, despite some limited recorded impact. Meanwhile, most projects are led by non-state actors as short-term crisis responses, raising sustainability concerns; education ministries implemented only 12% of the 101 projects.

Technology supported learning during COVID-19, but millions were left out. During school closures, 95% of education ministries carried out some form of distance learning, potentially reaching over 1 billion students globally. Many of the resources used during the pandemic were first developed in response to previous emergencies or rural education, with some countries building on decades of experience with remote learning. Sierra Leone revived the Radio Teaching Programme, developed during the Ebola crisis, one week after schools closed. Mexico expanded content from its Telesecundaria programme to all levels of education. However, at least half a billion, or 31% of students worldwide – mostly the poorest (72%) and those in rural areas (70%) – could not be reached by remote learning. Although 91% of countries used online learning platforms to deliver distance learning during school closures, the platforms only reached a quarter of students globally. For the rest, low-tech interventions such as radio and television were largely used, in combination with paper-based materials and mobile phones for increased interactivity.

Some countries are expanding existing platforms to reach marginalized groups. Less than half of all countries developed long-term strategies for increasing their resilience and the sustainability of interventions as part of their COVID-19 response plans. Many have abandoned distance learning platforms developed during COVID-19, while others are repurposing them to reach marginalized learners. The digital platform set up in Ukraine during the pandemic was expanded once the war broke out in 2022, allowing 85% of schools to complete the academic year.

EQUITY AND INCLUSION: ACCESS TO CONTENT

Technology facilitates content creation and adaptation. Open educational resources (OERs) encourage the reuse and repurposing of materials to cut development time, avoid duplication of work and make materials more context-specific or relevant to learners. They also significantly reduce the cost of access to content. In the US state of North Dakota, an initial investment of USD 110,000 to shift to OERs led to savings of over USD 1 million in student costs. Social media increases access to user-generated content. YouTube, a major player in both formal and informal learning, is used by about 80% of the world’s top 113 universities. Moreover, collaborative digital tools can improve the diversity and quality of content creation. In South Africa, the Siyavule initiative supported tutor collaboration on the creation of primary and secondary education textbooks.

Digitization of educational content simplifies access and distribution. Many countries, including Bhutan and Rwanda, have created static digital versions of traditional textbooks to increase availability. Others, including India and Sweden, have produced digital textbooks that encourage interactivity and multimodal learning. Digital libraries and educational content repositories such as the National Academic Digital Library of Ethiopia, National Digital Library of India and Teachers Portal in Bangladesh help teachers and learners find relevant materials. Learning management platforms, which have become a key part of the contemporary learning environment, help organize content by integrating digital resources into course structures.

Open access resources help overcome barriers. Open universities and MOOCs can eliminate time, location and cost barriers to access. In Indonesia, where low participation in tertiary education is largely attributed to geographical challenges, MOOCs play an important role in expanding access to post-secondary learning. During COVID-19, MOOC enrolment surged, with the top three providers adding as many users in April 2020 as in all of 2019. Technology can also remove language barriers. Translation tools help connect teachers and learners from various countries and increase the accessibility of courses by non-native students.

Ensuring and assessing the quality of digital content is difficult. The sheer quantity of content and its decentralized production pose logistical challenges for evaluation. Several strategies have been implemented to address this. China established specific quality criteria for MOOCs to be nationally recognized. The European Union developed its OpenupED quality label. India strengthened the link between non-formal and formal education. Micro-credentials are increasingly used to ensure that institution and learner both meet minimum standards. Some platforms aim to improve quality by recentralizing content production. YouTube, for example, has been funnelling financing and resources to a few trusted providers and partnering with well-established education institutions.

Technology may reinforce existing inequality in both access to and production of content. Privileged groups still produce most content. A study of higher-education repositories with OER collections found that nearly 90% were created in Europe or North America; 92% of the material in the OER Commons global library is in English. This influences who has access to digital content. MOOCs, for example, mainly benefit educated learners – studies have shown around 80% of participants on major platforms already have a tertiary degree – and those from richer countries. The disparity is due to divides in digital skills, internet access, language and course design. Regional MOOCs cater to local needs and languages but can also worsen inequality.

TEACHING AND LEARNING

Technology has been used to support teaching and learning in multiple ways. Digital technology offers two broad types of opportunities. First, it can improve instruction by addressing quality gaps, increasing opportunities to practise, increasing available time and personalizing instruction. Second, it can engage learners by varying how content is represented, stimulating interaction and prompting collaboration. Systematic reviews over the past two decades on technology’s impact on learning find small to medium-sized positive effects compared to traditional instruction. However, evaluations do not always isolate technology’s impact in an intervention, making it difficult to attribute positive effects to technology alone rather than to other factors, such as added instruction time, resources or teacher support. Technology companies can have disproportionate influence on evidence production. For example, Pearson funded studies contesting independent analysis that showed its products had no impact.

The prevalence of ICT use in classrooms is not high, even in the world’s richest countries. The 2018 PISA found that only about 10% of 15-year-old students in over 50 participating education systems used digital devices for more than an hour a week in mathematics and science lessons, on average (Figure 2) . The 2018 International Computer and Information Literacy Study (ICILS) showed that in the 12 participating education systems, simulation and modelling software in classrooms was available to just over one third of students, with country levels ranging from 8% in Italy to 91% in Finland.

Figure 2: Even in upper-middle- and high-income countries, technology use in mathematics and science classrooms is limited

Percentage of 15-year-old students who used digital devices for at least one hour per week in mathematics or science classroom lessons, selected upper-middle- and high-income countries, 2018 Source: 2018 PISA database.

Recorded lessons can address teacher quality gaps and improve teacher time allocation. In China, lesson recordings from high-quality urban teachers were delivered to 100 million rural students. An impact evaluation showed improvements in Chinese skills by 32% and a 38% long-term reduction in the rural–urban earning gap. However, just delivering materials without contextualizing and providing support is insufficient. In Peru, the One Laptop Per Child programme distributed over 1 million laptops loaded with content, but no positive impact on learning resulted, partly due to the focus on provision of devices instead of the quality of pedagogical integration.

Enhancing technology-aided instruction with personalization can improve some types of learning. Personalized adaptive software generates analytics that can help teachers track student progress, identify error patterns, provide differentiated feedback and reduce workload on routine tasks. Evaluations of the use of a personalized adaptive software in India documented learning gains in after-school settings and for low-performing students. However, not all widely used software interventions have strong evidence of positive effects compared to teacher-led instruction. A meta-analysis of studies on an AI learning and assessment system that has been used by over 25 million students in the United States found it was no better than traditional classroom teaching in improving outcomes.

Varied interaction and visual representation can enhance student engagement. A meta-analysis of 43 studies published from 2008 to 2019 found that digital games improved cognitive and behavioural outcomes in mathematics. Interactive whiteboards can support teaching and learning if well integrated in pedagogy; but in the United Kingdom, despite large-scale adoption, they were mostly used to replace blackboards. Augmented, mixed or virtual reality used as an experiential learning tool for repeated practice in life-like conditions in technical, vocational and scientific subjects is not always as effective as real-life training but may be superior to other digital methods, such as video demonstrations.

Technology offers teachers low-cost and convenient ways to communicate with parents. The Colombian Institute of Family Welfare’s distance education initiative, which targeted 1.7 million disadvantaged children, relied on social media platforms to relay guidance to caregivers on pedagogical activities at home. However, uptake and effectiveness of behavioural interventions targeting caregivers are limited by parental education levels, as well as lack of time and material resources.

Student use of technology in classrooms and at home can be distracting, disrupting learning. A meta-analysis of research on student mobile phone use and its impact on education outcomes, covering students from pre-primary to higher education in 14 countries, found a small negative effect, and a larger one at the university level. Studies using PISA data indicate a negative association between ICT use and student performance beyond a threshold of moderate use. Teachers perceive tablet and phone use as hampering classroom management. More than one in three teachers in seven countries participating in the 2018 ICILS agreed that ICT use in classrooms distracted students. Online learning relies on student ability to self-regulate and may put low-performing and younger learners at increased risk of disengagement.

DIGITAL SKILLS

The definition of digital skills has been evolving along with digital technology. An analysis for this report shows that 54% of countries have identified digital skills standards for learners. The Digital Competence Framework for Citizens (DigComp), developed on behalf of the European Commission, has five competence areas: information and data literacy, communication and collaboration, digital content creation, safety, and problem-solving. Some countries have adopted digital skills frameworks developed by non-state, mostly commercial, actors. The International Computer Driving Licence (ICDL) has been promoted as a ‘digital skills standard’ but is associated mainly with Microsoft applications. Kenya and Thailand have endorsed the ICDL as the digital literacy standard for use in schools.

Digital skills are unequally distributed. In the 27 European Union (EU) countries, 54% of adults had at least basic digital skills in 2021. In Brazil, 31% of adults had at least basic skills, but the level was twice as high in urban as in rural areas, three times as high among those in the labour force as among those outside it, and nine times as high in the top socioeconomic group as in the two bottom groups. The overall gender gap in digital skills is small, but wider in specific skills. In 50 countries, 6.5% of males and 3.2% of females could write a computer program. In Belgium, Hungary and Switzerland, no more than 2 women for every 10 men could program; in Albania, Malaysia and Palestine, 9 women for every 10 men could do so. According to the 2018 PISA, 5% of 15-year-olds with the strongest reading skills but 24% of those with the weakest ones were at risk of being misled by a typical phishing email.

Formal skills training may not be the main way of acquiring digital skills. About one quarter of adults in EU countries, ranging from 16% in Italy to 40% in Sweden, had acquired skills through a ‘formalised educational institution’. Informal learning, such as self-study and informal assistance from colleagues, relatives and friends, was used by twice as many. Still, formal education is important: In 2018, those with tertiary education in Europe were twice as likely (18%) as those with upper secondary education (9%) to engage in free online training or self-study to improve their computer, software or application use. Solid mastery of literacy and numeracy skills is positively associated with mastery of at least some digital skills.

A curriculum content mapping of 16 education systems showed that Greece and Portugal dedicated less than 10% of the curriculum to data and media literacy while Estonia and the Republic of Korea embedded both in half their curricula. In some countries, media literacy in curricula is explicitly connected to critical thinking in subject disciplines, as under Georgia’s New School Model. Asia is characterized by a protectionist approach to media literacy that prioritizes information control over education. But in the Philippines, the Association for Media and Information Literacy successfully advocated for incorporation of media and information literacy in the curriculum, and it is now a core subject in grades 11 and 12.

Digital skills in communication and collaboration matter in hybrid learning arrangements. Argentina promoted teamwork skills as part of a platform for programming and robotics competitions in primary and secondary education. Mexico offers teachers and students digital education resources and tools for remote collaboration, peer learning and knowledge sharing. Ethical digital behaviour includes rules, conventions and standards to be learned, understood and practised by digital users when using digital spaces. Digital communication’s anonymity, invisibility, asynchronicity and minimization of authority can make it difficult for individuals to understand its complexities.

Competences in digital content creation include selecting appropriate delivery formats and creating copy, audio, video and visual assets; integrating digital content; and respecting copyright and licences. The ubiquitous use of social media has turned content creation into a skill with direct application in electronic commerce. In Indonesia, the Siberkreasi platform counts collaborative engagement among its core activities. The Kenya Copyright Board collaborates closely with universities to provide copyright education and conducts frequent training sessions for students in the visual arts and ICT.

Education systems need to strengthen preventive measures and respond to many safety challenges, from passwords to permissions, helping learners understand the implications of their online presence and digital footprint. In Brazil, 29% of schools have conducted debates or lectures on privacy and data protection. In New Zealand, the Te Mana Tūhono (Power of Connectivity) programme delivers digital protection and security services to almost 2,500 state and state-integrated schools. A systematic review of interventions in Australia, Italy, Spain and the United States estimated that the average programme had a 76% chance of reducing cyberbullying perpetration. In Wales, United Kingdom, the government has advised schools how to prepare for and respond to harmful viral online content and hoaxes.

The definition of problem-solving skills varies widely among education systems. Many countries perceive them in terms of coding and programming and as part of a computer science curriculum that includes computational thinking, algorithm use and automation. A global review estimated that 43% of students in high-income countries, 62% in upper-middle-income, 5% in lower-middle-income but no students in low-income countries take computer science as compulsory in primary and/or secondary education. Only 20% of education systems require schools to offer computer science as an elective or core course. Non-state actors often support coding and programming skills. In Chile, Code.org has partnered with the government to provide educational resources in computer science.

EDUCATION MANAGEMENT

Education management information systems focus on efficiency and effectiveness. Education reforms have been characterized by increased school autonomy, target setting and results-based performance, all of which require more data. By one measure, since the 1990s, the number of policies making reference to data, statistics and information has increased by 13 times in high-income, 9 times in upper-middle-income, and 5 times in low- and lower-middle-income countries. But only 54% of countries globally – and as low as 22% in sub-Saharan Africa – have unique student identification mechanisms.

Geospatial data can support education management. Geographical information systems help address equity and efficiency in infrastructure and resource distribution in education systems. School mapping has been used to foster diversity and reduce inequality of opportunity. Ireland links three databases to decide in which of its 314 planning areas to build new schools. Geospatial data can identify areas where children live too far from the nearest school. For instance, it has been estimated that 5% of the population in Guatemala and 41% in the United Republic of Tanzania live more than 3 kilometres away from the nearest primary school.

Education management information systems struggle with data integration. In 2017, Malaysia introduced the Education Data Repository as part of its 2019–23 ICT Transformation Plan to progressively integrate its 350 education data systems and applications scattered across institutions. By 2019, it had integrated 12 of its main data systems, aiming for full integration through a single data platform by the end of 2023. In New Zealand, schools had been procuring student management systems independently and lack of interoperability between them was preventing authorities from tracking student progress. In 2019, the government began setting up the National Learner Repository and Data Exchange to be hosted in cloud data centres, but deployment was paused in 2021 due to cybersecurity concerns. European countries have been addressing interoperability concerns collectively to facilitate data sharing between countries and across multiple applications used in higher-education management through the EMREX project.

Computer-based assessments and computer adaptive testing have been replacing many paper-based assessments. They reduce test administration costs, improve measurement quality and provide rapid scoring. As more examinations shift online, the need for online cheating detection and proctoring tools has also increased. While these can reduce cheating, their effectiveness should be weighed against fairness and psychological effects. Evidence on the quality and usefulness of technology-based assessments has started to emerge, but much less is known about cost efficiency. Among 34 papers on technology-based assessments reviewed for this report, transparent data on cost were lacking.

Learning analytics can increase formative feedback and enable early detection systems. In China, learning analytics has been used to identify learners’ difficulties, predict learning trajectories and manage teacher resources. In the United States, Course Signals is a system used to flag the likelihood of a student not passing a course; educators can then target them for additional support. However, learning analytics requires all actors to have sufficient data literacy. Successful education systems typically have absorptive capacity, including strong school leaders and confident teachers willing to innovate. Yet often seemingly trivial issues, such as maintenance and repair, are ignored or underestimated.

ACCESS TO TECHNOLOGY: EQUITY, EFFICIENCY AND SUSTAINABILITY

Access to electricity and devices is highly unequal between and within countries. In 2021, almost 9% of the global population – and more than 70% of people in rural sub-Saharan Africa – lacked access to electricity. Globally, one in four primary schools do not have electricity. A 2018 study in Cambodia, Ethiopia, Kenya, Myanmar, Nepal and Niger found that 31% of public schools were on grid and 9% were off grid, with only 16% enjoying uninterrupted power supply. Globally, 46% of households had a computer at home in 2020; the share of schools with computers for pedagogical purposes was 47% in primary, 62% in lower secondary and 76% in upper secondary education. There were at most 10 computers per 100 students in Brazil and Morocco but 160 computers per 100 students in Luxembourg, according to the 2018 PISA.

Internet access, a vital enabler of economic, social and cultural rights, is also unequal. In 2022, two in three people globally used the internet. In late 2021, 55% of the world’s population had mobile broadband access. In low- and middle-income countries, 16% less women than men used mobile internet in 2021. An estimated 3.2 billion people do not use mobile internet services despite being covered by a mobile broadband network. Globally, 40% of primary, 50% of lower secondary and 65% of upper secondary schools are connected to the internet. In India, 53% of private unaided and 44% of private aided schools are connected, compared with only 14% of government schools.

Various policies are used to improve access to devices. Some one in five countries have policies granting subsidies or deductions to buy devices. One-to-one technology programmes were established in 30% of countries at one time; currently only 15% of countries pursue such programmes. A number of upper-middle- and high-income countries are shifting from providing devices to allowing students to use their own devices in school. Jamaica adopted a Bring Your Own Device policy framework in 2020 to aim for sustainability.

Some countries champion free and open source software. Education institutions with complex ICT infrastructure, such as universities, can benefit from open source software to add new solutions or functionalities. By contrast, proprietary software does not permit sharing and has vendor locks that hinder interoperability, exchange and updates. In India, the National e-Governance Plan makes it mandatory for all software applications and services used in government to be built on open source software to achieve efficiency, transparency, reliability and affordability.

Countries are committed to universal internet provision at home and in school. About 85% of countries have policies to improve school or learner connectivity and 38% have laws on universal internet provision. A review of 72 low- and middle-income countries found that 29 had used universal service funds to reduce costs for underserved groups. In Kyrgyzstan, renegotiated contracts helped cut prices by nearly half and almost doubled internet speed. In Costa Rica, the Hogares Conectados (Connected Households) programme, which provided an internet cost subsidy to the poorest 60% of households with school-age children, helped reduce the share of unconnected households from 41% in 2016 to 13% in 2019. Zero-rating, or providing free internet access for education or other purposes, has been used, especially during COVID-19, but is not without problems, as it violates the net neutrality principle.

Education technology is often underutilized. In the United States, an average of 67% of education software licences were unused and 98% were not used intensively. According to the EdTech Genome Project, 85% of some 7,000 pedagogical tools, which cost USD 13 billion, were ‘either a poor fit or implemented incorrectly’. Less than one in five of the top 100 education technology tools used in classrooms met the requirements of the US Every Student Succeeds Act. Research had been published for 39% of these tools but the research was aligned with the act in only 26% of cases.

Evidence needs to drive education technology decisions. A review in the United Kingdom found that only 7% of education technology companies had conducted randomized controlled trials, 12% had used third-party certification and 18% had engaged in academic studies. An online survey of teachers and administrators in 17 US states showed that only 11% requested peer-reviewed evidence prior to adopting education technology. Recommendations influence purchase decisions, yet ratings can be manipulated through fake reviews disseminated on social media. Few governments try to fill the evidence gap, so demand has grown for independent reviews. Edtech Tulna, a partnership between a private think tank and a public university in India, offers quality standards, an evaluation toolkit and publicly available expert reviews.

Education technology procurement decisions need to take economic, social and environmental sustainability into account. With respect to economic considerations, it is estimated that initial investment in education technology accounts for just 25% or less of the eventual total cost. Regarding social concerns, procurement processes need to address equity, accessibility, local ownership and appropriation. In France, the Territoires Numériques Educatifs (Digital Educational Territories) initiative was criticized because not all subsidized equipment met local needs, and local governments were left out of the decisions on which equipment to purchase. Both issues have since been addressed. Concerning environmental considerations, it has been estimated that extending the lifespan of all laptops in the European Union by a year would save the equivalent of taking almost 1 million cars off the road in terms of CO2 emissions.

Regulation needs to address risks in education technology procurement. Public procurement is vulnerable to collusion and corruption. In 2019, Brazil’s Comptroller General of the Union found irregularities in the electronic bidding process for the purchase of 1.3 million computers, laptops and notebooks for state and municipal public schools. Decentralizing public procurement to local governments is one way to balance some of the risks. Indonesia has used its SIPLah e-commerce platform to support school-level procurement processes. However, decentralization is vulnerable to weak organizational capacity. A survey of administrators in 54 US school districts found that they had rarely carried out needs assessments.

GOVERNANCE AND REGULATION

Governance of the education technology system is fragmented. A department or an agency responsible for education technology has been identified in 82% of countries. Placing education ministries in charge of education technology strategies and plans could help ensure that decisions are primarily based on pedagogical principles. However, this is the case in just 58% of countries. In Kenya, the 2019 National Information, Communications and Technology Policy led the Ministry of Information, Communications and Technology to integrate ICT at all levels of education.

Participation is often limited in the development of education technology strategies and plans. Nepal established a Steering and a Coordination Committee under the 2013–17 ICT in Education Master Plan for intersectoral and inter-agency coordination and cooperation in its implementation. Including administrators, teachers and students can help bridge the knowledge gap with decision makers to ensure that education technology choices are appropriate. In 2022, only 41% of US education sector leaders agreed that they were regularly included in planning and strategic conversations about technology.

The private sector’s commercial interests can clash with government equity, quality and efficiency goals. In India, the government alerted families about the hidden costs of free online content. Other risks relate to data use and protection, privacy, interoperability and lock-in effects, whereby students and teachers are compelled to use specific software or platforms. Google, Apple and Microsoft produce education platforms tied to particular hardware and operating systems.

Privacy risks to children make their learning environment unsafe. One analysis found that 89% of 163 education technology products recommended for children’s learning during the COVID-19 pandemic could or did watch children outside school hours or education settings. In addition, 39 of 42 governments providing online education during the pandemic fostered uses that ‘risked or infringed’ upon children’s rights. Data used for predictive algorithms can bias predictions and decisions and lead to discrimination, privacy violations and exclusion of disadvantaged groups. The Cyberspace Administration of China and the Ministry of Education introduced regulations in 2019 requiring parental consent before devices powered by AI, such as cameras and headbands, could be used with students in schools and required data to be encrypted.

Children’s exposure to screen time has increased. A survey of screen time of parents of 3- to 8-year-olds in Australia, China, Italy, Sweden and the United States found that their children’s screen exposure increased by 50 minutes during the pandemic for both education and leisure. Extended screen time can negatively affect self-control and emotional stability, increasing anxiety and depression. Few countries have strict regulations on screen time. In China, the Ministry of Education limited the use of digital devices as teaching tools to 30% of overall teaching time. Less than one in four countries are banning the use of smartphones in schools. Italy and the United States have banned the use of specific tools or social media from schools. Cyberbullying and online abuse are rarely defined as offences but can fall under existing laws, such as stalking laws as in Australia and harassment laws in Indonesia.

Monitoring of data protection law implementation is needed. Only 16% of countries explicitly guarantee data privacy in education by law and 29% have a relevant policy, mainly in Europe and Northern America. The number of cyberattacks in education is rising. Such attacks increase exposure to theft of identity and other personal data, but capacity and funds to address the issue are often insufficient. Globally, 5% of all ransomware attacks targeted the education sector in 2022, accounting for more than 30% of cybersecurity breaches. Regulations on sharing children’s personal information are rare but are starting to emerge under the EU’s General Data Protection Regulation. China and Japan have binding instruments on protecting children’s data and information.

Technology has an impact on the teaching profession. Technology allows teachers to choose, modify and generate educational materials. Personalized learning platforms offer teachers customized learning paths and insights based on student data. During the COVID-19 pandemic, France facilitated access to 17 online teaching resource banks mapped against the national curriculum. The Republic of Korea temporarily eased copyright restrictions for teachers. Online teacher-student collaboration platforms provide access to support services, facilitate work team creation, allow participation in virtual sessions and promote sharing of learning materials.

Obstacles to integrating technology in education prevent teachers from fully embracing it. Inadequate digital infrastructure and lack of devices hinder teachers’ ability to integrate technology in their practice. A survey in 165 countries during the pandemic found that two in five teachers used their own devices, and almost one third of schools had only one device for education use. Some teachers lack training to use digital devices effectively. Older teachers may struggle to keep up with rapidly changing technology. The 2018 Teaching and Learning International Survey (TALIS) found that older teachers in 48 education systems had weaker skills and lower self-efficacy in using ICT. Some teachers may lack confidence. Only 43% of lower secondary school teachers in the 2018 TALIS said they felt prepared to use technology for teaching after training, and 78% of teachers in the 2018 ICILS were not confident in using technology for assessment.

Education systems support teachers in developing technology-related professional competencies. About half of education systems worldwide have ICT standards for teachers in a competency framework, teacher training framework, development plan or strategy. Education systems set up annual digital education days for teachers, promote OER, support the exchange of experiences and resources between teachers, and offer training. One quarter of education systems have legislation to ensure teachers are trained in technology, either through initial or in-service training. Some 84% of education systems have strategies for in-service teacher professional development, compared with 72% for pre-service teacher education in technology. Teachers can identify their development needs using digital self-assessment tools such as that provided by the Centre for Innovation in Brazilian Education.

Technology is changing teacher training. Technology is used to create flexible learning environments, engage teachers in collaborative learning, support coaching and mentoring, increase reflective practice, and improve subject or pedagogical knowledge. Distance education programmes have promoted teacher learning in South Africa and even equalled the impact of in-person training in Ghana. Virtual communities have emerged, primarily through social networks, for communication and resource sharing. About 80% of teachers surveyed in the Caribbean belonged to professional WhatsApp groups and 44% used instant messaging to collaborate at least once a week. In Senegal, the Reading for All programme used in-person and online coaching. Teachers considered face-to-face coaching more useful, but online coaching cost 83% less and still achieved a significant, albeit small, improvement in how teachers guided students’ reading practice. In Flanders, Belgium, KlasCement, a teacher community network created by a non-profit and now run by the Ministry of Education, expanded access to digital education and provided a platform for discussions on distance education during the pandemic.

Many actors support teacher professional development in ICT. Universities, teacher training institutions and research institutes provide specialized training, research opportunities and partnerships with schools for professional development in ICT. In Rwanda, universities collaborated with teachers and the government to develop the ICT Essentials for Teachers course. Teacher unions also advocate for policies that support teachers. The Confederation of Education Workers of the Argentine Republic established the right of teachers to disconnect. Civil society organizations, including the Carey Institute for Global Good, offer support through initiatives such as providing OER and online courses for refugee teachers in Chad, Kenya, Lebanon and Niger.

• Reference Manager
• Simple TEXT file

People also looked at

Review article, a framework for research on education with technology.

• Alder Graduate School of Education, Redwood City, CA, United States

Educational software offers the potential for greatly enhanced student learning. The current availability and political will for trying new approaches means that there is currently much interest in and expenditure on technology for education. After reviewing some of the relevant issues, a framework that builds upon Marr and Poggio's (1977) levels of explanation is presented. The research itself should draw upon existing cognitive, educational, and social research; much existing research is applicable. Guidelines for those conducting research and those wishing to acquire technology are presented.

While the phrases: blended learning, computer-assisted instruction, computer supported education, edutainment, e-learning, flipped classrooms, intelligent tutoring systems, interactive learning environments, personalized learning, serious games, teaching machines, etc., are relatively recent labels, people have been thinking about the cognitive processes upon which these phrases rely for millenia. Many of the issues now faced have been addressed before (e.g., how to provide teachers with clear actionable information about how their students are doing, creating enough content for students), though with modern computers some of the difficulties faced by, for example, the teaching machines of the 1950s, can be addressed more easily (e.g., allowing multiple response formats for questions).

The different labels that have been used over the decades and those currently used by different stakeholders convey subtle differences of focus and also reflect different marketing strategies. To avoid these a more generic phrase will be used here: Education with Technology , abbreviated EwT. This was chosen to stress that the emphasis is on education and that technology provides a method for implementing some aspects of education.

Suppes (1966) notes while Alexander the Great was able to have personalized tutoring from Aristotle, this privilege is not available to many. He argued that if the wisdom and skills of Aristotle could be delivered by a computer, this could be scaled to benefit many students. Training millions of people to become Aristotle-like personal tutors is not economically feasible. However, if computer software could be developed to perform like Aristotle for some tasks, the additional costs of scaling this up to allow many to benefit is relatively small if the hardware is in place and if the same program is suitable for many. Computer software is a very scalable technology. In the future there will be more technologies that can be used for education. Part of the success of any new system will be if it scales as well as computer software. For example, holodecks might be used in education ( Thornburg, 2014 ), virtual reality glasses are already on the market, and neural implants designed to improve cognition are being built (e.g., https://www.neuralink.com/ ), but these would likely be expensive to scale-up. Nootropics (drugs designed to improve cognition) could become part of education discourse, and could be cost effective, but their use raises some ethical/health concerns. Future technologies could provide a radically different way to gain new information. For now, designing computer software is the most scalable technology available.

This first section is largely US focused. This is because the manuscript arose out of concerns for research and practice in the US. The framework is proposed for all EwT. The AIED (Artificial Intelligence in Education) conferences provide good snapshots of the relevant international research ( André, 2017 ; Conati et al., 2015 ).

Learning Before Computers

People have been interested in the cognitive processes that underlie learning and memory for centuries (e.g., Yates, 1966 ; Carruthers, 1990 ; Rubin, 1995 ; Small, 1997 ). Different theories of how people learn and remember have lead to different theories of how people should be taught ( Roediger, 1980 ). For example, if you assume that memory works like a file cabinet, where memories exist unaltered, memory errors result from not finding the “right” file. It follows that educational approaches should attempt to put knowledge into these cabinets until they are filled and teach retrieval techniques. With this metaphor, memory errors of omission (forgetting) may be seen as failures of retrieval, but elaborate errors of commission (confabulation) should be rare. Other memory metaphors (e.g., a sponge, wax tablet, a paleontologist recreating a dinosaur) suggest different reasons for memory errors and different pedagogies.

Over the centuries memory researchers have examined both internal memory mechanisms and external memory devices, and how they interact. Rubin (1995) describes how internal memory aides, or mnemonics , were taught to those needing to recite long passages. An example is the method of loci where the person visualizes to be remembered information on a well learned path and then mentally travels along this path when needing to recite the information. These skills were viewed so important that their name is associated with a Titan in Greek mythology, Mnemosyne , the mother of the muses (the nine Muses, whose father was Zeus , were the sources of knowledge in the arts, sciences, and literature). In modern education there has been less emphasis on teaching students how to remember than on what to remember, though some recent textbooks include sections on how to learn the book's content (e.g., Nolen-Hoeksema et al., 2015 ).

External aides have also been used to facilitate learning and memory. For example, Yates (1966) describes several theaters that were designed to help enhance memory (e.g., Giulio Camillo's Memory Theater, the Globe Theater). These were designed so that the actors could mentally place what they later would need to recall in different parts of the theater. Many also believed that these theaters were designed to provide some additional magical value for memory. Usually technology should be considered an external aide, but it is important for students to be trained to use the technology. Distinguishing internal and external memory aides can be complicated. Clark (2008) argues that if a technology is always available and always relied upon that it is only biological prejudice that prevents someone saying that the technology is part of the person's mind. With technologies like Google Glass, neural implants, and nootropics, differentiating internal and external is complicated.

Language and writing are social (inter-personal) technologies that are important for education. Small (1997) describes how these were used in the creation of the first books. In oral traditions stories waned and flowed with the orator's and contemporary society's influence, but with books the story could remain unaltered for generations. People no longer had to rely on stories passed through many people as accurate representations of the original events. Human knowledge of Atlantis will have gone through many iterations before Plato wrote about it, but since then his writings have become record.

With the printing press, more people could read the same book. Most books are not personalized for each individual, but individuals with the economic means could choose which books they read. There have been attempts to personalize books and to introduce some control for the reader. Borges (1941/1998) describes this approach when critiquing Herbert Quain's fictitious novel April March . Borges (1941/1998 , p. 109) used the schematic in Figure 1 to show how a reader could navigate through Quain's novel. After a shared introduction ( z ) the reader chooses one of three y options, and for each of these the reader chooses one of three x options. An example of a complete story would be z → y 2 → x 5 . This branching became popular in the 1970s with a genre of literature called gamebooks , where readers chose a path through the book by skipping pages. Two people could read the same book, but have different stories. Quain's novel, if it had existed, would have allowed readers to choose twice among three alternatives for nine possible paths. Borges (1941/1998 , p. 110) said “gods and demiurges” could create systems with infinite paths. Near infinite branching is at the heart of many digital first person adventure games. Within education, so-called serious games also often use this branching to create different stories for different readers. One question is whether any positive aspects of allowing students the autonomy to choose their path outweigh any negative aspects of missing out on educational information from the paths that are missed.

Figure 1 . Borges' schematic of Quain's novel April March . The reader can choose among nine different paths.

Three other important technologies for education are radio, film, and television (e.g., Cuban, 1986 ; Ferster, 2014 , 2016 ). These allowed what became known as edutainment to be heard and seen by millions. Initially there was much optimism. Thomas Edison declared in 1913 that “Books will soon be obsolete in our schools …. Our school system will be completely changed in 10 years” (as cited in Ferster, 2014 , p. 32). Optimism is repeated by some with the introduction of every new technology. There have been successes, but books are not obsolete. These media allow one production of a lecture to be provided to thousands of students, and the lecture is preserved for future students. These are mass media versions of the “sage-on-the-stage” approach to education ( Ferster, 2014 ). Television meant users could simply switch on their edutainment. Over 500 million people did this for Carl Sagan's Cosmos in the 1980s. Shows for children, like Sesame Street , also have had large impacts.

The choice of content–when left to the whims of viewing statistics and advertisers' target markets–often will not lead to positive educational messages being broadcast. Sagan notes how society's choice for what to present via a variety of social media is unfortunate:

An extraterrestrial being, newly arrived on Earth–scrutinizing what we mainly present to our children in television, radio, movies, newspaper, magazines, the comics, and many books–might easily conclude that we are intent on teaching them murder, rape, cruelty, superstition, credulity, and consumerism. We keep at it, and through constant repetition many of them finally get it. What kind of society could we create if, instead, we drummed into them science and a sense of hope?

( Sagan, 1996 , p. 39)

He would find little solace with the content of the internet.

There are practical issues linking massed produced material onto formal courses. This can be done more easily when the material is for just a single course. In the United Kingdom, where the Open University has pioneered large-scale well-respected distance education since 1969, lectures were often on radio and television, and sometimes late at night. Nowadays students download their materials and this is also done with many massive open online courses (MOOCs). The Open University is a good example of an education system adapting their methods for distance learning with advances in technology. Home schooling has seen similar changes in relation to technology. The International Association for K–12 Online Learning (iNACOL, http://www.inacol.org/ , “K–12” refers to the US grades kindergarten through 12th grade, which corresponds approximately to 5–17 years old), which began focused on home schooling, is now one of the main EdTech societies in the US. There are many large-scale courses available via the internet like Coursera, edX, Udemy, and Udacity, and stand-alone bits of knowledge that are available and used as part of educational courses including material from the Khan Academy, Wikipedia, and several YouTube (and YouTube-like) channels.

Another educational technology that pre-dates modern computers is teaching machines. In the 1920s and 1930s Pressey began creating machines to help to teach students. Figure 2 shows a schematic of one of this teaching machines taken from a 1930 patent (submitted in 1928). Pressey presented these machines at American Psychological Association conferences and began selling them with the promise of “the freeing of teacher and pupil from educational drudgery and incompetence” ( Pressey, 1933 , p. 583). His teaching machines did not become popular. Benjamin (1988) and Skinner (1958) say part of the reason was the culture in the US at the time: “the world of education was not ready for them” ( Skinner, 1958 , p. 969). At the time Pressey was marketing these machines there was a surplus of teachers so there was less need for time-saving technology. Pressey blamed lack of sales on the overall economic depression ( Ferster, 2014 , p. 60).

Figure 2 . From a 1930 patent by Pressey. The answer is D. James Ogle(thorpe) founded the colony in Georgia. From US patent “Machine for intelligence tests” (US 1749226 A).

When Skinner re-introduced teaching machines with some modifications in the late 1950s and US culture was more receptive. More teachers were needed to cope with the baby-boom and the US had just seen the Soviets launch Sputnik . There was a realization that the US needed to catch up with other countries particularly with respect to science education. President Eisenhower coined the phrase the Sputnik Crisis to describe this.

Skinner's (1958) approach differed from Pressey's, though they both assumed student-centered learning. Some of the differences were due to advances in learning theory within behaviorism as well as other areas of psychological research (e.g., Vygotsky's, 1978 work suggests a step-by-step approach through each individual student's zone of proximal development), and part Skinner's own nuanced approach. While Pressey was careful to say his machines would be a tool to help the teacher, Skinner was more comfortable saying his machines could do tasks formerly reserved for teachers: “the effect upon each student is surprisingly like that of a private tutor” ( Skinner, 1958 , p. 971). His claim lead to the popular press suggesting that his machines could lead to robots teaching students in classrooms like the research assistants taught pigeons in Skinner's lab.

Despite initial commercial success for some of the teaching machines of this era, their popularity faded. Benjamin (1988) , Ferster (2014) , and others discuss many of the factors that negatively affected their popularity. Three stand out: fear of technology, costs, and effectiveness. The first is that some people worried about students being taught by machines. While the Sputnik Crisis lead some to embrace technology, many feared that machines could create a dystopian future. These teaching machines were being marketed at around the time that Ginsberg (1955/2014 , p. 17) tapped into this fear referencing the Canaanite god of child sacrifice in his classic poem Howl : “Moloch whose mind is pure machinery!” Ginsberg was describing the dark-side of over-industrialization. The second reason is economic. These machines were expensive and more important for today's arguments, building each new machine was expensive so these did not scale-up as well as today's software solutions. Third, much of the research was showing that these machines were not as effective as initially promised.

Education With Computers

Papert (1993) began The Children's Machine by asking readers to imagine two groups of time travelers from the nineteenth century. The first are surgeons who are shown a modern surgery. Almost everything will appear new. The other group of time travelers are teachers who are shown classrooms of students sitting in rows listening to a teacher. While they would notice some differences in the classrooms, much would appear familiar. Ferster (2014 , p. 1) repeated this thought-experiment two decades later: “a nineteenth-century visitor would feel quite at home in a modern classroom, even at our most elite institutions of higher learning.” Why would modern classrooms seem so familiar to the nineteenth century guests? Is it that education got it right back then and that further advances were not necessary? Papert argued in letter to President Carter that education can be radically different and better if technology is embraced. He said: “Unless we do this, tomorrow will continue to be the prisoner of the primitivity of yesterday” ( Papert, 1980 ).

There are several ways to classify different types of interactions students can have with an educational computer system. Atkinson (1968) and Suppes (1966) describe three: drill and practice, tutorial systems, and dialogue systems. Drill and practice can be seen as computer extensions of most of the early teaching machines. Students could take, at their leisure, practice quizzes and be provided with immediate feedback. Given the value of practice, testing, and feedback (e.g., Roediger et al., 2010 ), that different students will be best served by items that vary in difficulty and pertain to different competencies ( Metcalfe, 2002 ), and that this is a monotonous task for teachers to do, drill and practice is an obvious part of the curriculum for computers to assist. Drill and practice systems allow students to evaluate their own knowledge efficiently.

The goal of tutorial systems goes beyond just allowing students to evaluate their knowledge. The goal is to teach students how to solve problems. The computer system can offer more interaction and feedback than a textbook or sage-on-the-stage edutainment, and more individualization than a teacher in front of a large class. For example, if most of a class has mastered calculating the area of a triangle, the remaining students could use a tutorial system to provide them with an alternative mode of teaching while the rest of the class learn a new task (which the students re-learning about triangles may or may not eventually be taught). While the teaching machines of the 1950s and 1960s could guide students, step-by-step, through different exercises, the computer allows many more steps and allows the student to progress down multiple pathways ( Ritter et al., 2016 ).

Dialogue systems allow a greater amount of interaction between the student and the system. Suppes ( 1966 , p. 219) gives the example of a student asking: “Why are demand curves always convex with respect to the origin?” Fifty years on there have been many advances in natural language processing. I entered this phrase into Google and it suggested several web pages, including quora.com , where the question:

What are the conditions under which a demand curve is convex? Explain with a few real life examples of goods with convex demand curves.

was asked and answered. Dialogue systems require some natural language processing. AutoTutor by Graesser and colleagues (for a review see Nye et al., 2014 ) is an excellent example of using language processing.

Papert (1980) describes another way to differentiate technology uses in education: auxiliary and fundamental computer uses. Auxiliary uses are where the computer is not changing the educational processes. The same (or very similar) activities are simply being presented in a different medium. These can be helpful, perhaps allowing individuals to work on their own activities and at their own pace, or making feedback more rapid. Using computers changes how the lesson is taught and the physical implementation of it, but there is not a major pedagogical shift. Fundamental uses change what is being taught and why . They enable students to learn information that they might not learn in a traditional classroom and learning is done in a manner that is a departure from the traditional pedagogy. Fundamental uses change the curriculum rather than implementing the same curriculum differently. While auxiliary uses can be beneficial and efficient, Papert argues that the fundamental uses have the potential to revolutionize education. In an essay written with former West Virginia governor Gaston Caperton, Papert describes how technology should be used not just to solve problems of “schools-as-they-are,” but to build schools into “schools-as-they-can-be” ( Papert and Caperton, 1999 ). This idea is at the core of the XQ Super School Project ( https://xqsuperschool.org/ ), who fund proposals to create innovative schools.

Because computers are part of modern society, they will remain in students' homes and classrooms. However, the progress for educational software is neither straight-forward nor without impediments. A common comparison is made with the wild west (e.g., Reingold, 2015 ). In the wild west people could make wild claims about the curing powers of anything (e.g., heroin was marketed as a cure for cancer, sluggishness, colds, tuberculosis, etc., www.narconon.org/drug-information/heroin-history.html , Accessed June 21, 2017). If a vendor sold a lot of the product, a lot of money could be made, and in that era some people viewed making a lot of money as an important indicator of success. It was difficult for the public then to verify any of the these claims. A customer might be choosing a remedy for a sick child based on hope and desperation. Cuban's (2001) book title, Oversold & Underused , summarizes the view of many about the impact of EwT. Schools want a computer system that will cover their entire curriculum and for all grades, and to improve scores on standardized tests immediately. They hope that there is just a switch to flip much like those in the wild west hoped a sip of a magical elixir would be a cure-all for any ailment. They hear a sales pitch that seems to offer this. It is important that those making decisions about EwT do not feel like they are making decisions out of desperation like the parents of a sick child in the wild west, but there is pressure on school administrators to have their schools move up in the rankings ( Foley and Goldstein, 2012 ; Muller, 2018 ) and offering hope without evidence is a popular and persuasive sales technique in unregulated markets.

Technology is advancing. The time traveling teachers from Papert's and Ferster's examples would be amazed to see the number of students with cell phones, the capabilities of these devices, the amounts the devices get used, the ubiquity of social media (and its impact), and in general the technology related behavior of these Digital Natives . Some aspects of these affect EwT. For example, the small screens put constraints on the amount of text that can be shown at any one time just as writing on paper vs. animal skin affected what was written ( Small, 1997 ). Detailed plots, long tables, and lengthy well-constructed arguments have been replaced by tweets .

Greenfield (2015) describes the phenomenon of Mind Change . The mind and the brain are adaptive to their environment. Many aspects of using computers (e.g., rapidly accessing lots of information in small pieces, social networking, “likes” on Facebook) make different demands on humans than traditional environments. She argues that it is important to research possible changes–some may be positive and some may negative–on the brains/minds of Digital Natives caused by new technologies. The EwT industry is betting that the positive effects greatly outweigh the negative effects, but consider the EwT approach of “asking Google.” Does the rapid access to (possibly accurate) related information change the way people create questions and evaluate answers? Does the impersonal way that people get feedback from electronic tutors affect how the graduates would handle workplace criticism? Does the anonymity of the internet affect us as social animals? EwT can be implemented in different ways and each of these may affect students in different ways.

In order to predict future use of EwT accurately and to develop EwT well it is necessary to understand how EwT and education in general are situated within political, social, and economic climates. Convincing people to change their behaviors can be difficult. Pressey argued that the poor economic period in which he was creating his teaching machines meant that they were not financially successful. In the late 1950s and 1960s, when Skinner and others were creating teaching machines, the economic situation was better. Further, in the aftermath of Sputnik there was a societal drive to increase education. Still, the teaching machines of this time failed to have a lasting impact.

Currently, while the US is in fairly good financial shape, there is uncertainty about Federal funding of education research. Further, the decision making about buying specific products is non-centralized. This means that an attractive sales pitch is critical to the product's success. The start-up mentality is also evident. Many products are being developed with the backing of venture capitalists who hope that they have bet on some successful ones. Success of a single product that a venture capitalist bets on usually is greater in financial terms than the amount lost by several failed bets. Whether this gambling ratio is appropriate for education is debatable. In this climate Papert's (1980) notion of having a few centers of research is welcomed, though how they are funded and if they can maintain independence are uncertain.

The political climate in the US and elsewhere is divided with respect to scientific evidence based decision making and argumentation. While people benefit from science (e.g., the popularity of cell phones), many people are not interested in the science itself. If the magazines at supermarket checkouts are any indication, then the public has more interest in where reality stars vacation than scientific progress. The division between decision making based on science and based on superstition has long existed ( MacKay, 1841/2012 ). Advanced technology has the potential for positive change, but it is necessary to make sure that the people embrace science over mysticism.

… people use electricity and still believe in the magic power of signs and exorcisms. The Pope of Rome broadcasts over the radio about the miraculous transformation of water into wine. Movie stars go to mediums. Aviators who pilot miraculous mechanisms created by man's genius wear amulets on their sweaters. What inexhaustible reserves they possess of darkness, ignorance, and savagery!

( Trotsky, 1933 , October, 1933)

Trotsky was talking about how the scientific conditions among much of the population in Germany helped Hitler come to power, but parallels can be made with other places and time periods when a sizable proportion of the population lacks trust in the scientific method and when leaders who do not use valid evidence for their decision making come to power. These are not good circumstances to use scientific results to convince many in the public of the value of using technology in education.

Importantly, some people do value technology and do believe in its potential. There will continue to be investment on EwT. It is important for the research and the education communities to help target this investment. There are lots of products, many without much evidence of effectiveness. To prevent survival of the loudest dictating the evolution of EwT, it is important to think carefully about a potential EwT research framework. In the interest of children's education decisions should be based on the available science, rather than on, using Trotsky's phrase, “darkness, ignorance, and savagery!”

A Research Framework

Billions of dollars are spent each year on technology for education. However, the current landscape is problematic. It is important to go beyond the wild west metaphor ( Reingold, 2015 ). This section is divided into three parts. The first section describes an attempt to discover whether a particular type of EwT is effective. The conclusion from this section is that identifying effective products is difficult. While many view a randomized controlled trial (RCT) of a product as the gold standard for evaluation research, here it is argued that there is a time and a place for RCTs, but that other research methods should also be used. The second section describes different levels of explanation. It is argued that EwT research should focus on the goals of the system and whether the underlying rules used to build the system effectively achieve these goals. These levels are based on an influential neuroscience framework put forward by Marr and Poggio (1977) . The third section provides more detail about how research could progress, and provides some examples.

Testing the Effectiveness of Any EwT System Is Difficult

The US Institute of Education Sciences (IES) rightly states that “well-designed and implemented randomized controlled trials are considered the “gold standard” for evaluating an intervention's effectiveness” ( https://ies.ed.gov/ncee/pubs/evidence_based/randomized.asp , Accessed June 22, 2017). However, it is often difficult to conduct an RCT of a product or any complex educational innovation in development. Many studies described as RCTs by their authors have significant problems and probably should not be called RCTs ( Ginsburg and Smith, 2016 ). Sullivan (2011) discusses how forcing a research question into an RCT can distance the study from the intended experiences of the product/system. The argument here is not to avoid full-program evaluations. These can be very important in providing evidence for the effectiveness of well-established products. A good example is Pane et al. (2014) study of Carnegie Learning ( Ritter et al., 2016 ). Randomization is useful, but is neither necessary nor sufficient for making causal inference ( Wright, 2006 ; Pearl, 2009 ; Deaton and Cartwright, in press ).

Performing experiments on components of the product can often be done more easily than evaluating whether and entire program works or not, and this approach can be beneficial for product development and may generalize to other products. Quasi-experiments still have their place, particularly with archival data. In the remainder of this section a study with the goal of evaluating the effectiveness of personalized learning (PL) is discussed to illustrate the difficulties of program evaluation.

The phrase “personalized learning” is often used to describe a wide variety of approaches ( Arney, 2015 ; Horn and Staker, 2015 ; Taylor and Gebre, 2016 ). The core elements are that individual students decide some of the content and pace of their own learning, and that the system (usually a computer) guides and may restrict choices. This has the important consequence of freeing up time and resources so that the teacher can work one-on-one with each student or with small groups of students when they are not working on computers.

The study has achieved much press and optimism from investors (e.g., www.chalkbeat.org/posts/us/2017/05/22/as-ed-reformers-urge-a-big-bet-on-personalized-learning-research-points-to-potential-rewards-and-risks/ . Accessed June 22, 2017). Pane et al. (2015) were funded to evaluate whether, in a nutshell, PL works. I describe several hypothetical “what ifs” and conclude that even if they had performed an RCT the results would have been difficult to interpret.

The authors highlighted how difficult a task it is to design a study to measure the effectiveness of PL and cautioned others not to over-interpret their results. Here is what they did. The schools in their PL condition had about 2 years of PL. These schools are described as those which “embrace personalized learning,” “have a high degree of integrated technology as part of their school designs,” are among “the country's best public charter schools,” and have gone “through a series of competitive selection processes” ( Pane et al., 2015 , p. 36). These descriptions make these schools sound great! From these descriptions it might be expected that if a random selection of students were sent to these schools overall this group would perform better (i.e., raise their test scores by more) than if these students had been sent to a random selection of other schools. Rather than choosing a comparison group of schools with similar positive characteristics, Pane et al. (2015) had the test vendor (NWEA) match each student in these select schools with students at a variety of schools that presumably, on the whole, do not have all the positive characteristics described above for the PL group.

Given that interest is in school effects on student outcomes the decision not to compare similar schools is problematic. Even without an intervention the expectation would be that the PL group's scores should increase more because according to the authors' descriptions these are better schools than most. A good analogy would be if you were comparing restaurants. In one condition you have restaurants that are “the country's best” and in the control condition you have a random sample of restaurants (the schools were matched on urban, suburban, and rural, so restaurants might be matched on serving French, Italian, or Spanish cuisines). You gave each restaurant the same set of ingredients. For the “best” group you also gave them a recently published cookbook. The restaurants prepare meals using only the ingredients that they were given. Judges grade these meals and the “best” restaurants get higher marks. The question is whether you would conclude the cookbook was the cause?

If the PL group of schools were shown to have a positive effect in a study with properly matched group of schools (or if schools were randomly allocated either to have PL for 2 years or to be in a control group), what would this tell us and what would be the next steps? The norm in science when trying to establish causation is to have the treatment nearly identical for all units. In this study, however, “innovation was encouraged” for the PL schools. The schools were “not adopting a single standardized model of personalized learning” ( Pane et al., 2015 , p. 3). While this may or may not be beneficial for the education of the students in these schools, it makes it difficult for the researchers. Because of the variation in what PL means to people and how it was implemented among the PL schools, coupled with variation in teaching philosophies among the set of control schools, it would be difficult to conclude anything other than these hodge podges of difficult to characterize pedagogies may differ. While the “positive” outcome on performance reported in Pane et al. (2015) would be predicted just because of how schools were sampled (there are also issues with respect to how students are allocated to schools), an RCT of a complex intervention would be unlikely to shed much light on why the intervention works. If an RCT (or a well designed matched-group study) showed substantial positive results, the next step would be to try to understand which components of the curriculum may be effective and research these components. An alternative is to begin with this research while these approaches are still being developed.

The purpose of the preceding paragraphs was not to criticize Pane et al. (2015) . They did well in their attempt to answer a difficult question: “Does the set of things called PL work in the schools that are funded to do it?” Consider a simpler research question:

What is the evidence that an EwT approach should work?

The remainder of this paper will explain what this question means in more detail and will argue for why it is a useful question for those seeking to purchase EwT and why it is a useful approach for developing research.

Levels of Explanation

Marr (1982/2010) reviewed vision research from the 1960s and 1970s. He marveled at research examining the physiology of vision, for example neuron firing patterns, but felt this did not provide a complete understanding of vision. To understand a system as complex as vision he argued that it was necessary to understand the system at multiple levels. He said that it was necessary to understand the goals of the system and the rules that the system used to achieve these goals in order to understand the system. Poggio (in the Afterword of Marr, 1982/2010 and in Poggio, 2012 ) discusses some changes that he recommends to the levels that he and Marr and had originally proposed ( Marr and Poggio, 1977 ). He discusses how any classification system is somewhat arbitrary and that alternative levels and labels may be more appropriate in other contexts. He also stressed the importance of understanding the relationships among levels. This is particularly important for developing EwT. Here are five levels proposed for understanding EwT that are adapted from Marr and Poggio's (1977) framework.

1. Decide the top-level GOALS of the system. These will often be related to student learning, but may also be skills acquisition, behavior modification, etc.

2. Decide WHAT is to be learned. This might be something specific like the physics of volcanoes or a specific component of socio-emotional learning, or it may be broad like all academic subjects included in state tests or improving maladaptive behavior. Once these top two levels are decided researchers and developers can concentrate on how to achieve the goals for what is to be learned.

3. The CORE features of the theory for how the goals can be achieved. These will likely be from pedagogical or learning science theories. These features would include a “soft core” that can be evaluated, refuted, and adapted. Researchers should continue to question whether these features produce the stated goals at the top level.

4. Decide the RULES or algorithm that will be used to set up the conditions that the core features of the theory predict will achieve the goals. These rules should be written in enough detail to allow them to be programmed into a computer language, to allow a carpenter to build a mechanical teaching machine, etc.

5. Decide the physical IMPLEMENTATION. For computer technology this would include choosing among tablets, smartphones, and desktops. The physical implementation will dictate, to some extent, how the rules are represented. With computers, this usually means the computer language used.

Once it is decided, for example, that the goal is for students to learn the physics of volcanoes, the researchers and developers would list some core features of the theory that they believe account for learning. Specific rules are developed that set up the conditions that these core features predict should increase learning. These are translated into a representation compatible with the physical implementation. This would be a top-down way to develop a product. Bottom-up development can also occur. Because of the widespread availability of computers and the scalability of software, some investors may only invest in computer software technology. The developers might then decide what they can build with this technology that may be profitable. The choice, for example, between building a product for foreign language learning vs. for learning physics may be based on what can be built with the physical device (e.g., foreign language tutorials usually require audio input/output and physics tutorials can be helped by allowing the user to manipulate diagrams).

Table 1 shows how the Pressey's and Skinner's teaching machines might fit within the proposed levels. At the top two levels the goal of the proposed technology is to increase student learning of the meanings of biology terms (other “whats” could be used). The teaching machines were built assuming the core features from the learning theories of behaviorism, developed by many of the psychologists of the time (e.g., Hull, Pavlov, Skinner, Thorndike, Watson). A rule that would increase the associations between biology terms and their definitions would be to present the definition to the students and have them respond with the term, until they are correct on each item. This is a “drill and practice” procedure as described before. How to represent this rule will depend on how it is physically implemented. Suppose that the accuracy of a student's response does not require perfect spelling of a term; it would be necessary to have that described in the rules. With teaching machines the students might be presented with the correct answer and have to judge whether they were correct. With computers it is possible to use approximate matching algorithms to allow for mis-spellings, but the designer would still need to state how close a spelling could be (for example, the Levenshtein distance [LD]). Partial credit could be given for some answers, and details of this rubric would be required. The choice of physical implementation could also be constrained by higher levels. For example, if it were necessary to show a video or play audio, then flashcards could not be used.

Table 1 . Levels for Teaching Machines.

The rules should be written out in detail. Consider a simple algorithm:

Let there be a set of items to be learned.

Let R j be the number of times the student “correctly” answers the j th item.

Loop until R j = 2 for all j .

  Sample one item from set of items such that R j < 2 and ask student.

  If the student is correct add one to R j .

Move onto next task.

These rules are simple enough that they could be implemented with different technologies, some of which are listed in the bottom row of Table 1 . The different implementations should all use the same rules as specified, but could differ on aspects that are not specified. For example, how the sampling of items with R j < 2 is done would vary depending on the implementation. Constructing a mechanical teaching machine for this purpose could be done, and the sampling would depend on how the gears work. A student using flashcards might have separate piles for the number of times the item was correctly answered (a pile for R j = 0, a pile for R j = 1, and a pile for R j = 2) and the student might progress through the R j = 0 then the R j = 1 piles in the order the cards are in, placing a card on the bottom of the appropriate pile after its use. Computer software might use a pseudo-random process to shuffle all R j < 2 items or create an order to optimize learning by spreading out semantically related items. If the sampling method turns out to be important for achieving the computational goals, then it should be specified by the rules, and this could constrain the choice of physical implementation.

These implementations can be judged on how well they implement the rules and the system's cost. Given the simplicity of this example all of these technologies should implement the rules accurately, though gears in teaching machines can break, flashcards get wet, and computers crash. Some implementations may have additional benefits built in, like making flashcards requires the student to write each item or that the computer can display information in innovative ways. The cost varies considerably among the different implementations. Creating mechanical teaching machines for each student would be expensive and would be limited in what else they could do. The flashcards would be cheap to produce. They are often produced by the students themselves. Assuming the student already has a computer, making the software for a good “drill and practice” task can be expensive for the original prototype, but making copies available for additional users can be done inexpensively.

It is worth stressing that there will be important issues constructing the input and output for computers vs. tablets vs. smartphones. Research to make sure these modes are compatible is necessary, but these likely could implement the underlying rules in a similar way, perhaps changing how information is displayed. Further, this type of research (called human computer interaction, human centered technology, or user design) is well known by software companies so most EdTech companies have people in place for this. Compatibility research to examine, for example, if there is an advantage taking the SAT on a computer or a tablet, is already done by testing organizations.

Papert's auxiliary-fundamental (or “school-as-it-is” vs. “school-as-it-can-be”) distinction is also interesting from the perspective of these levels. The auxilary uses would have the same upper levels as their traditional classroom counterparts. They might also assume the same core features of the learning theory and may even use the same rules. The physical implementation would differ from the traditional teaching curriculum. Auxiliary uses can still be valuable as the new technology may mean students learn more efficiently (e.g., by rapid feedback, having more one-on-one time with the teacher), but WHAT they learn would be the same. An example would be taking a well-constructed textbook, an item bank for each chapter, videos of excellent lectures, and changing the physical implementation of these so that they are delivered on a computer screen and through headphones. For fundamental uses WHAT students learn is different from the traditional curriculum. They may even introduce new GOALS.

As discussed earlier, PL is currently receiving a lot of attention. While there is not a single PL approach, Table 2 shows a generic PL approach. Often PL is designed to improve student learning, broadly defined. The WHAT is often most of the academic curriculum. This was true for the sample of PL schools in Pane et al. (2015) . Table 2 shows three of the core features for why PL is assumed to increase student learning. The first feature, individualization, is that students are taught what is optimal for them. What is best for one person is not best for everyone. For example, the information should not be too easy or too difficult; it should be within what is called the zone of proximal development ( Vygotsky, 1978 ) or the region of proximal learning ( Metcalfe, 2002 ). The material might also be individualized to the students' interests and learning styles. The second core feature is that allowing students choice in itself is important for their growth. However, because students do not generally choose the items that will maximize learning (e.g., Metcalfe, 2002 ), this feature can conflict with the first feature. The third feature is what is often called competency based education, where students progress based on their performance rather than their age.

Table 2 . Levels for Personalized Learning.

The rules for achieving these core features (which in turn are assumed to achieve the over-arching goal) with PL are often: allowing the student some control over the pace and content of their curriculum and using feedback to help them make good choices. A more detailed set of rules would be necessary to describe any specific approach. Suppose the system allowed students to choose a module and once within a module the students could decide (with feedback from the computer) whether they knew the module well enough to move onto the next module. Students would use the computer to try to learn the tasks, receive feedback on their progress, and then decide if they think they know the information well enough to move to the next task.

  Study information (amount and method determined by student)

  Take assessment

  Computer estimates student achievement

  Receive feedback

    If achievement estimate less than proficient, do not allow student to leave loop.

    If achievement estimate at least proficient, allow student to leave loop.

     Student decides whether to leave loop or repeat. Move to next task.

It can be beneficial to draw causal diagrams, which are called directed graphs in the branch of mathematics called graph theory, as in Figure 3 . This is just a single module. It would be nested with many others into a course. This module may have pre-requisites or be a pre-requisite for other modules. It is important to consider how a localized causal diagram like Figure 3 fits within a larger causal network (for more details about using graphs to attribute causality see Pearl, 2009 ; Pearl et al., 2016 ).

Figure 3 . A single module. The student can go back to look at content and then repeat the test, over and over. This is called a directed cyclical graph.

It is worth noting that this particular set of rules is compatible with different procedures being inside the box labeled “Look At.” This might involve using drill and practice, watching instructional videos (edutainment), reading web pages, natural language tutors, etc. A student might also choose not to review the information and just re-take the test. There are also many options for how to produce a score. Often this will be a percentage correct, but educational measurement experts discuss many alternatives. There are advantages and disadvantages to not constrain the rule specifics, particularly when the system is designed to cover many different academic subjects. If these specifics are not stated in the rules then they can be varied. Evaluations should, however, take into account this variability. The implementation level in Table 2 lists three possibilities. It is possible to implement the rules without a computer, using for example professional human tutors (peers can sometimes also be used). This might be practical for some tasks, like learning to drive, but this would be prohibitively expensive to educate all students throughout their curricula. This is why most people view computers as the most practical way to implement the rules listed in Table 2 and thereby to achieve the goals. For this reason PL has become closely associated with EwT.

An Approach to EwT

The purpose of this section is to provide recommendations for two groups of people and some example research questions. The groups are: (a) individual researchers, and (b) people buying EwT products.

Individual Researchers

Figure 3 isolates a small number of relationships so that they can be more easily understood in isolation. This is a common approach to science: examine the phenomenon “in the simplest possible context that is not entirely trivial, and later generalize” ( Cox and Donnelly, 2011 , p. 5). This allows researchers to identify causal relationships. The difficulty of this approach is that how things operate in complex contexts can be different than how they operate in a simple context. McGuire (1973 , p. 452) discusses this as his second Kōan for research: “In this nettle chaos, we discern this pattern, truth.” In most naturally occurring situations the variables in which the researchers are interested will likely covary with many other variables. Untangling these relations (and “nettle chaos” seems a good metaphor for having many relationships among variables) is statistically difficult. McGuire describes how this approach requires statistical expertise, but that it should be guided by theory. The following is a list, influenced by McGuire's (1973) , for researchers to consider.

1. For researchers focusing just on a small part of the system, like that depicted in Figure 3, the relationship between just two variables can be complex. Often the relationships are non-linear and vary depending on so called moderator variables.

2. If your theory predicts that some intervention affects test scores, describe what else it should affect and what it should not affect. And then measure these. The famous geneticist and statistician, Ronald Fisher, summed this up nicely: “Make your theories elaborate” (see Cochran, 1965 , p. 252).

3. If your primary effect of interest is long-term, like graduating high school, it can be useful to include mediator or proxy variables. These are variables that if affected signal that the long term variables should also be affected. For example, with graduating high school, mediator variables would include lower delinquency which has a causal influence on graduation. Proxy variables are common in medicine. If you are interested in whether some drug given to people in their forties reduces the risk of heart attacks later in life, you might measure blood pressure in the weeks after giving the drug to subjects and extrapolate that by reducing blood pressure in the short term this drug also reduces the probability of a heart attack in the long term.

4. When trying to show how a single “nettle plant” works within “nettle chaos” it may be useful to draw all the variables/constructs that you are interested in and use arrows to show how they may be connected. When many variables are all inter-related, trying to understand causal and associative relationships from a complex graph can be difficult. Pearl (2009) is a key reference for identifying causes from graphs (see also Morgan and Winship, 2007 ; Imbens and Rubin, 2015 ; Pearl et al., 2016 ; Steiner et al., 2017 ). Simulation methods can also be useful to show the predicted outcomes from these diagrams.

5. The focus of much science is on causal relationships, but in some cases associative relationships are also important. Researchers should not confuse these and should use appropriate methods for investigating each of these. It is important to avoid causal words, like “influence,” “effect,” and “impact,” when the study was not designed to estimate causal relations ( Wright, 2003 ).

6. Particularly when studying a complex system it is necessary to have theory guide analyses to avoid data fishing/mining problems. The relationships assumed for the rules and core features should help to constrain the statistical analyses.

Caveat Emptor : What the Buyer Should Know

Modern EwT consumers face the difficult situation when deciding whether to acquire EwT products and if so which ones. One solution would be to have regulations on what can be sold, but this seems unlikely in the current political climate in the US. An alternative is to have consumers demand more verifiable information from vendors before they spend money. If EwT research is done well and consumers expect vendors to provide certain information, then we can move beyond the wild west situation without further regulations. The following list are aspects of the product that ideally a consumer should be told. At present most vendors will not have answers to all of these, but hopefully future research will provide them with answers. This list is based on how judges in the US are told to decide whether to accept expert testimony on scientific and technical matters. These guidelines are adapted from the Federal Rules of Evidence and primarily from three US Supreme Court decisions. These are collectively called the Daubert Trilogy ( Daubert, 1993 ; Joiner, 1997 ; Kumho, 1999 ).

1. The evidence supporting the product's effectiveness should be generally agreed upon by those in the relevant field (e.g., learning scientists).

2. The studies that provide the evidence should be based on sound scientific principles. Daubert discusses Popper's (1959/2002) use of falsification to demarcate science from non-science. According to Popper a good scientific theory should have withstood studies that could have falsified it. If so, it is said to have attained a degree of corroboration. Additional aspects of the scientific value of the supporting evidence should also be considered.

3. The effect sizes (or error rates) of any effects should be known. The vendor should be able to predict effect sizes for your school, should reference the uncertainty of these estimates, and should be able to say how these estimates were calculated. The variables (e.g., school and student variables) that moderate efficacy should be known and the situations where it is not predicted to be effective should be discussed. The intervals for the estimates may be very broad, but the uncertainty of estimates should not be hidden.

4. The evidence that the vendor uses to support their claims should be published in peer-reviewed journals that adhere to scientific principles. It is important for consumers to realize that that being published in a good journal does not imply the finding is accurate. There are many inaccurate findings in good journals ( Ioannidis, 2005 ). However, the peer review process does prevent much junk science from being disseminated.

5. The distance between the research and the conclusions should not be too great. This might relate to how well the rules match with core features and these with goals, whether the conditions used for the supporting evidence are very different from the school setting, or whether the sample in the studies is very different from the intended group of students.

One of the arguments against the Daubert Trilogy is that it requires judges to make complex judgments about the value of scientific research when they do not receive much training on this. Differentiating junk science from reputable science can be difficult. Those making IT decisions for school systems are in a similar situation. They may face enthusiastic vendors and need to differentiate circumstance from pomp.

Example Research Questions

A few example research questions are presented to provide a flavor of the type of research that can address whether a product should work. For illustration the following list will focus on the type of EwT depicted Figure 3 .

How does the software estimate whether the student is likely to have reached the desired performance level? Is the assessment fair, valid, and reliable? How do the scores given to students affect how they decide to navigate the system and how do they affect students' beliefs about how much they know?

Bad choices

People learn from making errors ( Metcalfe, 2017 ) and given that many EwT systems require students to make choices, some of these will be bad choices. Can these be identified and types of bad choices classified? Can students be taught to make better choices and are there ways to ensure that students will learn as much as possible from their bad choices?

Computer software can provide a large amount of data (i.e., the log files). While exploratory data mining might suggest some associations, the data are messy and exploratory atheoretical mining can be problematic (massive “nettle chaos” with statistical analyses with many researcher degrees of freedom). Analysis based on theories of the students' cognitive processes could direct specific statistical questions.

How does choosing one's own path affect confidence? Are students happier with the task if they believe that they have chosen it? Are they likely to engage in the task more? Is there less mind-wandering? These could all mediate the effect of agency.

“Modern technology will dramatically improve education!” attracts headlines, but educators have read headlines like this before. Technology has the potential to improve education and it might someday revolutionize education, but to date research evidence has failed to keep pace with optimistic rhetoric. The computer is different than past technologies because students are already learning about computers and many have computers at home (and in their pockets). Even compared with a decade ago, children are more immersed in computing technologies (i.e., many are Digital Natives ). There are still pitfalls and it is possible (though unlikely) that EwT could fade as some other technologies have. It is important to learn from the history of EwT so that the field does not succumb to the same problems and to understand the current environment so that other potential problems can be addressed. There is much investment both financially and by many schools changing how they teach students, so there are many people wanting this to work. The goal of this paper was to put forward a research framework to increase the likelihood of success and to maximize positive impacts.

The best way to avoid pitfalls is to accumulate evidence about the effectiveness of products, submit the research for peer-review, and show how continued improvements to products are helping students. It is important to show those contemplating EwT that its adoption is a good investment. This requires more evidence–of the type described in the list for what consumers should ask vendors–to show that using these new technologies is financially responsible.

A research framework was put forward that will help with these goals. The focus should be to show that the different components of the system work. Studies should not just look at whether an innovative program improves end-of-year test scores, but whether the individual parts of this program influence many of the facets that co-vary with and influence test scores. This will help the field to evolve and to show why products work. The focus should not be on specific products, because new versions of them will arrive with new technologies (at the implementation level), but on the rules that the products implement and on whether these rules lead to the goals of the system. It is important for EwT to have evidence to withstand criticism. It is important that researchers and developers continue to strive for Suppes' goal to provide Aristotle-like EwT tutors for all students.

Author Contributions

The author confirms being the sole contributor of this work and approved it for publication.

My position is funded in part by the Chan Zuckerberg Initiative.

Conflict of Interest Statement

The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

The author's work on personalized learning is funded by the Chan Zuckerberg Initiative (CZI). The conclusions presented here are the author's, and do not represent those of CZI. Thanks to Heather Kirkpatrick and Kristin Smith Alvarez for much discussion and valuable comments on previous drafts.

André, E., Baker, R., Hu, X., Rodrigo, M. T., and du Boulay, B. (eds.). (2017). Artificial Intelligence in Education . Gewerbestrasse: Springer.

Google Scholar

Arney, L. (2015). Go Blended! A Handbook for Blending Technology in Schools . San Francisco, CA: Jossey-Bass.

Atkinson, R. C. (1968). Computerized instruction and the learning process. Amer. Psychol. 23, 225–239. doi: 10.1037/h0020791

PubMed Abstract | CrossRef Full Text | Google Scholar

Benjamin, L. T. Jr. (1988). A history of teaching machines. Am. Psychol. 43, 703–712. doi: 10.1037/0003-066X.43.9.703

CrossRef Full Text | Google Scholar

Borges, J. L. (1941/1998). Collected Fictions . New York, NY: Penguin Books.

Carruthers, M. (1990). The Book of Memory: A Study of Memory in Medieval Culture . New York, NY: Cambridge University Press.

Clark, A. (2008). Supersizing the Mind: Embodiment, Action, and Cognitive Extension . New York, NY: Oxford University Press.

Cochran, W. G. (1965). The planning of observational studies of human populations. J. R. Stat. Soc. Ser. A 128, 234–266. doi: 10.2307/2344179

Conati, C., Heffernan, N., Mitrovic, A., and Verdejo, M. (eds.). (2015). Artificial Intelligence in Education . Gewerbestrasse: Springer.

Cox, D. R., and Donnelly, C. A. (2011). Principles of Applied Statistics . Cambridge: Cambridge University Press.

Cuban, L. (1986). Teachers and Machines: The Classroom use of Technology Since 1920 . New York, NY: Teachers College Press.

Cuban, L. (2001). Oversold and Underused: Computers in the Classroom . Cambridge, MA: Harvard University Press.

Daubert (1993). Daubert v. Merrell Dow Pharmaceuticals, Inc . 509 US 579, 589.

Deaton, A., and Cartwright, N. (in press). Understanding misunderstanding randomized controlled trials. Soc. Sci. Med . doi: 10.1016/j.socscimed.2017.12.005

Ferster, B. (2014). Teaching Machines: Learning From the Intersection of Education and Technology . Baltimore, MD: Johns Hopkins Press.

Ferster, B. (2016). Sage on the Screen: Education, Media, and How We Learn . Baltimore, MD: Johns Hopkins Press.

Foley, B., and Goldstein, H. (2012). Measuring Success: League Tables in the Public Sector . London: British Academy.

Ginsberg, A. (1955/2014). Howl and Other Poems . Mansefield Centre, CT: Martino Publishing.

Ginsburg, A., and Smith, M. S. (2016). Do Randomized Controlled Trials Meet the “Gold Standard”? A Study of the Usefulness of RCTS in the What Works Clearinghouse . Technical report. American Enterprise Institute.

Greenfield, S. (2015). Mind Change: How Digital Technologies Are Leaving Their Mark on Our Brains . New York, NY: Random House.

Horn, M. B., and Staker, H. (2015). Blended: Using Disruptive Innovation to Improve Schools . San Francisco, CA: Jossey-Bass.

Imbens, G. W., and Rubin, D. B. (2015). Causal Inference for Statistics, Social, and Biomedical Sciences: An Introduction . New York, NY: Cambridge University Press.

Ioannidis, J. P. (2005). Why most published research findings are false. PLoS Med. 2:e124. doi: 10.1371/journal.pmed.0020124

Joiner (1997). General Electric Co. v. Joiner . 522 U.S. 136.

Kumho (1999). Kumho Tire Co. v. Carmichael . 526 U.S. 137.

MacKay, C. (1841/2012). Extraordinary Popular Delusions and the Madness of Crowds . San Bernardino, CA: Renaissance Classics.

Marr, D. (1982/2010). Vision: A Computational Investigation Into the Human Representation and Processing of Visual Information . Cambridge, MA: MIT Press.

Marr, D., and Poggio, T. (1977). From understanding computation to understanding neural circuitry. Neurosci. Res. Prog. Bull. 15, 470–488.

McGuire, W. J. (1973). The yin and yang of progress in social psychology: seven koan. J. Pers. Soc. Psychol. 26, 446–456. doi: 10.1037/h0034345

Metcalfe, J. (2002). Is study time allocated selectively to a region of proximal learning? J. Exp. Psychol. Gen. 131, 349–363. doi: 10.1037/0096-3445.131.3.349

Metcalfe, J. (2017). Learning from errors. Annu. Rev. Psychol. 68, 465–489. doi: 10.1146/annurev-psych-010416-044022

Morgan, S. L., and Winship, C. (2007). Counterfactuals and Causal Inference: Methods and Principles for Social Research . Cambridge: Cambridge University Press.

Muller, J. Z. (2018). The Tyranny of Metrics . Princeton, NJ: Princeton University Press.

Nolen-Hoeksema, S., Fredrickson, B. L., Loftus, G. R., and Lutz, C. (2015). Atkinson and Hilgard's Introduction to Psychology, 16th Edn . Wadsworth: Cengage Learning.

Nye, B. D., Graesser, A. C., and Hu, X. (2014). AutoTutor and family: a review of 17 years of natural language tutoring. Int. J. Artif. Intell. Educ. 24, 427–469. doi: 10.1007/s40593-014-0029-5

Pane, J. F., Griffin, B., McCaffrey, D. F., and Karam, R. (2014). Effectiveness of cognitive tutor algebra I at scale. Educ. Eval. Pol. Anal. 36, 127–144. doi: 10.3102/0162373713507480

Pane, J. F., Steiner, E. D., Baird, M. D., and Hamilton, L. S. (2015). Continued Progress: Promising Evidence on Personalized Learning . Technical Report RR-1365-BMGF, RAND Corporation.

Papert, S. (1980). Paper for the president's commission for a national agenda for the 80s . Available online at: http://www.papert.org/articles/president_paper.html

Papert, S. (1993). The Children's Machine: Rethinking School in the Age of the Computer . New York, NY: Basic Books.

Papert, S., and Caperton, G. (1999). Vision for Education: The Caperton-Papert Platform . St. Louis, MO: Essay written 91st Annual National Governors' Association.

Pearl, J. (2009). Causality: Models, Reasoning, and Inference, 2nd Edn . New York, NY: Cambridge University Press.

Pearl, J., Glymour, M., and Jewell, N. P. (2016). Causal Inference in Statistics: A Primer . Chichester: Wiley.

Poggio, T. (2012). The Levels of Understanding Framework, Revised . Technical Report MIT-CSAIL-TR-2012-014, CBCL-308, MIT, Cambridge, MA.

Popper, K. (1959/2002). The Logic of Scientific Discovery . Milton Park: Routledge.

Pressey, S. L. (1933). Psychology and the New Education . New York, NY: Harper and Brothers Publishers.

Reingold, J. (2015). Why ed tech is currently ‘the wild wild west’. Fortune . Available online at: http://fortune.com/2015/11/04/ed-tech-at-fortune-global-forum-2015

Ritter, S., Yudelson, M., Fancsali, S. E., and Berman, S. R. (2016). “Mastery learning works at scale,” in Proceedings of the Third ACM Conference on Learning @ Scale (Edinburgh: ACM), 71–79.

Roediger, H. L. III. (1980). Memory metaphors in cognitive psychology. Mem. Cogn. 8, 231–246.

PubMed Abstract | Google Scholar

Roediger, H. L. III., Agarwal, P. K., Kang, S. H. K., and Marsh, E. J. (2010). “Benefits of testing memory: best practices and boundary conditions,” in New Frontiers in Applied Memory , eds G. M. Davies and D. B. Wright (Brighton: Psychology Press), 13–49.

Rubin, D. C. (1995). Memory in Oral Traditions: The Cognitive Psychology of Epic, Ballads, and Counting-out Rhymes . New York, NY: Oxford University Press.

Sagan, C. (1996). The Demon-Haunted World: Science as a Candle in the Dark . New York, NY: Ballantine Books.

Skinner, B. F. (1958). Teaching machines. Science 128, 969–977. doi: 10.1126/science.128.3330.969

Small, J. P. (1997). Wax Tablets of the Mind: Cognitive Studies of Memory and Literacy in Classical Antiquity . London: Routledge.

Steiner, P. M., Kim, Y., Hall, C. E., and Su, D. (2017). Graphical models for quasi-experimental designs. Sociol. Methods Res . 46, 155–188. doi: 10.1177/0049124115582272

Sullivan, G. M. (2011). Getting off the “gold standard”: randomized controlled trials and education research. J. Grad. Med. Educ. 3, 285–289. doi: 10.4300/JGME-D-11-00147.1

Suppes, P. (1966). The uses of computers in education. Sci. Am. 215, 20–220. doi: 10.1038/scientificamerican0966-206

Taylor, R. D., and Gebre, A. (2016). “Teacher–student relationships and personalized learning: implications of person and contextual variables,” in Handbook on Personalized Learning for States, Districts, and Schools , eds M. Murphy, S. Redding, and J. S. Twyman (Charlotte, NC: Information Age Publishing, Inc.), 205–220.

Thornburg, D. (2014). From the Campfire to the Holodeck: Creating Engaging and Powerful 21st Century Learning Environments . San Francisco, CA: Jossey-Bass.

Trotsky, L. (1933). “What is national socialism? [English translation],” in Trotsky Internet Archive, originally The Modern Thinker . Available online at: https://www.marxists.org/archive/trotsky/germany/1933/330610.htm

Vygotsky, L. S. (1978). Mind in Society: The Development of Higher Psychological Processes . Boston, MA: Harvard University Press.

Wright, D. B. (2003). Making friends with your data: improving how statistics are conducted and reported. Brit. J. Educ. Psychol. 73, 123–136. doi: 10.1348/000709903762869950

Wright, D. B. (2006). Causal and associative hypotheses in psychology: examples from eyewitness testimony research. Psychol. Public Policy Law 12, 190–213. doi: 10.1037/1076-8971.12.2.190

Yates, F. A. (1966). The Art of Memory . Chicago, IL: University of Chicago Press.

Keywords: technology, CAI, EdTech, cognition, personalized learning, blended learning, cognitive tutor

Citation: Wright DB (2018) A Framework for Research on Education With Technology. Front. Educ . 3:21. doi: 10.3389/feduc.2018.00021

Received: 30 October 2017; Accepted: 20 March 2018; Published: 12 April 2018.

Reviewed by:

Copyright © 2018 Wright. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY) . The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Daniel B. Wright, [email protected]

Publication

Global education monitoring report summary, 2023: technology in education: a tool on whose terms? (hin)

The new 2023 GEM Report on  Technology in education: A tool on whose terms?  addresses the use of technology in education around the world through the lenses of relevance, equity, scalability and sustainability.

It argues that education systems should always ensure that learners’ interests are placed at the center and that digital technologies are used to support an education based on human interaction rather than aiming at substituting it. The report looks at ways in which technology can help reach disadvantaged learners but also ensure more knowledge reaches more learners in more engaging and cheaper formats. It focuses on how quality can be improved, both in teaching and learning basic skills, and in developing the digital skills needed in daily life. It recognizes the role of technology in system management with special reference to assessment data and other education management information.

Related items

• Guidelines and tools
• Policy Advice
• Country page: India
• Topics: Flagship report
• Region: Asia and the Pacific
• UNESCO Office in New Delhi
• See more add

More on this subject

Other recent publications

New global data reveal education technology’s impact on learning

The promise of technology in the classroom is great: enabling personalized, mastery-based learning; saving teacher time; and equipping students with the digital skills they will need  for 21st-century careers. Indeed, controlled pilot studies have shown meaningful improvements in student outcomes through personalized blended learning. 1 John F. Pane et al., “How does personalized learning affect student achievement?,” RAND Corporation, 2017, rand.org. During this time of school shutdowns and remote learning , education technology has become a lifeline for the continuation of learning.

As school systems begin to prepare for a return to the classroom , many are asking whether education technology should play a greater role in student learning beyond the immediate crisis and what that might look like. To help inform the answer to that question, this article analyzes one important data set: the 2018 Programme for International Student Assessment (PISA), published in December 2019 by the Organisation for Economic Co-operation and Development (OECD).

Every three years, the OECD uses PISA to test 15-year-olds around the world on math, reading, and science. What makes these tests so powerful is that they go beyond the numbers, asking students, principals, teachers, and parents a series of questions about their attitudes, behaviors, and resources. An optional student survey on information and communications technology (ICT) asks specifically about technology use—in the classroom, for homework, and more broadly.

In 2018, more than 340,000 students in 51 countries took the ICT survey, providing a rich data set for analyzing key questions about technology use in schools. How much is technology being used in schools? Which technologies are having a positive impact on student outcomes? What is the optimal amount of time to spend using devices in the classroom and for homework? How does this vary across different countries and regions?

From other studies we know that how education technology is used, and how it is embedded in the learning experience, is critical to its effectiveness. This data is focused on extent and intensity of use, not the pedagogical context of each classroom. It cannot therefore answer questions on the eventual potential of education technology—but it can powerfully tell us the extent to which that potential is being realized today in classrooms around the world.

Five key findings from the latest results help answer these questions and suggest potential links between technology and student outcomes:

• The type of device matters—some are associated with worse student outcomes.
• Geography matters—technology is associated with higher student outcomes in the United States than in other regions.
• Who is using the technology matters—technology in the hands of teachers is associated with higher scores than technology in the hands of students.
• Intensity matters—students who use technology intensely or not at all perform better than those with moderate use.
• A school system’s current performance level matters—in lower-performing school systems, technology is associated with worse results.

This analysis covers only one source of data, and it should be interpreted with care alongside other relevant studies. Nonetheless, the 2018 PISA results suggest that systems aiming to improve student outcomes should take a more nuanced and cautious approach to deploying technology once students return to the classroom. It is not enough add devices to the classroom, check the box, and hope for the best.

What can we learn from the latest PISA results?

How will the use, and effectiveness, of technology change post-covid-19.

The PISA assessment was carried out in 2018 and published in December 2019. Since its publication, schools and students globally have been quite suddenly thrust into far greater reliance on technology. Use of online-learning websites and adaptive software has expanded dramatically. Khan Academy has experienced a 250 percent surge in traffic; smaller sites have seen traffic grow fivefold or more. Hundreds of thousands of teachers have been thrown into the deep end, learning to use new platforms, software, and systems. No one is arguing that the rapid cobbling together of remote learning under extreme time pressure represents best-practice use of education technology. Nonetheless, a vast experiment is underway, and innovations often emerge in times of crisis. At this point, it is unclear whether this represents the beginning of a new wave of more widespread and more effective technology use in the classroom or a temporary blip that will fade once students and teachers return to in-person instruction. It is possible that a combination of software improvements, teacher capability building, and student familiarity will fundamentally change the effectiveness of education technology in improving student outcomes. It is also possible that our findings will continue to hold true and technology in the classroom will continue to be a mixed blessing. It is therefore critical that ongoing research efforts track what is working and for whom and, just as important, what is not. These answers will inform the project of reimagining a better education for all students in the aftermath of COVID-19.

PISA data have their limitations. First, these data relate to high-school students, and findings may not be applicable in elementary schools or postsecondary institutions. Second, these are single-point observational data, not longitudinal experimental data, which means that any links between technology and results should be interpreted as correlation rather than causation. Third, the outcomes measured are math, science, and reading test results, so our analysis cannot assess important soft skills and nonacademic outcomes.

It is also worth noting that technology for learning has implications beyond direct student outcomes, both positive and negative. PISA cannot address these broader issues, and neither does this paper.

But PISA results, which we’ve broken down into five key findings, can still provide powerful insights. The assessment strives to measure the understanding and application of ideas, rather than the retention of facts derived from rote memorization, and the broad geographic coverage and sample size help elucidate the reality of what is happening on the ground.

Finding 1: The type of device matters

The evidence suggests that some devices have more impact than others on outcomes (Exhibit 1). Controlling for student socioeconomic status, school type, and location, 2 Specifically, we control for a composite indicator for economic, social, and cultural status (ESCS) derived from questions about general wealth, home possessions, parental education, and parental occupation; for school type “Is your school a public or a private school” (SC013); and for school location (SC001) where the options are a village, hamlet or rural area (fewer than 3,000 people), a small town (3,000 to about 15,000 people), a town (15,000 to about 100,000 people), a city (100,000 to about 1,000,000 people), and a large city (with more than 1,000,000 people). the use of data projectors 3 A projector is any device that projects computer output, slides, or other information onto a screen in the classroom. and internet-connected computers in the classroom is correlated with nearly a grade-level-better performance on the PISA assessment (assuming approximately 40 PISA points to every grade level). 4 Students were specifically asked (IC009), “Are any of these devices available for you to use at school?,” with the choices being “Yes, and I use it,” “Yes, but I don’t use it,” and “No.” We compared the results for students who have access to and use each device with those who do not have access. The full text for each device in our chart was as follows: Data projector, eg, for slide presentations; Internet-connected school computers; Desktop computer; Interactive whiteboard, eg, SmartBoard; Portable laptop or notebook; and Tablet computer, eg, iPad, BlackBerry PlayBook.

On the other hand, students who use laptops and tablets in the classroom have worse results than those who do not. For laptops, the impact of technology varies by subject; students who use laptops score five points lower on the PISA math assessment, but the impact on science and reading scores is not statistically significant. For tablets, the picture is clearer—in every subject, students who use tablets in the classroom perform a half-grade level worse than those who do not.

Some technologies are more neutral. At the global level, there is no statistically significant difference between students who use desktop computers and interactive whiteboards in the classroom and those who do not.

Finding 2: Geography matters

Looking more closely at the reading results, which were the focus of the 2018 assessment, 5 PISA rotates between focusing on reading, science, and math. The 2018 assessment focused on reading. This means that the total testing time was two hours for each student, of which one hour was reading focused. we can see that the relationship between technology and outcomes varies widely by country and region (Exhibit 2). For example, in all regions except the United States (representing North America), 6 The United States is the only country that took the ICT Familiarity Questionnaire survey in North America; thus, we are comparing it as a country with the other regions. students who use laptops in the classroom score between five and 12 PISA points lower than students who do not use laptops. In the United States, students who use laptops score 17 PISA points higher than those who do not. It seems that US students and teachers are doing something different with their laptops than those in other regions. Perhaps this difference is related to learning curves that develop as teachers and students learn how to get the most out of devices. A proxy to assess this learning curve could be penetration—71 percent of US students claim to be using laptops in the classroom, compared with an average of 37 percent globally. 7 The rate of use excludes nulls. The United States measures higher than any other region in laptop use by students in the classroom. US = 71 percent, Asia = 40 percent, EU = 35 percent, Latin America = 31 percent, MENA = 21 percent, Non-EU Europe = 41 percent. We observe a similar pattern with interactive whiteboards in non-EU Europe. In every other region, interactive whiteboards seem to be hurting results, but in non-EU Europe they are associated with a lift of 21 PISA points, a total that represents a half-year of learning. In this case, however, penetration is not significantly higher than in other developed regions.

Finding 3: It matters whether technology is in the hands of teachers or students

The survey asks students whether the teacher, student, or both were using technology. Globally, the best results in reading occur when only the teacher is using the device, with some benefit in science when both teacher and students use digital devices (Exhibit 3). Exclusive use of the device by students is associated with significantly lower outcomes everywhere. The pattern is similar for science and math.

Again, the regional differences are instructive. Looking again at reading, we note that US students are getting significant lift (three-quarters of a year of learning) from either just teachers or teachers and students using devices, while students alone using a device score significantly lower (half a year of learning) than students who do not use devices at all. Exclusive use of devices by the teacher is associated with better outcomes in Europe too, though the size of the effect is smaller.

Finding 4: Intensity of use matters

PISA also asked students about intensity of use—how much time they spend on devices, 8 PISA rotates between focusing on reading, science, and math. The 2018 assessment focused on reading. This means that the total testing time was two hours for each student, of which one hour was reading focused. both in the classroom and for homework. The results are stark: students who either shun technology altogether or use it intensely are doing better, with those in the middle flailing (Exhibit 4).

The regional data show a dramatic picture. In the classroom, the optimal amount of time to spend on devices is either “none at all” or “greater than 60 minutes” per subject per week in every region and every subject (this is the amount of time associated with the highest student outcomes, controlling for student socioeconomic status, school type, and location). In no region is a moderate amount of time (1–30 minutes or 31–60 minutes) associated with higher student outcomes. There are important differences across subjects and regions. In math, the optimal amount of time is “none at all” in every region. 9 The United States is the only country that took the ICT Familiarity Questionnaire survey in North America; thus, we are comparing it as a country with the other regions. In reading and science, however, the optimal amount of time is greater than 60 minutes for some regions: Asia and the United States for reading, and the United States and non-EU Europe for science.

The pattern for using devices for homework is slightly less clear cut. Students in Asia, the Middle East and North Africa (MENA), and non-EU Europe score highest when they spend “no time at all” on devices for their homework, while students spending a moderate amount of time (1–60 minutes) score best in Latin America and the European Union. Finally, students in the United States who spend greater than 60 minutes are getting the best outcomes.

One interpretation of these data is that students need to get a certain familiarity with technology before they can really start using it to learn. Think of typing an essay, for example. When students who mostly write by hand set out to type an essay, their attention will be focused on the typing rather than the essay content. A competent touch typist, however, will get significant productivity gains by typing rather than handwriting.

Would you like to learn more about our Social Sector Practice ?

Finding 5: the school systems’ overall performance level matters.

Diving deeper into the reading outcomes, which were the focus of the 2018 assessment, we can see the magnitude of the impact of device use in the classroom. In Asia, Latin America, and Europe, students who spend any time on devices in their literacy and language arts classrooms perform about a half-grade level below those who spend none at all. In MENA, they perform more than a full grade level lower. In the United States, by contrast, more than an hour of device use in the classroom is associated with a lift of 17 PISA points, almost a half-year of learning improvement (Exhibit 5).

At the country level, we see that those who are on what we would call the “poor-to-fair” stage of the school-system journey 10 Michael Barber, Chinezi Chijoke, and Mona Mourshed, “ How the world’s most improved school systems keep getting better ,” November 2010. have the worst relationships between technology use and outcomes. For every poor-to-fair system taking the survey, the amount of time on devices in the classroom associated with the highest student scores is zero minutes. Good and great systems are much more mixed. Students in some very highly performing systems (for example, Estonia and Chinese Taipei) perform highest with no device use, but students in other systems (for example, Japan, the United States, and Australia) are getting the best scores with over an hour of use per week in their literacy and language arts classrooms (Exhibit 6). These data suggest that multiple approaches are effective for good-to-great systems, but poor-to-fair systems—which are not well equipped to use devices in the classroom—may need to rethink whether technology is the best use of their resources.

What are the implications for students, teachers, and systems?

Looking across all these results, we can say that the relationship between technology and outcomes in classrooms today is mixed, with variation by device, how that device is used, and geography. Our data do not permit us to draw strong causal conclusions, but this section offers a few hypotheses, informed by existing literature and our own work with school systems, that could explain these results.

First, technology must be used correctly to be effective. Our experience in the field has taught us that it is not enough to “add technology” as if it were the missing, magic ingredient. The use of tech must start with learning goals, and software selection must be based on and integrated with the curriculum. Teachers need support to adapt lesson plans to optimize the use of technology, and teachers should be using the technology themselves or in partnership with students, rather than leaving students alone with devices. These lessons hold true regardless of geography. Another ICT survey question asked principals about schools’ capacity using digital devices. Globally, students performed better in schools where there were sufficient numbers of devices connected to fast internet service; where they had adequate software and online support platforms; and where teachers had the skills, professional development, and time to integrate digital devices in instruction. This was true even accounting for student socioeconomic status, school type, and location.

COVID-19 and student learning in the United States: The hurt could last a lifetime

Second, technology must be matched to the instructional environment and context. One of the most striking findings in the latest PISA assessment is the extent to which technology has had a different impact on student outcomes in different geographies. This corroborates the findings of our 2010 report, How the world’s most improved school systems keep getting better . Those findings demonstrated that different sets of interventions were needed at different stages of the school-system reform journey, from poor-to-fair to good-to-great to excellent. In poor-to-fair systems, limited resources and teacher capabilities as well as poor infrastructure and internet bandwidth are likely to limit the benefits of student-based technology. Our previous work suggests that more prescriptive, teacher-based approaches and technologies (notably data projectors) are more likely to be effective in this context. For example, social enterprise Bridge International Academies equips teachers across several African countries with scripted lesson plans using e-readers. In general, these systems would likely be better off investing in teacher coaching than in a laptop per child. For administrators in good-to-great systems, the decision is harder, as technology has quite different impacts across different high-performing systems.

Third, technology involves a learning curve at both the system and student levels. It is no accident that the systems in which the use of education technology is more mature are getting more positive impact from tech in the classroom. The United States stands out as the country with the most mature set of education-technology products, and its scale enables companies to create software that is integrated with curricula. 11 Common Core State Standards sought to establish consistent educational standards across the United States. While these have not been adopted in all states, they cover enough states to provide continuity and consistency for software and curriculum developers. A similar effect also appears to operate at the student level; those who dabble in tech may be spending their time learning the tech rather than using the tech to learn. This learning curve needs to be built into technology-reform programs.

Taken together, these results suggest that systems that take a comprehensive, data-informed approach may achieve learning gains from thoughtful use of technology in the classroom. The best results come when significant effort is put into ensuring that devices and infrastructure are fit for purpose (fast enough internet service, for example), that software is effective and integrated with curricula, that teachers are trained and given time to rethink lesson plans integrating technology, that students have enough interaction with tech to use it effectively, and that technology strategy is cognizant of the system’s position on the school-system reform journey. Online learning and education technology are currently providing an invaluable service by enabling continued learning over the course of the pandemic; this does not mean that they should be accepted uncritically as students return to the classroom.

Jake Bryant is an associate partner in McKinsey’s Washington, DC, office; Felipe Child is a partner in the Bogotá office; Emma Dorn is the global Education Practice manager in the Silicon Valley office; and Stephen Hall is an associate partner in the Dubai office.

The authors wish to thank Fernanda Alcala, Sujatha Duraikkannan, and Samuel Huang for their contributions to this article.

Explore a career with us

Related articles.

Safely back to school after coronavirus closures

How the world’s most improved school systems keep getting better

Billions are spent on educational technology, but we don’t know if it works

Professor of Reading and Children’s Development, The Open University

Disclosure statement

Natalia Kucirkova receives funding from the Norwegian Research Council and The Jacobs Foundation. She works in WiKIT AS, which is a university spin-off concerned with EdTech evidence. She is affiliated with the University of Stavanger, The Open University and University College London.

The Open University provides funding as a founding partner of The Conversation UK.

View all partners

During the COVID lockdowns, schools and universities worldwide relied on education technology – edtech – to keep students learning. They used online platforms to give lessons, mark work and send feedback, used apps to teach and introduced students to programs that let them work together on projects.

In the aftermath of school closures, the market for edtech has kept on growing. The value of the sector is projected to rise to US$132.4 billion globally by 2032 (£106 billion).

The problem is that we don’t know very much about how effective many edtech apps or programs are – or if they are effective at all .

And some effects may be negative. Some of the so-called educational apps advertised to families show many adverts to children. They may use manipulative features to keep children on screens without teaching them anything new.

This technology is here to stay and will remain a significant part of how children learn – so knowing whether it works is imperative.

Assessing and addressing the quality of edtech is a significant task, especially when it is already so widely used. For edtech under development, a valuable option is to foster closer collaboration between tech developers and scientists who study learning to embed existing research and knowledge into the design.

Research consultancy firms can carry out swift assessments to provide edtech developers with information on how well what they are offering works. Transparency and integrity in the research process is vital, though, to prevent bias. Ways of ensuring this include pre-registration : reporting that a study is going to take place before it happens.

Partnerships with schools could also provide valuable feedback . However, minimum standards of quality and ethical considerations would need to be assured before technologies are sent to schools.

Setting a standard

When it comes to edtech that is already available, what is really needed is some kind of standardised metric to assess how well it works.

But establishing minimum standards for the effect of edtech is easier said than done. There is, historically, a lack of standardised metrics for assessing educational impact within impact economics – the study of how businesses create financial returns while ensuring positive social or environmental outcomes.

Without standardisation, there are too many ways to assess edtech. A review commissioned by the UK government of evaluation criteria and standards for edtech analysed 74 methods for assessing their quality.

Similarly, I carried out a research study with colleagues on available criteria to assess the effectiveness and efficacy of edtech produced specifically for schools. We found 65 different frameworks for evaluating whether these school-specific offerings work.

The abundance of evaluation possibilities can be confusing for edtech businesses. The multitude of options makes it difficult to ascertain the quality of their products. It is confusing to investors too, especially those who want to prioritise not only edtech’s return on investment but also a return on education and community.

Read more: Schools are using research to try to improve children's learning – but it's not working

A yardstick that establishes the minimum quality requirements for a edtech product to be used in schools is crucial to ensure technology does more good and no harm. The creation of a yardstick needs to take into account both the product quality and the process of using the technology – whether it works for diverse populations and diverse learning environments.

The independent verification of evidence is vital , considering that any company can simply “generate” a study with the data they daily collect on users. In my research work with colleagues, I have argued for a focus on the rigour and validity of various research types.

New initiatives, such as the International Certification of Evidence of Impact in Education , have begun to consolidate the different research approaches, standards and certifications related to evidence of edtech impact globally. Ultimately, the goal is to make it easier for schools and parents to navigate the thousands of educational apps and online platforms available.

Whether individual countries will create the legal and institutional frameworks to enforce any of the standards remains to be seen. Countries will need to select standards that suit both their economic and educational agendas. An important shift is needed so that schools can strategically select edtech they know will help children’s learning.

• Young people
• educational apps
• Keep me on trend

Content Coordinator

Lecturer / Senior Lecturer - Marketing

Assistant Editor - 1 year cadetship

Executive Dean, Faculty of Health

Lecturer/Senior Lecturer, Earth System Science (School of Science)

'I've learned from amazing professors who study the social impact of technology'

Tanvi Namjoshi: Computer Science

A&S Communications

Tanvi Namjoshi

Computer Science Basking Ridge, N.J.

Why did you choose Cornell?

I was drawn to Cornell and the College of Arts & Sciences because of the diversity of studies and the flexible path it offered. I came in as a computer science major, but I was also really passionate about policy and ethics, and at Cornell, these subjects weren’t taught as mutually exclusive but rather two sides of the same coin. As a scholar in the Milstein Program in Technology and Humanity, I saw Cornell’s investment in interdisciplinary studies and have had opportunities to learn from amazing professors who study the social impact of technology. I also believe that the diversity of Cornell’s academic programs draws in a student body that is unmatched – everyone has unique interests and backgrounds that they are invested in and that they are excited to share with you, either through events run by student organizations or simply over lunch.

What is your main extracurricular activity and why is it important to you? 

My main extracurricular activity has been serving on the executive board for Women in Computing at Cornell (WICC) where we work to make the computer and information science departments more inclusive and accessible to women and other gender minorities. In WICC, I found a group of women who were excited to give advice about classes, jobs and navigating the college environment. This sense of community is extremely important for women, who are underrepresented in computer science. Through the years I’ve served various roles from faculty relations director, to social chair, to vice president. It has been extremely rewarding to run impactful events and to be a leader supporting other women as they make their visions for the CIS community come to life. My most cherished memory of WICC has been running a program called “Lunch Bunch.” Each week, a group of students has the opportunity to eat lunch with a rotating guest professor, reducing the divide between faculty and students. As a facilitator, I learned about many female professors’ paths to academia, research interests and motivations. Most importantly, hearing their stories, I realized that much about what they loved in computer science resonated with my own interests. This showed me that I would enjoy research – and it led me to start my research journey! 

What Cornell memory do you treasure the most?      

Taking the final bow at the first Nazaqat showcase during my junior year. It was the culmination of a year of hard work as both a dancer and a leader in the club. Seeing the support of the Cornell community, our alumni and our families made the hours of rehearsals and planning worth it. I have studied Kathak, the Indian classical dance form Nazaqat performs, for 12 years of my life – finding a club at Cornell where I could keep up with my dance practice truly changed my Cornell experience. 

What are the most valuable skills you gained from your Arts & Sciences education?      

The power to approach problems from different perspectives. As someone in a technical field, my instinct is to look for solutions in formulas and mathematical analysis. However, my humanities classes have taught me to step back and look at how social values and our laws may create or entrench these problems in society. My design thinking classes and independent studies have taught me how to interview those affected by the problem and look for solutions in their answers. And of course, if these techniques reveal that technology could be a part of the solution, my computer science classes have equipped me with the technical skills to innovate. 

What have you accomplished as a Cornell student that you are most proud of?

I have participated in research at Cornell through three different departments: government, science and technology studies and computer science. I studied the behavior of candidates under different voting algorithms with Prof. Jon Kleinberg, worked with the Brooks School Tech Policy Institute and explored how generative AI affects art through interviews with local artists. Jumping into research is exciting, but also difficult coming from studying a subject only through coursework. My research journey has included many different fields and methodologies, but I am proud of my work in each project as it has all informed the work I want to do in graduate school. 

Every year, our faculty nominate graduating Arts & Sciences students to be featured as part of our Extraordinary Journeys series.  Read more about the Class of 202 4.

More News from A&S

Committee to recommend final expressive activity policy

American Sign Language has found a growing home on the Hill

Extraordinary Journeys: The Class of 2024

BTPI will research relationship between Bitcoin and financial freedom

The Impact of Technology on Education

Technology has revolutionized education, transforming traditional classrooms into dynamic, interactive learning environments. Digital tools and resources, such as online courses, educational apps, and virtual reality, have made learning more accessible and engaging for students worldwide. These advancements break down geographical barriers, allowing learners from all corners of the globe to access quality education and a wealth of information at their fingertips.

One of the most significant benefits of technology in education is the ability to personalize learning experiences. Students can progress at their own pace, focusing on their unique strengths and addressing individual challenges. Adaptive learning platforms and AI-driven tools provide customized feedback and resources tailored to each student’s needs, making learning more effective and enjoyable.

Additionally, technology empowers educators to employ innovative teaching methods and collaborate with peers globally. Teachers can use multimedia presentations, gamified learning experiences, and virtual labs to make lessons more engaging. Instant feedback tools and online assessments streamline the evaluation process, helping educators track progress and adjust their teaching strategies. Overall, technology enhances the educational experience, preparing students for a technology-driven future and equipping them with essential skills for the modern world.

Share this:

Written by Asifkhan

Leave a reply cancel reply.

You must be logged in to post a comment.

Impact of Dynamic Learning in the Contemporary Physics Classroom

Native speaker.

© Copyright 2024 Cambridge. All Rights Reserved.

Username or Email Address

Remember Me

Don't have an account? Register

Forgot password?

Enter your account data and we will send you a link to reset your password.

Your password reset link appears to be invalid or expired.

Privacy policy.

To use social login you have to agree with the storage and handling of your data by this website.

Add to Collection

Public collection title

Private collection title

No Collections

Here you'll find all collections you've created before.

Report Post

Please log in to report posts

Robot-phobia could exasperate hotel, restaurant labor shortage

Using more robots to close labor gaps in the hospitality industry may backfire and cause more human workers to quit, according to a Washington State University study.

The study, involving more than 620 lodging and food service employees, found that "robot-phobia" -- specifically the fear that robots and technology will take human jobs -- increased workers' job insecurity and stress, leading to greater intentions to leave their jobs. The impact was more pronounced with employees who had real experience working with robotic technology. It also affected managers in addition to frontline workers. The findings were published in the International Journal of Contemporary Hospitality Management .

"The turnover rate in the hospitality industry ranks among the highest across all non-farm sectors, so this is an issue that companies need to take seriously," said lead author Bamboo Chen, a hospitality researcher in WSU's Carson College of Business. "The findings seem to be consistent across sectors and across both frontline employees and managers. For everyone, regardless of their position or sector, robot-phobia has a real impact."

Food service and lodging industries were hit particularly hard by the pandemic lockdowns, and many businesses are still struggling to find enough workers. For example, the accommodation workforce in April 2024 was still 9.2% below what it was in February 2020, according to U.S. Bureau of Labor Statistics. The ongoing labor shortage has inspired some employers to turn to robotic technology to fill the gap.

While other studies have focused on customers' comfort with robots, this study focuses on how the technology impacted hospitality workers. Chen and WSU colleague Ruying Cai surveyed 321 lodging and 308 food service employees from across the U.S., asking a range of questions about their jobs and attitudes toward robots. The survey defined "robots" broadly to include a range of robotic and automation technologies, such as human-like robot servers and automated robotic arms as well as self-service kiosks and tabletop devices.

Analyzing the survey data, the researchers found that having a higher degree of robot-phobia was connected to greater feelings of job insecurity and stress -- which were then correlated with "turnover intention" or workers' plans to leave their jobs. Those fears did not decrease with familiarity: employees who had more actual engagement with robotic technology in their daily jobs had higher fears that it would make human workers obsolete.

Perception also played a role. The employees who viewed robots as being more capable and efficient also ranked higher in turnover intention.

Robots and automation can be good ways to help augment service, Chen said, as they can handle tedious tasks humans typically do not like doing such as washing dishes or handling loads of hotel laundry. But the danger comes if the robotic additions cause more human workers to quit. The authors point out this can create a "negative feedback loop" that can make the hospitality labor shortage worse.

Chen recommended that employers communicate not only the benefits but the limitations of the technology -- and place a particular emphasis on the role human workers play.

"When you're introducing a new technology, make sure not to focus just on how good or efficient it will be. Instead, focus on how people and the technology can work together," he said.

• Brain-Computer Interfaces
• Robotics Research
• Engineering
• Engineering and Construction
• Artificial Intelligence
• Neural Interfaces
• Industrial Relations
• Disaster Plan
• Education and Employment
• Humanoid robot
• Industrial robot
• Industrial relations
• Nanorobotics
• Robotic surgery
• Commercial fishing

Story Source:

Materials provided by Washington State University . Original written by Sara Zaske. Note: Content may be edited for style and length.

Journal Reference :

• Chun-Chu (Bamboo) Chen, Ruiying Cai. Are robots stealing our jobs? Examining robot-phobia as a job stressor in the hospitality workplace . International Journal of Contemporary Hospitality Management , 2024; DOI: 10.1108/IJCHM-09-2023-1454

Cite This Page :

Explore More

• Record Low Antarctic Sea Ice: Climate Change
• Brain 'Assembloids' Mimic Blood-Brain Barrier
• 'Doomsday' Glacier: Catastrophic Melting
• Blueprints of Self-Assembly
• Meerkat Chit-Chat
• First Glimpse of an Exoplanet's Interior
• High-Efficiency Photonic Integrated Circuit
• Life Expectancy May Increase by 5 Years by 2050
• Toward a Successful Vaccine for HIV
• Highly Efficient Thermoelectric Materials

Trending Topics

Strange & offbeat.