articles about educational tools

Along with Stanford news and stories, show me:

• Student information
• Faculty/Staff information

We want to provide announcements, events, leadership messages and resources that are relevant to you. Your selection is stored in a browser cookie which you can remove at any time using “Clear all personalization” below.

Image credit: Claire Scully

New advances in technology are upending education, from the recent debut of new artificial intelligence (AI) chatbots like ChatGPT to the growing accessibility of virtual-reality tools that expand the boundaries of the classroom. For educators, at the heart of it all is the hope that every learner gets an equal chance to develop the skills they need to succeed. But that promise is not without its pitfalls.

“Technology is a game-changer for education – it offers the prospect of universal access to high-quality learning experiences, and it creates fundamentally new ways of teaching,” said Dan Schwartz, dean of Stanford Graduate School of Education (GSE), who is also a professor of educational technology at the GSE and faculty director of the Stanford Accelerator for Learning . “But there are a lot of ways we teach that aren’t great, and a big fear with AI in particular is that we just get more efficient at teaching badly. This is a moment to pay attention, to do things differently.”

For K-12 schools, this year also marks the end of the Elementary and Secondary School Emergency Relief (ESSER) funding program, which has provided pandemic recovery funds that many districts used to invest in educational software and systems. With these funds running out in September 2024, schools are trying to determine their best use of technology as they face the prospect of diminishing resources.

Here, Schwartz and other Stanford education scholars weigh in on some of the technology trends taking center stage in the classroom this year.

AI in the classroom

In 2023, the big story in technology and education was generative AI, following the introduction of ChatGPT and other chatbots that produce text seemingly written by a human in response to a question or prompt. Educators immediately worried that students would use the chatbot to cheat by trying to pass its writing off as their own. As schools move to adopt policies around students’ use of the tool, many are also beginning to explore potential opportunities – for example, to generate reading assignments or coach students during the writing process.

AI can also help automate tasks like grading and lesson planning, freeing teachers to do the human work that drew them into the profession in the first place, said Victor Lee, an associate professor at the GSE and faculty lead for the AI + Education initiative at the Stanford Accelerator for Learning. “I’m heartened to see some movement toward creating AI tools that make teachers’ lives better – not to replace them, but to give them the time to do the work that only teachers are able to do,” he said. “I hope to see more on that front.”

He also emphasized the need to teach students now to begin questioning and critiquing the development and use of AI. “AI is not going away,” said Lee, who is also director of CRAFT (Classroom-Ready Resources about AI for Teaching), which provides free resources to help teach AI literacy to high school students across subject areas. “We need to teach students how to understand and think critically about this technology.”

Immersive environments

The use of immersive technologies like augmented reality, virtual reality, and mixed reality is also expected to surge in the classroom, especially as new high-profile devices integrating these realities hit the marketplace in 2024.

The educational possibilities now go beyond putting on a headset and experiencing life in a distant location. With new technologies, students can create their own local interactive 360-degree scenarios, using just a cell phone or inexpensive camera and simple online tools.

“This is an area that’s really going to explode over the next couple of years,” said Kristen Pilner Blair, director of research for the Digital Learning initiative at the Stanford Accelerator for Learning, which runs a program exploring the use of virtual field trips to promote learning. “Students can learn about the effects of climate change, say, by virtually experiencing the impact on a particular environment. But they can also become creators, documenting and sharing immersive media that shows the effects where they live.”

Integrating AI into virtual simulations could also soon take the experience to another level, Schwartz said. “If your VR experience brings me to a redwood tree, you could have a window pop up that allows me to ask questions about the tree, and AI can deliver the answers.”

Gamification

Another trend expected to intensify this year is the gamification of learning activities, often featuring dynamic videos with interactive elements to engage and hold students’ attention.

“Gamification is a good motivator, because one key aspect is reward, which is very powerful,” said Schwartz. The downside? Rewards are specific to the activity at hand, which may not extend to learning more generally. “If I get rewarded for doing math in a space-age video game, it doesn’t mean I’m going to be motivated to do math anywhere else.”

Gamification sometimes tries to make “chocolate-covered broccoli,” Schwartz said, by adding art and rewards to make speeded response tasks involving single-answer, factual questions more fun. He hopes to see more creative play patterns that give students points for rethinking an approach or adapting their strategy, rather than only rewarding them for quickly producing a correct response.

Data-gathering and analysis

The growing use of technology in schools is producing massive amounts of data on students’ activities in the classroom and online. “We’re now able to capture moment-to-moment data, every keystroke a kid makes,” said Schwartz – data that can reveal areas of struggle and different learning opportunities, from solving a math problem to approaching a writing assignment.

But outside of research settings, he said, that type of granular data – now owned by tech companies – is more likely used to refine the design of the software than to provide teachers with actionable information.

The promise of personalized learning is being able to generate content aligned with students’ interests and skill levels, and making lessons more accessible for multilingual learners and students with disabilities. Realizing that promise requires that educators can make sense of the data that’s being collected, said Schwartz – and while advances in AI are making it easier to identify patterns and findings, the data also needs to be in a system and form educators can access and analyze for decision-making. Developing a usable infrastructure for that data, Schwartz said, is an important next step.

With the accumulation of student data comes privacy concerns: How is the data being collected? Are there regulations or guidelines around its use in decision-making? What steps are being taken to prevent unauthorized access? In 2023 K-12 schools experienced a rise in cyberattacks, underscoring the need to implement strong systems to safeguard student data.

Technology is “requiring people to check their assumptions about education,” said Schwartz, noting that AI in particular is very efficient at replicating biases and automating the way things have been done in the past, including poor models of instruction. “But it’s also opening up new possibilities for students producing material, and for being able to identify children who are not average so we can customize toward them. It’s an opportunity to think of entirely new ways of teaching – this is the path I hope to see.”

REALIZING THE PROMISE:

Leading up to the 75th anniversary of the UN General Assembly, this “Realizing the promise: How can education technology improve learning for all?” publication kicks off the Center for Universal Education’s first playbook in a series to help improve education around the world.

It is intended as an evidence-based tool for ministries of education, particularly in low- and middle-income countries, to adopt and more successfully invest in education technology.

While there is no single education initiative that will achieve the same results everywhere—as school systems differ in learners and educators, as well as in the availability and quality of materials and technologies—an important first step is understanding how technology is used given specific local contexts and needs.

The surveys in this playbook are designed to be adapted to collect this information from educators, learners, and school leaders and guide decisionmakers in expanding the use of technology.  

Introduction

While technology has disrupted most sectors of the economy and changed how we communicate, access information, work, and even play, its impact on schools, teaching, and learning has been much more limited. We believe that this limited impact is primarily due to technology being been used to replace analog tools, without much consideration given to playing to technology’s comparative advantages. These comparative advantages, relative to traditional “chalk-and-talk” classroom instruction, include helping to scale up standardized instruction, facilitate differentiated instruction, expand opportunities for practice, and increase student engagement. When schools use technology to enhance the work of educators and to improve the quality and quantity of educational content, learners will thrive.

Further, COVID-19 has laid bare that, in today’s environment where pandemics and the effects of climate change are likely to occur, schools cannot always provide in-person education—making the case for investing in education technology.

Here we argue for a simple yet surprisingly rare approach to education technology that seeks to:

• Understand the needs, infrastructure, and capacity of a school system—the diagnosis;
• Survey the best available evidence on interventions that match those conditions—the evidence; and
• Closely monitor the results of innovations before they are scaled up—the prognosis.

RELATED CONTENT

Podcast: How education technology can improve learning for all students

To make ed tech work, set clear goals, review the evidence, and pilot before you scale

The framework.

Our approach builds on a simple yet intuitive theoretical framework created two decades ago by two of the most prominent education researchers in the United States, David K. Cohen and Deborah Loewenberg Ball. They argue that what matters most to improve learning is the interactions among educators and learners around educational materials. We believe that the failed school-improvement efforts in the U.S. that motivated Cohen and Ball’s framework resemble the ed-tech reforms in much of the developing world to date in the lack of clarity improving the interactions between educators, learners, and the educational material. We build on their framework by adding parents as key agents that mediate the relationships between learners and educators and the material (Figure 1).

Figure 1: The instructional core

Adapted from Cohen and Ball (1999)

As the figure above suggests, ed-tech interventions can affect the instructional core in a myriad of ways. Yet, just because technology can do something, it does not mean it should. School systems in developing countries differ along many dimensions and each system is likely to have different needs for ed-tech interventions, as well as different infrastructure and capacity to enact such interventions.

The diagnosis:

How can school systems assess their needs and preparedness.

A useful first step for any school system to determine whether it should invest in education technology is to diagnose its:

• Specific needs to improve student learning (e.g., raising the average level of achievement, remediating gaps among low performers, and challenging high performers to develop higher-order skills);
• Infrastructure to adopt technology-enabled solutions (e.g., electricity connection, availability of space and outlets, stock of computers, and Internet connectivity at school and at learners’ homes); and
• Capacity to integrate technology in the instructional process (e.g., learners’ and educators’ level of familiarity and comfort with hardware and software, their beliefs about the level of usefulness of technology for learning purposes, and their current uses of such technology).

Before engaging in any new data collection exercise, school systems should take full advantage of existing administrative data that could shed light on these three main questions. This could be in the form of internal evaluations but also international learner assessments, such as the Program for International Student Assessment (PISA), the Trends in International Mathematics and Science Study (TIMSS), and/or the Progress in International Literacy Study (PIRLS), and the Teaching and Learning International Study (TALIS). But if school systems lack information on their preparedness for ed-tech reforms or if they seek to complement existing data with a richer set of indicators, we developed a set of surveys for learners, educators, and school leaders. Download the full report to see how we map out the main aspects covered by these surveys, in hopes of highlighting how they could be used to inform decisions around the adoption of ed-tech interventions.

The evidence:

How can school systems identify promising ed-tech interventions.

There is no single “ed-tech” initiative that will achieve the same results everywhere, simply because school systems differ in learners and educators, as well as in the availability and quality of materials and technologies. Instead, to realize the potential of education technology to accelerate student learning, decisionmakers should focus on four potential uses of technology that play to its comparative advantages and complement the work of educators to accelerate student learning (Figure 2). These comparative advantages include:

• Scaling up quality instruction, such as through prerecorded quality lessons.
• Facilitating differentiated instruction, through, for example, computer-adaptive learning and live one-on-one tutoring.
• Expanding opportunities to practice.
• Increasing learner engagement through videos and games.

Figure 2: Comparative advantages of technology

Here we review the evidence on ed-tech interventions from 37 studies in 20 countries*, organizing them by comparative advantage. It’s important to note that ours is not the only way to classify these interventions (e.g., video tutorials could be considered as a strategy to scale up instruction or increase learner engagement), but we believe it may be useful to highlight the needs that they could address and why technology is well positioned to do so.

When discussing specific studies, we report the magnitude of the effects of interventions using standard deviations (SDs). SDs are a widely used metric in research to express the effect of a program or policy with respect to a business-as-usual condition (e.g., test scores). There are several ways to make sense of them. One is to categorize the magnitude of the effects based on the results of impact evaluations. In developing countries, effects below 0.1 SDs are considered to be small, effects between 0.1 and 0.2 SDs are medium, and those above 0.2 SDs are large (for reviews that estimate the average effect of groups of interventions, called “meta analyses,” see e.g., Conn, 2017; Kremer, Brannen, & Glennerster, 2013; McEwan, 2014; Snilstveit et al., 2015; Evans & Yuan, 2020.)

*In surveying the evidence, we began by compiling studies from prior general and ed-tech specific evidence reviews that some of us have written and from ed-tech reviews conducted by others. Then, we tracked the studies cited by the ones we had previously read and reviewed those, as well. In identifying studies for inclusion, we focused on experimental and quasi-experimental evaluations of education technology interventions from pre-school to secondary school in low- and middle-income countries that were released between 2000 and 2020. We only included interventions that sought to improve student learning directly (i.e., students’ interaction with the material), as opposed to interventions that have impacted achievement indirectly, by reducing teacher absence or increasing parental engagement. This process yielded 37 studies in 20 countries (see the full list of studies in Appendix B).

Scaling up standardized instruction

One of the ways in which technology may improve the quality of education is through its capacity to deliver standardized quality content at scale. This feature of technology may be particularly useful in three types of settings: (a) those in “hard-to-staff” schools (i.e., schools that struggle to recruit educators with the requisite training and experience—typically, in rural and/or remote areas) (see, e.g., Urquiola & Vegas, 2005); (b) those in which many educators are frequently absent from school (e.g., Chaudhury, Hammer, Kremer, Muralidharan, & Rogers, 2006; Muralidharan, Das, Holla, & Mohpal, 2017); and/or (c) those in which educators have low levels of pedagogical and subject matter expertise (e.g., Bietenbeck, Piopiunik, & Wiederhold, 2018; Bold et al., 2017; Metzler & Woessmann, 2012; Santibañez, 2006) and do not have opportunities to observe and receive feedback (e.g., Bruns, Costa, & Cunha, 2018; Cilliers, Fleisch, Prinsloo, & Taylor, 2018). Technology could address this problem by: (a) disseminating lessons delivered by qualified educators to a large number of learners (e.g., through prerecorded or live lessons); (b) enabling distance education (e.g., for learners in remote areas and/or during periods of school closures); and (c) distributing hardware preloaded with educational materials.

Prerecorded lessons

Technology seems to be well placed to amplify the impact of effective educators by disseminating their lessons. Evidence on the impact of prerecorded lessons is encouraging, but not conclusive. Some initiatives that have used short instructional videos to complement regular instruction, in conjunction with other learning materials, have raised student learning on independent assessments. For example, Beg et al. (2020) evaluated an initiative in Punjab, Pakistan in which grade 8 classrooms received an intervention that included short videos to substitute live instruction, quizzes for learners to practice the material from every lesson, tablets for educators to learn the material and follow the lesson, and LED screens to project the videos onto a classroom screen. After six months, the intervention improved the performance of learners on independent tests of math and science by 0.19 and 0.24 SDs, respectively but had no discernible effect on the math and science section of Punjab’s high-stakes exams.

One study suggests that approaches that are far less technologically sophisticated can also improve learning outcomes—especially, if the business-as-usual instruction is of low quality. For example, Naslund-Hadley, Parker, and Hernandez-Agramonte (2014) evaluated a preschool math program in Cordillera, Paraguay that used audio segments and written materials four days per week for an hour per day during the school day. After five months, the intervention improved math scores by 0.16 SDs, narrowing gaps between low- and high-achieving learners, and between those with and without educators with formal training in early childhood education.

Yet, the integration of prerecorded material into regular instruction has not always been successful. For example, de Barros (2020) evaluated an intervention that combined instructional videos for math and science with infrastructure upgrades (e.g., two “smart” classrooms, two TVs, and two tablets), printed workbooks for students, and in-service training for educators of learners in grades 9 and 10 in Haryana, India (all materials were mapped onto the official curriculum). After 11 months, the intervention negatively impacted math achievement (by 0.08 SDs) and had no effect on science (with respect to business as usual classes). It reduced the share of lesson time that educators devoted to instruction and negatively impacted an index of instructional quality. Likewise, Seo (2017) evaluated several combinations of infrastructure (solar lights and TVs) and prerecorded videos (in English and/or bilingual) for grade 11 students in northern Tanzania and found that none of the variants improved student learning, even when the videos were used. The study reports effects from the infrastructure component across variants, but as others have noted (Muralidharan, Romero, & Wüthrich, 2019), this approach to estimating impact is problematic.

A very similar intervention delivered after school hours, however, had sizeable effects on learners’ basic skills. Chiplunkar, Dhar, and Nagesh (2020) evaluated an initiative in Chennai (the capital city of the state of Tamil Nadu, India) delivered by the same organization as above that combined short videos that explained key concepts in math and science with worksheets, facilitator-led instruction, small groups for peer-to-peer learning, and occasional career counseling and guidance for grade 9 students. These lessons took place after school for one hour, five times a week. After 10 months, it had large effects on learners’ achievement as measured by tests of basic skills in math and reading, but no effect on a standardized high-stakes test in grade 10 or socio-emotional skills (e.g., teamwork, decisionmaking, and communication).

Drawing general lessons from this body of research is challenging for at least two reasons. First, all of the studies above have evaluated the impact of prerecorded lessons combined with several other components (e.g., hardware, print materials, or other activities). Therefore, it is possible that the effects found are due to these additional components, rather than to the recordings themselves, or to the interaction between the two (see Muralidharan, 2017 for a discussion of the challenges of interpreting “bundled” interventions). Second, while these studies evaluate some type of prerecorded lessons, none examines the content of such lessons. Thus, it seems entirely plausible that the direction and magnitude of the effects depends largely on the quality of the recordings (e.g., the expertise of the educator recording it, the amount of preparation that went into planning the recording, and its alignment with best teaching practices).

These studies also raise three important questions worth exploring in future research. One of them is why none of the interventions discussed above had effects on high-stakes exams, even if their materials are typically mapped onto the official curriculum. It is possible that the official curricula are simply too challenging for learners in these settings, who are several grade levels behind expectations and who often need to reinforce basic skills (see Pritchett & Beatty, 2015). Another question is whether these interventions have long-term effects on teaching practices. It seems plausible that, if these interventions are deployed in contexts with low teaching quality, educators may learn something from watching the videos or listening to the recordings with learners. Yet another question is whether these interventions make it easier for schools to deliver instruction to learners whose native language is other than the official medium of instruction.

Distance education

Technology can also allow learners living in remote areas to access education. The evidence on these initiatives is encouraging. For example, Johnston and Ksoll (2017) evaluated a program that broadcasted live instruction via satellite to rural primary school students in the Volta and Greater Accra regions of Ghana. For this purpose, the program also equipped classrooms with the technology needed to connect to a studio in Accra, including solar panels, a satellite modem, a projector, a webcam, microphones, and a computer with interactive software. After two years, the intervention improved the numeracy scores of students in grades 2 through 4, and some foundational literacy tasks, but it had no effect on attendance or classroom time devoted to instruction, as captured by school visits. The authors interpreted these results as suggesting that the gains in achievement may be due to improving the quality of instruction that children received (as opposed to increased instructional time). Naik, Chitre, Bhalla, and Rajan (2019) evaluated a similar program in the Indian state of Karnataka and also found positive effects on learning outcomes, but it is not clear whether those effects are due to the program or due to differences in the groups of students they compared to estimate the impact of the initiative.

In one context (Mexico), this type of distance education had positive long-term effects. Navarro-Sola (2019) took advantage of the staggered rollout of the telesecundarias (i.e., middle schools with lessons broadcasted through satellite TV) in 1968 to estimate its impact. The policy had short-term effects on students’ enrollment in school: For every telesecundaria per 50 children, 10 students enrolled in middle school and two pursued further education. It also had a long-term influence on the educational and employment trajectory of its graduates. Each additional year of education induced by the policy increased average income by nearly 18 percent. This effect was attributable to more graduates entering the labor force and shifting from agriculture and the informal sector. Similarly, Fabregas (2019) leveraged a later expansion of this policy in 1993 and found that each additional telesecundaria per 1,000 adolescents led to an average increase of 0.2 years of education, and a decline in fertility for women, but no conclusive evidence of long-term effects on labor market outcomes.

It is crucial to interpret these results keeping in mind the settings where the interventions were implemented. As we mention above, part of the reason why they have proven effective is that the “counterfactual” conditions for learning (i.e., what would have happened to learners in the absence of such programs) was either to not have access to schooling or to be exposed to low-quality instruction. School systems interested in taking up similar interventions should assess the extent to which their learners (or parts of their learner population) find themselves in similar conditions to the subjects of the studies above. This illustrates the importance of assessing the needs of a system before reviewing the evidence.

Preloaded hardware

Technology also seems well positioned to disseminate educational materials. Specifically, hardware (e.g., desktop computers, laptops, or tablets) could also help deliver educational software (e.g., word processing, reference texts, and/or games). In theory, these materials could not only undergo a quality assurance review (e.g., by curriculum specialists and educators), but also draw on the interactions with learners for adjustments (e.g., identifying areas needing reinforcement) and enable interactions between learners and educators.

In practice, however, most initiatives that have provided learners with free computers, laptops, and netbooks do not leverage any of the opportunities mentioned above. Instead, they install a standard set of educational materials and hope that learners find them helpful enough to take them up on their own. Students rarely do so, and instead use the laptops for recreational purposes—often, to the detriment of their learning (see, e.g., Malamud & Pop-Eleches, 2011). In fact, free netbook initiatives have not only consistently failed to improve academic achievement in math or language (e.g., Cristia et al., 2017), but they have had no impact on learners’ general computer skills (e.g., Beuermann et al., 2015). Some of these initiatives have had small impacts on cognitive skills, but the mechanisms through which those effects occurred remains unclear.

To our knowledge, the only successful deployment of a free laptop initiative was one in which a team of researchers equipped the computers with remedial software. Mo et al. (2013) evaluated a version of the One Laptop per Child (OLPC) program for grade 3 students in migrant schools in Beijing, China in which the laptops were loaded with a remedial software mapped onto the national curriculum for math (similar to the software products that we discuss under “practice exercises” below). After nine months, the program improved math achievement by 0.17 SDs and computer skills by 0.33 SDs. If a school system decides to invest in free laptops, this study suggests that the quality of the software on the laptops is crucial.

To date, however, the evidence suggests that children do not learn more from interacting with laptops than they do from textbooks. For example, Bando, Gallego, Gertler, and Romero (2016) compared the effect of free laptop and textbook provision in 271 elementary schools in disadvantaged areas of Honduras. After seven months, students in grades 3 and 6 who had received the laptops performed on par with those who had received the textbooks in math and language. Further, even if textbooks essentially become obsolete at the end of each school year, whereas laptops can be reloaded with new materials for each year, the costs of laptop provision (not just the hardware, but also the technical assistance, Internet, and training associated with it) are not yet low enough to make them a more cost-effective way of delivering content to learners.

Evidence on the provision of tablets equipped with software is encouraging but limited. For example, de Hoop et al. (2020) evaluated a composite intervention for first grade students in Zambia’s Eastern Province that combined infrastructure (electricity via solar power), hardware (projectors and tablets), and educational materials (lesson plans for educators and interactive lessons for learners, both loaded onto the tablets and mapped onto the official Zambian curriculum). After 14 months, the intervention had improved student early-grade reading by 0.4 SDs, oral vocabulary scores by 0.25 SDs, and early-grade math by 0.22 SDs. It also improved students’ achievement by 0.16 on a locally developed assessment. The multifaceted nature of the program, however, makes it challenging to identify the components that are driving the positive effects. Pitchford (2015) evaluated an intervention that provided tablets equipped with educational “apps,” to be used for 30 minutes per day for two months to develop early math skills among students in grades 1 through 3 in Lilongwe, Malawi. The evaluation found positive impacts in math achievement, but the main study limitation is that it was conducted in a single school.

Facilitating differentiated instruction

Another way in which technology may improve educational outcomes is by facilitating the delivery of differentiated or individualized instruction. Most developing countries massively expanded access to schooling in recent decades by building new schools and making education more affordable, both by defraying direct costs, as well as compensating for opportunity costs (Duflo, 2001; World Bank, 2018). These initiatives have not only rapidly increased the number of learners enrolled in school, but have also increased the variability in learner’ preparation for schooling. Consequently, a large number of learners perform well below grade-based curricular expectations (see, e.g., Duflo, Dupas, & Kremer, 2011; Pritchett & Beatty, 2015). These learners are unlikely to get much from “one-size-fits-all” instruction, in which a single educator delivers instruction deemed appropriate for the middle (or top) of the achievement distribution (Banerjee & Duflo, 2011). Technology could potentially help these learners by providing them with: (a) instruction and opportunities for practice that adjust to the level and pace of preparation of each individual (known as “computer-adaptive learning” (CAL)); or (b) live, one-on-one tutoring.

Computer-adaptive learning

One of the main comparative advantages of technology is its ability to diagnose students’ initial learning levels and assign students to instruction and exercises of appropriate difficulty. No individual educator—no matter how talented—can be expected to provide individualized instruction to all learners in his/her class simultaneously . In this respect, technology is uniquely positioned to complement traditional teaching. This use of technology could help learners master basic skills and help them get more out of schooling.

Although many software products evaluated in recent years have been categorized as CAL, many rely on a relatively coarse level of differentiation at an initial stage (e.g., a diagnostic test) without further differentiation. We discuss these initiatives under the category of “increasing opportunities for practice” below. CAL initiatives complement an initial diagnostic with dynamic adaptation (i.e., at each response or set of responses from learners) to adjust both the initial level of difficulty and rate at which it increases or decreases, depending on whether learners’ responses are correct or incorrect.

Existing evidence on this specific type of programs is highly promising. Most famously, Banerjee et al. (2007) evaluated CAL software in Vadodara, in the Indian state of Gujarat, in which grade 4 students were offered two hours of shared computer time per week before and after school, during which they played games that involved solving math problems. The level of difficulty of such problems adjusted based on students’ answers. This program improved math achievement by 0.35 and 0.47 SDs after one and two years of implementation, respectively. Consistent with the promise of personalized learning, the software improved achievement for all students. In fact, one year after the end of the program, students assigned to the program still performed 0.1 SDs better than those assigned to a business as usual condition. More recently, Muralidharan, et al. (2019) evaluated a “blended learning” initiative in which students in grades 4 through 9 in Delhi, India received 45 minutes of interaction with CAL software for math and language, and 45 minutes of small group instruction before or after going to school. After only 4.5 months, the program improved achievement by 0.37 SDs in math and 0.23 SDs in Hindi. While all learners benefited from the program in absolute terms, the lowest performing learners benefited the most in relative terms, since they were learning very little in school.

We see two important limitations from this body of research. First, to our knowledge, none of these initiatives has been evaluated when implemented during the school day. Therefore, it is not possible to distinguish the effect of the adaptive software from that of additional instructional time. Second, given that most of these programs were facilitated by local instructors, attempts to distinguish the effect of the software from that of the instructors has been mostly based on noncausal evidence. A frontier challenge in this body of research is to understand whether CAL software can increase the effectiveness of school-based instruction by substituting part of the regularly scheduled time for math and language instruction.

Live one-on-one tutoring

Recent improvements in the speed and quality of videoconferencing, as well as in the connectivity of remote areas, have enabled yet another way in which technology can help personalization: live (i.e., real-time) one-on-one tutoring. While the evidence on in-person tutoring is scarce in developing countries, existing studies suggest that this approach works best when it is used to personalize instruction (see, e.g., Banerjee et al., 2007; Banerji, Berry, & Shotland, 2015; Cabezas, Cuesta, & Gallego, 2011).

There are almost no studies on the impact of online tutoring—possibly, due to the lack of hardware and Internet connectivity in low- and middle-income countries. One exception is Chemin and Oledan (2020)’s recent evaluation of an online tutoring program for grade 6 students in Kianyaga, Kenya to learn English from volunteers from a Canadian university via Skype ( videoconferencing software) for one hour per week after school. After 10 months, program beneficiaries performed 0.22 SDs better in a test of oral comprehension, improved their comfort using technology for learning, and became more willing to engage in cross-cultural communication. Importantly, while the tutoring sessions used the official English textbooks and sought in part to help learners with their homework, tutors were trained on several strategies to teach to each learner’s individual level of preparation, focusing on basic skills if necessary. To our knowledge, similar initiatives within a country have not yet been rigorously evaluated.

Expanding opportunities for practice

A third way in which technology may improve the quality of education is by providing learners with additional opportunities for practice. In many developing countries, lesson time is primarily devoted to lectures, in which the educator explains the topic and the learners passively copy explanations from the blackboard. This setup leaves little time for in-class practice. Consequently, learners who did not understand the explanation of the material during lecture struggle when they have to solve homework assignments on their own. Technology could potentially address this problem by allowing learners to review topics at their own pace.

Practice exercises

Technology can help learners get more out of traditional instruction by providing them with opportunities to implement what they learn in class. This approach could, in theory, allow some learners to anchor their understanding of the material through trial and error (i.e., by realizing what they may not have understood correctly during lecture and by getting better acquainted with special cases not covered in-depth in class).

Existing evidence on practice exercises reflects both the promise and the limitations of this use of technology in developing countries. For example, Lai et al. (2013) evaluated a program in Shaanxi, China where students in grades 3 and 5 were required to attend two 40-minute remedial sessions per week in which they first watched videos that reviewed the material that had been introduced in their math lessons that week and then played games to practice the skills introduced in the video. After four months, the intervention improved math achievement by 0.12 SDs. Many other evaluations of comparable interventions have found similar small-to-moderate results (see, e.g., Lai, Luo, Zhang, Huang, & Rozelle, 2015; Lai et al., 2012; Mo et al., 2015; Pitchford, 2015). These effects, however, have been consistently smaller than those of initiatives that adjust the difficulty of the material based on students’ performance (e.g., Banerjee et al., 2007; Muralidharan, et al., 2019). We hypothesize that these programs do little for learners who perform several grade levels behind curricular expectations, and who would benefit more from a review of foundational concepts from earlier grades.

We see two important limitations from this research. First, most initiatives that have been evaluated thus far combine instructional videos with practice exercises, so it is hard to know whether their effects are driven by the former or the latter. In fact, the program in China described above allowed learners to ask their peers whenever they did not understand a difficult concept, so it potentially also captured the effect of peer-to-peer collaboration. To our knowledge, no studies have addressed this gap in the evidence.

Second, most of these programs are implemented before or after school, so we cannot distinguish the effect of additional instructional time from that of the actual opportunity for practice. The importance of this question was first highlighted by Linden (2008), who compared two delivery mechanisms for game-based remedial math software for students in grades 2 and 3 in a network of schools run by a nonprofit organization in Gujarat, India: one in which students interacted with the software during the school day and another one in which students interacted with the software before or after school (in both cases, for three hours per day). After a year, the first version of the program had negatively impacted students’ math achievement by 0.57 SDs and the second one had a null effect. This study suggested that computer-assisted learning is a poor substitute for regular instruction when it is of high quality, as was the case in this well-functioning private network of schools.

In recent years, several studies have sought to remedy this shortcoming. Mo et al. (2014) were among the first to evaluate practice exercises delivered during the school day. They evaluated an initiative in Shaanxi, China in which students in grades 3 and 5 were required to interact with the software similar to the one in Lai et al. (2013) for two 40-minute sessions per week. The main limitation of this study, however, is that the program was delivered during regularly scheduled computer lessons, so it could not determine the impact of substituting regular math instruction. Similarly, Mo et al. (2020) evaluated a self-paced and a teacher-directed version of a similar program for English for grade 5 students in Qinghai, China. Yet, the key shortcoming of this study is that the teacher-directed version added several components that may also influence achievement, such as increased opportunities for teachers to provide students with personalized assistance when they struggled with the material. Ma, Fairlie, Loyalka, and Rozelle (2020) compared the effectiveness of additional time-delivered remedial instruction for students in grades 4 to 6 in Shaanxi, China through either computer-assisted software or using workbooks. This study indicates whether additional instructional time is more effective when using technology, but it does not address the question of whether school systems may improve the productivity of instructional time during the school day by substituting educator-led with computer-assisted instruction.

Increasing learner engagement

Another way in which technology may improve education is by increasing learners’ engagement with the material. In many school systems, regular “chalk and talk” instruction prioritizes time for educators’ exposition over opportunities for learners to ask clarifying questions and/or contribute to class discussions. This, combined with the fact that many developing-country classrooms include a very large number of learners (see, e.g., Angrist & Lavy, 1999; Duflo, Dupas, & Kremer, 2015), may partially explain why the majority of those students are several grade levels behind curricular expectations (e.g., Muralidharan, et al., 2019; Muralidharan & Zieleniak, 2014; Pritchett & Beatty, 2015). Technology could potentially address these challenges by: (a) using video tutorials for self-paced learning and (b) presenting exercises as games and/or gamifying practice.

Video tutorials

Technology can potentially increase learner effort and understanding of the material by finding new and more engaging ways to deliver it. Video tutorials designed for self-paced learning—as opposed to videos for whole class instruction, which we discuss under the category of “prerecorded lessons” above—can increase learner effort in multiple ways, including: allowing learners to focus on topics with which they need more help, letting them correct errors and misconceptions on their own, and making the material appealing through visual aids. They can increase understanding by breaking the material into smaller units and tackling common misconceptions.

In spite of the popularity of instructional videos, there is relatively little evidence on their effectiveness. Yet, two recent evaluations of different versions of the Khan Academy portal, which mainly relies on instructional videos, offer some insight into their impact. First, Ferman, Finamor, and Lima (2019) evaluated an initiative in 157 public primary and middle schools in five cities in Brazil in which the teachers of students in grades 5 and 9 were taken to the computer lab to learn math from the platform for 50 minutes per week. The authors found that, while the intervention slightly improved learners’ attitudes toward math, these changes did not translate into better performance in this subject. The authors hypothesized that this could be due to the reduction of teacher-led math instruction.

More recently, Büchel, Jakob, Kühnhanss, Steffen, and Brunetti (2020) evaluated an after-school, offline delivery of the Khan Academy portal in grades 3 through 6 in 302 primary schools in Morazán, El Salvador. Students in this study received 90 minutes per week of additional math instruction (effectively nearly doubling total math instruction per week) through teacher-led regular lessons, teacher-assisted Khan Academy lessons, or similar lessons assisted by technical supervisors with no content expertise. (Importantly, the first group provided differentiated instruction, which is not the norm in Salvadorian schools). All three groups outperformed both schools without any additional lessons and classrooms without additional lessons in the same schools as the program. The teacher-assisted Khan Academy lessons performed 0.24 SDs better, the supervisor-led lessons 0.22 SDs better, and the teacher-led regular lessons 0.15 SDs better, but the authors could not determine whether the effects across versions were different.

Together, these studies suggest that instructional videos work best when provided as a complement to, rather than as a substitute for, regular instruction. Yet, the main limitation of these studies is the multifaceted nature of the Khan Academy portal, which also includes other components found to positively improve learner achievement, such as differentiated instruction by students’ learning levels. While the software does not provide the type of personalization discussed above, learners are asked to take a placement test and, based on their score, educators assign them different work. Therefore, it is not clear from these studies whether the effects from Khan Academy are driven by its instructional videos or to the software’s ability to provide differentiated activities when combined with placement tests.

Games and gamification

Technology can also increase learner engagement by presenting exercises as games and/or by encouraging learner to play and compete with others (e.g., using leaderboards and rewards)—an approach known as “gamification.” Both approaches can increase learner motivation and effort by presenting learners with entertaining opportunities for practice and by leveraging peers as commitment devices.

There are very few studies on the effects of games and gamification in low- and middle-income countries. Recently, Araya, Arias Ortiz, Bottan, and Cristia (2019) evaluated an initiative in which grade 4 students in Santiago, Chile were required to participate in two 90-minute sessions per week during the school day with instructional math software featuring individual and group competitions (e.g., tracking each learner’s standing in his/her class and tournaments between sections). After nine months, the program led to improvements of 0.27 SDs in the national student assessment in math (it had no spillover effects on reading). However, it had mixed effects on non-academic outcomes. Specifically, the program increased learners’ willingness to use computers to learn math, but, at the same time, increased their anxiety toward math and negatively impacted learners’ willingness to collaborate with peers. Finally, given that one of the weekly sessions replaced regular math instruction and the other one represented additional math instructional time, it is not clear whether the academic effects of the program are driven by the software or the additional time devoted to learning math.

The prognosis:

How can school systems adopt interventions that match their needs.

Here are five specific and sequential guidelines for decisionmakers to realize the potential of education technology to accelerate student learning.

1. Take stock of how your current schools, educators, and learners are engaging with technology .

Carry out a short in-school survey to understand the current practices and potential barriers to adoption of technology (we have included suggested survey instruments in the Appendices); use this information in your decisionmaking process. For example, we learned from conversations with current and former ministers of education from various developing regions that a common limitation to technology use is regulations that hold school leaders accountable for damages to or losses of devices. Another common barrier is lack of access to electricity and Internet, or even the availability of sufficient outlets for charging devices in classrooms. Understanding basic infrastructure and regulatory limitations to the use of education technology is a first necessary step. But addressing these limitations will not guarantee that introducing or expanding technology use will accelerate learning. The next steps are thus necessary.

“In Africa, the biggest limit is connectivity. Fiber is expensive, and we don’t have it everywhere. The continent is creating a digital divide between cities, where there is fiber, and the rural areas.  The [Ghanaian] administration put in schools offline/online technologies with books, assessment tools, and open source materials. In deploying this, we are finding that again, teachers are unfamiliar with it. And existing policies prohibit students to bring their own tablets or cell phones. The easiest way to do it would have been to let everyone bring their own device. But policies are against it.” H.E. Matthew Prempeh, Minister of Education of Ghana, on the need to understand the local context.

2. Consider how the introduction of technology may affect the interactions among learners, educators, and content .

Our review of the evidence indicates that technology may accelerate student learning when it is used to scale up access to quality content, facilitate differentiated instruction, increase opportunities for practice, or when it increases learner engagement. For example, will adding electronic whiteboards to classrooms facilitate access to more quality content or differentiated instruction? Or will these expensive boards be used in the same way as the old chalkboards? Will providing one device (laptop or tablet) to each learner facilitate access to more and better content, or offer students more opportunities to practice and learn? Solely introducing technology in classrooms without additional changes is unlikely to lead to improved learning and may be quite costly. If you cannot clearly identify how the interactions among the three key components of the instructional core (educators, learners, and content) may change after the introduction of technology, then it is probably not a good idea to make the investment. See Appendix A for guidance on the types of questions to ask.

3. Once decisionmakers have a clear idea of how education technology can help accelerate student learning in a specific context, it is important to define clear objectives and goals and establish ways to regularly assess progress and make course corrections in a timely manner .

For instance, is the education technology expected to ensure that learners in early grades excel in foundational skills—basic literacy and numeracy—by age 10? If so, will the technology provide quality reading and math materials, ample opportunities to practice, and engaging materials such as videos or games? Will educators be empowered to use these materials in new ways? And how will progress be measured and adjusted?

4. How this kind of reform is approached can matter immensely for its success.

It is easy to nod to issues of “implementation,” but that needs to be more than rhetorical. Keep in mind that good use of education technology requires thinking about how it will affect learners, educators, and parents. After all, giving learners digital devices will make no difference if they get broken, are stolen, or go unused. Classroom technologies only matter if educators feel comfortable putting them to work. Since good technology is generally about complementing or amplifying what educators and learners already do, it is almost always a mistake to mandate programs from on high. It is vital that technology be adopted with the input of educators and families and with attention to how it will be used. If technology goes unused or if educators use it ineffectually, the results will disappoint—no matter the virtuosity of the technology. Indeed, unused education technology can be an unnecessary expenditure for cash-strapped education systems. This is why surveying context, listening to voices in the field, examining how technology is used, and planning for course correction is essential.

5. It is essential to communicate with a range of stakeholders, including educators, school leaders, parents, and learners .

Technology can feel alien in schools, confuse parents and (especially) older educators, or become an alluring distraction. Good communication can help address all of these risks. Taking care to listen to educators and families can help ensure that programs are informed by their needs and concerns. At the same time, deliberately and consistently explaining what technology is and is not supposed to do, how it can be most effectively used, and the ways in which it can make it more likely that programs work as intended. For instance, if teachers fear that technology is intended to reduce the need for educators, they will tend to be hostile; if they believe that it is intended to assist them in their work, they will be more receptive. Absent effective communication, it is easy for programs to “fail” not because of the technology but because of how it was used. In short, past experience in rolling out education programs indicates that it is as important to have a strong intervention design as it is to have a solid plan to socialize it among stakeholders.

Beyond reopening: A leapfrog moment to transform education?

On September 14, the Center for Universal Education (CUE) will host a webinar to discuss strategies, including around the effective use of education technology, for ensuring resilient schools in the long term and to launch a new education technology playbook “Realizing the promise: How can education technology improve learning for all?”

file-pdf Full Playbook – Realizing the promise: How can education technology improve learning for all? file-pdf References file-pdf Appendix A – Instruments to assess availability and use of technology file-pdf Appendix B – List of reviewed studies file-pdf Appendix C – How may technology affect interactions among students, teachers, and content?

About the Authors

Alejandro j. ganimian, emiliana vegas, frederick m. hess.

• Media Relations
• Terms and Conditions
• Privacy Policy

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• My Account Login
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Review Article
• Open access
• Published: 12 February 2024

Education reform and change driven by digital technology: a bibliometric study from a global perspective

• Chengliang Wang 1 ,
• Xiaojiao Chen 1 ,
• Teng Yu   ORCID: orcid.org/0000-0001-5198-7261 2 , 3 ,
• Yidan Liu 1 , 4 &
• Yuhui Jing 1  

Humanities and Social Sciences Communications volume  11 , Article number:  256 ( 2024 ) Cite this article

6986 Accesses

1 Citations

1 Altmetric

Metrics details

• Development studies
• Science, technology and society

Amidst the global digital transformation of educational institutions, digital technology has emerged as a significant area of interest among scholars. Such technologies have played an instrumental role in enhancing learner performance and improving the effectiveness of teaching and learning. These digital technologies also ensure the sustainability and stability of education during the epidemic. Despite this, a dearth of systematic reviews exists regarding the current state of digital technology application in education. To address this gap, this study utilized the Web of Science Core Collection as a data source (specifically selecting the high-quality SSCI and SCIE) and implemented a topic search by setting keywords, yielding 1849 initial publications. Furthermore, following the PRISMA guidelines, we refined the selection to 588 high-quality articles. Using software tools such as CiteSpace, VOSviewer, and Charticulator, we reviewed these 588 publications to identify core authors (such as Selwyn, Henderson, Edwards), highly productive countries/regions (England, Australia, USA), key institutions (Monash University, Australian Catholic University), and crucial journals in the field ( Education and Information Technologies , Computers & Education , British Journal of Educational Technology ). Evolutionary analysis reveals four developmental periods in the research field of digital technology education application: the embryonic period, the preliminary development period, the key exploration, and the acceleration period of change. The study highlights the dual influence of technological factors and historical context on the research topic. Technology is a key factor in enabling education to transform and upgrade, and the context of the times is an important driving force in promoting the adoption of new technologies in the education system and the transformation and upgrading of education. Additionally, the study identifies three frontier hotspots in the field: physical education, digital transformation, and professional development under the promotion of digital technology. This study presents a clear framework for digital technology application in education, which can serve as a valuable reference for researchers and educational practitioners concerned with digital technology education application in theory and practice.

Similar content being viewed by others

A bibliometric analysis of knowledge mapping in Chinese education digitalization research from 2012 to 2022

Digital transformation and digital literacy in the context of complexity within higher education institutions: a systematic literature review

Education big data and learning analytics: a bibliometric analysis

Introduction.

Digital technology has become an essential component of modern education, facilitating the extension of temporal and spatial boundaries and enriching the pedagogical contexts (Selwyn and Facer, 2014 ). The advent of mobile communication technology has enabled learning through social media platforms (Szeto et al. 2015 ; Pires et al. 2022 ), while the advancement of augmented reality technology has disrupted traditional conceptions of learning environments and spaces (Perez-Sanagustin et al., 2014 ; Kyza and Georgiou, 2018 ). A wide range of digital technologies has enabled learning to become a norm in various settings, including the workplace (Sjöberg and Holmgren, 2021 ), home (Nazare et al. 2022 ), and online communities (Tang and Lam, 2014 ). Education is no longer limited to fixed locations and schedules, but has permeated all aspects of life, allowing learning to continue at any time and any place (Camilleri and Camilleri, 2016 ; Selwyn and Facer, 2014 ).

The advent of digital technology has led to the creation of several informal learning environments (Greenhow and Lewin, 2015 ) that exhibit divergent form, function, features, and patterns in comparison to conventional learning environments (Nygren et al. 2019 ). Consequently, the associated teaching and learning processes, as well as the strategies for the creation, dissemination, and acquisition of learning resources, have undergone a complete overhaul. The ensuing transformations have posed a myriad of novel issues, such as the optimal structuring of teaching methods by instructors and the adoption of appropriate learning strategies by students in the new digital technology environment. Consequently, an examination of the principles that underpin effective teaching and learning in this environment is a topic of significant interest to numerous scholars engaged in digital technology education research.

Over the course of the last two decades, digital technology has made significant strides in the field of education, notably in extending education time and space and creating novel educational contexts with sustainability. Despite research attempts to consolidate the application of digital technology in education, previous studies have only focused on specific aspects of digital technology, such as Pinto and Leite’s ( 2020 ) investigation into digital technology in higher education and Mustapha et al.’s ( 2021 ) examination of the role and value of digital technology in education during the pandemic. While these studies have provided valuable insights into the practical applications of digital technology in particular educational domains, they have not comprehensively explored the macro-mechanisms and internal logic of digital technology implementation in education. Additionally, these studies were conducted over a relatively brief period, making it challenging to gain a comprehensive understanding of the macro-dynamics and evolutionary process of digital technology in education. Some studies have provided an overview of digital education from an educational perspective but lack a precise understanding of technological advancement and change (Yang et al. 2022 ). Therefore, this study seeks to employ a systematic scientific approach to collate relevant research from 2000 to 2022, comprehend the internal logic and development trends of digital technology in education, and grasp the outstanding contribution of digital technology in promoting the sustainability of education in time and space. In summary, this study aims to address the following questions:

RQ1: Since the turn of the century, what is the productivity distribution of the field of digital technology education application research in terms of authorship, country/region, institutional and journal level?

RQ2: What is the development trend of research on the application of digital technology in education in the past two decades?

RQ3: What are the current frontiers of research on the application of digital technology in education?

Literature review

Although the term “digital technology” has become ubiquitous, a unified definition has yet to be agreed upon by scholars. Because the meaning of the word digital technology is closely related to the specific context. Within the educational research domain, Selwyn’s ( 2016 ) definition is widely favored by scholars (Pinto and Leite, 2020 ). Selwyn ( 2016 ) provides a comprehensive view of various concrete digital technologies and their applications in education through ten specific cases, such as immediate feedback in classes, orchestrating teaching, and community learning. Through these specific application scenarios, Selwyn ( 2016 ) argues that digital technology encompasses technologies associated with digital devices, including but not limited to tablets, smartphones, computers, and social media platforms (such as Facebook and YouTube). Furthermore, Further, the behavior of accessing the internet at any location through portable devices can be taken as an extension of the behavior of applying digital technology.

The evolving nature of digital technology has significant implications in the field of education. In the 1890s, the focus of digital technology in education was on comprehending the nuances of digital space, digital culture, and educational methodologies, with its connotations aligned more towards the idea of e-learning. The advent and subsequent widespread usage of mobile devices since the dawn of the new millennium have been instrumental in the rapid expansion of the concept of digital technology. Notably, mobile learning devices such as smartphones and tablets, along with social media platforms, have become integral components of digital technology (Conole and Alevizou, 2010 ; Batista et al. 2016 ). In recent times, the burgeoning application of AI technology in the education sector has played a vital role in enriching the digital technology lexicon (Banerjee et al. 2021 ). ChatGPT, for instance, is identified as a novel educational technology that has immense potential to revolutionize future education (Rospigliosi, 2023 ; Arif, Munaf and Ul-Haque, 2023 ).

Pinto and Leite ( 2020 ) conducted a comprehensive macroscopic survey of the use of digital technologies in the education sector and identified three distinct categories, namely technologies for assessment and feedback, mobile technologies, and Information Communication Technologies (ICT). This classification criterion is both macroscopic and highly condensed. In light of the established concept definitions of digital technology in the educational research literature, this study has adopted the characterizations of digital technology proposed by Selwyn ( 2016 ) and Pinto and Leite ( 2020 ) as crucial criteria for analysis and research inclusion. Specifically, this criterion encompasses several distinct types of digital technologies, including Information and Communication Technologies (ICT), Mobile tools, eXtended Reality (XR) Technologies, Assessment and Feedback systems, Learning Management Systems (LMS), Publish and Share tools, Collaborative systems, Social media, Interpersonal Communication tools, and Content Aggregation tools.

Methodology and materials

Research method: bibliometric.

The research on econometric properties has been present in various aspects of human production and life, yet systematic scientific theoretical guidance has been lacking, resulting in disorganization. In 1969, British scholar Pritchard ( 1969 ) proposed “bibliometrics,” which subsequently emerged as an independent discipline in scientific quantification research. Initially, Pritchard defined bibliometrics as “the application of mathematical and statistical methods to books and other media of communication,” however, the definition was not entirely rigorous. To remedy this, Hawkins ( 2001 ) expanded Pritchard’s definition to “the quantitative analysis of the bibliographic features of a body of literature.” De Bellis further clarified the objectives of bibliometrics, stating that it aims to analyze and identify patterns in literature, such as the most productive authors, institutions, countries, and journals in scientific disciplines, trends in literary production over time, and collaboration networks (De Bellis, 2009 ). According to Garfield ( 2006 ), bibliometric research enables the examination of the history and structure of a field, the flow of information within the field, the impact of journals, and the citation status of publications over a longer time scale. All of these definitions illustrate the unique role of bibliometrics as a research method for evaluating specific research fields.

This study uses CiteSpace, VOSviewer, and Charticulator to analyze data and create visualizations. Each of these three tools has its own strengths and can complement each other. CiteSpace and VOSviewer use set theory and probability theory to provide various visualization views in fields such as keywords, co-occurrence, and co-authors. They are easy to use and produce visually appealing graphics (Chen, 2006 ; van Eck and Waltman, 2009 ) and are currently the two most widely used bibliometric tools in the field of visualization (Pan et al. 2018 ). In this study, VOSviewer provided the data necessary for the Performance Analysis; Charticulator was then used to redraw using the tabular data exported from VOSviewer (for creating the chord diagram of country collaboration); this was to complement the mapping process, while CiteSpace was primarily utilized to generate keyword maps and conduct burst word analysis.

Data retrieval

This study selected documents from the Science Citation Index Expanded (SCIE) and Social Science Citation Index (SSCI) in the Web of Science Core Collection as the data source, for the following reasons:

(1) The Web of Science Core Collection, as a high-quality digital literature resource database, has been widely accepted by many researchers and is currently considered the most suitable database for bibliometric analysis (Jing et al. 2023a ). Compared to other databases, Web of Science provides more comprehensive data information (Chen et al. 2022a ), and also provides data formats suitable for analysis using VOSviewer and CiteSpace (Gaviria-Marin et al. 2019 ).

(2) The application of digital technology in the field of education is an interdisciplinary research topic, involving technical knowledge literature belonging to the natural sciences and education-related literature belonging to the social sciences. Therefore, it is necessary to select Science Citation Index Expanded (SCIE) and Social Science Citation Index (SSCI) as the sources of research data, ensuring the comprehensiveness of data while ensuring the reliability and persuasiveness of bibliometric research (Hwang and Tsai, 2011 ; Wang et al. 2022 ).

After establishing the source of research data, it is necessary to determine a retrieval strategy (Jing et al. 2023b ). The choice of a retrieval strategy should consider a balance between the breadth and precision of the search formula. That is to say, it should encompass all the literature pertaining to the research topic while excluding irrelevant documents as much as possible. In light of this, this study has set a retrieval strategy informed by multiple related papers (Mustapha et al. 2021 ; Luo et al. 2021 ). The research by Mustapha et al. ( 2021 ) guided us in selecting keywords (“digital” AND “technolog*”) to target digital technology, while Luo et al. ( 2021 ) informed the selection of terms (such as “instruct*,” “teach*,” and “education”) to establish links with the field of education. Then, based on the current application of digital technology in the educational domain and the scope of selection criteria, we constructed the final retrieval strategy. Following the general patterns of past research (Jing et al. 2023a , 2023b ), we conducted a specific screening using the topic search (Topics, TS) function in Web of Science. For the specific criteria used in the screening for this study, please refer to Table 1 .

Literature screening

Literature acquired through keyword searches may contain ostensibly related yet actually unrelated works. Therefore, to ensure the close relevance of literature included in the analysis to the research topic, it is often necessary to perform a manual screening process to identify the final literature to be analyzed, subsequent to completing the initial literature search.

The manual screening process consists of two steps. Initially, irrelevant literature is weeded out based on the title and abstract, with two members of the research team involved in this phase. This stage lasted about one week, resulting in 1106 articles being retained. Subsequently, a comprehensive review of the full text is conducted to accurately identify the literature required for the study. To carry out the second phase of manual screening effectively and scientifically, and to minimize the potential for researcher bias, the research team established the inclusion criteria presented in Table 2 . Three members were engaged in this phase, which took approximately 2 weeks, culminating in the retention of 588 articles after meticulous screening. The entire screening process is depicted in Fig. 1 , adhering to the PRISMA guidelines (Page et al. 2021 ).

The process of obtaining and filtering the necessary literature data for research.

Data standardization

Nguyen and Hallinger ( 2020 ) pointed out that raw data extracted from scientific databases often contains multiple expressions of the same term, and not addressing these synonymous expressions could affect research results in bibliometric analysis. For instance, in the original data, the author list may include “Tsai, C. C.” and “Tsai, C.-C.”, while the keyword list may include “professional-development” and “professional development,” which often require merging. Therefore, before analyzing the selected literature, a data disambiguation process is necessary to standardize the data (Strotmann and Zhao, 2012 ; Van Eck and Waltman, 2019 ). This study adopted the data standardization process proposed by Taskin and Al ( 2019 ), mainly including the following standardization operations:

Firstly, the author and source fields in the data are corrected and standardized to differentiate authors with similar names.

Secondly, the study checks whether the journals to which the literature belongs have been renamed in the past over 20 years, so as to avoid the influence of periodical name change on the analysis results.

Finally, the keyword field is standardized by unifying parts of speech and singular/plural forms of keywords, which can help eliminate redundant entries in the knowledge graph.

Performance analysis (RQ1)

This section offers a thorough and detailed analysis of the state of research in the field of digital technology education. By utilizing descriptive statistics and visual maps, it provides a comprehensive overview of the development trends, authors, countries, institutions, and journal distribution within the field. The insights presented in this section are of great significance in advancing our understanding of the current state of research in this field and identifying areas for further investigation. The use of visual aids to display inter-country cooperation and the evolution of the field adds to the clarity and coherence of the analysis.

Time trend of the publications

To understand a research field, it is first necessary to understand the most basic quantitative information, among which the change in the number of publications per year best reflects the development trend of a research field. Figure 2 shows the distribution of publication dates.

Time trend of the publications on application of digital technology in education.

From the Fig. 2 , it can be seen that the development of this field over the past over 20 years can be roughly divided into three stages. The first stage was from 2000 to 2007, during which the number of publications was relatively low. Due to various factors such as technological maturity, the academic community did not pay widespread attention to the role of digital technology in expanding the scope of teaching and learning. The second stage was from 2008 to 2019, during which the overall number of publications showed an upward trend, and the development of the field entered an accelerated period, attracting more and more scholars’ attention. The third stage was from 2020 to 2022, during which the number of publications stabilized at around 100. During this period, the impact of the pandemic led to a large number of scholars focusing on the role of digital technology in education during the pandemic, and research on the application of digital technology in education became a core topic in social science research.

Analysis of authors

An analysis of the author’s publication volume provides information about the representative scholars and core research strengths of a research area. Table 3 presents information on the core authors in adaptive learning research, including name, publication number, and average number of citations per article (based on the analysis and statistics from VOSviewer).

Variations in research foci among scholars abound. Within the field of digital technology education application research over the past two decades, Neil Selwyn stands as the most productive author, having published 15 papers garnering a total of 1027 citations, resulting in an average of 68.47 citations per paper. As a Professor at the Faculty of Education at Monash University, Selwyn concentrates on exploring the application of digital technology in higher education contexts (Selwyn et al. 2021 ), as well as related products in higher education such as Coursera, edX, and Udacity MOOC platforms (Bulfin et al. 2014 ). Selwyn’s contributions to the educational sociology perspective include extensive research on the impact of digital technology on education, highlighting the spatiotemporal extension of educational processes and practices through technological means as the greatest value of educational technology (Selwyn, 2012 ; Selwyn and Facer, 2014 ). In addition, he provides a blueprint for the development of future schools in 2030 based on the present impact of digital technology on education (Selwyn et al. 2019 ). The second most productive author in this field, Henderson, also offers significant contributions to the understanding of the important value of digital technology in education, specifically in the higher education setting, with a focus on the impact of the pandemic (Henderson et al. 2015 ; Cohen et al. 2022 ). In contrast, Edwards’ research interests focus on early childhood education, particularly the application of digital technology in this context (Edwards, 2013 ; Bird and Edwards, 2015 ). Additionally, on the technical level, Edwards also mainly prefers digital game technology, because it is a digital technology that children are relatively easy to accept (Edwards, 2015 ).

Analysis of countries/regions and organization

The present study aimed to ascertain the leading countries in digital technology education application research by analyzing 75 countries related to 558 works of literature. Table 4 depicts the top ten countries that have contributed significantly to this field in terms of publication count (based on the analysis and statistics from VOSviewer). Our analysis of Table 4 data shows that England emerged as the most influential country/region, with 92 published papers and 2401 citations. Australia and the United States secured the second and third ranks, respectively, with 90 papers (2187 citations) and 70 papers (1331 citations) published. Geographically, most of the countries featured in the top ten publication volumes are situated in Australia, North America, and Europe, with China being the only exception. Notably, all these countries, except China, belong to the group of developed nations, suggesting that economic strength is a prerequisite for fostering research in the digital technology education application field.

This study presents a visual representation of the publication output and cooperation relationships among different countries in the field of digital technology education application research. Specifically, a chord diagram is employed to display the top 30 countries in terms of publication output, as depicted in Fig. 3 . The chord diagram is composed of nodes and chords, where the nodes are positioned as scattered points along the circumference, and the length of each node corresponds to the publication output, with longer lengths indicating higher publication output. The chords, on the other hand, represent the cooperation relationships between any two countries, and are weighted based on the degree of closeness of the cooperation, with wider chords indicating closer cooperation. Through the analysis of the cooperation relationships, the findings suggest that the main publishing countries in this field are engaged in cooperative relationships with each other, indicating a relatively high level of international academic exchange and research internationalization.

In the diagram, nodes are scattered along the circumference of a circle, with the length of each node representing the volume of publications. The weighted arcs connecting any two points on the circle are known as chords, representing the collaborative relationship between the two, with the width of the arc indicating the closeness of the collaboration.

Further analyzing Fig. 3 , we can extract more valuable information, enabling a deeper understanding of the connections between countries in the research field of digital technology in educational applications. It is evident that certain countries, such as the United States, China, and England, display thicker connections, indicating robust collaborative relationships in terms of productivity. These thicker lines signify substantial mutual contributions and shared objectives in certain sectors or fields, highlighting the interconnectedness and global integration in these areas. By delving deeper, we can also explore potential future collaboration opportunities through the chord diagram, identifying possible partners to propel research and development in this field. In essence, the chord diagram successfully encapsulates and conveys the multi-dimensionality of global productivity and cooperation, allowing for a comprehensive understanding of the intricate inter-country relationships and networks in a global context, providing valuable guidance and insights for future research and collaborations.

An in-depth examination of the publishing institutions is provided in Table 5 , showcasing the foremost 10 institutions ranked by their publication volume. Notably, Monash University and Australian Catholic University, situated in Australia, have recorded the most prolific publications within the digital technology education application realm, with 22 and 10 publications respectively. Moreover, the University of Oslo from Norway is featured among the top 10 publishing institutions, with an impressive average citation count of 64 per publication. It is worth highlighting that six institutions based in the United Kingdom were also ranked within the top 10 publishing institutions, signifying their leading position in this area of research.

Analysis of journals

Journals are the main carriers for publishing high-quality papers. Some scholars point out that the two key factors to measure the influence of journals in the specified field are the number of articles published and the number of citations. The more papers published in a magazine and the more citations, the greater its influence (Dzikowski, 2018 ). Therefore, this study utilized VOSviewer to statistically analyze the top 10 journals with the most publications in the field of digital technology in education and calculated the average citations per article (see Table 6 ).

Based on Table 6 , it is apparent that the highest number of articles in the domain of digital technology in education research were published in Education and Information Technologies (47 articles), Computers & Education (34 articles), and British Journal of Educational Technology (32 articles), indicating a higher article output compared to other journals. This underscores the fact that these three journals concentrate more on the application of digital technology in education. Furthermore, several other journals, such as Technology Pedagogy and Education and Sustainability, have published more than 15 articles in this domain. Sustainability represents the open access movement, which has notably facilitated research progress in this field, indicating that the development of open access journals in recent years has had a significant impact. Although there is still considerable disagreement among scholars on the optimal approach to achieve open access, the notion that research outcomes should be accessible to all is widely recognized (Huang et al. 2020 ). On further analysis of the research fields to which these journals belong, except for Sustainability, it is evident that they all pertain to educational technology, thus providing a qualitative definition of the research area of digital technology education from the perspective of journals.

Temporal keyword analysis: thematic evolution (RQ2)

The evolution of research themes is a dynamic process, and previous studies have attempted to present the developmental trajectory of fields by drawing keyword networks in phases (Kumar et al. 2021 ; Chen et al. 2022b ). To understand the shifts in research topics across different periods, this study follows past research and, based on the significant changes in the research field and corresponding technological advancements during the outlined periods, divides the timeline into four stages (the first stage from January 2000 to December 2005, the second stage from January 2006 to December 2011, the third stage from January 2012 to December 2017; and the fourth stage from January 2018 to December 2022). The division into these four stages was determined through a combination of bibliometric analysis and literature review, which presented a clear trajectory of the field’s development. The research analyzes the keyword networks for each time period (as there are only three articles in the first stage, it was not possible to generate an appropriate keyword co-occurrence map, hence only the keyword co-occurrence maps from the second to the fourth stages are provided), to understand the evolutionary track of the digital technology education application research field over time.

2000.1–2005.12: germination period

From January 2000 to December 2005, digital technology education application research was in its infancy. Only three studies focused on digital technology, all of which were related to computers. Due to the popularity of computers, the home became a new learning environment, highlighting the important role of digital technology in expanding the scope of learning spaces (Sutherland et al. 2000 ). In specific disciplines and contexts, digital technology was first favored in medical clinical practice, becoming an important tool for supporting the learning of clinical knowledge and practice (Tegtmeyer et al. 2001 ; Durfee et al. 2003 ).

2006.1–2011.12: initial development period

Between January 2006 and December 2011, it was the initial development period of digital technology education research. Significant growth was observed in research related to digital technology, and discussions and theoretical analyses about “digital natives” emerged. During this phase, scholars focused on the debate about “how to use digital technology reasonably” and “whether current educational models and school curriculum design need to be adjusted on a large scale” (Bennett and Maton, 2010 ; Selwyn, 2009 ; Margaryan et al. 2011 ). These theoretical and speculative arguments provided a unique perspective on the impact of cognitive digital technology on education and teaching. As can be seen from the vocabulary such as “rethinking”, “disruptive pedagogy”, and “attitude” in Fig. 4 , many scholars joined the calm reflection and analysis under the trend of digital technology (Laurillard, 2008 ; Vratulis et al. 2011 ). During this phase, technology was still undergoing dramatic changes. The development of mobile technology had already caught the attention of many scholars (Wong et al. 2011 ), but digital technology represented by computers was still very active (Selwyn et al. 2011 ). The change in technological form would inevitably lead to educational transformation. Collins and Halverson ( 2010 ) summarized the prospects and challenges of using digital technology for learning and educational practices, believing that digital technology would bring a disruptive revolution to the education field and bring about a new educational system. In addition, the term “teacher education” in Fig. 4 reflects the impact of digital technology development on teachers. The rapid development of technology has widened the generation gap between teachers and students. To ensure smooth communication between teachers and students, teachers must keep up with the trend of technological development and establish a lifelong learning concept (Donnison, 2009 ).

In the diagram, each node represents a keyword, with the size of the node indicating the frequency of occurrence of the keyword. The connections represent the co-occurrence relationships between keywords, with a higher frequency of co-occurrence resulting in tighter connections.

2012.1–2017.12: critical exploration period

During the period spanning January 2012 to December 2017, the application of digital technology in education research underwent a significant exploration phase. As can be seen from Fig. 5 , different from the previous stage, the specific elements of specific digital technology have started to increase significantly, including the enrichment of technological contexts, the greater variety of research methods, and the diversification of learning modes. Moreover, the temporal and spatial dimensions of the learning environment were further de-emphasized, as noted in previous literature (Za et al. 2014 ). Given the rapidly accelerating pace of technological development, the education system in the digital era is in urgent need of collaborative evolution and reconstruction, as argued by Davis, Eickelmann, and Zaka ( 2013 ).

In the domain of digital technology, social media has garnered substantial scholarly attention as a promising avenue for learning, as noted by Pasquini and Evangelopoulos ( 2016 ). The implementation of social media in education presents several benefits, including the liberation of education from the restrictions of physical distance and time, as well as the erasure of conventional educational boundaries. The user-generated content (UGC) model in social media has emerged as a crucial source for knowledge creation and distribution, with the widespread adoption of mobile devices. Moreover, social networks have become an integral component of ubiquitous learning environments (Hwang et al. 2013 ). The utilization of social media allows individuals to function as both knowledge producers and recipients, which leads to a blurring of the conventional roles of learners and teachers. On mobile platforms, the roles of learners and teachers are not fixed, but instead interchangeable.

In terms of research methodology, the prevalence of empirical studies with survey designs in the field of educational technology during this period is evident from the vocabulary used, such as “achievement,” “acceptance,” “attitude,” and “ict.” in Fig. 5 . These studies aim to understand learners’ willingness to adopt and attitudes towards new technologies, and some seek to investigate the impact of digital technologies on learning outcomes through quasi-experimental designs (Domínguez et al. 2013 ). Among these empirical studies, mobile learning emerged as a hot topic, and this is not surprising. First, the advantages of mobile learning environments over traditional ones have been empirically demonstrated (Hwang et al. 2013 ). Second, learners born around the turn of the century have been heavily influenced by digital technologies and have developed their own learning styles that are more open to mobile devices as a means of learning. Consequently, analyzing mobile learning as a relatively novel mode of learning has become an important issue for scholars in the field of educational technology.

The intervention of technology has led to the emergence of several novel learning modes, with the blended learning model being the most representative one in the current phase. Blended learning, a novel concept introduced in the information age, emphasizes the integration of the benefits of traditional learning methods and online learning. This learning mode not only highlights the prominent role of teachers in guiding, inspiring, and monitoring the learning process but also underlines the importance of learners’ initiative, enthusiasm, and creativity in the learning process. Despite being an early conceptualization, blended learning’s meaning has been expanded by the widespread use of mobile technology and social media in education. The implementation of new technologies, particularly mobile devices, has resulted in the transformation of curriculum design and increased flexibility and autonomy in students’ learning processes (Trujillo Maza et al. 2016 ), rekindling scholarly attention to this learning mode. However, some scholars have raised concerns about the potential drawbacks of the blended learning model, such as its significant impact on the traditional teaching system, the lack of systematic coping strategies and relevant policies in several schools and regions (Moskal et al. 2013 ).

2018.1–2022.12: accelerated transformation period

The period spanning from January 2018 to December 2022 witnessed a rapid transformation in the application of digital technology in education research. The field of digital technology education research reached a peak period of publication, largely influenced by factors such as the COVID-19 pandemic (Yu et al. 2023 ). Research during this period was built upon the achievements, attitudes, and social media of the previous phase, and included more elements that reflect the characteristics of this research field, such as digital literacy, digital competence, and professional development, as depicted in Fig. 6 . Alongside this, scholars’ expectations for the value of digital technology have expanded, and the pursuit of improving learning efficiency and performance is no longer the sole focus. Some research now aims to cultivate learners’ motivation and enhance their self-efficacy by applying digital technology in a reasonable manner, as demonstrated by recent studies (Beardsley et al. 2021 ; Creely et al. 2021 ).

The COVID-19 pandemic has emerged as a crucial backdrop for the digital technology’s role in sustaining global education, as highlighted by recent scholarly research (Zhou et al. 2022 ; Pan and Zhang, 2020 ; Mo et al. 2022 ). The online learning environment, which is supported by digital technology, has become the primary battleground for global education (Yu, 2022 ). This social context has led to various studies being conducted, with some scholars positing that the pandemic has impacted the traditional teaching order while also expanding learning possibilities in terms of patterns and forms (Alabdulaziz, 2021 ). Furthermore, the pandemic has acted as a catalyst for teacher teaching and technological innovation, and this viewpoint has been empirically substantiated (Moorhouse and Wong, 2021 ). Additionally, some scholars believe that the pandemic’s push is a crucial driving force for the digital transformation of the education system, serving as an essential mechanism for overcoming the system’s inertia (Romero et al. 2021 ).

The rapid outbreak of the pandemic posed a challenge to the large-scale implementation of digital technologies, which was influenced by a complex interplay of subjective and objective factors. Objective constraints included the lack of infrastructure in some regions to support digital technologies, while subjective obstacles included psychological resistance among certain students and teachers (Moorhouse, 2021 ). These factors greatly impacted the progress of online learning during the pandemic. Additionally, Timotheou et al. ( 2023 ) conducted a comprehensive systematic review of existing research on digital technology use during the pandemic, highlighting the critical role played by various factors such as learners’ and teachers’ digital skills, teachers’ personal attributes and professional development, school leadership and management, and administration in facilitating the digitalization and transformation of schools.

The current stage of research is characterized by the pivotal term “digital literacy,” denoting a growing interest in learners’ attitudes and adoption of emerging technologies. Initially, the term “literacy” was restricted to fundamental abilities and knowledge associated with books and print materials (McMillan, 1996 ). However, with the swift advancement of computers and digital technology, there have been various attempts to broaden the scope of literacy beyond its traditional meaning, including game literacy (Buckingham and Burn, 2007 ), information literacy (Eisenberg, 2008 ), and media literacy (Turin and Friesem, 2020 ). Similarly, digital literacy has emerged as a crucial concept, and Gilster and Glister ( 1997 ) were the first to introduce this concept, referring to the proficiency in utilizing technology and processing digital information in academic, professional, and daily life settings. In practical educational settings, learners who possess higher digital literacy often exhibit an aptitude for quickly mastering digital devices and applying them intelligently to education and teaching (Yu, 2022 ).

The utilization of digital technology in education has undergone significant changes over the past two decades, and has been a crucial driver of educational reform with each new technological revolution. The impact of these changes on the underlying logic of digital technology education applications has been noticeable. From computer technology to more recent developments such as virtual reality (VR), augmented reality (AR), and artificial intelligence (AI), the acceleration in digital technology development has been ongoing. Educational reforms spurred by digital technology development continue to be dynamic, as each new digital innovation presents new possibilities and models for teaching practice. This is especially relevant in the post-pandemic era, where the importance of technological progress in supporting teaching cannot be overstated (Mughal et al. 2022 ). Existing digital technologies have already greatly expanded the dimensions of education in both time and space, while future digital technologies aim to expand learners’ perceptions. Researchers have highlighted the potential of integrated technology and immersive technology in the development of the educational metaverse, which is highly anticipated to create a new dimension for the teaching and learning environment, foster a new value system for the discipline of educational technology, and more effectively and efficiently achieve the grand educational blueprint of the United Nations’ Sustainable Development Goals (Zhang et al. 2022 ; Li and Yu, 2023 ).

Hotspot evolution analysis (RQ3)

The examination of keyword evolution reveals a consistent trend in the advancement of digital technology education application research. The emergence and transformation of keywords serve as indicators of the varying research interests in this field. Thus, the utilization of the burst detection function available in CiteSpace allowed for the identification of the top 10 burst words that exhibited a high level of burst strength. This outcome is illustrated in Table 7 .

According to the results presented in Table 7 , the explosive terminology within the realm of digital technology education research has exhibited a concentration mainly between the years 2018 and 2022. Prior to this time frame, the emerging keywords were limited to “information technology” and “computer”. Notably, among them, computer, as an emergent keyword, has always had a high explosive intensity from 2008 to 2018, which reflects the important position of computer in digital technology and is the main carrier of many digital technologies such as Learning Management Systems (LMS) and Assessment and Feedback systems (Barlovits et al. 2022 ).

Since 2018, an increasing number of research studies have focused on evaluating the capabilities of learners to accept, apply, and comprehend digital technologies. As indicated by the use of terms such as “digital literacy” and “digital skill,” the assessment of learners’ digital literacy has become a critical task. Scholarly efforts have been directed towards the development of literacy assessment tools and the implementation of empirical assessments. Furthermore, enhancing the digital literacy of both learners and educators has garnered significant attention. (Nagle, 2018 ; Yu, 2022 ). Simultaneously, given the widespread use of various digital technologies in different formal and informal learning settings, promoting learners’ digital skills has become a crucial objective for contemporary schools (Nygren et al. 2019 ; Forde and OBrien, 2022 ).

Since 2020, the field of applied research on digital technology education has witnessed the emergence of three new hotspots, all of which have been affected to some extent by the pandemic. Firstly, digital technology has been widely applied in physical education, which is one of the subjects that has been severely affected by the pandemic (Parris et al. 2022 ; Jiang and Ning, 2022 ). Secondly, digital transformation has become an important measure for most schools, especially higher education institutions, to cope with the impact of the pandemic globally (García-Morales et al. 2021 ). Although the concept of digital transformation was proposed earlier, the COVID-19 pandemic has greatly accelerated this transformation process. Educational institutions must carefully redesign their educational products to face this new situation, providing timely digital learning methods, environments, tools, and support systems that have far-reaching impacts on modern society (Krishnamurthy, 2020 ; Salas-Pilco et al. 2022 ). Moreover, the professional development of teachers has become a key mission of educational institutions in the post-pandemic era. Teachers need to have a certain level of digital literacy and be familiar with the tools and online teaching resources used in online teaching, which has become a research hotspot today. Organizing digital skills training for teachers to cope with the application of emerging technologies in education is an important issue for teacher professional development and lifelong learning (Garzón-Artacho et al. 2021 ). As the main organizers and practitioners of emergency remote teaching (ERT) during the pandemic, teachers must put cognitive effort into their professional development to ensure effective implementation of ERT (Romero-Hall and Jaramillo Cherrez, 2022 ).

The burst word “digital transformation” reveals that we are in the midst of an ongoing digital technology revolution. With the emergence of innovative digital technologies such as ChatGPT and Microsoft 365 Copilot, technology trends will continue to evolve, albeit unpredictably. While the impact of these advancements on school education remains uncertain, it is anticipated that the widespread integration of technology will significantly affect the current education system. Rejecting emerging technologies without careful consideration is unwise. Like any revolution, the technological revolution in the education field has both positive and negative aspects. Detractors argue that digital technology disrupts learning and memory (Baron, 2021 ) or causes learners to become addicted and distracted from learning (Selwyn and Aagaard, 2020 ). On the other hand, the prudent use of digital technology in education offers a glimpse of a golden age of open learning. Educational leaders and practitioners have the opportunity to leverage cutting-edge digital technologies to address current educational challenges and develop a rational path for the sustainable and healthy growth of education.

Discussion on performance analysis (RQ1)

The field of digital technology education application research has experienced substantial growth since the turn of the century, a phenomenon that is quantifiably apparent through an analysis of authorship, country/region contributions, and institutional engagement. This expansion reflects the increased integration of digital technologies in educational settings and the heightened scholarly interest in understanding and optimizing their use.

Discussion on authorship productivity in digital technology education research

The authorship distribution within digital technology education research is indicative of the field’s intellectual structure and depth. A primary figure in this domain is Neil Selwyn, whose substantial citation rate underscores the profound impact of his work. His focus on the implications of digital technology in higher education and educational sociology has proven to be seminal. Selwyn’s research trajectory, especially the exploration of spatiotemporal extensions of education through technology, provides valuable insights into the multifaceted role of digital tools in learning processes (Selwyn et al. 2019 ).

Other notable contributors, like Henderson and Edwards, present diversified research interests, such as the impact of digital technologies during the pandemic and their application in early childhood education, respectively. Their varied focuses highlight the breadth of digital technology education research, encompassing pedagogical innovation, technological adaptation, and policy development.

Discussion on country/region-level productivity and collaboration

At the country/region level, the United Kingdom, specifically England, emerges as a leading contributor with 92 published papers and a significant citation count. This is closely followed by Australia and the United States, indicating a strong English-speaking research axis. Such geographical concentration of scholarly output often correlates with investment in research and development, technological infrastructure, and the prevalence of higher education institutions engaging in cutting-edge research.

China’s notable inclusion as the only non-Western country among the top contributors to the field suggests a growing research capacity and interest in digital technology in education. However, the lower average citation per paper for China could reflect emerging engagement or different research focuses that may not yet have achieved the same international recognition as Western counterparts.

The chord diagram analysis furthers this understanding, revealing dense interconnections between countries like the United States, China, and England, which indicates robust collaborations. Such collaborations are fundamental in addressing global educational challenges and shaping international research agendas.

Discussion on institutional-level contributions to digital technology education

Institutional productivity in digital technology education research reveals a constellation of universities driving the field forward. Monash University and the Australian Catholic University have the highest publication output, signaling Australia’s significant role in advancing digital education research. The University of Oslo’s remarkable average citation count per publication indicates influential research contributions, potentially reflecting high-quality studies that resonate with the broader academic community.

The strong showing of UK institutions, including the University of London, The Open University, and the University of Cambridge, reinforces the UK’s prominence in this research field. Such institutions are often at the forefront of pedagogical innovation, benefiting from established research cultures and funding mechanisms that support sustained inquiry into digital education.

Discussion on journal publication analysis

An examination of journal outputs offers a lens into the communicative channels of the field’s knowledge base. Journals such as Education and Information Technologies , Computers & Education , and the British Journal of Educational Technology not only serve as the primary disseminators of research findings but also as indicators of research quality and relevance. The impact factor (IF) serves as a proxy for the quality and influence of these journals within the academic community.

The high citation counts for articles published in Computers & Education suggest that research disseminated through this medium has a wide-reaching impact and is of particular interest to the field. This is further evidenced by its significant IF of 11.182, indicating that the journal is a pivotal platform for seminal work in the application of digital technology in education.

The authorship, regional, and institutional productivity in the field of digital technology education application research collectively narrate the evolution of this domain since the turn of the century. The prominence of certain authors and countries underscores the importance of socioeconomic factors and existing academic infrastructure in fostering research productivity. Meanwhile, the centrality of specific journals as outlets for high-impact research emphasizes the role of academic publishing in shaping the research landscape.

As the field continues to grow, future research may benefit from leveraging the collaborative networks that have been elucidated through this analysis, perhaps focusing on underrepresented regions to broaden the scope and diversity of research. Furthermore, the stabilization of publication numbers in recent years invites a deeper exploration into potential plateaus in research trends or saturation in certain sub-fields, signaling an opportunity for novel inquiries and methodological innovations.

Discussion on the evolutionary trends (RQ2)

The evolution of the research field concerning the application of digital technology in education over the past two decades is a story of convergence, diversification, and transformation, shaped by rapid technological advancements and shifting educational paradigms.

At the turn of the century, the inception of digital technology in education was largely exploratory, with a focus on how emerging computer technologies could be harnessed to enhance traditional learning environments. Research from this early period was primarily descriptive, reflecting on the potential and challenges of incorporating digital tools into the educational setting. This phase was critical in establishing the fundamental discourse that would guide subsequent research, as it set the stage for understanding the scope and impact of digital technology in learning spaces (Wang et al. 2023 ).

As the first decade progressed, the narrative expanded to encompass the pedagogical implications of digital technologies. This was a period of conceptual debates, where terms like “digital natives” and “disruptive pedagogy” entered the academic lexicon, underscoring the growing acknowledgment of digital technology as a transformative force within education (Bennett and Maton, 2010 ). During this time, the research began to reflect a more nuanced understanding of the integration of technology, considering not only its potential to change where and how learning occurred but also its implications for educational equity and access.

In the second decade, with the maturation of internet connectivity and mobile technology, the focus of research shifted from theoretical speculations to empirical investigations. The proliferation of digital devices and the ubiquity of social media influenced how learners interacted with information and each other, prompting a surge in studies that sought to measure the impact of these tools on learning outcomes. The digital divide and issues related to digital literacy became central concerns, as scholars explored the varying capacities of students and educators to engage with technology effectively.

Throughout this period, there was an increasing emphasis on the individualization of learning experiences, facilitated by adaptive technologies that could cater to the unique needs and pacing of learners (Jing et al. 2023a ). This individualization was coupled with a growing recognition of the importance of collaborative learning, both online and offline, and the role of digital tools in supporting these processes. Blended learning models, which combined face-to-face instruction with online resources, emerged as a significant trend, advocating for a balance between traditional pedagogies and innovative digital strategies.

The later years, particularly marked by the COVID-19 pandemic, accelerated the necessity for digital technology in education, transforming it from a supplementary tool to an essential platform for delivering education globally (Mo et al. 2022 ; Mustapha et al. 2021 ). This era brought about an unprecedented focus on online learning environments, distance education, and virtual classrooms. Research became more granular, examining not just the pedagogical effectiveness of digital tools, but also their role in maintaining continuity of education during crises, their impact on teacher and student well-being, and their implications for the future of educational policy and infrastructure.

Across these two decades, the research field has seen a shift from examining digital technology as an external addition to the educational process, to viewing it as an integral component of curriculum design, instructional strategies, and even assessment methods. The emergent themes have broadened from a narrow focus on specific tools or platforms to include wider considerations such as data privacy, ethical use of technology, and the environmental impact of digital tools.

Moreover, the field has moved from considering the application of digital technology in education as a primarily cognitive endeavor to recognizing its role in facilitating socio-emotional learning, digital citizenship, and global competencies. Researchers have increasingly turned their attention to the ways in which technology can support collaborative skills, cultural understanding, and ethical reasoning within diverse student populations.

In summary, the past over twenty years in the research field of digital technology applications in education have been characterized by a progression from foundational inquiries to complex analyses of digital integration. This evolution has mirrored the trajectory of technology itself, from a facilitative tool to a pervasive ecosystem defining contemporary educational experiences. As we look to the future, the field is poised to delve into the implications of emerging technologies like AI, AR, and VR, and their potential to redefine the educational landscape even further. This ongoing metamorphosis suggests that the application of digital technology in education will continue to be a rich area of inquiry, demanding continual adaptation and forward-thinking from educators and researchers alike.

Discussion on the study of research hotspots (RQ3)

The analysis of keyword evolution in digital technology education application research elucidates the current frontiers in the field, reflecting a trajectory that is in tandem with the rapidly advancing digital age. This landscape is sculpted by emergent technological innovations and shaped by the demands of an increasingly digital society.

Interdisciplinary integration and pedagogical transformation

One of the frontiers identified from recent keyword bursts includes the integration of digital technology into diverse educational contexts, particularly noted with the keyword “physical education.” The digitalization of disciplines traditionally characterized by physical presence illustrates the pervasive reach of technology and signifies a push towards interdisciplinary integration where technology is not only a facilitator but also a transformative agent. This integration challenges educators to reconceptualize curriculum delivery to accommodate digital tools that can enhance or simulate the physical aspects of learning.

Digital literacy and skills acquisition

Another pivotal frontier is the focus on “digital literacy” and “digital skill”, which has intensified in recent years. This suggests a shift from mere access to technology towards a comprehensive understanding and utilization of digital tools. In this realm, the emphasis is not only on the ability to use technology but also on critical thinking, problem-solving, and the ethical use of digital resources (Yu, 2022 ). The acquisition of digital literacy is no longer an additive skill but a fundamental aspect of modern education, essential for navigating and contributing to the digital world.

Educational digital transformation

The keyword “digital transformation” marks a significant research frontier, emphasizing the systemic changes that education institutions must undergo to align with the digital era (Romero et al. 2021 ). This transformation includes the redesigning of learning environments, pedagogical strategies, and assessment methods to harness digital technology’s full potential. Research in this area explores the complexity of institutional change, addressing the infrastructural, cultural, and policy adjustments needed for a seamless digital transition.

Engagement and participation

Further exploration into “engagement” and “participation” underscores the importance of student-centered learning environments that are mediated by technology. The current frontiers examine how digital platforms can foster collaboration, inclusivity, and active learning, potentially leading to more meaningful and personalized educational experiences. Here, the use of technology seeks to support the emotional and cognitive aspects of learning, moving beyond the transactional view of education to one that is relational and interactive.

Professional development and teacher readiness

As the field evolves, “professional development” emerges as a crucial area, particularly in light of the pandemic which necessitated emergency remote teaching. The need for teacher readiness in a digital age is a pressing frontier, with research focusing on the competencies required for educators to effectively integrate technology into their teaching practices. This includes familiarity with digital tools, pedagogical innovation, and an ongoing commitment to personal and professional growth in the digital domain.

Pandemic as a catalyst

The recent pandemic has acted as a catalyst for accelerated research and application in this field, particularly in the domains of “digital transformation,” “professional development,” and “physical education.” This period has been a litmus test for the resilience and adaptability of educational systems to continue their operations in an emergency. Research has thus been directed at understanding how digital technologies can support not only continuity but also enhance the quality and reach of education in such contexts.

Ethical and societal considerations

The frontier of digital technology in education is also expanding to consider broader ethical and societal implications. This includes issues of digital equity, data privacy, and the sociocultural impact of technology on learning communities. The research explores how educational technology can be leveraged to address inequities and create more equitable learning opportunities for all students, regardless of their socioeconomic background.

Innovation and emerging technologies

Looking forward, the frontiers are set to be influenced by ongoing and future technological innovations, such as artificial intelligence (AI) (Wu and Yu, 2023 ; Chen et al. 2022a ). The exploration into how these technologies can be integrated into educational practices to create immersive and adaptive learning experiences represents a bold new chapter for the field.

In conclusion, the current frontiers of research on the application of digital technology in education are multifaceted and dynamic. They reflect an overarching movement towards deeper integration of technology in educational systems and pedagogical practices, where the goals are not only to facilitate learning but to redefine it. As these frontiers continue to expand and evolve, they will shape the educational landscape, requiring a concerted effort from researchers, educators, policymakers, and technologists to navigate the challenges and harness the opportunities presented by the digital revolution in education.

Conclusions and future research

Conclusions.

The utilization of digital technology in education is a research area that cuts across multiple technical and educational domains and continues to experience dynamic growth due to the continuous progress of technology. In this study, a systematic review of this field was conducted through bibliometric techniques to examine its development trajectory. The primary focus of the review was to investigate the leading contributors, productive national institutions, significant publications, and evolving development patterns. The study’s quantitative analysis resulted in several key conclusions that shed light on this research field’s current state and future prospects.

(1) The research field of digital technology education applications has entered a stage of rapid development, particularly in recent years due to the impact of the pandemic, resulting in a peak of publications. Within this field, several key authors (Selwyn, Henderson, Edwards, etc.) and countries/regions (England, Australia, USA, etc.) have emerged, who have made significant contributions. International exchanges in this field have become frequent, with a high degree of internationalization in academic research. Higher education institutions in the UK and Australia are the core productive forces in this field at the institutional level.

(2) Education and Information Technologies , Computers & Education , and the British Journal of Educational Technology are notable journals that publish research related to digital technology education applications. These journals are affiliated with the research field of educational technology and provide effective communication platforms for sharing digital technology education applications.

(3) Over the past two decades, research on digital technology education applications has progressed from its early stages of budding, initial development, and critical exploration to accelerated transformation, and it is currently approaching maturity. Technological progress and changes in the times have been key driving forces for educational transformation and innovation, and both have played important roles in promoting the continuous development of education.

(4) Influenced by the pandemic, three emerging frontiers have emerged in current research on digital technology education applications, which are physical education, digital transformation, and professional development under the promotion of digital technology. These frontier research hotspots reflect the core issues that the education system faces when encountering new technologies. The evolution of research hotspots shows that technology breakthroughs in education’s original boundaries of time and space create new challenges. The continuous self-renewal of education is achieved by solving one hotspot problem after another.

The present study offers significant practical implications for scholars and practitioners in the field of digital technology education applications. Firstly, it presents a well-defined framework of the existing research in this area, serving as a comprehensive guide for new entrants to the field and shedding light on the developmental trajectory of this research domain. Secondly, the study identifies several contemporary research hotspots, thus offering a valuable decision-making resource for scholars aiming to explore potential research directions. Thirdly, the study undertakes an exhaustive analysis of published literature to identify core journals in the field of digital technology education applications, with Sustainability being identified as a promising open access journal that publishes extensively on this topic. This finding can potentially facilitate scholars in selecting appropriate journals for their research outputs.

Limitation and future research

Influenced by some objective factors, this study also has some limitations. First of all, the bibliometrics analysis software has high standards for data. In order to ensure the quality and integrity of the collected data, the research only selects the periodical papers in SCIE and SSCI indexes, which are the core collection of Web of Science database, and excludes other databases, conference papers, editorials and other publications, which may ignore some scientific research and original opinions in the field of digital technology education and application research. In addition, although this study used professional software to carry out bibliometric analysis and obtained more objective quantitative data, the analysis and interpretation of data will inevitably have a certain subjective color, and the influence of subjectivity on data analysis cannot be completely avoided. As such, future research endeavors will broaden the scope of literature screening and proactively engage scholars in the field to gain objective and state-of-the-art insights, while minimizing the adverse impact of personal subjectivity on research analysis.

Data availability

The datasets analyzed during the current study are available in the Dataverse repository: https://doi.org/10.7910/DVN/F9QMHY

Alabdulaziz MS (2021) COVID-19 and the use of digital technology in mathematics education. Educ Inf Technol 26(6):7609–7633. https://doi.org/10.1007/s10639-021-10602-3

Arif TB, Munaf U, Ul-Haque I (2023) The future of medical education and research: is ChatGPT a blessing or blight in disguise? Med Educ Online 28. https://doi.org/10.1080/10872981.2023.2181052

Banerjee M, Chiew D, Patel KT, Johns I, Chappell D, Linton N, Cole GD, Francis DP, Szram J, Ross J, Zaman S (2021) The impact of artificial intelligence on clinical education: perceptions of postgraduate trainee doctors in London (UK) and recommendations for trainers. BMC Med Educ 21. https://doi.org/10.1186/s12909-021-02870-x

Barlovits S, Caldeira A, Fesakis G, Jablonski S, Koutsomanoli Filippaki D, Lázaro C, Ludwig M, Mammana MF, Moura A, Oehler DXK, Recio T, Taranto E, Volika S(2022) Adaptive, synchronous, and mobile online education: developing the ASYMPTOTE learning environment. Mathematics 10:1628. https://doi.org/10.3390/math10101628

Article   Google Scholar  

Baron NS(2021) Know what? How digital technologies undermine learning and remembering J Pragmat 175:27–37. https://doi.org/10.1016/j.pragma.2021.01.011

Batista J, Morais NS, Ramos F (2016) Researching the use of communication technologies in higher education institutions in Portugal. https://doi.org/10.4018/978-1-5225-0571-6.ch057

Beardsley M, Albó L, Aragón P, Hernández-Leo D (2021) Emergency education effects on teacher abilities and motivation to use digital technologies. Br J Educ Technol 52. https://doi.org/10.1111/bjet.13101

Bennett S, Maton K(2010) Beyond the “digital natives” debate: towards a more nuanced understanding of students’ technology experiences J Comput Assist Learn 26:321–331. https://doi.org/10.1111/j.1365-2729.2010.00360.x

Buckingham D, Burn A (2007) Game literacy in theory and practice 16:323–349

Google Scholar  

Bulfin S, Pangrazio L, Selwyn N (2014) Making “MOOCs”: the construction of a new digital higher education within news media discourse. In: The International Review of Research in Open and Distributed Learning 15. https://doi.org/10.19173/irrodl.v15i5.1856

Camilleri MA, Camilleri AC(2016) Digital learning resources and ubiquitous technologies in education Technol Knowl Learn 22:65–82. https://doi.org/10.1007/s10758-016-9287-7

Chen C(2006) CiteSpace II: detecting and visualizing emerging trends and transient patterns in scientific literature J Am Soc Inf Sci Technol 57:359–377. https://doi.org/10.1002/asi.20317

Chen J, Dai J, Zhu K, Xu L(2022) Effects of extended reality on language learning: a meta-analysis Front Psychol 13:1016519. https://doi.org/10.3389/fpsyg.2022.1016519

Article   PubMed   PubMed Central   Google Scholar  

Chen J, Wang CL, Tang Y (2022b) Knowledge mapping of volunteer motivation: a bibliometric analysis and cross-cultural comparative study. Front Psychol 13. https://doi.org/10.3389/fpsyg.2022.883150

Cohen A, Soffer T, Henderson M(2022) Students’ use of technology and their perceptions of its usefulness in higher education: International comparison J Comput Assist Learn 38(5):1321–1331. https://doi.org/10.1111/jcal.12678

Collins A, Halverson R(2010) The second educational revolution: rethinking education in the age of technology J Comput Assist Learn 26:18–27. https://doi.org/10.1111/j.1365-2729.2009.00339.x

Conole G, Alevizou P (2010) A literature review of the use of Web 2.0 tools in higher education. Walton Hall, Milton Keynes, UK: the Open University, retrieved 17 February

Creely E, Henriksen D, Crawford R, Henderson M(2021) Exploring creative risk-taking and productive failure in classroom practice. A case study of the perceived self-efficacy and agency of teachers at one school Think Ski Creat 42:100951. https://doi.org/10.1016/j.tsc.2021.100951

Davis N, Eickelmann B, Zaka P(2013) Restructuring of educational systems in the digital age from a co-evolutionary perspective J Comput Assist Learn 29:438–450. https://doi.org/10.1111/jcal.12032

De Belli N (2009) Bibliometrics and citation analysis: from the science citation index to cybermetrics, Scarecrow Press. https://doi.org/10.1111/jcal.12032

Domínguez A, Saenz-de-Navarrete J, de-Marcos L, Fernández-Sanz L, Pagés C, Martínez-Herráiz JJ(2013) Gamifying learning experiences: practical implications and outcomes Comput Educ 63:380–392. https://doi.org/10.1016/j.compedu.2012.12.020

Donnison S (2009) Discourses in conflict: the relationship between Gen Y pre-service teachers, digital technologies and lifelong learning. Australasian J Educ Technol 25. https://doi.org/10.14742/ajet.1138

Durfee SM, Jain S, Shaffer K (2003) Incorporating electronic media into medical student education. Acad Radiol 10:205–210. https://doi.org/10.1016/s1076-6332(03)80046-6

Dzikowski P(2018) A bibliometric analysis of born global firms J Bus Res 85:281–294. https://doi.org/10.1016/j.jbusres.2017.12.054

van Eck NJ, Waltman L(2009) Software survey: VOSviewer, a computer program for bibliometric mapping Scientometrics 84:523–538 https://doi.org/10.1007/s11192-009-0146-3

Edwards S(2013) Digital play in the early years: a contextual response to the problem of integrating technologies and play-based pedagogies in the early childhood curriculum Eur Early Child Educ Res J 21:199–212. https://doi.org/10.1080/1350293x.2013.789190

Edwards S(2015) New concepts of play and the problem of technology, digital media and popular-culture integration with play-based learning in early childhood education Technol Pedagogy Educ 25:513–532 https://doi.org/10.1080/1475939x.2015.1108929

Article   MathSciNet   Google Scholar  

Eisenberg MB(2008) Information literacy: essential skills for the information age DESIDOC J Libr Inf Technol 28:39–47. https://doi.org/10.14429/djlit.28.2.166

Forde C, OBrien A (2022) A literature review of barriers and opportunities presented by digitally enhanced practical skill teaching and learning in health science education. Med Educ Online 27. https://doi.org/10.1080/10872981.2022.2068210

García-Morales VJ, Garrido-Moreno A, Martín-Rojas R (2021) The transformation of higher education after the COVID disruption: emerging challenges in an online learning scenario. Front Psychol 12. https://doi.org/10.3389/fpsyg.2021.616059

Garfield E(2006) The history and meaning of the journal impact factor JAMA 295:90. https://doi.org/10.1001/jama.295.1.90

Article   PubMed   Google Scholar  

Garzón-Artacho E, Sola-Martínez T, Romero-Rodríguez JM, Gómez-García G(2021) Teachers’ perceptions of digital competence at the lifelong learning stage Heliyon 7:e07513. https://doi.org/10.1016/j.heliyon.2021.e07513

Gaviria-Marin M, Merigó JM, Baier-Fuentes H(2019) Knowledge management: a global examination based on bibliometric analysis Technol Forecast Soc Change 140:194–220. https://doi.org/10.1016/j.techfore.2018.07.006

Gilster P, Glister P (1997) Digital literacy. Wiley Computer Pub, New York

Greenhow C, Lewin C(2015) Social media and education: reconceptualizing the boundaries of formal and informal learning Learn Media Technol 41:6–30. https://doi.org/10.1080/17439884.2015.1064954

Hawkins DT(2001) Bibliometrics of electronic journals in information science Infor Res 7(1):7–1. http://informationr.net/ir/7-1/paper120.html

Henderson M, Selwyn N, Finger G, Aston R(2015) Students’ everyday engagement with digital technology in university: exploring patterns of use and “usefulness J High Educ Policy Manag 37:308–319 https://doi.org/10.1080/1360080x.2015.1034424

Huang CK, Neylon C, Hosking R, Montgomery L, Wilson KS, Ozaygen A, Brookes-Kenworthy C (2020) Evaluating the impact of open access policies on research institutions. eLife 9. https://doi.org/10.7554/elife.57067

Hwang GJ, Tsai CC(2011) Research trends in mobile and ubiquitous learning: a review of publications in selected journals from 2001 to 2010 Br J Educ Technol 42:E65–E70. https://doi.org/10.1111/j.1467-8535.2011.01183.x

Hwang GJ, Wu PH, Zhuang YY, Huang YM(2013) Effects of the inquiry-based mobile learning model on the cognitive load and learning achievement of students Interact Learn Environ 21:338–354. https://doi.org/10.1080/10494820.2011.575789

Jiang S, Ning CF (2022) Interactive communication in the process of physical education: are social media contributing to the improvement of physical training performance. Universal Access Inf Soc, 1–10. https://doi.org/10.1007/s10209-022-00911-w

Jing Y, Zhao L, Zhu KK, Wang H, Wang CL, Xia Q(2023) Research landscape of adaptive learning in education: a bibliometric study on research publications from 2000 to 2022 Sustainability 15:3115–3115. https://doi.org/10.3390/su15043115

Jing Y, Wang CL, Chen Y, Wang H, Yu T, Shadiev R (2023b) Bibliometric mapping techniques in educational technology research: a systematic literature review. Educ Inf Technol 1–29. https://doi.org/10.1007/s10639-023-12178-6

Krishnamurthy S (2020) The future of business education: a commentary in the shadow of the Covid-19 pandemic. J Bus Res. https://doi.org/10.1016/j.jbusres.2020.05.034

Kumar S, Lim WM, Pandey N, Christopher Westland J (2021) 20 years of electronic commerce research. Electron Commer Res 21:1–40

Kyza EA, Georgiou Y(2018) Scaffolding augmented reality inquiry learning: the design and investigation of the TraceReaders location-based, augmented reality platform Interact Learn Environ 27:211–225. https://doi.org/10.1080/10494820.2018.1458039

Laurillard D(2008) Technology enhanced learning as a tool for pedagogical innovation J Philos Educ 42:521–533. https://doi.org/10.1111/j.1467-9752.2008.00658.x

Li M, Yu Z (2023) A systematic review on the metaverse-based blended English learning. Front Psychol 13. https://doi.org/10.3389/fpsyg.2022.1087508

Luo H, Li G, Feng Q, Yang Y, Zuo M (2021) Virtual reality in K-12 and higher education: a systematic review of the literature from 2000 to 2019. J Comput Assist Learn. https://doi.org/10.1111/jcal.12538

Margaryan A, Littlejohn A, Vojt G(2011) Are digital natives a myth or reality? University students’ use of digital technologies Comput Educ 56:429–440. https://doi.org/10.1016/j.compedu.2010.09.004

McMillan S(1996) Literacy and computer literacy: definitions and comparisons Comput Educ 27:161–170. https://doi.org/10.1016/s0360-1315(96)00026-7

Mo CY, Wang CL, Dai J, Jin P (2022) Video playback speed influence on learning effect from the perspective of personalized adaptive learning: a study based on cognitive load theory. Front Psychology 13. https://doi.org/10.3389/fpsyg.2022.839982

Moorhouse BL (2021) Beginning teaching during COVID-19: newly qualified Hong Kong teachers’ preparedness for online teaching. Educ Stud 1–17. https://doi.org/10.1080/03055698.2021.1964939

Moorhouse BL, Wong KM (2021) The COVID-19 Pandemic as a catalyst for teacher pedagogical and technological innovation and development: teachers’ perspectives. Asia Pac J Educ 1–16. https://doi.org/10.1080/02188791.2021.1988511

Moskal P, Dziuban C, Hartman J (2013) Blended learning: a dangerous idea? Internet High Educ 18:15–23

Mughal MY, Andleeb N, Khurram AFA, Ali MY, Aslam MS, Saleem MN (2022) Perceptions of teaching-learning force about Metaverse for education: a qualitative study. J. Positive School Psychol 6:1738–1745

Mustapha I, Thuy Van N, Shahverdi M, Qureshi MI, Khan N (2021) Effectiveness of digital technology in education during COVID-19 pandemic. a bibliometric analysis. Int J Interact Mob Technol 15:136

Nagle J (2018) Twitter, cyber-violence, and the need for a critical social media literacy in teacher education: a review of the literature. Teach Teach Education 76:86–94

Nazare J, Woolf A, Sysoev I, Ballinger S, Saveski M, Walker M, Roy D (2022) Technology-assisted coaching can increase engagement with learning technology at home and caregivers’ awareness of it. Comput Educ 188:104565

Nguyen UP, Hallinger P (2020) Assessing the distinctive contributions of simulation & gaming to the literature, 1970-2019: a bibliometric review. Simul Gaming 104687812094156. https://doi.org/10.1177/1046878120941569

Nygren H, Nissinen K, Hämäläinen R, Wever B(2019) Lifelong learning: formal, non-formal and informal learning in the context of the use of problem-solving skills in technology-rich environments Br J Educ Technol 50:1759–1770. https://doi.org/10.1111/bjet.12807

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, Moher D (2021) The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. Int J Surg 88:105906

Pan SL, Zhang S(2020) From fighting COVID-19 pandemic to tackling sustainable development goals: an opportunity for responsible information systems research Int J Inf Manage 55:102196. https://doi.org/10.1016/j.ijinfomgt.2020.102196

Pan X, Yan E, Cui M, Hua W(2018) Examining the usage, citation, and diffusion patterns of bibliometric mapping software: a comparative study of three tools J Informetr 12:481–493. https://doi.org/10.1016/j.joi.2018.03.005

Parris Z, Cale L, Harris J, Casey A (2022) Physical activity for health, covid-19 and social media: what, where and why?. Movimento, 28. https://doi.org/10.22456/1982-8918.122533

Pasquini LA, Evangelopoulos N (2016) Sociotechnical stewardship in higher education: a field study of social media policy documents. J Comput High Educ 29:218–239

Pérez-Sanagustín M, Hernández-Leo D, Santos P, Delgado Kloos C, Blat J(2014) Augmenting reality and formality of informal and non-formal settings to enhance blended learning IEEE Trans Learn Technol 7:118–131. https://doi.org/10.1109/TLT.2014.2312719

Pinto M, Leite C (2020) Digital technologies in support of students learning in Higher Education: literature review. Digital Education Review 343–360. https://doi.org/10.1344/der.2020.37.343-360

Pires F, Masanet MJ, Tomasena JM, Scolari CA(2022) Learning with YouTube: beyond formal and informal through new actors, strategies and affordances Convergence 28(3):838–853. https://doi.org/10.1177/1354856521102054

Pritchard A (1969) Statistical bibliography or bibliometrics 25:348

Romero M, Romeu T, Guitert M, Baztán P (2021) Digital transformation in higher education: the UOC case. In ICERI2021 Proceedings (pp. 6695–6703). IATED https://doi.org/10.21125/iceri.2021.1512

Romero-Hall E, Jaramillo Cherrez N (2022) Teaching in times of disruption: faculty digital literacy in higher education during the COVID-19 pandemic. Innovations in Education and Teaching International 1–11. https://doi.org/10.1080/14703297.2022.2030782

Rospigliosi PA(2023) Artificial intelligence in teaching and learning: what questions should we ask of ChatGPT? Interactive Learning Environments 31:1–3. https://doi.org/10.1080/10494820.2023.2180191

Salas-Pilco SZ, Yang Y, Zhang Z(2022) Student engagement in online learning in Latin American higher education during the COVID-19 pandemic: a systematic review. Br J Educ Technol 53(3):593–619. https://doi.org/10.1111/bjet.13190

Selwyn N(2009) The digital native-myth and reality In Aslib proceedings 61(4):364–379. https://doi.org/10.1108/00012530910973776

Selwyn N(2012) Making sense of young people, education and digital technology: the role of sociological theory Oxford Review of Education 38:81–96. https://doi.org/10.1080/03054985.2011.577949

Selwyn N, Facer K(2014) The sociology of education and digital technology: past, present and future Oxford Rev Educ 40:482–496. https://doi.org/10.1080/03054985.2014.933005

Selwyn N, Banaji S, Hadjithoma-Garstka C, Clark W(2011) Providing a platform for parents? Exploring the nature of parental engagement with school Learning Platforms J Comput Assist Learn 27:314–323. https://doi.org/10.1111/j.1365-2729.2011.00428.x

Selwyn N, Aagaard J (2020) Banning mobile phones from classrooms-an opportunity to advance understandings of technology addiction, distraction and cyberbullying. Br J Educ Technol 52. https://doi.org/10.1111/bjet.12943

Selwyn N, O’Neill C, Smith G, Andrejevic M, Gu X (2021) A necessary evil? The rise of online exam proctoring in Australian universities. Media Int Austr 1329878X2110058. https://doi.org/10.1177/1329878x211005862

Selwyn N, Pangrazio L, Nemorin S, Perrotta C (2019) What might the school of 2030 be like? An exercise in social science fiction. Learn, Media Technol 1–17. https://doi.org/10.1080/17439884.2020.1694944

Selwyn, N (2016) What works and why?* Understanding successful technology enabled learning within institutional contexts 2016 Final report Appendices (Part B). Monash University Griffith University

Sjöberg D, Holmgren R (2021) Informal workplace learning in swedish police education-a teacher perspective. Vocations and Learning. https://doi.org/10.1007/s12186-021-09267-3

Strotmann A, Zhao D (2012) Author name disambiguation: what difference does it make in author-based citation analysis? J Am Soc Inf Sci Technol 63:1820–1833

Article   CAS   Google Scholar  

Sutherland R, Facer K, Furlong R, Furlong J(2000) A new environment for education? The computer in the home. Comput Educ 34:195–212. https://doi.org/10.1016/s0360-1315(99)00045-7

Szeto E, Cheng AY-N, Hong J-C(2015) Learning with social media: how do preservice teachers integrate YouTube and Social Media in teaching? Asia-Pac Educ Res 25:35–44. https://doi.org/10.1007/s40299-015-0230-9

Tang E, Lam C(2014) Building an effective online learning community (OLC) in blog-based teaching portfolios Int High Educ 20:79–85. https://doi.org/10.1016/j.iheduc.2012.12.002

Taskin Z, Al U(2019) Natural language processing applications in library and information science Online Inf Rev 43:676–690. https://doi.org/10.1108/oir-07-2018-0217

Tegtmeyer K, Ibsen L, Goldstein B(2001) Computer-assisted learning in critical care: from ENIAC to HAL Crit Care Med 29:N177–N182. https://doi.org/10.1097/00003246-200108001-00006

Article   CAS   PubMed   Google Scholar  

Timotheou S, Miliou O, Dimitriadis Y, Sobrino SV, Giannoutsou N, Cachia R, Moné AM, Ioannou A(2023) Impacts of digital technologies on education and factors influencing schools' digital capacity and transformation: a literature review. Educ Inf Technol 28(6):6695–6726. https://doi.org/10.1007/s10639-022-11431-8

Trujillo Maza EM, Gómez Lozano MT, Cardozo Alarcón AC, Moreno Zuluaga L, Gamba Fadul M (2016) Blended learning supported by digital technology and competency-based medical education: a case study of the social medicine course at the Universidad de los Andes, Colombia. Int J Educ Technol High Educ 13. https://doi.org/10.1186/s41239-016-0027-9

Turin O, Friesem Y(2020) Is that media literacy?: Israeli and US media scholars’ perceptions of the field J Media Lit Educ 12:132–144. https://doi.org/10.1007/s11192-009-0146-3

Van Eck NJ, Waltman L (2019) VOSviewer manual. Universiteit Leiden

Vratulis V, Clarke T, Hoban G, Erickson G(2011) Additive and disruptive pedagogies: the use of slowmation as an example of digital technology implementation Teach Teach Educ 27:1179–1188. https://doi.org/10.1016/j.tate.2011.06.004

Wang CL, Dai J, Xu LJ (2022) Big data and data mining in education: a bibliometrics study from 2010 to 2022. In 2022 7th International Conference on Cloud Computing and Big Data Analytics ( ICCCBDA ) (pp. 507-512). IEEE. https://doi.org/10.1109/icccbda55098.2022.9778874

Wang CL, Dai J, Zhu KK, Yu T, Gu XQ (2023) Understanding the continuance intention of college students toward new E-learning spaces based on an integrated model of the TAM and TTF. Int J Hum-Comput Int 1–14. https://doi.org/10.1080/10447318.2023.2291609

Wong L-H, Boticki I, Sun J, Looi C-K(2011) Improving the scaffolds of a mobile-assisted Chinese character forming game via a design-based research cycle Comput Hum Behav 27:1783–1793. https://doi.org/10.1016/j.chb.2011.03.005

Wu R, Yu Z (2023) Do AI chatbots improve students learning outcomes? Evidence from a meta-analysis. Br J Educ Technol. https://doi.org/10.1111/bjet.13334

Yang D, Zhou J, Shi D, Pan Q, Wang D, Chen X, Liu J (2022) Research status, hotspots, and evolutionary trends of global digital education via knowledge graph analysis. Sustainability 14:15157–15157. https://doi.org/10.3390/su142215157

Yu T, Dai J, Wang CL (2023) Adoption of blended learning: Chinese university students’ perspectives. Humanit Soc Sci Commun 10:390. https://doi.org/10.3390/su142215157

Yu Z (2022) Sustaining student roles, digital literacy, learning achievements, and motivation in online learning environments during the COVID-19 pandemic. Sustainability 14:4388. https://doi.org/10.3390/su14084388

Za S, Spagnoletti P, North-Samardzic A(2014) Organisational learning as an emerging process: the generative role of digital tools in informal learning practices Br J Educ Technol 45:1023–1035. https://doi.org/10.1111/bjet.12211

Zhang X, Chen Y, Hu L, Wang Y (2022) The metaverse in education: definition, framework, features, potential applications, challenges, and future research topics. Front Psychol 13:1016300. https://doi.org/10.3389/fpsyg.2022.1016300

Zhou M, Dzingirai C, Hove K, Chitata T, Mugandani R (2022) Adoption, use and enhancement of virtual learning during COVID-19. Education and Information Technologies. https://doi.org/10.1007/s10639-022-10985-x

Download references

Acknowledgements

This research was supported by the Zhejiang Provincial Social Science Planning Project, “Mechanisms and Pathways for Empowering Classroom Teaching through Learning Spaces under the Strategy of High-Quality Education Development”, the 2022 National Social Science Foundation Education Youth Project “Research on the Strategy of Creating Learning Space Value and Empowering Classroom Teaching under the background of ‘Double Reduction’” (Grant No. CCA220319) and the National College Student Innovation and Entrepreneurship Training Program of China (Grant No. 202310337023).

Author information

Authors and affiliations.

College of Educational Science and Technology, Zhejiang University of Technology, Zhejiang, China

Chengliang Wang, Xiaojiao Chen, Yidan Liu & Yuhui Jing

Graduate School of Business, Universiti Sains Malaysia, Minden, Malaysia

Department of Management, The Chinese University of Hong Kong, Hong Kong, China

College of Humanities and Social Sciences, Beihang University, Beijing, China

You can also search for this author in PubMed   Google Scholar

Contributions

Conceptualization: Y.J., C.W.; methodology, C.W.; software, C.W., Y.L.; writing-original draft preparation, C.W., Y.L.; writing-review and editing, T.Y., Y.L., C.W.; supervision, X.C., T.Y.; project administration, Y.J.; funding acquisition, X.C., Y.L. All authors read and approved the final manuscript. All authors have read and approved the re-submission of the manuscript.

Corresponding author

Correspondence to Yuhui Jing .

Ethics declarations

Ethical approval.

Ethical approval was not required as the study did not involve human participants.

Informed consent

Informed consent was not required as the study did not involve human participants.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Wang, C., Chen, X., Yu, T. et al. Education reform and change driven by digital technology: a bibliometric study from a global perspective. Humanit Soc Sci Commun 11 , 256 (2024). https://doi.org/10.1057/s41599-024-02717-y

Download citation

Received : 11 July 2023

Accepted : 17 January 2024

Published : 12 February 2024

DOI : https://doi.org/10.1057/s41599-024-02717-y

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

This article is cited by

A meta-analysis of learners’ continuance intention toward online education platforms.

• Chengliang Wang

Education and Information Technologies (2024)

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

The Pros and Cons of 7 Digital Teaching Tools

Explore more.

• Classroom Management
• Course Materials
• Digital Learning

O ne highlight of the last 18 months has been the level of experimentation I’ve seen among educators. They’ve explored new ways to teach in different environments and new technologies to keep students involved and engaged. As we move forward, it’s important that we learn from all this experimentation so we may deliver a learning experience that’s better than what we entered the pandemic delivering.

Simply rejecting all digital modes of teaching once you’re back in a physical classroom is not in your students’—or your own—best interest. There are many benefits to virtual learning that are worth keeping. In my own teaching, I’ve tried to incorporate the best learnings of pandemic teaching by using a four-step framework —struggle, structure, systemize, and synthesize—alongside different digital teaching methods and technologies that I’ve found work for me.

The precise way you use each digital tool and the extent to which you combine digital instruction with in-person instruction will of course depend on the needs of each specific course you teach. But to help you start thinking about how digital tools can remain useful to you, here’s a summary of the advantages and disadvantages of seven of the most common ones. I also share when I use each one to help spur your thinking.

1. Recorded Lecture Videos

Recording yourself giving lectures is perhaps the simplest digital approach. While these video recordings are easy to create and effective for sharing information quickly, production value is often less than ideal and the videos can be less than engaging. Overall, this approach doesn’t nearly reach the full potential that can be accomplished with digital learning.

Advantages of recorded lectures:

They let students consume course material on their own schedule and at their own pace, which students like.

They are more accessible—you can speed them up or slow them down—and you can easily add additional accessibility features, such as automated closed captioning or transcriptions.

Students can fast forward through material they already understand and rewind or rewatch material they are struggling with, unlike in a live lecture when wandering attention can mean missing a crucial point.

Disadvantages of recorded lectures:

They can be less than engaging.

They’re not interactive.

When I use them:

Truthfully, I don’t use them very often. They can be useful for exceptional circumstances that make it impossible for everyone to be in a live lecture.

Occasionally, I use them to set up a mini-case to initiate problem-solving thinking, or to provide information about key framework ideas before class discussion, but I tend to use edited video lessons (see next section) for that purpose.

2. Edited Video Lessons

Webinar: designing better courses.

Robert D. Austin recently delivered an HBP Education webinar, entitled Designing Better Courses: Blending the Best of Pre- and Post-Pandemic Pedagogy , to discuss his course design process and detail how—and why—he mixes digital technologies both asynchronously and synchronously throughout his courses. Watch the full webinar recording here .

Advantages of video lessons:

All the advantages of recorded lectures (e.g., self-paced).

Students have an opportunity to watch several short videos in a row, as their schedules permit.

Graphics and other illustrations can be useful for clarifying concepts.

Disadvantages of video lessons:

They’re more engaging than recorded lectures, but still not interactive.

Producing these videos requires extra time and effort.

To set up initial problem situations or present useful framework materials.

To add new information that may cause students to reconsider previous conclusions.

To teach a mechanical analysis approach, such as how to calculate a net present value.

3. Zoom Sessions

When courses are held fully remotely or in a hybrid setting (with some students participating in person and some participating virtually), most class sessions and discussions happen over Zoom or a similar videoconferencing platform. These live virtual sessions can allow for a synchronous learning experience enhanced by other digital tools, such as whiteboards and other display technologies, but they cannot be considered an exact replacement for in-person discussions.

For those teaching fully in person, Zoom can still be used for things like bringing in guests from afar and for exercises that involve the use of groups in the form of breakout rooms. I run a negotiation exercise for one of my classes that is actually a lot easier to run in Zoom than in person, because it involves rapid transitions between breakout groups and larger class discussion. Zoom is also great for students to use in coordinating project work outside of class.

Advantages of Zoom sessions:

Students can synchronously interact with each other remotely.

Technology allows for unique modes of interaction and discussion, such as breakout rooms , which can be configured instantaneously, as well as chat channels.

It’s easy to invite remote guest speakers who would otherwise be unable to travel to campus.

Disadvantages of Zoom sessions:

Students and educators alike can experience Zoom fatigue.

It can be hard to read interpersonal cues from those who are remote.

While Zoom calls are interactive, they still lack valuable opportunities for casual social interaction.

There’s no real substitute for students walking in the hall together, chatting about pretty much anything. At least not yet.

For case discussions that include remote guests.

For exercises that need fast transitions in and out of groups.

For group-based project work.

4. Online Discussion Boards

Many instructors have tried to replace in-person discussions with asynchronous online discussion boards. In my experience, however, online discussion boards are best used in conjunction with synchronous discussion (via Zoom or in person). You can pick up points or concepts introduced in an online discussion and use them as jumping-off points for a synchronous discussion—giving credit to the students who raised them, of course. It’s a flow that I find leads to greater understanding of the material.

Advantages of online discussion boards:

They encourage student interaction.

Students can participate on their own time.

There’s generally no limit to the number of ideas students can contribute—meaning more students can participate in these discussions.

Shy students reluctant to engage in live sessions can build confidence with online contributions, especially if you pick up their points and credit them in synchronous discussions.

Disadvantages of online discussion boards:

Although instructors can drop comments and questions into online chat, it’s harder to actively guide and focus the discussions (because you’re not constantly there), so there’s no guarantee that students will arrive at the desired conclusions.

Multiple unrelated, branching discussions can occur at once, making things confusing or unfocused.

Students may not enjoy these types of discussions; they can feel forced or unnatural.

To start students thinking in a particular direction with the intention of bringing it all home in synchronous discussions.

To allow shy students opportunities to make contributions and gain confidence that may carry over into live sessions.

To surface ideas that I want to pick up on and add to in subsequent synchronous discussions.

5. Simulations

Simulations, like case studies, are a way to immerse students in a very specific experience—but with simulations, information is unfolding in real time. We can then ask students to do the work of extracting generalizable propositions, frameworks, theories, and so forth under our guidance.

Advantages of simulations:

They invite students to interact directly with the course material—and often each other—to solve the types of problems they may encounter in a real business environment.

Students have the opportunity to take direct control of their learning . They reach their own conclusions, then connect those learnings to framework material you present to rescue them from their struggle with it—to help them structure and systemize.

They have narrative elements and cause students to change their minds; students tend to remember lessons from simulations in much the same way they remember an impactful dramatic experience.

They give students experience in organizing and making meaning from information that arrives in real time and out of any helpful order.

Disadvantages of simulations:

They can take up a lot of time; in my view, the real learning from a simulation happens in a debrief and you need to take the time to distill out general lessons , especially when the models that underlie a simulation are complex.

Preparing a simulation for use can be effort intensive for instructors.

Very much in the same situations I use cases—when I want to present specific problems or situations from which I want students to derive general lessons.

To mix learning modes, as a break from and enhancement of cases.

Sometimes, in conjunction with cases, to show students that it can be harder than they think to “walk the talk”—to do what they said they would do in a case discussion when confronted with a problem unfolding in real time.

6. Multimedia Content

There’s also a lot of great multimedia content available—and this is yet another way to mix things up and shift modes to keep students interested. Using video elements in multimedia cases , for example, allows students see and hear case protagonists as opposed to just reading quotations.

Advantages of multimedia content:

Multimedia experiences offer a change of pace, and they’re often highly engaging.

Disadvantages of multimedia content:

They still don’t facilitate casual social interaction.

When I use it:

When I want to offer alternative modes for introducing problems or management situations, much like my use cases for simulations.

7. Curated Content

Many of us were using curated third-party content—anything from TED Talks to podcasts to YouTube tutorial videos—before the pandemic. But going virtual has prompted me to search around and use even more curated material. This kind of content can be used for a variety of desired outcomes: to help students explore case studies more deeply, for example, or to complete projects in virtual workspaces, such as Miro or Google Jamboard, for which students may need a how-to assist.

Advantages of curated content:

It’s often quite engaging, and much of it is very professionally done.

Once you have located good content, there is relatively little an instructor needs to do other than cue it up.

Disadvantages of curated content:

When you use too much of this type of content, students can think that you haven’t prepared for their specific needs.

Some content isn’t research based, or it can even put forth theoretical ideas that are unsupported or flawed. You must verify the quality of the content for yourself.

Pretty much anywhere—interwoven amid asynchronous edited video content or in synchronous classes, whether online or in person.

Pulling This All Together: An Example

The thought of putting all of these pieces—and there are a lot of them—together can feel like assembling a difficult puzzle. But by taking a fresh look at these technologies and thinking through how these use cases may support your course objectives, you can land on some really powerful learning experiences for your students.

Here is an example of how I tried to get the mix right for a course called Managing Innovation that I teach in Ivey’s Accelerated MBA program.

Robert D. Austin, “ Designing Better Courses: Blending the Best of Pre- and Post-Pandemic Pedagogy ,” Harvard Business Publishing Education, July 21, 2021. Accessed September 8, 2021.

To step through this in more detail, watch the video below to hear me talking though this sample implementation.

The New Normal of Teaching Includes Digital Tools

No matter how enticing it may be to return to your previous “normal”—a normal in which perhaps you didn’t incorporate all that many technologies or tools in your teaching—there are many benefits to virtual learning that are worth keeping, from better accessibility for all students to more opportunities for experiential learning that sticks.

By carefully considering the pros and cons of each available technology, you can choose the digital tools that will best support your lesson plans, making each stage of your course as effective and memorable for your students as possible.

TELL US WHAT YOU THINK: Do you use other technologies in your online, hybrid, or in-person courses that aren’t on this list? We want to hear from you. Email us at [email protected] .

Robert D. Austin is a professor of information systems at Ivey Business School and an affiliated faculty member at Harvard Medical School. He has published widely, authoring nine books, more than 50 cases and notes, three Harvard online products, and two popular massive open online courses (MOOCs) running on the Coursera platform.

Related Articles

We use cookies to understand how you use our site and to improve your experience, including personalizing content. Learn More . By continuing to use our site, you accept our use of cookies and revised Privacy Policy .

Education portal for digital productivity online

Home » Articles

Creating Inclusive Learning Environments with Technology

Ever heard the term ‘inclusive learning environments’? These are places where everyone feels like they belong, no matter who they are or how they learn…

Creating a Culture of Curiosity in the Classroom

Curiosity isn’t just a fleeting feeling; it’s the spark that ignites the fire of learning. Classrooms buzz with excitement as students eagerly explore new ideas,…

Mentoring in Education: Benefits and Practices

Have you ever thought about the incredible power of mentoring in education? Picture a world where every student, teacher, parent, and educational professional has access…

Supporting Educators’ Well-being: Mental Health at the Heart of Education

Teaching isn’t just about delivering lessons; it’s an intentional act of balancing academic rigor with emotional support. From juggling curriculum demands to navigating the complexities…

Future Skills: Preparing Students for the Fourth Industrial Revolution

There is a world in the near future where robots collaborate with humans, where data is the new currency, and where innovation is the name…

Entrepreneurship in the Educational Sector: Identifying Opportunities and Overcoming Challenges

From interactive apps to virtual classrooms, there’s a whole universe of digital tools transforming the way we teach and learn. But here’s the thing– behind…

Educational Leadership: Keys to Empathetic and Effective Management

In the dynamic realm of education, leadership isn’t just about directives; it’s about empathy-driven guidance that resonates across diverse stakeholders.  Imagine a digital landscape where…

Agile Methods in Education: Speeding Up Learning Without Sacrificing Quality

It is an open secret that traditional education has its flaws. The one-size-fits-all approach often leaves students feeling uninspired and teachers overwhelmed. But there is…

Become an Insider

Sign up today to receive premium content.

The Evolution of Technology in K–12 Classrooms: 1659 to Today

Alexander Huls is a Toronto-based writer whose work has appeared in  The New York Times ,  Popular Mechanics ,  Esquire ,  The Atlantic  and elsewhere.

In the 21st century, it can feel like advanced technology is changing the K–12 classroom in ways we’ve never seen before. But the truth is, technology and education have a long history of evolving together to dramatically change how students learn.

With more innovations surely headed our way, why not look back at how we got to where we are today, while looking forward to how educators can continue to integrate new technologies into their learning?

DISCOVER:  Special education departments explore advanced tech in their classrooms.

Using Technology in the K–12 Classroom: A History

1659: magic lantern.

• Inventor:  Christiaan Huygens
• A Brief History:  An ancestor of the slide projector, the magic lantern projected glass slides with light from oil lamps or candles. In the 1680s, the technology was brought to the education space to show detailed anatomical illustrations, which were difficult to sketch on a chalkboard.
• Interesting Fact:  Huygens initially regretted his creation, thinking it was too frivolous.

1795: Pencil

• Inventor:  Nicolas-Jacques Conté
• A Brief History : Versions of the pencil can be traced back hundreds of years, but what’s considered the modern pencil is credited to Conté, a scientist in Napoleon Bonaparte’s army. It made its impact on the classroom, however, when it began to be mass produced in the 1900s.
• Interesting Fact:  The Aztecs used a form of graphite pencil in the 13th century.

1801: Chalkboard

• Inventor:  James Pillans
• A Brief History:  Pillans — a headmaster at a high school in Edinburgh, Scotland — created the first front-of-class chalkboard, or “blackboard,” to better teach his students geography with large maps. Prior to his creation, educators worked with students on smaller, individual pieces of wood or slate. In the 1960s, the creation was upgraded to a green board, which became a familiar fixture in every classroom.
• Interesting Fact:  Before chalkboards were commercially manufactured, some were made do-it-yourself-style with ingredients like pine board, egg whites and charred potatoes.

1888: Ballpoint Pen

• Inventory:  John L. Loud
• A Brief History:  John L. Loud invented and patented the first ballpoint pen after seeking to create a tool that could write on leather. It was not a commercial success. Fifty years later, following the lapse of Loud’s patent, Hungarian journalist László Bíró invented a pen with a quick-drying special ink that wouldn’t smear thanks to a rolling ball in its nib.
• Interesting Fact:  When ballpoint pens debuted in the U.S., they were so popular that Gimbels, the department store selling them, made $81 million in today’s money within six months.

LEARN MORE:  Logitech Pen works with Chromebooks to combine digital and physical learning.

1950s: Overhead Projector

• Inventor:  Roger Appeldorn
• A Brief History:  Overhead projects were used during World War II for mission briefings. However, 3M employee Appeldorn is credited with creating not only a projectable transparent film, but also the overhead projectors that would find a home in classrooms for decades.
• Interesting Fact:  Appeldorn’s creation is the predecessor to today’s  bright and efficient laser projectors .

1959: Photocopier

• Inventor:  Chester Carlson
• A Brief History:  Because of his arthritis, patent attorney and inventor Carlson wanted to create a less painful alternative to making carbon copies. Between 1938 and 1947, working with The Haloid Photographic Company, Carlson perfected the process of electrophotography, which led to development of the first photocopy machines.
• Interesting Fact:  Haloid and Carlson named their photocopying process xerography, which means “dry writing” in Greek. Eventually, Haloid renamed its company (and its flagship product line) Xerox .

1967: Handheld Calculator

• Inventor:   Texas Instruments
• A Brief History:  As recounted in our  history of the calculator , Texas Instruments made calculators portable with a device that weighed 45 ounces and featured a small keyboard with 18 keys and a visual display of 12 decimal digits.
• Interesting Fact:  The original 1967 prototype of the device can be found in the Smithsonian Institution’s  National Museum of American History .

1981: The Osborne 1 Laptop

• Inventor:  Adam Osborne, Lee Felsenstein
• A Brief History:  Osborne, a computer book author, teamed up with computer engineer Felsenstein to create a portable computer that would appeal to general consumers. In the process, they provided the technological foundation that made modern one-to-one devices — like Chromebooks — a classroom staple.
• Interesting Fact:  At 24.5 pounds, the Osborne 1 was about as big and heavy as a sewing machine, earning it the current classification of a “luggable” computer, rather than a laptop.

1990: World Wide Web

• Inventor:  Tim Berners-Lee
• A Brief History:  In the late 1980s, British scientist Berners-Lee created the World Wide Web to enable information sharing between scientists and academics. It wasn’t long before the Web could connect anyone, anywhere to a wealth of information, and it was soon on its way to powering the modern classroom.
• Interesting Fact:  The first web server Berners-Lee created was so new, he had to put a sign on the computer that read, “This machine is a server. DO NOT POWER IT DOWN!”

Click the banner  to access customized K–12 technology content when you sign up as an Insider.

What Technology Is Used in Today’s K–12 Classrooms?

Technology has come so far that modern classrooms are more technologically advanced than many science labs were two decades ago. Students have access to digital textbooks,  personal devices , collaborative  cloud-based tools , and  interactive whiteboards . Emerging technologies now being introduced to K–12 classrooms include voice assistants, virtual reality devices and 3D printers.

Perhaps the most important thing about ed tech in K–12 isn’t what the technology is, but how it’s used.

How to Integrate Technology into K–12 Classrooms

The first step to integrating technology into the K–12 classroom is  figuring out which solution to integrate , given the large variety of tools available to educators. That variety comes with benefits — like the ability to align tech with district objectives and grade level — but also brings challenges.

“It’s difficult to know how to choose the appropriate digital tool or resource,” says Judi Harris, professor and Pavey Family Chair in Educational Technology at the William & Mary School of Education. “Teachers need some familiarity with the tools so that they understand the potential advantages and disadvantages.”

Judi Harris Professor and Pavey Family Chair in Educational Technology, William and Mary School of Education

K–12 IT leaders should also be careful not to focus too much on technology implementation at the expense of curriculum-based learning needs. “What districts need to ask themselves is not only whether they’re going to adopt a technology, but how they’re going to adopt it,” says Royce Kimmons, associate professor of instructional psychology and technology at Brigham Young University.

In other words, while emerging technologies may be exciting, acquiring them without proper consideration of their role in improving classroom learning will likely result in mixed student outcomes. For effective integration, educators should ask themselves, in what ways would the tech increase or support a student’s productivity and learning outcomes? How will it improve engagement?

Integrating ed tech also requires some practical know-how. “Teachers need to be comfortable and confident with the tools they ask students to use,” says Harris.

Professional development for new technologies is crucial, as are supportive IT teams, tech providers with generous onboarding programs and technology integration specialists. Harris also points to initiatives like YES: Youth and Educators Succeeding, a nonprofit organization that prepares students to act as resident experts and classroom IT support.

KEEP READING:  What is the continued importance of professional development in K–12 education?

But as educational technology is rolled out and integrated, it’s important to keep academic goals in sight. “We should never stop focusing on how to best understand and help the learner to achieve those learning objectives,” says Harris.

That should continue to be the case as the technology timeline unfolds, something Harris has witnessed firsthand during her four decades in the field. “It’s been an incredible thing to watch and to participate in,” she notes. “The great majority of teachers are extremely eager to learn and to do anything that will help their students learn better.”

• Professional Development

Related Articles

Unlock white papers, personalized recommendations and other premium content for an in-depth look at evolving IT

Copyright © 2024 CDW LLC 200 N. Milwaukee Avenue , Vernon Hills, IL 60061 Do Not Sell My Personal Information

Cellphones in Schools: A Huge Nuisance and a Powerful Teaching Tool

• Share article

When Nelann Taylor lets her high school students whip out their smartphones and dive into tools like Duolingo, Quizlet, Kahoot, and Flipgrid, she knows she may be in for a classroom management headache. Some of her students “have really figured out how to self-correct and just say, ‘Well, I know that I can’t be on my own phone right now’ ” unless it is for classwork, she said. But others take advantage of the freedom to start scrolling through text messages, and Taylor has to tell them put the devices away. Cellphones are both a powerful learning tool and huge distractions for kids. Figuring out how to make the most of them is “really tricky,” said Taylor, a fan of technology in the classroom who teaches high school Spanish and web design in Louisiana’s St. John the Baptist Parish Schools. “It’s always a work in progress.” Educators like Taylor have struggled with whether to ban phones, let kids use them for classwork, or some combination of the two for more than a decade. But the need to figure out how to use cellphones for learning, rather than letting them become a distraction, has gotten more urgent since kids returned from pandemic-driven virtual learning, experts and educators say. “I think the transition from trying to learn at home using devices and having perhaps multiple devices, being distracted by them, trying to focus attention on learning, and then transition back into the classroom has been really difficult,” said Christine Elgersma, the senior editor for social media and learning resources at Common Sense Media, a nonprofit organization that focuses on children, technology, and media. There are some good practices, including having a schoolwide policy on devices that’s clearly communicated to students and parents at the beginning of the school year. Being vehemently anti-cellphone may backfire, Elgersma warned. Allowing kids to use the devices for classwork is a way to acknowledge that, “these are really cool tools, and that some of what kids are doing on their phones is really impressive and creative and important to them,” she said. “We don’t want to discount how woven into the fabric of their lives these devices are.” At Kansas’ Springhill Middle School, students are expected to put their phones in their lockers as soon as school begins, and not take them out until the end of the day, unless a teacher plans to use the devices in a lesson, said Trevor Goertzen, the school’s principal. A National Association of Secondary School Principals digital principal of the year, Goertzen is a champion of tech in the classroom. But he thinks it’s too easy for kids to get distracted by entertainment or social media if they have access to their phones all day. All his students have MacBooks, he said, which can be used for just about any classroom activity requiring a device. Teachers have permission to allow cellphones occasionally for specific purposes, but “most teachers realize it’s not worth opening the door for them to use their phones.”

‘Teach kids to manage their technology’

But Stevie Frank, a 5th grade humanities teacher at Zionsville West Middle School in Whitestown, Ind., views cellphones as a great student engagement tool. Her students can keep their phones with them during class, as long as they have notifications turned off, so they’re not interrupted by a dinging noise. And she incorporates them into her class assignments. For instance, Frank sets up stations around the room where kids read passages and tackle questions on, say, an author’s purpose. To check to see if their answers are right, students use their phones to scan a QR code, and up pops a video of Frank explaining the correct answer. “It’s one of those things where I was like, ‘How can I be at 12 stations at once?’ ” Frank said. “And I’m like, ‘Wait a minute, I can!’ ” Frank’s students also use their phones to record podcasts, since they tend to have better microphones than school-issued devices do. Recently, for instance, she had groups of students choose books about different identities and then create a podcast exploring themes that the text raised. One group picked a book about a person experiencing homelessness and interviewed a staffer at a local shelter for their podcast. Naturally, there are times when students use their cellphones to go off task, Frank said. But that’s all part of the lesson. She said kids need to figure out how to voluntarily distance themselves from their devices. “You’ve got to teach the kids how to manage their technology and if we’re not going to do it in school, where’s it going to be done?” Frank said. A certified yoga teacher, she’s talked to her students about mindfulness, the importance of being present in the moment, and how technology can distract from those things. If a kid has a particularly tough time putting their phone away, or keeps getting distracted while using a school laptop, Frank will ask if they’d rather have a paper copy of the assignment, or if they’d like to put their phone on their desk. Giving students the choice to disengage from their phones helps “get their buy-in,” Frank said. “They’re like, ‘yup, I need to do that.’ ” Another advantage of using a phone for class assignments: Students are already familiar with how they operate, said Kristin Daley Conti, a science teacher at Tantasqua Regional Junior High School in central Massachusetts. Her attitude on cellphones in school is essentially, “if you can’t beat ‘em, join ‘em.” So if her students want to use their phones to, say, time how long it takes ice to melt, she’s fine with that. Many of her students also used the cameras on their phones for a project last year on ecosystems. Students chose an outdoor area near the school and took pictures of the spot once a week, then looked at how the biodiversity in its ecosystem changed over time. Students snapped photos of flowers, squirrels, plants, insects, frogs, and more and then shared them in a digital journal that was also accessible to parents. Daley Conti’s advice to teachers who are considering using cellphones in their classroom: Listen to kids’ ideas. Ask them questions like, “Do you think we’re using our phones too much?” or “Could we use our phones in class responsibly?” “If you’re thinking about incorporating cellphone use, hear from the experts,” she said.

A version of this article appeared in the March 23, 2022 edition of Education Week as Cellphones in Schools: Huge Nuisance And Powerful Teaching

Sign Up for EdWeek Tech Leader

Edweek top school jobs.

Sign Up & Sign In

You are using an outdated browser. Upgrade your browser today or install Google Chrome Frame to better experience this site.

• Professional learning

Teach. Learn. Grow.

Teach. learn. grow. the education blog.

75 digital tools and apps teachers can use to support formative assessment in the classroom

There is no shortage of  formative assessment strategies, techniques, and tools  available to teachers who use formative instructional practice in their classrooms. Here is an extensive list of 75 digital tools, apps, and platforms that can help you and your students use formative assessment to elicit evidence of learning. We didn’t just add any old tool to this list. Here are the criteria we used for those that made the cut:

• Supports formative instructional strategies and ways to activate learners to be resources for themselves and peers
• Is free or awful close to it (under $10 per year, where possible)
• Allows both students and teachers to take the activator role when possible (sometimes teachers need to get things started)

Before you dig into the tools, I encourage you to spend some time thinking about exactly what you want to accomplish with your students.  “How to pick the right digital tool: Start with your learning goal” by Erin Beard can help you wrap your head around goals and guide you in choosing the best tool for the task. “27 easy formative assessment strategies for gathering evidence of student learning” can help you decide what strategies work best for you and your students.

Record audio and video

• Animoto  Gives students the ability to make a 30-second video of what they learned in a lesson.
• AudioNote  A combination of a voice recorder and notepad, it captures both audio and notes for student collaboration.
• Edpuzzle  Helps you use video (your own, or one from Khan Academy, YouTube, and more) to track student understanding.
• Flip  Lets students use 15-second to 5-minute videos to respond to prompts. Teachers and peers can provide feedback.
• QuickVoice Recorder  Allows you to record classes, discussions, or audio for projects. Sync your recordings to your computer easily for use in presentations.
• Vocaroo  Creates audio recordings without the need for software. Embed the recording into slideshows, presentations, or websites.
• WeVideo Lets you use video creatively to engage students in learning. Teachers and students alike can make videos.

Create quizzes, polls, and surveys

• Crowdsignal  Lets you create online polls, quizzes, and questions. Students can use smartphones, tablets, and computers to provide their answers, and information can be culled for reports.
• Edulastic  Allows you to make standards-aligned assessments and get instant feedback.
• FreeOnlineSurveys  Helps you create surveys, quizzes, forms, and polls.
• Gimkit  Lets you write real-time quizzes. And it was designed by a high school student!
• Kahoot!  A game-based classroom response system that lets you create quizzes using internet content.
• MicroPoll  Helps you create polls, embed them into websites, and analyze responses.
• Naiku  Lets you write quizzes students can answer using their mobile devices.
• Obsurvey  Designed to make surveys, polls, and questionnaires.
• Poll Everywhere  Lets you create a feedback poll or ask questions and see results in real time. Allows students to respond in various ways. With open-ended questions, you can capture data and spin up tag clouds to aggregate responses.
• Poll Maker  Offers unique features, like allowing multiple answers to one question.
• ProProfs  Helps you make quizzes, polls, and surveys.
• Quia  Lets you create games, quizzes, surveys, and more. Access a database of existing quizzes from other educators.
• Quizalize  Helps you create quizzes and homework.
• Quizizz  Guides you through designing quizzes and lets you include students in the quiz-writing process.
• Quizlet  Lets you make flashcards, tests, quizzes, and study games that are mobile friendly.
• Survey Hero  Designed to build questionnaires and surveys.
• SurveyMonkey  Helpful for online polls and surveys.
• SurveyPlanet  Also helpful for online polls and surveys.
• Triventy  Lets you create quizzes students take in real time using individual devices.
• Yacapaca  Helps you write and assign quizzes.
• Zoho Survey  Allows you to make mobile-friendly surveys and see results in real time.

Brainstorm, mind map, and collaborate

• AnswerGarden  A tool for online brainstorming and collaboration.
• Coggle  A mind-mapping tool designed to help you understand student thinking.
• Conceptboard  Software that facilitates team collaboration in a visual format, similar to mind mapping but using visual and text inputs.
• Dotstorming  A whiteboard app that allows digital sticky notes to be posted and voted on. This tool is best for generating class discussion and brainstorming on different topics and questions.
• Educreations Whiteboard  A whiteboard app that lets students share what they know.
• iBrainstorm  Lets students collaborate on projects using a stylus or their finger.
• Miro  Allows whole-class collaboration in real time.
• Padlet  Provides a blank canvas for students to create and design collaborative projects.
• ShowMe Interactive Whiteboard  Another whiteboard tool to check understanding.
• XMind  Mind-mapping software for use on desktop computers and laptops.

Present, engage, and inspire

• BrainPOP Lets you use prerecorded videos on countless topics to shape your lesson plan, then use quizzes to see what stuck.
• Buncee  Helps students and teachers visualize, communicate, and engage with classroom concepts.
• Five Card Flickr  Uses the tag feature from photos in Flickr to foster visual thinking.
• PlayPosit  Allows you to add formative assessment features to a video from a library or popular sites, such as YouTube and Vimeo, to survey what students know about a topic.
• RabbleBrowser  Allows a leader to facilitate a collaborative browsing experience.
• Random Name/Word Picker  Facilitates random name picking. You can also add a list of keywords and use the tool to prompt students to guess words by providing definitions.
• Socrative  Uses exercises and games to engage students with a topic.
• Adobe Express  Lets you add graphics and visuals to exit tickets.
• Typeform  Helps you add graphical elements to polls.

Generate word or tag clouds

• EdWordle Generates word clouds from any entered text to help aggregate responses and facilitate discussion. Word clouds are pictures composed of a cloud of smaller words that form a clue to the topic.
• Tagxedo Allows you to examine student consensus and facilitate dialogues.
• Wordables Helps you elicit evidence of learning or determine background knowledge about a topic.
• WordArt Includes a feature that allows the user to make each word an active link to connect to websites, including YouTube.

Get real-time feedback

• Formative Lets you assign activities, receive results in real time, and provide immediate feedback.
• GoSoapBox Works with the bring-your-own-device model and includes an especially intriguing feature: a confusion meter.
• IXL Breaks down options by grade level and content area.
• Kaizena Gives students real-time feedback on work they upload. You can use a highlighter or give verbal feedback. You can also attach resources.
• Mentimeter Allows you to use mobile phones or tablets to vote on any question a teacher asks, increasing student engagement.
• Pear Deck Lets you plan and build interactive presentations that students can participate in via their smart device. It also offers unique question types.
• Plickers Allows you to collect real-time formative assessment data without the need for student devices.
• Quick Key Helps you with accurate marking, instant grading, and immediate feedback.

Foster family communication  

• Remind Lets you text students and stay in touch with families.
• Seesaw Helps you improve family communication and makes formative assessment easy, while students can use the platform to document their learning.
• Voxer Lets you send recordings so families can hear how their students are doing, students can chat about their work, and you can provide feedback.

Strengthen teacher-to-student or student-to-student communication

• Biblionasium Lets you view books students have read, create reading challenges, and track progress. Students can also review and recommend books to their peers.
• Classkick Helps you post assignments for students, and both you and your students’ peers can provide feedback. Students can also monitor their progress and work.
• ForAllRubrics Lets you import, create, and score rubrics on your tablet or smartphone. Collect data offline, compute scores automatically, and print or save the rubrics as a PDF or spreadsheet.
• Lino A virtual cork board of sticky notes, it lets students ask questions or make comments on their learning.
• Online Stopwatch Provides dozens of themed digital classroom timers to use during small- and whole-group discussions.
• Peergrade Helps you create assignments and upload rubrics. You can also anonymously assign peer review work. Students can upload and review work using the corresponding rubric.
• Spiral Gives you access to formative assessment feedback.
• Verso Lets you set up learning using a URL. Space is provided for directions. Students can add their assignment, post comments, and respond to comments. You can group responses and check engagement levels.
• VoiceThread Allows you to create and share conversations on documents, diagrams, videos, pictures, and more.

Keep the conversation going with live chats

• Yo Teach A backchannel site great for keeping the conversation going with students.
• Chatzy Supports live, online chats in a private setting.

Create and store documents or assignments

• Google Forms A Google Drive app that allows you to create documents students can collaborate on in real time using smartphones, tablets, and laptops.
• Piazza Lets you upload lectures, assignments, and homework; pose and respond to student questions; and poll students about class content. This tool is better suited for older students as it mimics post-secondary class instructional formats.

There are several resources for learning more about formative assessment and responsive instruction strategies. Consider our  formative practices workshops , where school and district teams can gain a better understanding of the role formative practice plays in instruction and the four foundational practices to use in the classroom. Or for a quick start, download our eBook  “Making it work: How formative assessment can supercharge your practice.”

Jump in, try new tools and methods, and have fun!

Recommended for you

27 easy formative assessment strategies for gathering evidence of student learning

What are classroom assessment standards, and how do they impact student learning?

3 tips for using data to drive instruction

Making it work: How formative assessment can supercharge your practice

Formative assessment isn’t new. But as our education system changes, our approaches to any instructional strategy must evolve. Learn how to put formative assessment to work in your classroom.

View the eBook

STAY CURRENT by subscribing to our newsletter

You are now signed up to receive our newsletter containing the latest news, blogs, and resources from nwea..

Advertisement

Artificial intelligence (AI) learning tools in K-12 education: A scoping review

• Open access
• Published: 06 January 2024

Cite this article

You have full access to this open access article

• Iris Heung Yue Yim   ORCID: orcid.org/0000-0002-5392-0092 1 &
• Jiahong Su   ORCID: orcid.org/0000-0002-9681-7677 2  

8261 Accesses

8 Altmetric

Explore all metrics

Artificial intelligence (AI) literacy is a global strategic objective in education. However, little is known about how AI should be taught. In this paper, 46 studies in academic conferences and journals are reviewed to investigate pedagogical strategies, learning tools, assessment methods in AI literacy education in K-12 contexts, and students’ learning outcomes. The investigation reveals that the promotion of AI literacy education has seen significant progress in the past two decades. This highlights that intelligent agents, including Google’s Teachable Machine, Learning ML, and Machine Learning for Kids, are age-appropriate tools for AI literacy education in K-12 contexts. Kindergarten students can benefit from learning tools such as PopBots, while software devices, such as Scratch and Python, which help to develop the computational thinking of AI algorithms, can be introduced to both primary and secondary schools. The research shows that project-based, human–computer collaborative learning and play- and game-based approaches, with constructivist methodologies, have been applied frequently in AI literacy education. Cognitive, affective, and behavioral learning outcomes, course satisfaction and soft skills acquisition have been reported. The paper informs educators of appropriate learning tools, pedagogical strategies, assessment methodologies in AI literacy education, and students’ learning outcomes. Research implications and future research directions within the K-12 context are also discussed.

Similar content being viewed by others

Artificial Intelligence in K-12 Education: eliciting and reflecting on Swedish teachers' understanding of AI and its implications for teaching & learning

Teaching and learning artificial intelligence: Insights from the literature

AI literacy in K-12: a systematic literature review

Avoid common mistakes on your manuscript.

Introduction

Artificial intelligence (AI) was defined in 1956 as “the science and engineering of creating intelligent machines” (McCarthy, 2004 , p.2). AI education is considered a driver of economic growth, future workforce development, and global competitiveness (Cetindamar et al., 2022 ; Sestino & De Mauro, 2022 ). Researchers’ interest in equipping students with AI knowledge, skills, and attitudes to thrive in an AI-rich future (Miao et al., 2021 ; Rina et al., 2022 ; Wang & Cheng, 2021 ) has given rise to the term “AI literacy”, which concerns the design and implementation of AI learning activities, learning tools and applications, and pedagogical models. Some educators focus on demonstrating machine learning through activities for mastering coding skills and AI concepts (Marques et al., 2020 ), while others suggest focusing on computational thinking and engagement in deductive and logical reasoning practices (Wong, 2020 ). In this paper, it is argued that AI education should be extended beyond universities to K-12 students.

There have been a number of recent studies of AI in the context of kindergartens (Su & Yang, 2022 ; Williams et al., 2019a , 2019b ), primary schools (Ali et al., 2019 ; Shamir & Levin, 2021 ), and secondary schools (Norouzi et al., 2020 ; Yoder et al., 2020 ). However, little is known about what and how AI should be taught (Su et al., 2023a ; Ng et al., 2023 ; Van Brummelen et al., 2021 ). One challenge is delivering AI content in an age-appropriate and effective manner (Su et al., 2023b ; Su & Yang, 2023 ). Despite the numerous AI learning tools available in K-12 contexts (Rizvi et al., 2023 ; Van Brummelen et al., 2021 ), such as Turtle Robot (Papert & Solomon, 1971 ), PopBots (Williams et al., 2019a ) and LearningML applications (Rodríguez-García et al., 2020 ), many educators are concerned about the suitability of these tools (Chiu & Chai, 2020 ; Su & Yang, 2023 ).

With the development of age-appropriate learning tools, AI concepts can be simplified via visual representation, such as block-based programming (Estevez et al., 2019 ). For example, Scratch, a high-level block-based programming language, allows students with limited reading ability to create computer programs by using illustrations and visual elements (such as icons and shapes) without having to rely on traditional written instructions (Park & Shin, 2021 ). AI tools and platforms, including Zhorai (Lin et al., 2020 ), Learning ML (Rodríguez-Garciá et al., 2021 ), Machine Learning for Kids (Sabuncuoglu, 2020 ), and Scratch (Li & Song, 2019 ), have a positive impact on students’ AI knowledge and skills. Chen et al. ( 2020 ) noted that despite the introduction of various learning tools to teach AI, there has not been enough discussion on how AI content should be taught and how tools should be used to support pedagogical strategies and related educational outcomes.

Theoretical model

The technology-based learning model of Hsu et al. ( 2012 ) is adopted and modified in this study; it has been widely used by other researchers conducting similar systematic reviews (Chang et al., 2018 , 2022 ; Darmawansah et al., 2023 ; Tu & Hwang, 2020 ), as shown in Fig. 1 . Hsu et al. ( 2012 ) suggested cross-analyzing academic research trends by examining the associations among three categories: research methods, research issues, and application domains. They argue, for example, that by exploring how the topic of a study may affect the selection of its sample and participants, a more thorough and comprehensive analysis can be conducted. Their proposed technology-based learning model has helped frame the research questions of the present study.

Modified technology-based learning model by the researchers of this review (adopted from Hsu et al., 2012 )

According to Hsu et al. ( 2012 ), “research methods”, “research issues”, and “application domains” are the three main categories to be considered in the development of a coding scheme to gauge research trends in the field of technology-based learning and education. In terms of research methods, a quantitative, qualitative, and mixed approach is employed in this study to construct the coding scheme for the review of the literature (McMillan & Schumacher, 2010 ). In terms of research issues, with reference to Chang et al. ( 2018 ), learning outcomes are categorized as cognitive, affective, behavioral, and skills acquisition outcomes. Finally, two application domains are pursued in this paper: (1) the pedagogical strategies commonly used in science courses, which were employed by Lai and Hwang ( 2015 ) and which include constructive, reflective, didactic, and unplugged pedagogies (Cope & Kalantzis, 2016 ), and (2) the learning tools, namely, hardware, software, intelligent agents, and unplugged strategies, which are coded as suggested by Ng and Chu ( 2021 ).

Research objectives

In this study, the literature on pedagogical strategies, assessment methods, learning tools, and learning outcomes in AI K-12 settings is studied. Four research questions are formulated.

RQ1: What are the potential learning tools identified in AI K-12 education?

Rq2: what pedagogical strategies are commonly proposed by studies on ai k-12 learning tools, rq3: what learning outcomes have been demonstrated in studies on ai k-12 learning tools.

RQ4: What are the research and assessment methods used in studies on AI K-12 learning tools?

This study follows the same four steps employed in other studies on AI literacy in K-12 (e.g., Ng et al., 2022 ; Su et al., 2022 ): (1) identifying relevant studies, (2) selecting and excluding eligible studies, (3) data analysis, and (4) reporting findings. In this study, the preferred reporting items for systematic reviews and meta-analyses (PRISMA) guidelines (Moher et al., 2015 ) are followed.

Identifying relevant studies

The electronic databases used for the literature search were ACM, EBSCO, Web of Science, and Scopus. The aim of this review is to provide a comprehensive K-12 education for learning tools, encompassing early childhood education and primary and secondary education. As the education systems of different countries may differ from each other, the search string used in the paper for K-12 includes from kindergarten to secondary school students. In addition, learning tools are defined as a variety of learning platforms and systems, educational applications and activities that can enhance the teaching process and support students in AI literacy learning. Therefore, the search strings are reflected with specific definitions for K-12 and learning tools to search for target articles and data, as shown in Table 1 .

Study selection and exclusion

To ensure the generalizability of the findings and to avoid biases in article selection, specific inclusion and exclusion criteria are employed in this study (Table  2 ).

As shown in Fig.  2 , a total of 326 articles were identified, 105 from EBSCO, 81 from Web of Science, 110 from Scopus, and 30 from ACM. The exclusion criteria were as follows: (1) studies that were irrelevant to the research topic (N = 251). For example, Bai and Yang ( 2019 ) were excluded since the research applied a deep learning technology recommendation system to improve teachers’ information technology ability. It was conducted in contexts other than those of AI literacy education, learning and instruction. Mahon et al. ( 2022 ) presented the design of an online machine learning and artificial intelligence course for secondary school students; however, they did not discuss in detail what type of learning tools can be used and how to support students’ AI literacy learning. A discussion paper by Karalekas et al. ( 2023 ), a theoretical paper by Leitner et al. ( 2023 ) and a scoping review by Marques et al. ( 2020 ) were also removed because they were not empirical studies, and they did not involve conducting any practical experiment. (2) Duplicate studies (N = 10), (3) studies that were not written in English (N = 4), (4) non research studies (N = 10), and other types of articles (N = 8). Finally, 46 studies were selected, as shown in Appendix 1 .

PRISMA diagram of included articles in the scoping review

The snowball method

To enhance the systematic search for relevant literature, the snowballing method as outlined by Sayers ( 2008 ) was employed. This involved tracing references in previously selected articles. The focus was on the references cited in the earlier selected articles as discovered through Google Scholar. Utilizing the snowballing method led to the identification of three additional articles that met the eligibility criteria described above.

Overview of selected studies

Table 3 presents an overview of the 46 selected studies, including the type of articles, year of publication, and educational level.

Publication trends

Forty-six articles were identified: 28 conference papers and 18 journal articles. The first article was published in 1995, and 39 articles have been published in the past 5 years, with a peak in 2021 (Fig. 3 ).

The trend of AI literacy education in K-12 contexts

Most research took place in the USA (N = 8), China (N = 7), Finland (N = 3), Hong Kong (N = 3), Israel (N = 3), Spain (N = 3), Australia (N = 2) and Japan (N=2). Others were conducted in Brazil, Denmark, Greece, Indonesia, New Zealand, Norway, Sweden, Thailand, and the UK. The locations of the remaining six articles are unknown.

Educational levels

Primary and secondary schools are both the most researched educational levels, each covering 44% of the selected articles, followed by kindergartens (11%) and K-12 education (2%).

These selected studies generally include samples of students of both genders and a wide range of ages, from 3-year-old kindergarten students (Vartiainen et al., 2020 ) to 20-year-old Danish high school students (Kaspersen et al., 2021 ). It also encompasses participants in science technology engineering mathematics (STEM) classes (Ho et al., 2019 ), high-performing students of the Scientists in School program (Heinze et al., 2010 ), students with and without an AI background (Yoder et al., 2020 ), and students from varying socioeconomic backgrounds (Kaspersen et al., 2021 ).

There were three AI-related research studies between 1995 and 2017, mostly adopting unplugged activities and games for AI teaching, which are different from research conducted after 2017. The first article was published by Scherz and Haberman ( 1995 ), who designed a special AI curriculum with the use of abstract data types and instructional models (e.g., graphs and decision trees) to teach AI concepts such as logic programming and AI systems to high school students in Israel. In another two studies, the use of programming robots (Heinze et al., 2010 ) and computer science unplugged activities (Lucas, 2009 ) were explored with Australian and New Zealand K-6 students, respectively. Since then, a greater variety of learning tools have been employed and expanded to European and Asian countries across all educational levels in K-12 settings. Appendix 1 provides an overview of the selected articles.

The potential learning tools identified in K-12 contexts were intelligent agents (N = 20), software-focused devices (N = 19), hardware-focused devices (N = 10), and unplugged activities (N = 6) (Fig. 4 and Table 4 ). In this section, intelligent agents, software devices, and hardware devices are discussed.

Summary of learning tools used in AI K-12 education

Intelligent agents

Intelligent agents, such as Google Teachable Machine, Learning ML, and Machine Learning for Kids, which make decisions based on environmental inputs by using their sensors and actuators, are the most popular learning tools for enhancing students’ computational thinking skills within K-12 contexts. Teachable Machine is a web-based tool developed by Google and is found to be more effective than are unplugged activities in kindergarten settings (Lucas, 2009 ; Vartiainen et al., 2020 ). In Vartiainen et al. ( 2020 ), children aged between 3 and 9 autonomously explored the input‒output relationship with Google Teachable Machine, which fostered their intellectual curiosity, developed their computational thinking, and enhanced their understanding of machine learning. In both primary (Toivonen et al., 2020 ; Melsión et al., 2021 ) and secondary schools (Kilhoffer et al., 2023 ; Martins et al., 2023 ), Google Teachable Machine has been employed, allowing students to use their webcams, images, or sounds without coding to develop their own machine learning classification models.

In addition, Learning ML has been employed for primary schools to create AI-driven solutions and models, for example, to teach the supervised machine learning principle (Voulgari et al., 2021 ; Rodríguez-Garciá et al., 2021 ), which simplifies abstract AI algorithms for primary school students. Machine Learning for Kids, which introduces the power of the IBM Watson engine for AI modelling (Fernández-Martínez et al., 2021 ), Cognimates (Sabuncuoglu, 2020 ; Fernández-Martínez et al., 2021 ), which allows students to practice coding, and Ecraft2Learn, which contains a deep learning functionality (Kahn et al., 2018 ), have also been used in secondary school classrooms. Intelligent agents often offer students hands-on experience to develop datasets and to build customized machine learning systems.

Software devices

Software devices are adopted to enable mostly primary and secondary school students to learn about computational thinking, including programming for sequences, rule-based and conditional mechanisms, as well as data science and machine learning using visual language. For example, Scratch, a block-based programming software, is frequently used in both primary (Dai et al., 2023 ; Li & Song, 2019 ; Shamir & Levin, 2021 ) and secondary schools (Estevez et al., 2019 ; Fernández-Martínez et al., 2021 ). Other software is used for visualizing and scaffolding abstract AI concepts through online games and experiences, such as Quick and Draw (Martins et al., 2023 ) and Music Box (Han et al., 2018 ). In primary schools, Kitten is used to teach block-based programming (Li & Song, 2019 ), whereas C++ and JavaScript are used for logical thinking and simulation (Gong et al., 2020 ). In secondary schools, researchers have often employed free online software and tools, such as Snap (Yoder et al., 2020 ) and Python (Gong et al., 2018 ; Norouzi et al., 2020 ), for algorithm automation, as well as RapidMiner for no-code data science learning (Sakulkueakulsuk et al., 2018 ). To introduce machine learning concepts to secondary school students, other researchers have focused on developing online games such as the Rock Paper Game (Kajiwara et al., 2023 ) and the 3D role-player video game Quest (Priya et al., 2022 ).

Hardware devices

In addition, hardware, such as robotics and physical artifacts, has also been used with built-in software to supplement students’ understanding of AI concepts. Williams et al. ( 2019a , 2019b ) introduced a preschool originated programming platform consisting of a social robot (PopBot) and a block-based programming interface. In Williams et al. ( 2019a ), 80 prekindergartens to second-grade children (aged four to seven) were asked to build their own LEGO robot characters by using DUPLO block programming. PopBot is used as a learning companion to demonstrate its human-like behavior and to demystify AI concepts to younger students.

The lawn bowling robot (Ho et al., 2019 ), Zhorai conversational robot (Lin et al., 2020 ), Micro: Bits (Lin et al., 2021 ), and Plush toys (Tseng et al., 2021 ) have been used in primary schools, while CUHKiCar (Chiu et al., 2021 ), the Alpha robot dog (Chai et al., 2020 ), Raspberry Pi Raspbian and a four-wheel drive chassis (Gong et al., 2018 ) have been used in secondary schools. For example, in Ho et al. ( 2019 ), grade six students built lawn-bowling robots for games and competitions while learning about the binary search and optimization algorithms of machine learning. Chiu et al. ( 2021 ) introduced the robotic CUHKiCar to secondary school students so that they could perform face-tracking and line following tasks.

As shown in Fig. 5 , the four orientations of pedagogy are summarized as authentic/constructive, reflective, didactic, and unplugged. While a total of 17 potential pedagogical strategies were identified within the four orientations in K-12 contexts (Table 5 ), authentic/constructive methodologies with project-based learning (N = 27) were the most popular pedagogy used across kindergartens (Williams et al., 2019a , 2019b ), primary schools (Toivonen et al., 2020 ; Rodríguez-Garciá et al., 2021 ), and secondary schools (Gong et al., 2018 ; Kilhoffer et al., 2023 ; Sakulkueakulsuk et al., 2018 ). When teaching AI to students with a diverse range of needs, the evidence demonstrates the positive impact of combining multiple pedagogical approaches in K-12 studies (Heinze et al., 2010 ; Lee et al., 2021 ; Williams et al., 2019a , 2019b ).

Four orientations of pedagogical strategies commonly used in AI K-12 education

First, authentic and constructive methodologies, project-based (N = 27), human-computer interaction (N = 7), and play-based active learning (N = 5) approaches have been commonly used in K-12 education. Offering hands-on opportunities to students to learn about real-world applications of AI is an example of project-based learning (Fernández-Martínez et al., 2021 ; Han et al., 2018 ; Williams et al., 2019a ). Other researchers have examined whether students can acquire AI knowledge on human-computer interactive experiences and have found that this does not require any prior knowledge of AI models, such as Zohari (Melsión et al., 2021 ) and Google Teachable Machine (Lin et al., 2020 ; Vartiainen et al., 2020 ). In addition, child-centered play-based learning can effectively engage students and encourage them to take the initiative to construct knowledge during the process of imaginative play (Heinze et al., 2010 ), which involves students adopting the role of AI developer, tester, and AI robot (Henry et al., 2021 ).

Pedagogical strategies in kindergartens

Researchers have often used project-based approaches (N = 3), human-computer interactions (N = 3), play-based learning (N = 1), and unplugged activities (N = 1) to teach younger students AI concepts. In a project-based learning approach, students learn by actively engaging in real-world projects. Williams et al. ( 2019a , 2019b ) used a hands-on project allowing prekindergarten and kindergarten students to acquire AI concepts, including knowledge-based systems, supervised machine learning, and AI generative music. Alternatively, Vartiainen et al. ( 2020 ) studied human-computer interactions that allowed students to freely explore the input‒output relationship with Google Teachable Machine to identify and to evaluate a problem and find a solution to it. Heinze et al. ( 2010 ) focused on imaginative play, which is relevant to young students, as play is associated with various levels of autonomy and provides an engaging introduction to AI and the formation of scientific concepts. Lucas ( 2009 ) used unplugged activities to teach the key concepts of computing, including data encoding, data compression, and error detection.

Pedagogical strategies in primary schools

Project-based learning is more frequently used in primary schools than in kindergartens: It has been reported as a learning approach in 14 of the 18 studies of primary school settings, compared to only three of the five studies in the kindergarten setting. Similarly, in primary school settings, studies have revealed a strong dependence on play/game-based (N = 5) and human-computer interaction learning approaches (N = 3).

Projects that demonstrate students’ improved AI knowledge have been conducted. Machine learning projects (Toivonen et al., 2020 ), LearningML projects (Rodríguez-Garciá et al., 2021 ), and “AI+” projects (Han et al., 2018 ) have been designed to demystify AI knowledge. Henry et al. ( 2021 ) integrated machine learning in role-playing games, while Shamir and Levin ( 2021 ) allowed students to play with AI chatbots to develop AI models and to construct a rule-based machine-learning system. Some researchers have designed learning programs that offer human-computer interaction activities to educate students about gender bias (Melsión et al., 2021 ) and the social impact of mistakes made by AI models in training datasets (Lin et al., 2020 ).

Pedagogical strategies in secondary schools

The project-based learning approach (N = 10) is also the most dominant in secondary schools, followed by collaborative learning (N = 5). First, project-based learning is used to engage students by applying their AI knowledge to solve real-world problems. Teachers have reported that AI projects and hands-on activities are effective in keeping students focused on tasks (Kilhoffer et al., 2023 ). For example, a smart car-themed AI project (Gong et al., 2018 ), the Redesign YouTube project (Fernández-Martínez et al., 2021 ), and the agriculture-based AI Challenge project (Sakulkueakulsuk et al., 2018 ) have been introduced to provide hands-on experience for students to connect their knowledge to their day-to-day lives. Through active exploration, such projects prompt secondary school students to contemplate the personal, social, economic, and ethical consequences of AI technologies (Kaspersen et al., 2021 ).

Second, collaborative learning allows students to work in groups to promote cognitive knowledge, as it engages them in scientific inquiry with the help of smart devices (Wan et al., 2020 ). Kaspersen et al. ( 2021 ) designed a collaborative learning tool, VotestratesML, together with a voting project allowing students to build machine learning models based on real-world voting data to predict results.

Of the 46 articles, 31 reported potential learning outcomes: (1) cognitive outcomes, (2) affective and behavioral outcomes, and (3) the level of course satisfaction and soft skills acquisition.

Cognitive outcomes

Thirty-one studies documented various degrees of positive cognitive outcomes. Students generally showed a basic understanding of AI, including AI rule-based systems (Ho et al., 2019 ), machine learning principles and applications (Han et al., 2018 ; Shamir & Levin, 2021 ), AI ethics (Melsión et al., 2021 ), and AI limitations (Lin et al., 2020 ). In Williams et al. ( 2019a ), 70% of prekindergarten and kindergarten students understood knowledge-based systems, whereas Vartiainen et al. ( 2020 ) found that, through AI learning tools, younger students developed their computational thinking and their understanding of machine-learning principles and applications. Then, Dai et al. ( 2023 ) reported that primary school students taught with analogy-based pedagogy (i.e., using humans as a reference to teach and learn AI) significantly outperformed primary school students taught with the conventional direct instructional approach in terms of developing their conceptual understanding and increasing their AI technical knowledge proficiency as well as their ethical awareness of AI. Other researchers have argued that primary school students have demonstrated their understanding of AI by constructing and applying machine-learning algorithms with the help of digital role-playing games (Voulgari et al., 2021 ) and project-based pedagogy (Shamir & Levin, 2021 ). Through designing and programming a robot, students increased their understanding of AI biases (Melsión et al., 2021 ). In secondary schools, researchers have also reported an increase in students’ knowledge of AI algorithms (Yoder et al., 2020 ) and machine learning concepts (Sakulkueakulsuk et al., 2018 ), as well as their recognition of AI patterns (Wan et al., 2020 ). For example, students understood the fundamental neural networks of machine learning concepts by developing a classification model of recycling images (Martins et al., 2023 ).

Affective and behavioral outcomes

Affective and behavioral outcomes have been identified in AI learning tool studies within K-12 contexts. In general, students’ motivation to learn AI (Han et al., 2018 ; Shamir & Levin, 2021 , 2022 ) and their interest in the course (Mariescu-Istodor & Jormanainen, 2019 ; Martins et al., 2023 ) were enhanced as a result of AI learning activities. Students’ perceptions of the relevance of AI to their life also increased (Kajiwara et al., 2023 ; Lin et al., 2021 ). Students scored high on self-efficacy (Kajiwara et al., 2023 ; Shamir & Levin, 2022 ) and confidence (Shamir & Levin, 2021 ) in training and validating an AI system. In Martins et al. ( 2023 ), over 45% of 108 secondary school student participants in the introductory course “Machine Learning for all” reported that they perceived AI learning as an enjoyable experience, and 63% of them hoped to learn more about machine learning in the future.

Moreover, students reported that they were highly motivated to explore the Teachable Machine (Vartiainen et al., 2020 ), to design the robotic arm and computer source codes (Ho et al., 2019 ), to draw animals and sea creatures for the machine learning project (Mariescu-Istodor & Jormanainen, 2019 ), and to predict the sweetness of mangoes by using machine learning models (Sakulkueakulsuk et al., 2018 ).

From the behavioral perspective, high student engagement was reported in project-based (Kaspersen et al., 2021 ; Shamir & Levin, 2021 ; Wan et al., 2020 ) and play/game-based (Heinze et al., 2010 ; Voulgari et al., 2021 ) settings. Primary students attended all sessions and expressed a desire to join an upcoming AI contingency course (Shamir & Levin, 2021 ), while secondary students were actively engaged in scientific inquiry (Wan et al., 2020 ). Students were also keen on recommending AI games to their friends (Voulgari et al., 2021 ). Therefore, a combination of play/game-based and project-based approaches may consolidate AI concepts through gameplay while enhancing students’ engagement in AI projects (Han et al., 2018 ).

Level of satisfaction and soft skills acquisition

Students’ level of satisfaction was found to be positively influenced by constructivist (e.g., project-based) and reflective (e.g., learning by design and learning by teaching) pedagogies (Ho et al., 2019 ; Shamir & Levin, 2021 , 2022 ). In Lin et al. ( 2020 ), students reported a high satisfaction level upon acquiring AI knowledge. Their computational thinking and subsequent project performance were also enhanced. All students completed the course and their AI tasks without any previous learning experience (Toivonen et al., 2020 ).

The findings from the selected articles reveal that a deep understanding of AI promotes the acquisition of various soft skills. Ali et al. ( 2019 ) found that students’ intellectual curiosity increased after engaging in the construction of an AI neuron. By using bulletin boards shared electronically and online chats for feedback, their collaboration and communication skills were also enhanced (Shamir & Levin, 2021 ). Moreover, students reported gaining problem solving and technical skills when working with AI systems, including coding, designing simple algorithms, and debugging in Scratch learning activities (Dai et al., 2023 ).

RQ4: What were the research and assessment methods used in AI K-12 learning tools studies?

In this section, an overview is presented of research methods and data collection procedures within K-12 contexts. Overall, researchers adopted a mixed method (N = 19), qualitative (N = 15) and quantitative methods (N = 12) in AI learning tools in K-12 research. Mixed methods are predominantly used in both primary school (e.g., Dai et al., 2023 ; Martins et al., 2023 ; Shamir & Levin, 2021 ; Toivonen et al., 2020 ) and secondary school contexts (e.g., Chiu et al., 2021 ; Estevez et al., 2019 ), whereas qualitative methods are commonly used in kindergartens (e.g., Heinze et al., 2010 ; Vartiainen et al., 2020 ), as shown in Table 6 .

A variety of assessment methods were used: questionnaires and surveys (N = 30), artifacts/performance-based evaluation (N = 15), interviews (N = 14), observations (N = 5), games assessment (N = 1), and field visits (N = 1) (Table 7 ). The two most commonly used data collection methods - questionnaires and surveys and artifacts/performance-based evaluation - are discussed in this section.

In terms of assessment methods, questionnaires and surveys (N = 30) and artifacts/performance-based evaluation (N = 17) are the two most commonly used data collection methods across K-12 contexts (Table  7 ).

Questionnaires and surveys are used in a quantitative methodology to understand the perception of robotics and theory of mind (e.g., knowledge access, content false belief and explicit false belief). For example, perception of robotics and theory of mind were used in kindergartens (Williams et al., 2019a , 2019b ).

Surveys were used to evaluate primary school students’ motivation (Lin et al., 2021 ), self-efficacy in AI learning (Shamir & Levin, 2022 ), and perceived knowledge and competence (Dai et al., 2023 ; Mariescu-Istodor & Jormanainen, 2019 ; Ng et al., 2022 ). In addition to Ali et al. ( 2019 ), who used the Torrance test for assessment, researchers also utilized pre- and posttests (Tseng et al., 2021 ) to compare the AI learning outcomes of control and treatment groups in primary school settings (Melsión et al., 2021 ). Others provided AI educational experience without stating the assessment method (Ho et al., 2019 ; Lee et al., 2020 ; Tseng et al., 2021 ). Heinze et al. ( 2010 ) conducted AI learning activities without assessing learning outcomes. Shamir and Levin ( 2022 ) designed a questionnaire based on “constructionist validated robotics learning” for machine learning construction (the questionnaire included statements such as " I can make a ML system ", " I can propose ideas for using ML to solve problems ."). Dai et al. ( 2023 ) used multiple choice questions (e.g., " Which of the following devices or systems is an intelligent agent? ") to evaluate the AI knowledge of primary school students according to Bloom’s Taxonomy.

In secondary schools, surveys are used to measure students’ information knowledge acquisition (Priya et al., 2022 ), perceived abilities (Chiu et al., 2021 ; Ng & Chu, 2021 ) and futuristic thinking, engagement, interactivity, and interdisciplinary thinking skills (Sakulkueakulsuk et al., 2018 ). For example, in Priya et al. ( 2022 ), surveys were used in the first phase of their study to test the knowledge gained by students in three AI areas, namely, supervised learning (e.g., " What is the underlying idea behind supervised learning ?"), gradient descent (e.g., " In gradient descent how do we reach optimum point? "), and KNN classifications (e.g., " Using underlying principle of KNN classification classify a fruit which is surrounded by 2 apples and 1 mango in its nearest neighbors. "). In the second phase of the study, surveys were used to evaluate students’ satisfaction with the design of the game “ML Quest”, which introduced machine learning concepts based on the quality factors of the technological acceptance model (e.g., “Visualizations displayed by ML-Game are relevant to the concept taught at each level” ).

Artifact-based/performance-based assessments are embedded in a large number of studies to evaluate learning outcomes. Through artifacts (e.g., Popbots), Williams et al., 2019a , 2019b ) evaluated kindergarteners’ knowledge and understanding of supervised machine learning. Ho et al. ( 2019 ) used a performance-based assessment to assess primary students’ understanding of optimal data training and its AI applications. The artifact analysis of Shamir and Levin ( 2021 ) involved the construction of a rule-based AI system, which included designing, understanding, and creating the AI neural network agent. Dai et al. ( 2023 ) used a drawing assessment to evaluate primary school students’ understanding of AI and its impact on their cognitive development using prompt questions (e.g., " What AI can do? What would you like to use AI for? ") to stimulate their thinking.

Moreover, Yoder et al. ( 2020 ) focused on secondary school students’ block-based programming artifacts to examine their knowledge of AI search algorithms and breadth-first search (BFS), as well as their understanding of the possibility of gender bias when using AI screening tools in recruitment. In Martins et al. ( 2023 ), machine learning model artifacts created by students were used as evidence to demonstrate their learning outcomes. The performance-based assessment was used to evaluate students’ ability to correctly label the recycling trash images in the classification process.

Discussion and conclusion

The results of this study are consistent with Kandlhofer et al. ( 2016 ), who found that a variety of learning tools have been designed to support various learning objectives for students from kindergarten to university. The previous literature also indicates that many learning tools, such as intelligent agents and software, are effective in facilitating adolescents’ and university students’ acquisition of computational thinking skills (Çakiroğlu et al., 2018 ; Van Brummelen et al., 2021 ), whereas the availability of such tools for kindergarten and primary students is often overlooked. Few researchers have investigated whether AI learning tools can bridge the learning gap of younger students (Zhou et al., 2020 ). This study revealed that without prior programming experience, these learning tools (such as Popbots, Teachable Machine, and Scratch) can help address the diverse needs of younger students across K-12 educational levels (Resnick et al., 2005 ), leading to a richer visual learning experience and improving instructional quality (Kaspersen et al., 2021 ; Long & Magerko, 2020 ).

Previous reviews have indicated that many pedagogies are suitable in AI education, although this was done without reference to students' learning outcomes (Sanui & Oyelere, 2020 ). The findings of this study enrich existing knowledge of the positive effects of authentic and constructivist pedagogies in affective, behavioral, and cognitive aspects, as well as students’ level of satisfaction in AI learning. This study reveals that multiple pedagogies, such as project-based learning, experiential learning, game-based learning, collaborative learning, and human–computer interaction, are widely used in K-12 educational settings. An emerging form of analogy-based pedagogy to evaluate the AI knowledge of primary school students by assessing their drawings is identified. The focus of this analogy-based pedagogical strategy is the comparison of humans and AI, where humans are gradually moved from an analogy and to a contrast to highlight the characteristics, mechanism, and learning procedures of AI. It demonstrates and reflects the dialogic quality of the relationship with shared enquiry and shared thinking among students and AI learning tools. This is significant given the new cognitive demand of the AI era, as it provokes a shift in the role of the students by thinking together and learning to learn together (Wegerif, 2011 ). In future studies, exploration of additional emerging pedagogies (Yim, 2023 ), the co-creation of arts-based possibility spaces (Burnard et al., 2022 ), and dialogic learning spaces (Wegerif, 2007 ) in AI literacy education can be considered.

In addition, educational tools and applications are used not only to contribute new ways of knowing and doing but also to embed learning tools at the center of the AI literacy activities and programs instead of playing a supporting role in the primary purpose of education. This is expanding to serve the human need for education. The use of multiple educational learning tools and pedagogical strategies may be influenced by various factors in the teaching process, including students’ gender, background knowledge, and educational setting, all of which may affect their learning styles and motivation to learn AI. These factors and issues can be explored in future studies.

In this review, it was found that some studies assessed students’ performance by using the Torrance test for creativity (Ali et al., 2019 ), an AI knowledge test (Ng et al., 2022 ; Wan et al., 2020 ), pre- and postsurveys (Chiu et al., 2021 ; Estevez et al., 2019 ), and comparisons between control and treatment groups (Dai et al., 2023 ; Melsión et al., 2021 ), while others used subjective measures, including self-report surveys. Although artifact-based and performance-based approaches have been increasingly adopted in data collection procedures, some researchers used them as evidence of learning, without scoring according to established marking criteria for assessment purposes. There is room for introducing objective and rubric-based evaluation mechanisms to assess the quality of suggested methodologies. However, the lack of agreement on assessment criteria and instructional feedback shows that further research is needed to support the wide application of AI teaching in K-12 classrooms.

Research implications

From this study, the use of intelligent agents is recommended, including Teachable Machines, Machine Learning for Kids, and Learning for ML. Kindergarten students can benefit from learning tools such as PopBots, while software devices such as Scratch and Python can be introduced to demystify core AI principles to primary school students and create AI-driven solutions and models for secondary school students. Although hardware such as robotics and physical artifacts are generally effective, they may be costly for scalability.

This review reveals that constructivism, constructionism, and computational thinking are instrumental in addressing AI literacy education. Unfortunately, little research has adopted theoretical frameworks or conceptual models of reference for AI curricula, educational activities, or the design of AI learning tools and applications. To guide teaching, learning and effectiveness in using AI learning tools within AI literacy education, AI literacy learning theoretical frameworks are needed to guide the teaching instruction of kindergarten, primary and secondary school students. Usability, AI ethics, and transparency must be addressed in tool design to ensure that issues pertaining to data privacy and security will not arise. Moreover, there is currently insufficient theory-based, rigorous research on the effectiveness of AI educational tools to meet the diverse learning needs of students. Children may be invited to codesign with application designers. Thus, researchers may conduct theory-based and outcome-oriented quantitative and qualitative research on AI educational tools, which may be significantly beneficial to students.

More evaluation and documented analysis regarding the effectiveness of learning tools should be conducted to inform stakeholders of the existing trends in the field, pedagogical strategies, and instructional methods for teacher professional development.

More research, analysis, and evidence are needed to determine the effectiveness of AI learning tools before they are scaled up based on a risk-benefit analysis. Researchers should also clearly define the educational settings in which specific AI learning tools are appropriate to support the effective delivery of AI content in the classroom.

Recommendations

For educators.

Aside from providing students with AI knowledge and skills that the market demands (Burgsteiner et al., 2016 ) and encouraging all citizens to be AI literate (Goel, 2017 ; Pedro et al., 2019 ), educators may promote holistic AI literacy education by considering humans, nonhumans (e.g., animals and machines) (Yim, 2023 ) and environmental elements (Miao & Shiohira, 2022 ) in their teaching content. Ethical questions should also be considered, including inclusivity, fairness, responsibility, transparency, data justice, and social responsibility (Crawford, 2021 ; Benjamin, 2019 ). To provide a roadmap for sustainable AI education implementation and development, it is essential to involve teachers in the design of learning tools and understand their perceptions regarding AI literacy education, as well as provide pedagogical strategies, resource development, and needs-based professional training for both preservice and in-service teachers.

For teachers

Children learn best at a certain stage of cognitive development (Ghazi & Ullah, 2015 ). It is recommended that the content of instruction is consistent with students’ cognitive developmental level, as it influences their readiness and ability to learn (Piaget, 2000 ). As a result, the technical and content depth of the educational learning tools should align with students’ age and the teaching objectives, and teachers should understand students’ cognitive development to plan age-appropriate activities with suitable learning tools. More collaboration among teachers with various pedagogical experiences across various educational levels may lead to more innovative and efficient teaching processes.

For researchers

Researchers should report evidence of the reliability. and validity of their findings where applicable since such data are crucial to evaluating the quality of their recommended learning tools or pedagogies. This can also aid other academics in updating their research on existing and developing pedagogical strategies. Researchers may consider designing and developing a standardized AI assessment mechanism that can be used across different grade levels to compare students’ AI literacy. This approach permits the standardization of assessment criteria and instructional feedback and thus better supports the wider application of AI teaching in K-12 classrooms.

References marked with an asterisk indicate reports included in this scoping review

*Ali, S., Payne, B. H., Williams, R., Park, H. W., & Breazeal, C. (2019, June). Constructionism, ethics, and creativity: Developing primary and middle school artificial intelligence education. In International workshop on education in artificial intelligence k-12 (eduai’19) (pp. 1–4).

Bai, H., & Yang, S. (2019, October). Research on the sustainable development model of information technology literacy of normal students based on deep learning recommendation system. In 2019 4th International conference on mechanical, control and computer engineering (icmcce) (pp. 837–8373). IEEE.

Bargh, J. A., & Schul, Y. (1980). On the cognitive benefits of teaching. Journal of Educational Psychology, 72 (5), 593.

Article   Google Scholar  

Battal, A., Afacan Adanır, G., & Gülbahar, Y. (2021). Computer science unplugged: A systematic literature review. Journal of Educational Technology Systems, 50 (1), 24–47.

Benjamin, R. (2020). Race after technology: Abolitionist tools for the new Jim code.

Berthelsen, D. (2009). Participatory learning. In Participatory learning in the early years: Research and pedagogy (pp. 1–11). Routledge.

Burgsteiner, H., Kandlhofer, M., & Steinbauer, G. (2016). Irobot: Teaching the basics of artificial intelligence in high schools. In Proceedings of the 30th AAAI conference on artificial intelligence (pp. 4126–4127). AAAI Press.

Burnard, P., Dragovic, T., Jasilek, S., Biddulph, J., Rolls, L., Durning, A., & Fenyvesi, K. (2022). The art of co-creating arts-based possibility spaces for fostering STE (A) M practices in primary education. In Arts-based methods in education around the world (pp. 247–281). River Publishers.

Çakiroğlu, Ü., Suiçmez, S. S., Kurtoğlu, Y. B., Sari, A., Yildiz, S., & Öztürk, M. (2018). Exploring perceived cognitive load in learning programming via Scratch. Research in Learning Technology, 26 , 1–20.

Cetindamar, D., Kitto, K., Wu, M., Zhang, Y., Abedin, B., & Knight, S. (2022). Explicating AI literacy of employees at digital workplaces. IEEE Transactions on Engineering Management, 71 , 810–823.

*Chai, C. S., Wang, X., & Xu, C. (2020). An extended theory of planned behavior for the modelling of Chinese secondary school students’ intention to learn Artificial Intelligence. Mathematics, 8 (11), 2089.

Chang, C. C., Hwang, G. J., & Tu, Y. F. (2022). Concept mapping in technology-supported K-12 education: A systematic review of selected SSCI publications from 2001 to 2020. Journal of Educational Computing Research, 60 (7), 1637–1662.

Chang, C. Y., Lai, C. L., & Hwang, G. J. (2018). Trends and research issues of mobile learning studies in nursing education: A review of academic publications from 1971 to 2016. Computers & Education, 116 , 28–48.

Chen, L., Chen, P., & Lin, Z. (2020). Artificial intelligence in education: A review. Ieee Access, 8 , 75264–75278.

Chiu, T. K. F., & Chai, C. S. (2020). Sustainable curriculum planning for artificial intelligence education: A self-determination theory perspective. Sustainability, 12 (14), 5568.

*Chiu, T. K. F., Meng, H., Chai, C. S., King, I., Wong, S., & Yam, Y. (2021). Creation and evaluation of a pre-tertiary Artificial Intelligence (AI) curriculum. IEEE Transactions on Education , 65 (1), 30–39.

Cope, B., & Kalantzis, M. (Eds.). (2016). A pedagogy of multiliteracies: Learning by design . Springer.

Google Scholar  

Crawford, K. (2021). The atlas of AI: Power, politics, and the planetary costs of artificial intelligence . Yale University Press.

Book   Google Scholar  

*Dai, Y., Lin, Z., Liu, A., Dai, D., & Wang, W. (2023). Effect of an analogy-based approach of artificial intelligence pedagogy in upper primary schools. Journal of Educational Computing Research . https://doi.org/10.1177/0735633123120134 .

Danniels, E., & Pyle, A. (2023). Inclusive play-based learning: Approaches from enacting kindergarten teachers. Early Childhood Education Journal, 51 (7), 1169–1179.

Darmawansah, D., Hwang, G. J., Chen, M. R. A., & Liang, J. C. (2023). Trends and research foci of robotics-based STEM education: A systematic review from diverse angles based on the technology-based learning model. International Journal of STEM Education, 10 (1), 1–24.

*Estevez, J., Garate, G., & Graña, M. (2019). Gentle introduction to artificial intelligence for high-school students using scratch . IEEE Access, 7 , 179027–179036.

*Fernández-Martínez, C., Hernán-Losada, I., & Fernández, A. (2021). Early introduction of AI in Spanish middle schools. A motivational study. KI-Künstliche Intelligenz, 35 (2), 163–170.

Finkelstein, J. E. (2006). Learning in real time: Synchronous teaching and learning online (Vol. 5). Wiley.

Ghazi, S. R., & Ullah, K. (2015). Concrete operational stage of Piaget’s cognitive development theory: An implication in learning general science. Gomal University Journal of Research, 31 (1), 78–89.

Goel, A. (2017). AI education for the world. AI Magazine, 38 (2), 3–4.

*Gong, X., Tang, Y., Liu, X., Jing, S., Cui, W., Liang, J., & Wang, F. Y. (2020, October). K-9 Artificial Intelligence education in Qingdao: Issues, challenges and suggestions. In 2020 IEEE international conference on networking, sensing and control (ICNSC) (pp. 1–6). IEEE.

*Gong, X., Wu, Y., Ye, Z., & Liu, X. (2018, June). Artificial Intelligence course design: iSTREAM-based visual cognitive smart vehicles. In 2018 IEEE intelligent vehicles symposium (IV) (pp. 1731–1735). IEEE.

Green, M. C. (2004). Transportation into narrative worlds: The role of prior knowledge and perceived realism. Discourse Processes, 38 (2), 247–266.

*Gunasilan, U. (2021). Debate as a learning activity for teaching programming: A case in the subject of machine learning. Higher Education, Skills and Work-Based Learning , 12 (4), 705–718.

*Han, X., Hu, F., Xiong, G., Liu, X., Gong, X., Niu, X., Shi, W., & Wang, X. (2018). Design of AI+ curriculum for primary and secondary schools in Qingdao. In 2018 Chinese automation congress (CAC) (pp. 4135–4140). IEEE.

*Heinze, C. A., Haase, J., & Higgins, H. (2010, July). An action research report from a multi-year approach to teaching artificial intelligence at the k-6 level. In 1st AAAI symposium on educational advances in artificial intelligence .

*Henry, J., Hernalesteen, A., & Collard, A. S. (2021). Teaching Artificial Intelligence to K-12 through a role-playing game questioning the intelligence concept. KI-Künstliche Intelligenz, 35 (2), 171–179.

*Ho, J. W., Scadding, M., Kong, S. C., Andone, D., Biswas, G., Hoppe, H. U., & Hsu, T. C. (2019, June). Classroom activities for teaching artificial intelligence to primary school students. In Proceedings of international conference on computational thinking education (pp. 157–159).

Hourcade, J. P. (2015). Child–computer interaction . Self, Iowa City.

Hsu, Y. C., Ho, H. N. J., Tsai, C. C., Hwang, G. J., Chu, H. C., Wang, C. Y., & Chen, N. S. (2012). Research trends in technology-based learning from 2000 to 2009: A content analysis of publications in selected journals. Journal of Educational Technology & Society, 15 (2), 354–370.

Hung, W., Jonassen, D. H., & Liu, R. (2008). Problem-based learning. In Handbook of research on educational communications and technology (Vol. 3(1), pp. 485–506). Erlbaum Associates.

*Kahn, K. M., Megasari, R., Piantari, E., & Junaeti, E. (2018). AI programming by children using snap! Block programming in a developing country. In EC-TEL practitioner proceedings 2018: 13th European conference on technology enhanced learning .

Kandlhofer, M., Steinbauer, G., Hirschmugl-Gaisch, S., & Huber, P. (2016). Artificial intelligence and computer science in education: From kindergarten to university. In 2016 IEEE frontiers in education conference (FIE) (pp. 1—9). IEEE.

Kajiwara, Y., Matsuoka, A., & Shinbo, F. (2023). Machine learning role playing game: Instructional design of AI education for age-appropriate in K-12 and beyond. Computers and Education: Artificial Intelligence, 5 , 100162.

Karalekas, G., Vologiannidis, S., & Kalomiros, J. (2023). Teaching machine learning in K-12 using robotics. Education Sciences, 13 (1), 67.

*Kaspersen, M. H., Bilstrup, K. E. K., Van Mechelen, M., Hjorth, A., Bouvin, N. O., & Petersen, M. G. (2021, June). VotestratesML: A high school learning tool for exploring machine learning and its societal implications. In FabLearn Europe/MakeEd 2021—an international conference on computing, design and making in education (pp. 1–10).

Keri, Z., & Elbatarny, H. S. (2021). The power of analogy-based learning in science. HAPS Educator, 25 (1), 13–20.

*Kilhoffer, Z., Zhou, Z., Wang, F., Tamton, F., Huang, Y., Kim, P., Yeh, T., & Wang, Y. (2023, May). “How technical do you get? I’m an English teacher”: Teaching and learning cybersecurity and AI ethics in high school. In 2023 IEEE symposium on security and privacy (SP) (pp. 2032–2032). IEEE.

Kolb, A. Y., & Kolb, D. A. (2009). Experiential learning theory: A dynamic, holistic approach to management learning, education and development. In The SAGE handbook of management learning, education and development (Vol. 7, p. 42). SAGE.

Lai, C. L., & Hwang, G. J. (2015). High school teachers’ perspectives on applying different mobile learning strategies to science courses: The national mobile learning program in Taiwan. International Journal of Mobile Learning and Organisation, 9 (2), 124–145.

*Lee, I., Ali, S., Zhang, H., DiPaola, D., & Breazeal, C. (2021, March). Developing middle school students’ AI literacy. In Proceedings of the 52nd ACM technical symposium on computer science education (pp. 191–197).

*Lee, S., Mott, B., Ottenbriet-Leftwich, A., Scribner, A., Taylor, S., Glazewski, K., Hmelo-Silver, C. E., & Lester, J. (2020, June). Designing a collaborative game-based learning environment for AI-infused inquiry learning in elementary school classrooms. In Proceedings of the 2020 ACM conference on innovation and technology in computer science education (p. 566).

Leitner, M., Greenwald, E., Wang, N., Montgomery, R., & Merchant, C. (2023). Designing game-based learning for high school artificial intelligence education. International Journal of Artificial Intelligence in Education, 33 , 384–398.

*Li, K., & Song, S. (2019, June). Application of artificial intelligence in primary and secondary schools: A case study of scratch. In International conference on applications and techniques in cyber security and intelligence (pp. 2026–2030). Springer.

*Lin, P., Van Brummelen, J., Lukin, G., Williams, R., & Breazeal, C. (2020, April). Zhorai: Designing a conversational agent for children to explore machine learning concepts. In Proceedings of the AAAI conference on artificial intelligence (Vol. 34(9), pp. 13381–13388).

*Lin, P. Y., Chai, C. S., Jong, M. S. Y., Dai, Y., Guo, Y., & Qin, J. (2021). Modeling the structural relationship among primary students’ motivation to learn artificial intelligence. Computers and Education: Artificial Intelligence, 2 , 100006.

Long, D., & Magerko, B. (2020, April). What is AI literacy? Competencies and design considerations. In Proceedings of the 2020 CHI conference on human factors in computing systems (pp. 1–16).

*Lucas, J. M. (2009). K-6 outreach using “Computer science unplugged”. Journal of Computing Sciences in Colleges, 24 (6), 62–63.

Lye, S. Y., & Koh, J. H. L. (2014). Review on teaching and learning of computational thinking through programming: What is next for K-12? Computers in Human Behavior, 41 , 51–61.

Mahon, J., Quille, K., Mac Namee, B., & Becker, B. A. (2022, March). A novel machine learning and artificial intelligence course for secondary school students. In Proceedings of the 53rd ACM technical symposium on computer science education (p. 1155).

*Mariescu-Istodor, R., & Jormanainen, I. (2019, November). Machine learning for high school students. In Proceedings of the 19th Koli calling international conference on computing education research (pp. 1–9).

Markham, T., Larmer, J., & Ravitz, J. (2003). Project based learning handbook: A guide to standards-focused project based learning for middle and high school teachers. Buck Institute for Education, Novato.

Marques, L. S., Gresse von Wangenheim, C., & Hauck, J. C. (2020). Teaching machine learning in school: A systematic mapping of the state of the art . Informatics in Education, 19 (2), 283–321.

*Martins, R. M., von Wangenheim, C. G., Rauber, M. F., & Hauck, J. C. (2023). Machine learning for all!—introducing machine learning in middle and high school . International Journal of Artificial Intelligence in Education . https://doi.org/10.1007/s40593-022-00325-y .

McCarthy, J. (2004). What is artificial intelligence? Engineering Materials and Design, 32 (3), 1–14.

McMillan, J. H., & Schumacher, S. (2010). Research in education: Evidence-based inquiry . Pearson.

*Melsión, G. I., Torre, I., Vidal, E., & Leite, I. (2021, June). Using explainability to help children understand gender bias in AI. In Proceedings of the interaction design and children , Athens, Greece (pp. 87–99).

Miao, F., Holmes, W., Huang, R., & Zhang, H. (2021). AI and education: A guidance for policymakers . UNESCO Publishing.

Miao, F., & Shiohira, K. (2022). K-12 AI curricula . UNESCO Publishing.

Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & Prisma Group. (2009). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. PLoS Medicine, 6 (7), e1000097.

*Ng, D. T. K., & Chu, S. K. W. (2021). Motivating students to learn AI through social networking sites: A case study in Hong Kong. Online Learning, 25 (1), 195–208.

Ng, D. T. K., Leung, J. K. L., Su, M. J., Yim, I. H. Y., Qiao, M. S., & Chu, S. K. W. (2023). AI literacy in K-16 classrooms . Springer.

*Ng, D. T. K., Luo, W. Y., Chan, H. M. Y., & Chu, S. K. W. (2022). An examination on primary students’ development in AI Literacy through digital story writing.  Computers and Education: Artificial Intelligence 3, 100054.

*Norouzi, N., Chaturvedi, S., & Rutledge, M. (2020, April). Lessons learned from teaching machine learning and natural language processing to high school students. In Proceedings of the AAAI conference on artificial intelligence (Vol. 34(9), pp. 13397–13403).

Papert, S., & Solomon, C. (1971). Twenty things to do with a computer. In S. Papert, C. Solomon, E. Soloway, & J. C. Spohrer (Eds.), Studying the novice programmer (pp. 3–28). Lawrence Erlbaum Associates.

Park, Y., & Shin, Y. (2021). Tooee: A novel scratch extension for K-12 big data and artificial intelligence education using text-based visual blocks. IEEE Access, 9 , 149630–149646.

Pedaste, M., Mäeots, M., Siiman, L. A., De Jong, T., Van Riesen, S. A., Kamp, E. T., Manoli, C. C., Zacharia, Z. C., & Tsourlidaki, E. (2015). Phases of inquiry-based learning: Definitions and the inquiry cycle. Educational Research Review, 14 , 47–61.

Pedro, F., Subosa, M., Rivas, A., & Valverde, P. (2019). Artificial intelligence in education: Challenges and opportunities for sustainable development. Roscongress Building Trust.

*Perach, S., & Alexandron, G. (2022, June). A blended-learning program for implementing a rigorous machine-learning curriculum in high-schools. In Proceedings of the 9th ACM conference on learning@ Scale (pp. 267–270).

Piaget, J. (2000). Piaget’s theory of cognitive development. In Childhood cognitive development: The essential readings (Vol. 2, pp. 33–47). Blackwell.

Priya, S., Bhadra, S., Chimalakonda, S., & Venigalla, A. S. M. (2022). ML-Quest: A game for introducing machine learning concepts to K-12 students. Interactive Learning Environments. DOI, 10 (1080/10494820), 2022.

Qin, J. J., Ma, F. G., & Guo, Y. M. (2019). Foundations of artificial intelligence for primary school.

Resnick, M., Myers, B., Nakakoji, K., Shneiderman, B., Pausch, R., Selker, T., & Eisenberg, M. (2005). Design principles for tools to support creative thinking. In Creativity support tools: Report from a US national science foundation sponsored workshop .

Rina, R., Kluzer, S., & Punie, Y. (2022). DigComp 2.2: The Digital competence framework for citizens-with new examples of knowledge, skills and attitudes (No. JRC128415). Joint Research Centre (Seville site).

Rizvi, S., Waite, J., & Sentance, S. (2023). Artificial Intelligence teaching and learning in K-12 from 2019 to 2022: A systematic literature review. Computers and Education: Artificial Intelligence, 4 , 100145.

*Rodríguez-García, J. D., Moreno-León, J., Román-González, M., & Robles, G. (2020, October). Introducing artificial intelligence fundamentals with LearningML: Artificial intelligence made easy. In 8th International conference on technological ecosystems for enhancing multiculturality (pp. 18–20).

*Rodríguez-García, J. D., Moreno-León, J., Román-González, M., & Robles, G. (2021, March). Evaluation of an online intervention to teach artificial intelligence with LearningML to 10–16-year-old students. In Proceedings of the 52nd ACM technical symposium on computer science education (pp. 177–183).

*Sabuncuoglu, A. (2020, June). Designing one year curriculum to teach artificial intelligence for middle school. In Proceedings of the 2020 ACM conference on innovation and technology in computer science education (pp. 96–102).

*Sakulkueakulsuk, B., Witoon, S., Ngarmkajornwiwat, P., Pataranutaporn, P., Surareungchai, W., Pataranutaporn, P., & Subsoontorn, P. (2018, December). Kids making AI: Integrating machine learning, gamification, and social context in STEM education. In 2018 IEEE international conference on teaching, assessment, and learning for engineering (TALE) (pp. 1005–1010). IEEE.

Sanusi, I. T., & Oyelere, S. S. (2020). Pedagogies of machine learning in K-12 context. In 2020 IEEE frontiers in education conference (FIE) (pp. 1–8). IEEE.

Sayers, A. (2008). Tips and tricks in performing a systematic review. British Journal of General Practice, 58 (547), 136–136.

*Scherz, Z., & Haberman, B. (1995, March). Logic programming based curriculum for high school students: The use of abstract data types. In  Proceedings of the 26th SIGCSE technical symposium on computer science education (pp. 331–335).

Sestino, A., & De Mauro, A. (2022). Leveraging artificial intelligence in business: Implications, applications and methods. Technology Analysis & Strategic Management, 34 (1), 16–29.

*Shamir, G., & Levin, I. (2021). Neural network construction practices in elementary school.  KI-Künstliche Intelligenz, 35 (2), 181–189.

*Shamir, G., & Levin, I. (2022). Teaching machine learning in elementary school. International Journal of Child–Computer Interaction, 31 , 100415.

Smerdon, B. A., Burkam, D. T., & Lee, V. E. (1999). Access to constructivist and didactic teaching: Who gets it? Where is it practiced? Teachers College Record, 101 (1), 5–34.

Smith, B. L., & MacGregor, J. T. (1992). What is collaborative learning. In Collaborative learning: A sourcebook for higher education . National Center on Postsecondary Teaching, Learning, and Assessment Publishing, Pennsylvania State University.

Su, J., & Yang, W. (2022). Artificial intelligence in early childhood education: A scoping review. Computers and Education: Artificial Intelligence, 3 , 100049.

Su, J., & Yang, W. (2023). AI literacy curriculum and its relation to children’s perceptions of robots and attitudes towards engineering and science: An intervention study in early childhood education. Journal of Computer Assisted Learning . https://doi.org/10.1111/jcal.12867

Su, J., Zhong, Y., & Ng, D. T. K. (2022). A meta-review of literature on educational approaches for teaching AI at the K-12 levels in the Asia-Pacific region. Computers and Education: Artificial Intelligence, 3 , 100065.

Su, J., Guo, K., Chen, X., & Chu, S. K. W. (2023). Teaching artificial intelligence in K–12 classrooms: a scoping review. Interactive Learning Environments , 1–20.

Su, J., Ng, D. T. K., & Chu, S. K. W. (2023). Artificial intelligence (AI) literacy in early childhood education: The challenges and opportunities. Computers and Education: Artificial Intelligence, 4 , 100124.

*Toivonen, T., Jormanainen, I., Kahila, J., Tedre, M., Valtonen, T., & Vartiainen, H. (2020, July). Co-designing machine learning apps in K-12 with primary school children. In  2020 IEEE 20th International Conference on Advanced Learning Technologies (ICALT) (pp. 308–310). IEEE.

Trybus, J. (2015). Game-based learning: What it is, why it works, and where it's going (p. 6). New Media Institute.

*Tseng, T., Murai, Y., Freed, N., Gelosi, D., Ta, T. D., & Kawahara, Y. (2021, June). PlushPal: Storytelling with interactive plush toys and machine learning. In Proceedings of the interaction design and children (pp. 236–245).

Tu, Y. F., & Hwang, G. J. (2020). Trends and research issues of mobile learning studies in hospitality, leisure, sport and tourism education: A review of academic publications from 2002 to 2017. Interactive Learning Environments, 28 (4), 385–403.

Van Brummelen, J., Heng, T., & Tabunshchyk, V. (2021, May). Teaching tech to talk: K-12 conversational artificial intelligence literacy curriculum and development tools. In Proceedings of the AAAI conference on artificial intelligence (Vol. 35(17), pp. 15655–15663).

*Vartiainen, H., Tedre, M., & Valtonen, T. (2020). Learning machine learning with very young children: Who is teaching whom ?. International Journal of Child-Computer Interaction, 25 , 100182.

*Voulgari, I., Zammit, M., Stouraitis, E., Liapis, A., & Yannakakis, G. (2021, June). Learn to machine learn: designing a game based approach for teaching machine learning to primary and secondary education students. In Proceedings of the interaction design and children (pp. 593–598).

*Wan, X., Zhou, X., Ye, Z., Mortensen, C. K., & Bai, Z. (2020, June). SmileyCluster: Supporting accessible machine learning in K-12 scientific discovery. In Proceedings of the interaction design and children conference (pp. 23–35).

Wang, T., & Cheng, E. C. K. (2021). An investigation of barriers to Hong Kong K-12 schools incorporating Artificial Intelligence in education. Computers and Education: Artificial Intelligence, 2 , 100031.

Wegerif, R. (2007). Dialogic education and technology: Expanding the space of learning (Vol. 7). Springer, London.

Wegerif, R. (2011). Towards a dialogic theory of how children learn to think. Thinking Skills and Creativity, 6 (3), 179–190.

*Williams, R., Park, H. W., & Breazeal, C. (2019b). A is for artificial intelligence: The impact of artificial intelligence activities on young children’s perceptions of robots. In Proceedings of the 2019 CHI conference on human factors in computing systems .

*Williams, R., Park, H. W., Oh, L., & Breazeal, C. (2019a, July). Popbots: Designing an artificial intelligence curriculum for early childhood education. In Proceedings of the AAAI conference on artificial intelligence (Vol. 33(1), pp. 9729–9736).

Wong, K.-C. (2020). Computational thinking and artificial intelligence education: A balanced approach using both classical AI and modern AI. In: Proceedings of international conference on computational thinking education 2020 (p. 108), The Education University of Hong Kong.

Yim, I. H. Y. (2023). Design of Artificial Intelligence (AI) education for primary schools: Arts-based approach (pp. 65–90). ISTES BOOKS.

*Yoder, S., Tatar, C., Aderemi, I., Boorugu, S., Jiang, S., & Akram, B. (2020). Gaining insight into effective teaching of AI problem-solving through CSEDM: A case study. In  5th Workshop on computer science educational data mining.

Zhou, X., Van Brummelen, J., & Lin, P. (2020). Designing AI learning experiences for K-12: Emerging works, future opportunities and a design framework. arXiv preprint. arXiv:2009.10228 .

Download references

Acknowledgements

The support and guidance provided by Professor Rupert Wegerif, Dr Annouchka Bayley, Dr Eleanor Dare and the University of Cambridge during this ongoing academic pursuit have been instrumental and greatly appreciated.

This research has not received any funding.

Author information

Authors and affiliations.

Faculty of Education, University of Cambridge, 184 Hills Road, Cambridge, CB2 8PQ, UK

Iris Heung Yue Yim

Faculty of Education, The University of Hong Kong, Pok Fu Lam, Hong Kong SAR, China

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Iris Heung Yue Yim .

Ethics declarations

Conflict of interest.

The authors declare no conflict of interest.

Informed consent

Not applicable.

Research involving human participants and/or animals

Ethical statements.

We hereby declare that this manuscript is the result of our independent creation under the reviewers’ comments. Except for the quoted contents, this manuscript does not contain any research achievements that have been published or written by other individuals or groups. We are the only authors of this manuscript. The legal responsibility of this statement shall be borne by us.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix 1: Overview of the selected articles

Rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Yim, I.H.Y., Su, J. Artificial intelligence (AI) learning tools in K-12 education: A scoping review. J. Comput. Educ. (2024). https://doi.org/10.1007/s40692-023-00304-9

Download citation

Received : 15 July 2023

Revised : 04 October 2023

Accepted : 25 October 2023

Published : 06 January 2024

DOI : https://doi.org/10.1007/s40692-023-00304-9

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Artificial intelligence literacy
• K-12 students
• AI literacy education
• Learning tools
• Find a journal
• Publish with us
• Track your research

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Springer Nature - PMC COVID-19 Collection

Impacts of digital technologies on education and factors influencing schools' digital capacity and transformation: A literature review

Stella timotheou.

1 CYENS Center of Excellence & Cyprus University of Technology (Cyprus Interaction Lab), Cyprus, CYENS Center of Excellence & Cyprus University of Technology, Nicosia-Limassol, Cyprus

Ourania Miliou

Yiannis dimitriadis.

2 Universidad de Valladolid (UVA), Spain, Valladolid, Spain

Sara Villagrá Sobrino

Nikoleta giannoutsou, romina cachia.

3 JRC - Joint Research Centre of the European Commission, Seville, Spain

Alejandra Martínez Monés

Andri ioannou, associated data.

Data sharing not applicable to this article as no datasets were generated or analysed during the current study.

Digital technologies have brought changes to the nature and scope of education and led education systems worldwide to adopt strategies and policies for ICT integration. The latter brought about issues regarding the quality of teaching and learning with ICTs, especially concerning the understanding, adaptation, and design of the education systems in accordance with current technological trends. These issues were emphasized during the recent COVID-19 pandemic that accelerated the use of digital technologies in education, generating questions regarding digitalization in schools. Specifically, many schools demonstrated a lack of experience and low digital capacity, which resulted in widening gaps, inequalities, and learning losses. Such results have engendered the need for schools to learn and build upon the experience to enhance their digital capacity and preparedness, increase their digitalization levels, and achieve a successful digital transformation. Given that the integration of digital technologies is a complex and continuous process that impacts different actors within the school ecosystem, there is a need to show how these impacts are interconnected and identify the factors that can encourage an effective and efficient change in the school environments. For this purpose, we conducted a non-systematic literature review. The results of the literature review were organized thematically based on the evidence presented about the impact of digital technology on education and the factors that affect the schools’ digital capacity and digital transformation. The findings suggest that ICT integration in schools impacts more than just students’ performance; it affects several other school-related aspects and stakeholders, too. Furthermore, various factors affect the impact of digital technologies on education. These factors are interconnected and play a vital role in the digital transformation process. The study results shed light on how ICTs can positively contribute to the digital transformation of schools and which factors should be considered for schools to achieve effective and efficient change.

Introduction

Digital technologies have brought changes to the nature and scope of education. Versatile and disruptive technological innovations, such as smart devices, the Internet of Things (IoT), artificial intelligence (AI), augmented reality (AR) and virtual reality (VR), blockchain, and software applications have opened up new opportunities for advancing teaching and learning (Gaol & Prasolova-Førland, 2021 ; OECD, 2021 ). Hence, in recent years, education systems worldwide have increased their investment in the integration of information and communication technology (ICT) (Fernández-Gutiérrez et al., 2020 ; Lawrence & Tar, 2018 ) and prioritized their educational agendas to adapt strategies or policies around ICT integration (European Commission, 2019 ). The latter brought about issues regarding the quality of teaching and learning with ICTs (Bates, 2015 ), especially concerning the understanding, adaptation, and design of education systems in accordance with current technological trends (Balyer & Öz, 2018 ). Studies have shown that despite the investment made in the integration of technology in schools, the results have not been promising, and the intended outcomes have not yet been achieved (Delgado et al., 2015 ; Lawrence & Tar, 2018 ). These issues were exacerbated during the COVID-19 pandemic, which forced teaching across education levels to move online (Daniel, 2020 ). Online teaching accelerated the use of digital technologies generating questions regarding the process, the nature, the extent, and the effectiveness of digitalization in schools (Cachia et al., 2021 ; König et al., 2020 ). Specifically, many schools demonstrated a lack of experience and low digital capacity, which resulted in widening gaps, inequalities, and learning losses (Blaskó et al., 2021 ; Di Pietro et al, 2020 ). Such results have engendered the need for schools to learn and build upon the experience in order to enhance their digital capacity (European Commission, 2020 ) and increase their digitalization levels (Costa et al., 2021 ). Digitalization offers possibilities for fundamental improvement in schools (OECD, 2021 ; Rott & Marouane, 2018 ) and touches many aspects of a school’s development (Delcker & Ifenthaler, 2021 ) . However, it is a complex process that requires large-scale transformative changes beyond the technical aspects of technology and infrastructure (Pettersson, 2021 ). Namely, digitalization refers to “ a series of deep and coordinated culture, workforce, and technology shifts and operating models ” (Brooks & McCormack, 2020 , p. 3) that brings cultural, organizational, and operational change through the integration of digital technologies (JISC, 2020 ). A successful digital transformation requires that schools increase their digital capacity levels, establishing the necessary “ culture, policies, infrastructure as well as digital competence of students and staff to support the effective integration of technology in teaching and learning practices ” (Costa et al, 2021 , p.163).

Given that the integration of digital technologies is a complex and continuous process that impacts different actors within the school ecosystem (Eng, 2005 ), there is a need to show how the different elements of the impact are interconnected and to identify the factors that can encourage an effective and efficient change in the school environment. To address the issues outlined above, we formulated the following research questions:

a) What is the impact of digital technologies on education?

b) Which factors might affect a school’s digital capacity and transformation?

In the present investigation, we conducted a non-systematic literature review of publications pertaining to the impact of digital technologies on education and the factors that affect a school’s digital capacity and transformation. The results of the literature review were organized thematically based on the evidence presented about the impact of digital technology on education and the factors which affect the schools’ digital capacity and digital transformation.

Methodology

The non-systematic literature review presented herein covers the main theories and research published over the past 17 years on the topic. It is based on meta-analyses and review papers found in scholarly, peer-reviewed content databases and other key studies and reports related to the concepts studied (e.g., digitalization, digital capacity) from professional and international bodies (e.g., the OECD). We searched the Scopus database, which indexes various online journals in the education sector with an international scope, to collect peer-reviewed academic papers. Furthermore, we used an all-inclusive Google Scholar search to include relevant key terms or to include studies found in the reference list of the peer-reviewed papers, and other key studies and reports related to the concepts studied by professional and international bodies. Lastly, we gathered sources from the Publications Office of the European Union ( https://op.europa.eu/en/home ); namely, documents that refer to policies related to digital transformation in education.

Regarding search terms, we first searched resources on the impact of digital technologies on education by performing the following search queries: “impact” OR “effects” AND “digital technologies” AND “education”, “impact” OR “effects” AND “ICT” AND “education”. We further refined our results by adding the terms “meta-analysis” and “review” or by adjusting the search options based on the features of each database to avoid collecting individual studies that would provide limited contributions to a particular domain. We relied on meta-analyses and review studies as these consider the findings of multiple studies to offer a more comprehensive view of the research in a given area (Schuele & Justice, 2006 ). Specifically, meta-analysis studies provided quantitative evidence based on statistically verifiable results regarding the impact of educational interventions that integrate digital technologies in school classrooms (Higgins et al., 2012 ; Tolani-Brown et al., 2011 ).

However, quantitative data does not offer explanations for the challenges or difficulties experienced during ICT integration in learning and teaching (Tolani-Brown et al., 2011 ). To fill this gap, we analyzed literature reviews and gathered in-depth qualitative evidence of the benefits and implications of technology integration in schools. In the analysis presented herein, we also included policy documents and reports from professional and international bodies and governmental reports, which offered useful explanations of the key concepts of this study and provided recent evidence on digital capacity and transformation in education along with policy recommendations. The inclusion and exclusion criteria that were considered in this study are presented in Table ​ Table1 1 .

Inclusion and exclusion criteria for the selection of resources on the impact of digital technologies on education

To ensure a reliable extraction of information from each study and assist the research synthesis we selected the study characteristics of interest (impact) and constructed coding forms. First, an overview of the synthesis was provided by the principal investigator who described the processes of coding, data entry, and data management. The coders followed the same set of instructions but worked independently. To ensure a common understanding of the process between coders, a sample of ten studies was tested. The results were compared, and the discrepancies were identified and resolved. Additionally, to ensure an efficient coding process, all coders participated in group meetings to discuss additions, deletions, and modifications (Stock, 1994 ). Due to the methodological diversity of the studied documents we began to synthesize the literature review findings based on similar study designs. Specifically, most of the meta-analysis studies were grouped in one category due to the quantitative nature of the measured impact. These studies tended to refer to student achievement (Hattie et al., 2014 ). Then, we organized the themes of the qualitative studies in several impact categories. Lastly, we synthesized both review and meta-analysis data across the categories. In order to establish a collective understanding of the concept of impact, we referred to a previous impact study by Balanskat ( 2009 ) which investigated the impact of technology in primary schools. In this context, the impact had a more specific ICT-related meaning and was described as “ a significant influence or effect of ICT on the measured or perceived quality of (parts of) education ” (Balanskat, 2009 , p. 9). In the study presented herein, the main impacts are in relation to learning and learners, teaching, and teachers, as well as other key stakeholders who are directly or indirectly connected to the school unit.

The study’s results identified multiple dimensions of the impact of digital technologies on students’ knowledge, skills, and attitudes; on equality, inclusion, and social integration; on teachers’ professional and teaching practices; and on other school-related aspects and stakeholders. The data analysis indicated various factors that might affect the schools’ digital capacity and transformation, such as digital competencies, the teachers’ personal characteristics and professional development, as well as the school’s leadership and management, administration, infrastructure, etc. The impacts and factors found in the literature review are presented below.

Impacts of digital technologies on students’ knowledge, skills, attitudes, and emotions

The impact of ICT use on students’ knowledge, skills, and attitudes has been investigated early in the literature. Eng ( 2005 ) found a small positive effect between ICT use and students' learning. Specifically, the author reported that access to computer-assisted instruction (CAI) programs in simulation or tutorial modes—used to supplement rather than substitute instruction – could enhance student learning. The author reported studies showing that teachers acknowledged the benefits of ICT on pupils with special educational needs; however, the impact of ICT on students' attainment was unclear. Balanskat et al. ( 2006 ) found a statistically significant positive association between ICT use and higher student achievement in primary and secondary education. The authors also reported improvements in the performance of low-achieving pupils. The use of ICT resulted in further positive gains for students, namely increased attention, engagement, motivation, communication and process skills, teamwork, and gains related to their behaviour towards learning. Evidence from qualitative studies showed that teachers, students, and parents recognized the positive impact of ICT on students' learning regardless of their competence level (strong/weak students). Punie et al. ( 2006 ) documented studies that showed positive results of ICT-based learning for supporting low-achieving pupils and young people with complex lives outside the education system. Liao et al. ( 2007 ) reported moderate positive effects of computer application instruction (CAI, computer simulations, and web-based learning) over traditional instruction on primary school student's achievement. Similarly, Tamim et al. ( 2011 ) reported small to moderate positive effects between the use of computer technology (CAI, ICT, simulations, computer-based instruction, digital and hypermedia) and student achievement in formal face-to-face classrooms compared to classrooms that did not use technology. Jewitt et al., ( 2011 ) found that the use of learning platforms (LPs) (virtual learning environments, management information systems, communication technologies, and information- and resource-sharing technologies) in schools allowed primary and secondary students to access a wider variety of quality learning resources, engage in independent and personalized learning, and conduct self- and peer-review; LPs also provide opportunities for teacher assessment and feedback. Similar findings were reported by Fu ( 2013 ), who documented a list of benefits and opportunities of ICT use. According to the author, the use of ICTs helps students access digital information and course content effectively and efficiently, supports student-centered and self-directed learning, as well as the development of a creative learning environment where more opportunities for critical thinking skills are offered, and promotes collaborative learning in a distance-learning environment. Higgins et al. ( 2012 ) found consistent but small positive associations between the use of technology and learning outcomes of school-age learners (5–18-year-olds) in studies linking the provision and use of technology with attainment. Additionally, Chauhan ( 2017 ) reported a medium positive effect of technology on the learning effectiveness of primary school students compared to students who followed traditional learning instruction.

The rise of mobile technologies and hardware devices instigated investigations into their impact on teaching and learning. Sung et al. ( 2016 ) reported a moderate effect on students' performance from the use of mobile devices in the classroom compared to the use of desktop computers or the non-use of mobile devices. Schmid et al. ( 2014 ) reported medium–low to low positive effects of technology integration (e.g., CAI, ICTs) in the classroom on students' achievement and attitude compared to not using technology or using technology to varying degrees. Tamim et al. ( 2015 ) found a low statistically significant effect of the use of tablets and other smart devices in educational contexts on students' achievement outcomes. The authors suggested that tablets offered additional advantages to students; namely, they reported improvements in students’ notetaking, organizational and communication skills, and creativity. Zheng et al. ( 2016 ) reported a small positive effect of one-to-one laptop programs on students’ academic achievement across subject areas. Additional reported benefits included student-centered, individualized, and project-based learning enhanced learner engagement and enthusiasm. Additionally, the authors found that students using one-to-one laptop programs tended to use technology more frequently than in non-laptop classrooms, and as a result, they developed a range of skills (e.g., information skills, media skills, technology skills, organizational skills). Haßler et al. ( 2016 ) found that most interventions that included the use of tablets across the curriculum reported positive learning outcomes. However, from 23 studies, five reported no differences, and two reported a negative effect on students' learning outcomes. Similar results were indicated by Kalati and Kim ( 2022 ) who investigated the effect of touchscreen technologies on young students’ learning. Specifically, from 53 studies, 34 advocated positive effects of touchscreen devices on children’s learning, 17 obtained mixed findings and two studies reported negative effects.

More recently, approaches that refer to the impact of gamification with the use of digital technologies on teaching and learning were also explored. A review by Pan et al. ( 2022 ) that examined the role of learning games in fostering mathematics education in K-12 settings, reported that gameplay improved students’ performance. Integration of digital games in teaching was also found as a promising pedagogical practice in STEM education that could lead to increased learning gains (Martinez et al., 2022 ; Wang et al., 2022 ). However, although Talan et al. ( 2020 ) reported a medium effect of the use of educational games (both digital and non-digital) on academic achievement, the effect of non-digital games was higher.

Over the last two years, the effects of more advanced technologies on teaching and learning were also investigated. Garzón and Acevedo ( 2019 ) found that AR applications had a medium effect on students' learning outcomes compared to traditional lectures. Similarly, Garzón et al. ( 2020 ) showed that AR had a medium impact on students' learning gains. VR applications integrated into various subjects were also found to have a moderate effect on students’ learning compared to control conditions (traditional classes, e.g., lectures, textbooks, and multimedia use, e.g., images, videos, animation, CAI) (Chen et al., 2022b ). Villena-Taranilla et al. ( 2022 ) noted the moderate effect of VR technologies on students’ learning when these were applied in STEM disciplines. In the same meta-analysis, Villena-Taranilla et al. ( 2022 ) highlighted the role of immersive VR, since its effect on students’ learning was greater (at a high level) across educational levels (K-6) compared to semi-immersive and non-immersive integrations. In another meta-analysis study, the effect size of the immersive VR was small and significantly differentiated across educational levels (Coban et al., 2022 ). The impact of AI on education was investigated by Su and Yang ( 2022 ) and Su et al. ( 2022 ), who showed that this technology significantly improved students’ understanding of AI computer science and machine learning concepts.

It is worth noting that the vast majority of studies referred to learning gains in specific subjects. Specifically, several studies examined the impact of digital technologies on students’ literacy skills and reported positive effects on language learning (Balanskat et al., 2006 ; Grgurović et al., 2013 ; Friedel et al., 2013 ; Zheng et al., 2016 ; Chen et al., 2022b ; Savva et al., 2022 ). Also, several studies documented positive effects on specific language learning areas, namely foreign language learning (Kao, 2014 ), writing (Higgins et al., 2012 ; Wen & Walters, 2022 ; Zheng et al., 2016 ), as well as reading and comprehension (Cheung & Slavin, 2011 ; Liao et al., 2007 ; Schwabe et al., 2022 ). ICTs were also found to have a positive impact on students' performance in STEM (science, technology, engineering, and mathematics) disciplines (Arztmann et al., 2022 ; Bado, 2022 ; Villena-Taranilla et al., 2022 ; Wang et al., 2022 ). Specifically, a number of studies reported positive impacts on students’ achievement in mathematics (Balanskat et al., 2006 ; Hillmayr et al., 2020 ; Li & Ma, 2010 ; Pan et al., 2022 ; Ran et al., 2022 ; Verschaffel et al., 2019 ; Zheng et al., 2016 ). Furthermore, studies documented positive effects of ICTs on science learning (Balanskat et al., 2006 ; Liao et al., 2007 ; Zheng et al., 2016 ; Hillmayr et al., 2020 ; Kalemkuş & Kalemkuş, 2022 ; Lei et al., 2022a ). Çelik ( 2022 ) also noted that computer simulations can help students understand learning concepts related to science. Furthermore, some studies documented that the use of ICTs had a positive impact on students’ achievement in other subjects, such as geography, history, music, and arts (Chauhan, 2017 ; Condie & Munro, 2007 ), and design and technology (Balanskat et al., 2006 ).

More specific positive learning gains were reported in a number of skills, e.g., problem-solving skills and pattern exploration skills (Higgins et al., 2012 ), metacognitive learning outcomes (Verschaffel et al., 2019 ), literacy skills, computational thinking skills, emotion control skills, and collaborative inquiry skills (Lu et al., 2022 ; Su & Yang, 2022 ; Su et al., 2022 ). Additionally, several investigations have reported benefits from the use of ICT on students’ creativity (Fielding & Murcia, 2022 ; Liu et al., 2022 ; Quah & Ng, 2022 ). Lastly, digital technologies were also found to be beneficial for enhancing students’ lifelong learning skills (Haleem et al., 2022 ).

Apart from gaining knowledge and skills, studies also reported improvement in motivation and interest in mathematics (Higgins et. al., 2019 ; Fadda et al., 2022 ) and increased positive achievement emotions towards several subjects during interventions using educational games (Lei et al., 2022a ). Chen et al. ( 2022a ) also reported a small but positive effect of digital health approaches in bullying and cyberbullying interventions with K-12 students, demonstrating that technology-based approaches can help reduce bullying and related consequences by providing emotional support, empowerment, and change of attitude. In their meta-review study, Su et al. ( 2022 ) also documented that AI technologies effectively strengthened students’ attitudes towards learning. In another meta-analysis, Arztmann et al. ( 2022 ) reported positive effects of digital games on motivation and behaviour towards STEM subjects.

Impacts of digital technologies on equality, inclusion and social integration

Although most of the reviewed studies focused on the impact of ICTs on students’ knowledge, skills, and attitudes, reports were also made on other aspects in the school context, such as equality, inclusion, and social integration. Condie and Munro ( 2007 ) documented research interventions investigating how ICT can support pupils with additional or special educational needs. While those interventions were relatively small scale and mostly based on qualitative data, their findings indicated that the use of ICTs enabled the development of communication, participation, and self-esteem. A recent meta-analysis (Baragash et al., 2022 ) with 119 participants with different disabilities, reported a significant overall effect size of AR on their functional skills acquisition. Koh’s meta-analysis ( 2022 ) also revealed that students with intellectual and developmental disabilities improved their competence and performance when they used digital games in the lessons.

Istenic Starcic and Bagon ( 2014 ) found that the role of ICT in inclusion and the design of pedagogical and technological interventions was not sufficiently explored in educational interventions with people with special needs; however, some benefits of ICT use were found in students’ social integration. The issue of gender and technology use was mentioned in a small number of studies. Zheng et al. ( 2016 ) reported a statistically significant positive interaction between one-to-one laptop programs and gender. Specifically, the results showed that girls and boys alike benefitted from the laptop program, but the effect on girls’ achievement was smaller than that on boys’. Along the same lines, Arztmann et al. ( 2022 ) reported no difference in the impact of game-based learning between boys and girls, arguing that boys and girls equally benefited from game-based interventions in STEM domains. However, results from a systematic review by Cussó-Calabuig et al. ( 2018 ) found limited and low-quality evidence on the effects of intensive use of computers on gender differences in computer anxiety, self-efficacy, and self-confidence. Based on their view, intensive use of computers can reduce gender differences in some areas and not in others, depending on contextual and implementation factors.

Impacts of digital technologies on teachers’ professional and teaching practices

Various research studies have explored the impact of ICT on teachers’ instructional practices and student assessment. Friedel et al. ( 2013 ) found that the use of mobile devices by students enabled teachers to successfully deliver content (e.g., mobile serious games), provide scaffolding, and facilitate synchronous collaborative learning. The integration of digital games in teaching and learning activities also gave teachers the opportunity to study and apply various pedagogical practices (Bado, 2022 ). Specifically, Bado ( 2022 ) found that teachers who implemented instructional activities in three stages (pre-game, game, and post-game) maximized students’ learning outcomes and engagement. For instance, during the pre-game stage, teachers focused on lectures and gameplay training, at the game stage teachers provided scaffolding on content, addressed technical issues, and managed the classroom activities. During the post-game stage, teachers organized activities for debriefing to ensure that the gameplay had indeed enhanced students’ learning outcomes.

Furthermore, ICT can increase efficiency in lesson planning and preparation by offering possibilities for a more collaborative approach among teachers. The sharing of curriculum plans and the analysis of students’ data led to clearer target settings and improvements in reporting to parents (Balanskat et al., 2006 ).

Additionally, the use and application of digital technologies in teaching and learning were found to enhance teachers’ digital competence. Balanskat et al. ( 2006 ) documented studies that revealed that the use of digital technologies in education had a positive effect on teachers’ basic ICT skills. The greatest impact was found on teachers with enough experience in integrating ICTs in their teaching and/or who had recently participated in development courses for the pedagogical use of technologies in teaching. Punie et al. ( 2006 ) reported that the provision of fully equipped multimedia portable computers and the development of online teacher communities had positive impacts on teachers’ confidence and competence in the use of ICTs.

Moreover, online assessment via ICTs benefits instruction. In particular, online assessments support the digitalization of students’ work and related logistics, allow teachers to gather immediate feedback and readjust to new objectives, and support the improvement of the technical quality of tests by providing more accurate results. Additionally, the capabilities of ICTs (e.g., interactive media, simulations) create new potential methods of testing specific skills, such as problem-solving and problem-processing skills, meta-cognitive skills, creativity and communication skills, and the ability to work productively in groups (Punie et al., 2006 ).

Impacts of digital technologies on other school-related aspects and stakeholders

There is evidence that the effective use of ICTs and the data transmission offered by broadband connections help improve administration (Balanskat et al., 2006 ). Specifically, ICTs have been found to provide better management systems to schools that have data gathering procedures in place. Condie and Munro ( 2007 ) reported impacts from the use of ICTs in schools in the following areas: attendance monitoring, assessment records, reporting to parents, financial management, creation of repositories for learning resources, and sharing of information amongst staff. Such data can be used strategically for self-evaluation and monitoring purposes which in turn can result in school improvements. Additionally, they reported that online access to other people with similar roles helped to reduce headteachers’ isolation by offering them opportunities to share insights into the use of ICT in learning and teaching and how it could be used to support school improvement. Furthermore, ICTs provided more efficient and successful examination management procedures, namely less time-consuming reporting processes compared to paper-based examinations and smooth communications between schools and examination authorities through electronic data exchange (Punie et al., 2006 ).

Zheng et al. ( 2016 ) reported that the use of ICTs improved home-school relationships. Additionally, Escueta et al. ( 2017 ) reported several ICT programs that had improved the flow of information from the school to parents. Particularly, they documented that the use of ICTs (learning management systems, emails, dedicated websites, mobile phones) allowed for personalized and customized information exchange between schools and parents, such as attendance records, upcoming class assignments, school events, and students’ grades, which generated positive results on students’ learning outcomes and attainment. Such information exchange between schools and families prompted parents to encourage their children to put more effort into their schoolwork.

The above findings suggest that the impact of ICT integration in schools goes beyond students’ performance in school subjects. Specifically, it affects a number of school-related aspects, such as equality and social integration, professional and teaching practices, and diverse stakeholders. In Table ​ Table2, 2 , we summarize the different impacts of digital technologies on school stakeholders based on the literature review, while in Table ​ Table3 3 we organized the tools/platforms and practices/policies addressed in the meta-analyses, literature reviews, EU reports, and international bodies included in the manuscript.

The impact of digital technologies on schools’ stakeholders based on the literature review

Tools/platforms and practices/policies addressed in the meta-analyses, literature reviews, EU reports, and international bodies included in the manuscript

Additionally, based on the results of the literature review, there are many types of digital technologies with different affordances (see, for example, studies on VR vs Immersive VR), which evolve over time (e.g. starting from CAIs in 2005 to Augmented and Virtual reality 2020). Furthermore, these technologies are linked to different pedagogies and policy initiatives, which are critical factors in the study of impact. Table ​ Table3 3 summarizes the different tools and practices that have been used to examine the impact of digital technologies on education since 2005 based on the review results.

Factors that affect the integration of digital technologies

Although the analysis of the literature review demonstrated different impacts of the use of digital technology on education, several authors highlighted the importance of various factors, besides the technology itself, that affect this impact. For example, Liao et al. ( 2007 ) suggested that future studies should carefully investigate which factors contribute to positive outcomes by clarifying the exact relationship between computer applications and learning. Additionally, Haßler et al., ( 2016 ) suggested that the neutral findings regarding the impact of tablets on students learning outcomes in some of the studies included in their review should encourage educators, school leaders, and school officials to further investigate the potential of such devices in teaching and learning. Several other researchers suggested that a number of variables play a significant role in the impact of ICTs on students’ learning that could be attributed to the school context, teaching practices and professional development, the curriculum, and learners’ characteristics (Underwood, 2009 ; Tamim et al., 2011 ; Higgins et al., 2012 ; Archer et al., 2014 ; Sung et al., 2016 ; Haßler et al., 2016 ; Chauhan, 2017 ; Lee et al., 2020 ; Tang et al., 2022 ).

Digital competencies

One of the most common challenges reported in studies that utilized digital tools in the classroom was the lack of students’ skills on how to use them. Fu ( 2013 ) found that students’ lack of technical skills is a barrier to the effective use of ICT in the classroom. Tamim et al. ( 2015 ) reported that students faced challenges when using tablets and smart mobile devices, associated with the technical issues or expertise needed for their use and the distracting nature of the devices and highlighted the need for teachers’ professional development. Higgins et al. ( 2012 ) reported that skills training about the use of digital technologies is essential for learners to fully exploit the benefits of instruction.

Delgado et al. ( 2015 ), meanwhile, reported studies that showed a strong positive association between teachers’ computer skills and students’ use of computers. Teachers’ lack of ICT skills and familiarization with technologies can become a constraint to the effective use of technology in the classroom (Balanskat et al., 2006 ; Delgado et al., 2015 ).

It is worth noting that the way teachers are introduced to ICTs affects the impact of digital technologies on education. Previous studies have shown that teachers may avoid using digital technologies due to limited digital skills (Balanskat, 2006 ), or they prefer applying “safe” technologies, namely technologies that their own teachers used and with which they are familiar (Condie & Munro, 2007 ). In this regard, the provision of digital skills training and exposure to new digital tools might encourage teachers to apply various technologies in their lessons (Condie & Munro, 2007 ). Apart from digital competence, technical support in the school setting has also been shown to affect teachers’ use of technology in their classrooms (Delgado et al., 2015 ). Ferrari et al. ( 2011 ) found that while teachers’ use of ICT is high, 75% stated that they needed more institutional support and a shift in the mindset of educational actors to achieve more innovative teaching practices. The provision of support can reduce time and effort as well as cognitive constraints, which could cause limited ICT integration in the school lessons by teachers (Escueta et al., 2017 ).

Teachers’ personal characteristics, training approaches, and professional development

Teachers’ personal characteristics and professional development affect the impact of digital technologies on education. Specifically, Cheok and Wong ( 2015 ) found that teachers’ personal characteristics (e.g., anxiety, self-efficacy) are associated with their satisfaction and engagement with technology. Bingimlas ( 2009 ) reported that lack of confidence, resistance to change, and negative attitudes in using new technologies in teaching are significant determinants of teachers’ levels of engagement in ICT. The same author reported that the provision of technical support, motivation support (e.g., awards, sufficient time for planning), and training on how technologies can benefit teaching and learning can eliminate the above barriers to ICT integration. Archer et al. ( 2014 ) found that comfort levels in using technology are an important predictor of technology integration and argued that it is essential to provide teachers with appropriate training and ongoing support until they are comfortable with using ICTs in the classroom. Hillmayr et al. ( 2020 ) documented that training teachers on ICT had an important effecton students’ learning.

According to Balanskat et al. ( 2006 ), the impact of ICTs on students’ learning is highly dependent on the teachers’ capacity to efficiently exploit their application for pedagogical purposes. Results obtained from the Teaching and Learning International Survey (TALIS) (OECD, 2021 ) revealed that although schools are open to innovative practices and have the capacity to adopt them, only 39% of teachers in the European Union reported that they are well or very well prepared to use digital technologies for teaching. Li and Ma ( 2010 ) and Hardman ( 2019 ) showed that the positive effect of technology on students’ achievement depends on the pedagogical practices used by teachers. Schmid et al. ( 2014 ) reported that learning was best supported when students were engaged in active, meaningful activities with the use of technological tools that provided cognitive support. Tamim et al. ( 2015 ) compared two different pedagogical uses of tablets and found a significant moderate effect when the devices were used in a student-centered context and approach rather than within teacher-led environments. Similarly, Garzón and Acevedo ( 2019 ) and Garzón et al. ( 2020 ) reported that the positive results from the integration of AR applications could be attributed to the existence of different variables which could influence AR interventions (e.g., pedagogical approach, learning environment, and duration of the intervention). Additionally, Garzón et al. ( 2020 ) suggested that the pedagogical resources that teachers used to complement their lectures and the pedagogical approaches they applied were crucial to the effective integration of AR on students’ learning gains. Garzón and Acevedo ( 2019 ) also emphasized that the success of a technology-enhanced intervention is based on both the technology per se and its characteristics and on the pedagogical strategies teachers choose to implement. For instance, their results indicated that the collaborative learning approach had the highest impact on students’ learning gains among other approaches (e.g., inquiry-based learning, situated learning, or project-based learning). Ran et al. ( 2022 ) also found that the use of technology to design collaborative and communicative environments showed the largest moderator effects among the other approaches.

Hattie ( 2008 ) reported that the effective use of computers is associated with training teachers in using computers as a teaching and learning tool. Zheng et al. ( 2016 ) noted that in addition to the strategies teachers adopt in teaching, ongoing professional development is also vital in ensuring the success of technology implementation programs. Sung et al. ( 2016 ) found that research on the use of mobile devices to support learning tends to report that the insufficient preparation of teachers is a major obstacle in implementing effective mobile learning programs in schools. Friedel et al. ( 2013 ) found that providing training and support to teachers increased the positive impact of the interventions on students’ learning gains. Trucano ( 2005 ) argued that positive impacts occur when digital technologies are used to enhance teachers’ existing pedagogical philosophies. Higgins et al. ( 2012 ) found that the types of technologies used and how they are used could also affect students’ learning. The authors suggested that training and professional development of teachers that focuses on the effective pedagogical use of technology to support teaching and learning is an important component of successful instructional approaches (Higgins et al., 2012 ). Archer et al. ( 2014 ) found that studies that reported ICT interventions during which teachers received training and support had moderate positive effects on students’ learning outcomes, which were significantly higher than studies where little or no detail about training and support was mentioned. Fu ( 2013 ) reported that the lack of teachers’ knowledge and skills on the technical and instructional aspects of ICT use in the classroom, in-service training, pedagogy support, technical and financial support, as well as the lack of teachers’ motivation and encouragement to integrate ICT on their teaching were significant barriers to the integration of ICT in education.

School leadership and management

Management and leadership are important cornerstones in the digital transformation process (Pihir et al., 2018 ). Zheng et al. ( 2016 ) documented leadership among the factors positively affecting the successful implementation of technology integration in schools. Strong leadership, strategic planning, and systematic integration of digital technologies are prerequisites for the digital transformation of education systems (Ređep, 2021 ). Management and leadership play a significant role in formulating policies that are translated into practice and ensure that developments in ICT become embedded into the life of the school and in the experiences of staff and pupils (Condie & Munro, 2007 ). Policy support and leadership must include the provision of an overall vision for the use of digital technologies in education, guidance for students and parents, logistical support, as well as teacher training (Conrads et al., 2017 ). Unless there is a commitment throughout the school, with accountability for progress at key points, it is unlikely for ICT integration to be sustained or become part of the culture (Condie & Munro, 2007 ). To achieve this, principals need to adopt and promote a whole-institution strategy and build a strong mutual support system that enables the school’s technological maturity (European Commission, 2019 ). In this context, school culture plays an essential role in shaping the mindsets and beliefs of school actors towards successful technology integration. Condie and Munro ( 2007 ) emphasized the importance of the principal’s enthusiasm and work as a source of inspiration for the school staff and the students to cultivate a culture of innovation and establish sustainable digital change. Specifically, school leaders need to create conditions in which the school staff is empowered to experiment and take risks with technology (Elkordy & Lovinelli, 2020 ).

In order for leaders to achieve the above, it is important to develop capacities for learning and leading, advocating professional learning, and creating support systems and structures (European Commission, 2019 ). Digital technology integration in education systems can be challenging and leadership needs guidance to achieve it. Such guidance can be introduced through the adoption of new methods and techniques in strategic planning for the integration of digital technologies (Ređep, 2021 ). Even though the role of leaders is vital, the relevant training offered to them has so far been inadequate. Specifically, only a third of the education systems in Europe have put in place national strategies that explicitly refer to the training of school principals (European Commission, 2019 , p. 16).

Connectivity, infrastructure, and government and other support

The effective integration of digital technologies across levels of education presupposes the development of infrastructure, the provision of digital content, and the selection of proper resources (Voogt et al., 2013 ). Particularly, a high-quality broadband connection in the school increases the quality and quantity of educational activities. There is evidence that ICT increases and formalizes cooperative planning between teachers and cooperation with managers, which in turn has a positive impact on teaching practices (Balanskat et al., 2006 ). Additionally, ICT resources, including software and hardware, increase the likelihood of teachers integrating technology into the curriculum to enhance their teaching practices (Delgado et al., 2015 ). For example, Zheng et al. ( 2016 ) found that the use of one-on-one laptop programs resulted in positive changes in teaching and learning, which would not have been accomplished without the infrastructure and technical support provided to teachers. Delgado et al. ( 2015 ) reported that limited access to technology (insufficient computers, peripherals, and software) and lack of technical support are important barriers to ICT integration. Access to infrastructure refers not only to the availability of technology in a school but also to the provision of a proper amount and the right types of technology in locations where teachers and students can use them. Effective technical support is a central element of the whole-school strategy for ICT (Underwood, 2009 ). Bingimlas ( 2009 ) reported that lack of technical support in the classroom and whole-school resources (e.g., failing to connect to the Internet, printers not printing, malfunctioning computers, and working on old computers) are significant barriers that discourage the use of ICT by teachers. Moreover, poor quality and inadequate hardware maintenance, and unsuitable educational software may discourage teachers from using ICTs (Balanskat et al., 2006 ; Bingimlas, 2009 ).

Government support can also impact the integration of ICTs in teaching. Specifically, Balanskat et al. ( 2006 ) reported that government interventions and training programs increased teachers’ enthusiasm and positive attitudes towards ICT and led to the routine use of embedded ICT.

Lastly, another important factor affecting digital transformation is the development and quality assurance of digital learning resources. Such resources can be support textbooks and related materials or resources that focus on specific subjects or parts of the curriculum. Policies on the provision of digital learning resources are essential for schools and can be achieved through various actions. For example, some countries are financing web portals that become repositories, enabling teachers to share resources or create their own. Additionally, they may offer e-learning opportunities or other services linked to digital education. In other cases, specific agencies of projects have also been set up to develop digital resources (Eurydice, 2019 ).

Administration and digital data management

The digital transformation of schools involves organizational improvements at the level of internal workflows, communication between the different stakeholders, and potential for collaboration. Vuorikari et al. ( 2020 ) presented evidence that digital technologies supported the automation of administrative practices in schools and reduced the administration’s workload. There is evidence that digital data affects the production of knowledge about schools and has the power to transform how schooling takes place. Specifically, Sellar ( 2015 ) reported that data infrastructure in education is developing due to the demand for “ information about student outcomes, teacher quality, school performance, and adult skills, associated with policy efforts to increase human capital and productivity practices ” (p. 771). In this regard, practices, such as datafication which refers to the “ translation of information about all kinds of things and processes into quantified formats” have become essential for decision-making based on accountability reports about the school’s quality. The data could be turned into deep insights about education or training incorporating ICTs. For example, measuring students’ online engagement with the learning material and drawing meaningful conclusions can allow teachers to improve their educational interventions (Vuorikari et al., 2020 ).

Students’ socioeconomic background and family support

Research show that the active engagement of parents in the school and their support for the school’s work can make a difference to their children’s attitudes towards learning and, as a result, their achievement (Hattie, 2008 ). In recent years, digital technologies have been used for more effective communication between school and family (Escueta et al., 2017 ). The European Commission ( 2020 ) presented data from a Eurostat survey regarding the use of computers by students during the pandemic. The data showed that younger pupils needed additional support and guidance from parents and the challenges were greater for families in which parents had lower levels of education and little to no digital skills.

In this regard, the socio-economic background of the learners and their socio-cultural environment also affect educational achievements (Punie et al., 2006 ). Trucano documented that the use of computers at home positively influenced students’ confidence and resulted in more frequent use at school, compared to students who had no home access (Trucano, 2005 ). In this sense, the socio-economic background affects the access to computers at home (OECD, 2015 ) which in turn influences the experience of ICT, an important factor for school achievement (Punie et al., 2006 ; Underwood, 2009 ). Furthermore, parents from different socio-economic backgrounds may have different abilities and availability to support their children in their learning process (Di Pietro et al., 2020 ).

Schools’ socioeconomic context and emergency situations

The socio-economic context of the school is closely related to a school’s digital transformation. For example, schools in disadvantaged, rural, or deprived areas are likely to lack the digital capacity and infrastructure required to adapt to the use of digital technologies during emergency periods, such as the COVID-19 pandemic (Di Pietro et al., 2020 ). Data collected from school principals confirmed that in several countries, there is a rural/urban divide in connectivity (OECD, 2015 ).

Emergency periods also affect the digitalization of schools. The COVID-19 pandemic led to the closure of schools and forced them to seek appropriate and connective ways to keep working on the curriculum (Di Pietro et al., 2020 ). The sudden large-scale shift to distance and online teaching and learning also presented challenges around quality and equity in education, such as the risk of increased inequalities in learning, digital, and social, as well as teachers facing difficulties coping with this demanding situation (European Commission, 2020 ).

Looking at the findings of the above studies, we can conclude that the impact of digital technologies on education is influenced by various actors and touches many aspects of the school ecosystem. Figure  1 summarizes the factors affecting the digital technologies’ impact on school stakeholders based on the findings from the literature review.

Factors that affect the impact of ICTs on education

The findings revealed that the use of digital technologies in education affects a variety of actors within a school’s ecosystem. First, we observed that as technologies evolve, so does the interest of the research community to apply them to school settings. Figure  2 summarizes the trends identified in current research around the impact of digital technologies on schools’ digital capacity and transformation as found in the present study. Starting as early as 2005, when computers, simulations, and interactive boards were the most commonly applied tools in school interventions (e.g., Eng, 2005 ; Liao et al., 2007 ; Moran et al., 2008 ; Tamim et al., 2011 ), moving towards the use of learning platforms (Jewitt et al., 2011 ), then to the use of mobile devices and digital games (e.g., Tamim et al., 2015 ; Sung et al., 2016 ; Talan et al., 2020 ), as well as e-books (e.g., Savva et al., 2022 ), to the more recent advanced technologies, such as AR and VR applications (e.g., Garzón & Acevedo, 2019 ; Garzón et al., 2020 ; Kalemkuş & Kalemkuş, 2022 ), or robotics and AI (e.g., Su & Yang, 2022 ; Su et al., 2022 ). As this evolution shows, digital technologies are a concept in flux with different affordances and characteristics. Additionally, from an instructional perspective, there has been a growing interest in different modes and models of content delivery such as online, blended, and hybrid modes (e.g., Cheok & Wong, 2015 ; Kazu & Yalçin, 2022 ; Ulum, 2022 ). This is an indication that the value of technologies to support teaching and learning as well as other school-related practices is increasingly recognized by the research and school community. The impact results from the literature review indicate that ICT integration on students’ learning outcomes has effects that are small (Coban et al., 2022 ; Eng, 2005 ; Higgins et al., 2012 ; Schmid et al., 2014 ; Tamim et al., 2015 ; Zheng et al., 2016 ) to moderate (Garzón & Acevedo, 2019 ; Garzón et al., 2020 ; Liao et al., 2007 ; Sung et al., 2016 ; Talan et al., 2020 ; Wen & Walters, 2022 ). That said, a number of recent studies have reported high effect sizes (e.g., Kazu & Yalçin, 2022 ).

Current work and trends in the study of the impact of digital technologies on schools’ digital capacity

Based on these findings, several authors have suggested that the impact of technology on education depends on several variables and not on the technology per se (Tamim et al., 2011 ; Higgins et al., 2012 ; Archer et al., 2014 ; Sung et al., 2016 ; Haßler et al., 2016 ; Chauhan, 2017 ; Lee et al., 2020 ; Lei et al., 2022a ). While the impact of ICTs on student achievement has been thoroughly investigated by researchers, other aspects related to school life that are also affected by ICTs, such as equality, inclusion, and social integration have received less attention. Further analysis of the literature review has revealed a greater investment in ICT interventions to support learning and teaching in the core subjects of literacy and STEM disciplines, especially mathematics, and science. These were the most common subjects studied in the reviewed papers often drawing on national testing results, while studies that investigated other subject areas, such as social studies, were limited (Chauhan, 2017 ; Condie & Munro, 2007 ). As such, research is still lacking impact studies that focus on the effects of ICTs on a range of curriculum subjects.

The qualitative research provided additional information about the impact of digital technologies on education, documenting positive effects and giving more details about implications, recommendations, and future research directions. Specifically, the findings regarding the role of ICTs in supporting learning highlight the importance of teachers’ instructional practice and the learning context in the use of technologies and consequently their impact on instruction (Çelik, 2022 ; Schmid et al., 2014 ; Tamim et al., 2015 ). The review also provided useful insights regarding the various factors that affect the impact of digital technologies on education. These factors are interconnected and play a vital role in the transformation process. Specifically, these factors include a) digital competencies; b) teachers’ personal characteristics and professional development; c) school leadership and management; d) connectivity, infrastructure, and government support; e) administration and data management practices; f) students’ socio-economic background and family support and g) the socioeconomic context of the school and emergency situations. It is worth noting that we observed factors that affect the integration of ICTs in education but may also be affected by it. For example, the frequent use of ICTs and the use of laptops by students for instructional purposes positively affect the development of digital competencies (Zheng et al., 2016 ) and at the same time, the digital competencies affect the use of ICTs (Fu, 2013 ; Higgins et al., 2012 ). As a result, the impact of digital technologies should be explored more as an enabler of desirable and new practices and not merely as a catalyst that improves the output of the education process i.e. namely student attainment.

Conclusions

Digital technologies offer immense potential for fundamental improvement in schools. However, investment in ICT infrastructure and professional development to improve school education are yet to provide fruitful results. Digital transformation is a complex process that requires large-scale transformative changes that presuppose digital capacity and preparedness. To achieve such changes, all actors within the school’s ecosystem need to share a common vision regarding the integration of ICTs in education and work towards achieving this goal. Our literature review, which synthesized quantitative and qualitative data from a list of meta-analyses and review studies, provided useful insights into the impact of ICTs on different school stakeholders and showed that the impact of digital technologies touches upon many different aspects of school life, which are often overlooked when the focus is on student achievement as the final output of education. Furthermore, the concept of digital technologies is a concept in flux as technologies are not only different among them calling for different uses in the educational practice but they also change through time. Additionally, we opened a forum for discussion regarding the factors that affect a school’s digital capacity and transformation. We hope that our study will inform policy, practice, and research and result in a paradigm shift towards more holistic approaches in impact and assessment studies.

Study limitations and future directions

We presented a review of the study of digital technologies' impact on education and factors influencing schools’ digital capacity and transformation. The study results were based on a non-systematic literature review grounded on the acquisition of documentation in specific databases. Future studies should investigate more databases to corroborate and enhance our results. Moreover, search queries could be enhanced with key terms that could provide additional insights about the integration of ICTs in education, such as “policies and strategies for ICT integration in education”. Also, the study drew information from meta-analyses and literature reviews to acquire evidence about the effects of ICT integration in schools. Such evidence was mostly based on the general conclusions of the studies. It is worth mentioning that, we located individual studies which showed different, such as negative or neutral results. Thus, further insights are needed about the impact of ICTs on education and the factors influencing the impact. Furthermore, the nature of the studies included in meta-analyses and reviews is different as they are based on different research methodologies and data gathering processes. For instance, in a meta-analysis, the impact among the studies investigated is measured in a particular way, depending on policy or research targets (e.g., results from national examinations, pre-/post-tests). Meanwhile, in literature reviews, qualitative studies offer additional insights and detail based on self-reports and research opinions on several different aspects and stakeholders who could affect and be affected by ICT integration. As a result, it was challenging to draw causal relationships between so many interrelating variables.

Despite the challenges mentioned above, this study envisaged examining school units as ecosystems that consist of several actors by bringing together several variables from different research epistemologies to provide an understanding of the integration of ICTs. However, the use of other tools and methodologies and models for evaluation of the impact of digital technologies on education could give more detailed data and more accurate results. For instance, self-reflection tools, like SELFIE—developed on the DigCompOrg framework- (Kampylis et al., 2015 ; Bocconi & Lightfoot, 2021 ) can help capture a school’s digital capacity and better assess the impact of ICTs on education. Furthermore, the development of a theory of change could be a good approach for documenting the impact of digital technologies on education. Specifically, theories of change are models used for the evaluation of interventions and their impact; they are developed to describe how interventions will work and give the desired outcomes (Mayne, 2015 ). Theory of change as a methodological approach has also been used by researchers to develop models for evaluation in the field of education (e.g., Aromatario et al., 2019 ; Chapman & Sammons, 2013 ; De Silva et al., 2014 ).

We also propose that future studies aim at similar investigations by applying more holistic approaches for impact assessment that can provide in-depth data about the impact of digital technologies on education. For instance, future studies could focus on different research questions about the technologies that are used during the interventions or the way the implementation takes place (e.g., What methodologies are used for documenting impact? How are experimental studies implemented? How can teachers be taken into account and trained on the technology and its functions? What are the elements of an appropriate and successful implementation? How is the whole intervention designed? On which learning theories is the technology implementation based?).

Future research could also focus on assessing the impact of digital technologies on various other subjects since there is a scarcity of research related to particular subjects, such as geography, history, arts, music, and design and technology. More research should also be done about the impact of ICTs on skills, emotions, and attitudes, and on equality, inclusion, social interaction, and special needs education. There is also a need for more research about the impact of ICTs on administration, management, digitalization, and home-school relationships. Additionally, although new forms of teaching and learning with the use of ICTs (e.g., blended, hybrid, and online learning) have initiated several investigations in mainstream classrooms, only a few studies have measured their impact on students’ learning. Additionally, our review did not document any study about the impact of flipped classrooms on K-12 education. Regarding teaching and learning approaches, it is worth noting that studies referred to STEM or STEAM did not investigate the impact of STEM/STEAM as an interdisciplinary approach to learning but only investigated the impact of ICTs on learning in each domain as a separate subject (science, technology, engineering, arts, mathematics). Hence, we propose future research to also investigate the impact of the STEM/STEAM approach on education. The impact of emerging technologies on education, such as AR, VR, robotics, and AI has also been investigated recently, but more work needs to be done.

Finally, we propose that future studies could focus on the way in which specific factors, e.g., infrastructure and government support, school leadership and management, students’ and teachers’ digital competencies, approaches teachers utilize in the teaching and learning (e.g., blended, online and hybrid learning, flipped classrooms, STEM/STEAM approach, project-based learning, inquiry-based learning), affect the impact of digital technologies on education. We hope that future studies will give detailed insights into the concept of schools’ digital transformation through further investigation of impacts and factors which influence digital capacity and transformation based on the results and the recommendations of the present study.

Acknowledgements

This project has received funding under Grant Agreement No Ref Ares (2021) 339036 7483039 as well as funding from the European Union’s Horizon 2020 Research and Innovation Program under Grant Agreement No 739578 and the Government of the Republic of Cyprus through the Deputy Ministry of Research, Innovation and Digital Policy. The UVa co-authors would like also to acknowledge funding from the European Regional Development Fund and the National Research Agency of the Spanish Ministry of Science and Innovation, under project grant PID2020-112584RB-C32.

Data availability statement

Declarations.

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

• Archer K, Savage R, Sanghera-Sidhu S, Wood E, Gottardo A, Chen V. Examining the effectiveness of technology use in classrooms: A tertiary meta-analysis. Computers & Education. 2014; 78 :140–149. doi: 10.1016/j.compedu.2014.06.001. [ CrossRef ] [ Google Scholar ]
• Aromatario O, Van Hoye A, Vuillemin A, Foucaut AM, Pommier J, Cambon L. Using theory of change to develop an intervention theory for designing and evaluating behavior change SDApps for healthy eating and physical exercise: The OCAPREV theory. BMC Public Health. 2019; 19 (1):1–12. doi: 10.1186/s12889-019-7828-4. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Arztmann, M., Hornstra, L., Jeuring, J., & Kester, L. (2022). Effects of games in STEM education: A meta-analysis on the moderating role of student background characteristics. Studies in Science Education , 1-37. 10.1080/03057267.2022.2057732
• Bado N. Game-based learning pedagogy: A review of the literature. Interactive Learning Environments. 2022; 30 (5):936–948. doi: 10.1080/10494820.2019.1683587. [ CrossRef ] [ Google Scholar ]
• Balanskat, A. (2009). Study of the impact of technology in primary schools – Synthesis Report. Empirica and European Schoolnet. Retrieved 30 June 2022 from: https://erte.dge.mec.pt/sites/default/files/Recursos/Estudos/synthesis_report_steps_en.pdf
• Balanskat, A. (2006). The ICT Impact Report: A review of studies of ICT impact on schools in Europe, European Schoolnet. Retrieved 30 June 2022 from:  https://en.unesco.org/icted/content/ict-impact-report-review-studies-ict-impact-schools-europe
• Balanskat, A., Blamire, R., & Kefala, S. (2006). The ICT impact report.  European Schoolnet . Retrieved from: http://colccti.colfinder.org/sites/default/files/ict_impact_report_0.pdf
• Balyer, A., & Öz, Ö. (2018). Academicians’ views on digital transformation in education. International Online Journal of Education and Teaching (IOJET), 5 (4), 809–830. Retrieved 30 June 2022 from  http://iojet.org/index.php/IOJET/article/view/441/295
• Baragash RS, Al-Samarraie H, Moody L, Zaqout F. Augmented reality and functional skills acquisition among individuals with special needs: A meta-analysis of group design studies. Journal of Special Education Technology. 2022; 37 (1):74–81. doi: 10.1177/0162643420910413. [ CrossRef ] [ Google Scholar ]
• Bates, A. W. (2015). Teaching in a digital age: Guidelines for designing teaching and learning . Open Educational Resources Collection . 6. Retrieved 30 June 2022 from: https://irl.umsl.edu/oer/6
• Bingimlas KA. Barriers to the successful integration of ICT in teaching and learning environments: A review of the literature. Eurasia Journal of Mathematics, Science and Technology Education. 2009; 5 (3):235–245. doi: 10.12973/ejmste/75275. [ CrossRef ] [ Google Scholar ]
• Blaskó Z, Costa PD, Schnepf SV. Learning losses and educational inequalities in Europe: Mapping the potential consequences of the COVID-19 crisis. Journal of European Social Policy. 2022; 32 (4):361–375. doi: 10.1177/09589287221091687. [ CrossRef ] [ Google Scholar ]
• Bocconi S, Lightfoot M. Scaling up and integrating the selfie tool for schools' digital capacity in education and training systems: Methodology and lessons learnt. European Training Foundation. 2021 doi: 10.2816/907029,JRC123936. [ CrossRef ] [ Google Scholar ]
• Brooks, D. C., & McCormack, M. (2020). Driving Digital Transformation in Higher Education . Retrieved 30 June 2022 from: https://library.educause.edu/-/media/files/library/2020/6/dx2020.pdf?la=en&hash=28FB8C377B59AFB1855C225BBA8E3CFBB0A271DA
• Cachia, R., Chaudron, S., Di Gioia, R., Velicu, A., & Vuorikari, R. (2021). Emergency remote schooling during COVID-19, a closer look at European families. Retrieved 30 June 2022 from  https://publications.jrc.ec.europa.eu/repository/handle/JRC125787
• Çelik B. The effects of computer simulations on students’ science process skills: Literature review. Canadian Journal of Educational and Social Studies. 2022; 2 (1):16–28. doi: 10.53103/cjess.v2i1.17. [ CrossRef ] [ Google Scholar ]
• Chapman, C., & Sammons, P. (2013). School Self-Evaluation for School Improvement: What Works and Why? . CfBT Education Trust. 60 Queens Road, Reading, RG1 4BS, England.
• Chauhan S. A meta-analysis of the impact of technology on learning effectiveness of elementary students. Computers & Education. 2017; 105 :14–30. doi: 10.1016/j.compedu.2016.11.005. [ CrossRef ] [ Google Scholar ]
• Chen, Q., Chan, K. L., Guo, S., Chen, M., Lo, C. K. M., & Ip, P. (2022a). Effectiveness of digital health interventions in reducing bullying and cyberbullying: a meta-analysis. Trauma, Violence, & Abuse , 15248380221082090. 10.1177/15248380221082090 [ PubMed ]
• Chen B, Wang Y, Wang L. The effects of virtual reality-assisted language learning: A meta-analysis. Sustainability. 2022; 14 (6):3147. doi: 10.3390/su14063147. [ CrossRef ] [ Google Scholar ]
• Cheok ML, Wong SL. Predictors of e-learning satisfaction in teaching and learning for school teachers: A literature review. International Journal of Instruction. 2015; 8 (1):75–90. doi: 10.12973/iji.2015.816a. [ CrossRef ] [ Google Scholar ]
• Cheung, A. C., & Slavin, R. E. (2011). The Effectiveness of Education Technology for Enhancing Reading Achievement: A Meta-Analysis. Center for Research and reform in Education .
• Coban, M., Bolat, Y. I., & Goksu, I. (2022). The potential of immersive virtual reality to enhance learning: A meta-analysis. Educational Research Review , 100452. 10.1016/j.edurev.2022.100452
• Condie, R., & Munro, R. K. (2007). The impact of ICT in schools-a landscape review. Retrieved 30 June 2022 from: https://oei.org.ar/ibertic/evaluacion/sites/default/files/biblioteca/33_impact_ict_in_schools.pdf
• Conrads, J., Rasmussen, M., Winters, N., Geniet, A., Langer, L., (2017). Digital Education Policies in Europe and Beyond: Key Design Principles for More Effective Policies. Redecker, C., P. Kampylis, M. Bacigalupo, Y. Punie (ed.), EUR 29000 EN, Publications Office of the European Union, Luxembourg, 10.2760/462941
• Costa P, Castaño-Muñoz J, Kampylis P. Capturing schools’ digital capacity: Psychometric analyses of the SELFIE self-reflection tool. Computers & Education. 2021; 162 :104080. doi: 10.1016/j.compedu.2020.104080. [ CrossRef ] [ Google Scholar ]
• Cussó-Calabuig R, Farran XC, Bosch-Capblanch X. Effects of intensive use of computers in secondary school on gender differences in attitudes towards ICT: A systematic review. Education and Information Technologies. 2018; 23 (5):2111–2139. doi: 10.1007/s10639-018-9706-6. [ CrossRef ] [ Google Scholar ]
• Daniel SJ. Education and the COVID-19 pandemic. Prospects. 2020; 49 (1):91–96. doi: 10.1007/s11125-020-09464-3. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Delcker J, Ifenthaler D. Teachers’ perspective on school development at German vocational schools during the Covid-19 pandemic. Technology, Pedagogy and Education. 2021; 30 (1):125–139. doi: 10.1080/1475939X.2020.1857826. [ CrossRef ] [ Google Scholar ]
• Delgado, A., Wardlow, L., O’Malley, K., & McKnight, K. (2015). Educational technology: A review of the integration, resources, and effectiveness of technology in K-12 classrooms. Journal of Information Technology Education Research , 14, 397. Retrieved 30 June 2022 from  http://www.jite.org/documents/Vol14/JITEv14ResearchP397-416Delgado1829.pdf
• De Silva MJ, Breuer E, Lee L, Asher L, Chowdhary N, Lund C, Patel V. Theory of change: A theory-driven approach to enhance the Medical Research Council's framework for complex interventions. Trials. 2014; 15 (1):1–13. doi: 10.1186/1745-6215-15-267. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Di Pietro G, Biagi F, Costa P, Karpiński Z, Mazza J. The likely impact of COVID-19 on education: Reflections based on the existing literature and recent international datasets. Publications Office of the European Union; 2020. [ Google Scholar ]
• Elkordy A, Lovinelli J. Competencies, Culture, and Change: A Model for Digital Transformation in K12 Educational Contexts. In: Ifenthaler D, Hofhues S, Egloffstein M, Helbig C, editors. Digital Transformation of Learning Organizations. Springer; 2020. pp. 203–219. [ Google Scholar ]
• Eng TS. The impact of ICT on learning: A review of research. International Education Journal. 2005; 6 (5):635–650. [ Google Scholar ]
• European Commission. (2020). Digital Education Action Plan 2021 – 2027. Resetting education and training for the digital age. Retrieved 30 June 2022 from  https://ec.europa.eu/education/sites/default/files/document-library-docs/deap-communication-sept2020_en.pdf
• European Commission. (2019). 2 nd survey of schools: ICT in education. Objective 1: Benchmark progress in ICT in schools . Retrieved 30 June 2022 from: https://data.europa.eu/euodp/data/storage/f/2019-03-19T084831/FinalreportObjective1-BenchmarkprogressinICTinschools.pdf
• Eurydice. (2019). Digital Education at School in Europe , Luxembourg: Publications Office of the European Union. Retrieved 30 June 2022 from: https://eacea.ec.europa.eu/national-policies/eurydice/content/digital-education-school-europe_en
• Escueta, M., Quan, V., Nickow, A. J., & Oreopoulos, P. (2017). Education technology: An evidence-based review. Retrieved 30 June 2022 from  https://ssrn.com/abstract=3031695
• Fadda D, Pellegrini M, Vivanet G, Zandonella Callegher C. Effects of digital games on student motivation in mathematics: A meta-analysis in K-12. Journal of Computer Assisted Learning. 2022; 38 (1):304–325. doi: 10.1111/jcal.12618. [ CrossRef ] [ Google Scholar ]
• Fernández-Gutiérrez M, Gimenez G, Calero J. Is the use of ICT in education leading to higher student outcomes? Analysis from the Spanish Autonomous Communities. Computers & Education. 2020; 157 :103969. doi: 10.1016/j.compedu.2020.103969. [ CrossRef ] [ Google Scholar ]
• Ferrari, A., Cachia, R., & Punie, Y. (2011). Educational change through technology: A challenge for obligatory schooling in Europe. Lecture Notes in Computer Science , 6964 , 97–110. Retrieved 30 June 2022  https://link.springer.com/content/pdf/10.1007/978-3-642-23985-4.pdf
• Fielding, K., & Murcia, K. (2022). Research linking digital technologies to young children’s creativity: An interpretive framework and systematic review. Issues in Educational Research , 32 (1), 105–125. Retrieved 30 June 2022 from  http://www.iier.org.au/iier32/fielding-abs.html
• Friedel, H., Bos, B., Lee, K., & Smith, S. (2013). The impact of mobile handheld digital devices on student learning: A literature review with meta-analysis. In Society for Information Technology & Teacher Education International Conference (pp. 3708–3717). Association for the Advancement of Computing in Education (AACE).
• Fu JS. ICT in education: A critical literature review and its implications. International Journal of Education and Development Using Information and Communication Technology (IJEDICT) 2013; 9 (1):112–125. [ Google Scholar ]
• Gaol FL, Prasolova-Førland E. Special section editorial: The frontiers of augmented and mixed reality in all levels of education. Education and Information Technologies. 2022; 27 (1):611–623. doi: 10.1007/s10639-021-10746-2. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Garzón J, Acevedo J. Meta-analysis of the impact of Augmented Reality on students’ learning gains. Educational Research Review. 2019; 27 :244–260. doi: 10.1016/j.edurev.2019.04.001. [ CrossRef ] [ Google Scholar ]
• Garzón, J., Baldiris, S., Gutiérrez, J., & Pavón, J. (2020). How do pedagogical approaches affect the impact of augmented reality on education? A meta-analysis and research synthesis. Educational Research Review , 100334. 10.1016/j.edurev.2020.100334
• Grgurović M, Chapelle CA, Shelley MC. A meta-analysis of effectiveness studies on computer technology-supported language learning. ReCALL. 2013; 25 (2):165–198. doi: 10.1017/S0958344013000013. [ CrossRef ] [ Google Scholar ]
• Haßler B, Major L, Hennessy S. Tablet use in schools: A critical review of the evidence for learning outcomes. Journal of Computer Assisted Learning. 2016; 32 (2):139–156. doi: 10.1111/jcal.12123. [ CrossRef ] [ Google Scholar ]
• Haleem A, Javaid M, Qadri MA, Suman R. Understanding the role of digital technologies in education: A review. Sustainable Operations and Computers. 2022; 3 :275–285. doi: 10.1016/j.susoc.2022.05.004. [ CrossRef ] [ Google Scholar ]
• Hardman J. Towards a pedagogical model of teaching with ICTs for mathematics attainment in primary school: A review of studies 2008–2018. Heliyon. 2019; 5 (5):e01726. doi: 10.1016/j.heliyon.2019.e01726. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Hattie J, Rogers HJ, Swaminathan H. The role of meta-analysis in educational research. In: Reid AD, Hart P, Peters MA, editors. A companion to research in education. Springer; 2014. pp. 197–207. [ Google Scholar ]
• Hattie J. Visible learning: A synthesis of over 800 meta-analyses relating to achievement. Routledge. 2008 doi: 10.4324/9780203887332. [ CrossRef ] [ Google Scholar ]
• Higgins S, Xiao Z, Katsipataki M. The impact of digital technology on learning: A summary for the education endowment foundation. Education Endowment Foundation and Durham University; 2012. [ Google Scholar ]
• Higgins, K., Huscroft-D’Angelo, J., & Crawford, L. (2019). Effects of technology in mathematics on achievement, motivation, and attitude: A meta-analysis. Journal of Educational Computing Research , 57(2), 283-319.
• Hillmayr D, Ziernwald L, Reinhold F, Hofer SI, Reiss KM. The potential of digital tools to enhance mathematics and science learning in secondary schools: A context-specific meta-analysis. Computers & Education. 2020; 153 (1038):97. doi: 10.1016/j.compedu.2020.103897. [ CrossRef ] [ Google Scholar ]
• Istenic Starcic A, Bagon S. ICT-supported learning for inclusion of people with special needs: Review of seven educational technology journals, 1970–2011. British Journal of Educational Technology. 2014; 45 (2):202–230. doi: 10.1111/bjet.12086. [ CrossRef ] [ Google Scholar ]
• Jewitt C, Clark W, Hadjithoma-Garstka C. The use of learning platforms to organise learning in English primary and secondary schools. Learning, Media and Technology. 2011; 36 (4):335–348. doi: 10.1080/17439884.2011.621955. [ CrossRef ] [ Google Scholar ]
• JISC. (2020). What is digital transformation?.  Retrieved 30 June 2022 from: https://www.jisc.ac.uk/guides/digital-strategy-framework-for-university-leaders/what-is-digital-transformation
• Kalati, A. T., & Kim, M. S. (2022). What is the effect of touchscreen technology on young children’s learning?: A systematic review. Education and Information Technologies , 1-19. 10.1007/s10639-021-10816-5
• Kalemkuş, J., & Kalemkuş, F. (2022). Effect of the use of augmented reality applications on academic achievement of student in science education: Meta-analysis review. Interactive Learning Environments , 1-18. 10.1080/10494820.2022.2027458
• Kao C-W. The effects of digital game-based learning task in English as a foreign language contexts: A meta-analysis. Education Journal. 2014; 42 (2):113–141. [ Google Scholar ]
• Kampylis P, Punie Y, Devine J. Promoting effective digital-age learning - a European framework for digitally competent educational organisations. JRC Technical Reports. 2015 doi: 10.2791/54070. [ CrossRef ] [ Google Scholar ]
• Kazu IY, Yalçin CK. Investigation of the effectiveness of hybrid learning on academic achievement: A meta-analysis study. International Journal of Progressive Education. 2022; 18 (1):249–265. doi: 10.29329/ijpe.2022.426.14. [ CrossRef ] [ Google Scholar ]
• Koh C. A qualitative meta-analysis on the use of serious games to support learners with intellectual and developmental disabilities: What we know, what we need to know and what we can do. International Journal of Disability, Development and Education. 2022; 69 (3):919–950. doi: 10.1080/1034912X.2020.1746245. [ CrossRef ] [ Google Scholar ]
• König J, Jäger-Biela DJ, Glutsch N. Adapting to online teaching during COVID-19 school closure: Teacher education and teacher competence effects among early career teachers in Germany. European Journal of Teacher Education. 2020; 43 (4):608–622. doi: 10.1080/02619768.2020.1809650. [ CrossRef ] [ Google Scholar ]
• Lawrence JE, Tar UA. Factors that influence teachers’ adoption and integration of ICT in teaching/learning process. Educational Media International. 2018; 55 (1):79–105. doi: 10.1080/09523987.2018.1439712. [ CrossRef ] [ Google Scholar ]
• Lee, S., Kuo, L. J., Xu, Z., & Hu, X. (2020). The effects of technology-integrated classroom instruction on K-12 English language learners’ literacy development: A meta-analysis. Computer Assisted Language Learning , 1-32. 10.1080/09588221.2020.1774612
• Lei, H., Chiu, M. M., Wang, D., Wang, C., & Xie, T. (2022a). Effects of game-based learning on students’ achievement in science: a meta-analysis. Journal of Educational Computing Research . 10.1177/07356331211064543
• Lei H, Wang C, Chiu MM, Chen S. Do educational games affect students' achievement emotions? Evidence from a meta-analysis. Journal of Computer Assisted Learning. 2022; 38 (4):946–959. doi: 10.1111/jcal.12664. [ CrossRef ] [ Google Scholar ]
• Liao YKC, Chang HW, Chen YW. Effects of computer application on elementary school student's achievement: A meta-analysis of students in Taiwan. Computers in the Schools. 2007; 24 (3–4):43–64. doi: 10.1300/J025v24n03_04. [ CrossRef ] [ Google Scholar ]
• Li Q, Ma X. A meta-analysis of the effects of computer technology on school students’ mathematics learning. Educational Psychology Review. 2010; 22 (3):215–243. doi: 10.1007/s10648-010-9125-8. [ CrossRef ] [ Google Scholar ]
• Liu, M., Pang, W., Guo, J., & Zhang, Y. (2022). A meta-analysis of the effect of multimedia technology on creative performance. Education and Information Technologies , 1-28. 10.1007/s10639-022-10981-1
• Lu Z, Chiu MM, Cui Y, Mao W, Lei H. Effects of game-based learning on students’ computational thinking: A meta-analysis. Journal of Educational Computing Research. 2022 doi: 10.1177/07356331221100740. [ CrossRef ] [ Google Scholar ]
• Martinez L, Gimenes M, Lambert E. Entertainment video games for academic learning: A systematic review. Journal of Educational Computing Research. 2022 doi: 10.1177/07356331211053848. [ CrossRef ] [ Google Scholar ]
• Mayne J. Useful theory of change models. Canadian Journal of Program Evaluation. 2015; 30 (2):119–142. doi: 10.3138/cjpe.230. [ CrossRef ] [ Google Scholar ]
• Moran J, Ferdig RE, Pearson PD, Wardrop J, Blomeyer RL., Jr Technology and reading performance in the middle-school grades: A meta-analysis with recommendations for policy and practice. Journal of Literacy Research. 2008; 40 (1):6–58. doi: 10.1080/10862960802070483. [ CrossRef ] [ Google Scholar ]
• OECD. (2015). Students, Computers and Learning: Making the Connection . PISA, OECD Publishing, Paris. Retrieved from: 10.1787/9789264239555-en
• OECD. (2021). OECD Digital Education Outlook 2021: Pushing the Frontiers with Artificial Intelligence, Blockchain and Robots. Retrieved from: https://www.oecd-ilibrary.org/education/oecd-digital-education-outlook-2021_589b283f-en
• Pan Y, Ke F, Xu X. A systematic review of the role of learning games in fostering mathematics education in K-12 settings. Educational Research Review. 2022; 36 :100448. doi: 10.1016/j.edurev.2022.100448. [ CrossRef ] [ Google Scholar ]
• Pettersson F. Understanding digitalization and educational change in school by means of activity theory and the levels of learning concept. Education and Information Technologies. 2021; 26 (1):187–204. doi: 10.1007/s10639-020-10239-8. [ CrossRef ] [ Google Scholar ]
• Pihir, I., Tomičić-Pupek, K., & Furjan, M. T. (2018). Digital transformation insights and trends. In Central European Conference on Information and Intelligent Systems (pp. 141–149). Faculty of Organization and Informatics Varazdin. Retrieved 30 June 2022 from https://www.proquest.com/conference-papers-proceedings/digital-transformation-insights-trends/docview/2125639934/se-2
• Punie, Y., Zinnbauer, D., & Cabrera, M. (2006). A review of the impact of ICT on learning. Working Paper prepared for DG EAC. Retrieved 30 June 2022 from: http://www.eurosfaire.prd.fr/7pc/doc/1224678677_jrc47246n.pdf
• Quah CY, Ng KH. A systematic literature review on digital storytelling authoring tool in education: January 2010 to January 2020. International Journal of Human-Computer Interaction. 2022; 38 (9):851–867. doi: 10.1080/10447318.2021.1972608. [ CrossRef ] [ Google Scholar ]
• Ran H, Kim NJ, Secada WG. A meta-analysis on the effects of technology's functions and roles on students' mathematics achievement in K-12 classrooms. Journal of computer assisted learning. 2022; 38 (1):258–284. doi: 10.1111/jcal.12611. [ CrossRef ] [ Google Scholar ]
• Ređep, N. B. (2021). Comparative overview of the digital preparedness of education systems in selected CEE countries. Center for Policy Studies. CEU Democracy Institute .
• Rott, B., & Marouane, C. (2018). Digitalization in schools–organization, collaboration and communication. In Digital Marketplaces Unleashed (pp. 113–124). Springer, Berlin, Heidelberg.
• Savva M, Higgins S, Beckmann N. Meta-analysis examining the effects of electronic storybooks on language and literacy outcomes for children in grades Pre-K to grade 2. Journal of Computer Assisted Learning. 2022; 38 (2):526–564. doi: 10.1111/jcal.12623. [ CrossRef ] [ Google Scholar ]
• Schmid RF, Bernard RM, Borokhovski E, Tamim RM, Abrami PC, Surkes MA, Wade CA, Woods J. The effects of technology use in postsecondary education: A meta-analysis of classroom applications. Computers & Education. 2014; 72 :271–291. doi: 10.1016/j.compedu.2013.11.002. [ CrossRef ] [ Google Scholar ]
• Schuele CM, Justice LM. The importance of effect sizes in the interpretation of research: Primer on research: Part 3. The ASHA Leader. 2006; 11 (10):14–27. doi: 10.1044/leader.FTR4.11102006.14. [ CrossRef ] [ Google Scholar ]
• Schwabe, A., Lind, F., Kosch, L., & Boomgaarden, H. G. (2022). No negative effects of reading on screen on comprehension of narrative texts compared to print: A meta-analysis. Media Psychology , 1-18. 10.1080/15213269.2022.2070216
• Sellar S. Data infrastructure: a review of expanding accountability systems and large-scale assessments in education. Discourse: Studies in the Cultural Politics of Education. 2015; 36 (5):765–777. doi: 10.1080/01596306.2014.931117. [ CrossRef ] [ Google Scholar ]
• Stock WA. Systematic coding for research synthesis. In: Cooper H, Hedges LV, editors. The handbook of research synthesis, 236. Russel Sage; 1994. pp. 125–138. [ Google Scholar ]
• Su, J., Zhong, Y., & Ng, D. T. K. (2022). A meta-review of literature on educational approaches for teaching AI at the K-12 levels in the Asia-Pacific region. Computers and Education: Artificial Intelligence , 100065. 10.1016/j.caeai.2022.100065
• Su J, Yang W. Artificial intelligence in early childhood education: A scoping review. Computers and Education: Artificial Intelligence. 2022; 3 :100049. doi: 10.1016/j.caeai.2022.100049. [ CrossRef ] [ Google Scholar ]
• Sung YT, Chang KE, Liu TC. The effects of integrating mobile devices with teaching and learning on students' learning performance: A meta-analysis and research synthesis. Computers & Education. 2016; 94 :252–275. doi: 10.1016/j.compedu.2015.11.008. [ CrossRef ] [ Google Scholar ]
• Talan T, Doğan Y, Batdı V. Efficiency of digital and non-digital educational games: A comparative meta-analysis and a meta-thematic analysis. Journal of Research on Technology in Education. 2020; 52 (4):474–514. doi: 10.1080/15391523.2020.1743798. [ CrossRef ] [ Google Scholar ]
• Tamim, R. M., Bernard, R. M., Borokhovski, E., Abrami, P. C., & Schmid, R. F. (2011). What forty years of research says about the impact of technology on learning: A second-order meta-analysis and validation study. Review of Educational research, 81 (1), 4–28. Retrieved 30 June 2022 from 10.3102/0034654310393361
• Tamim, R. M., Borokhovski, E., Pickup, D., Bernard, R. M., & El Saadi, L. (2015). Tablets for teaching and learning: A systematic review and meta-analysis. Commonwealth of Learning. Retrieved from: http://oasis.col.org/bitstream/handle/11599/1012/2015_Tamim-et-al_Tablets-for-Teaching-and-Learning.pdf
• Tang C, Mao S, Xing Z, Naumann S. Improving student creativity through digital technology products: A literature review. Thinking Skills and Creativity. 2022; 44 :101032. doi: 10.1016/j.tsc.2022.101032. [ CrossRef ] [ Google Scholar ]
• Tolani-Brown, N., McCormac, M., & Zimmermann, R. (2011). An analysis of the research and impact of ICT in education in developing country contexts. In ICTs and sustainable solutions for the digital divide: Theory and perspectives (pp. 218–242). IGI Global.
• Trucano, M. (2005). Knowledge Maps: ICTs in Education. Washington, DC: info Dev / World Bank. Retrieved 30 June 2022 from  https://files.eric.ed.gov/fulltext/ED496513.pdf
• Ulum H. The effects of online education on academic success: A meta-analysis study. Education and Information Technologies. 2022; 27 (1):429–450. doi: 10.1007/s10639-021-10740-8. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Underwood, J. D. (2009). The impact of digital technology: A review of the evidence of the impact of digital technologies on formal education. Retrieved 30 June 2022 from: http://dera.ioe.ac.uk/id/eprint/10491
• Verschaffel, L., Depaepe, F., & Mevarech, Z. (2019). Learning Mathematics in metacognitively oriented ICT-Based learning environments: A systematic review of the literature. Education Research International , 2019 . 10.1155/2019/3402035
• Villena-Taranilla R, Tirado-Olivares S, Cózar-Gutiérrez R, González-Calero JA. Effects of virtual reality on learning outcomes in K-6 education: A meta-analysis. Educational Research Review. 2022; 35 :100434. doi: 10.1016/j.edurev.2022.100434. [ CrossRef ] [ Google Scholar ]
• Voogt J, Knezek G, Cox M, Knezek D, ten Brummelhuis A. Under which conditions does ICT have a positive effect on teaching and learning? A call to action. Journal of Computer Assisted Learning. 2013; 29 (1):4–14. doi: 10.1111/j.1365-2729.2011.00453.x. [ CrossRef ] [ Google Scholar ]
• Vuorikari, R., Punie, Y., & Cabrera, M. (2020). Emerging technologies and the teaching profession: Ethical and pedagogical considerations based on near-future scenarios  (No. JRC120183). Joint Research Centre. Retrieved 30 June 2022 from: https://publications.jrc.ec.europa.eu/repository/handle/JRC120183
• Wang LH, Chen B, Hwang GJ, Guan JQ, Wang YQ. Effects of digital game-based STEM education on students’ learning achievement: A meta-analysis. International Journal of STEM Education. 2022; 9 (1):1–13. doi: 10.1186/s40594-022-00344-0. [ CrossRef ] [ Google Scholar ]
• Wen X, Walters SM. The impact of technology on students’ writing performances in elementary classrooms: A meta-analysis. Computers and Education Open. 2022; 3 :100082. doi: 10.1016/j.caeo.2022.100082. [ CrossRef ] [ Google Scholar ]
• Zheng B, Warschauer M, Lin CH, Chang C. Learning in one-to-one laptop environments: A meta-analysis and research synthesis. Review of Educational Research. 2016; 86 (4):1052–1084. doi: 10.3102/0034654316628645. [ CrossRef ] [ Google Scholar ]

• Our Mission

10 Teacher Picks for Best Tech Tools

Teachers and administrators from pre-K through 12th grade named these tools their top picks for this year and beyond.

If there was ever a year for teachers to beg, borrow, and steal good ideas, this is it. All good teachers know this is how we get better, and this is a curated list of tech tools that I have begged and borrowed from others—and I didn’t have to steal any of these as there has never been a year when teachers were more generous.

Based on my own teaching of college students as well as the responses of 1,461 virtual learning academy participants—pre-K to 12 teachers and administrators—to survey questions on impactful tools that I conducted from May to December 2020, and over 70 webinars and virtual learning sessions I’ve conducted in that time, these are the top teacher-tested tech tools I have identified. My learning has happened with my own students and through my own mistakes, and I received great ideas from educators across the U.S. and around the world.

I will continue to use these tools and recommend their use regardless of student level or how we deliver education moving forward.

Top Tech Tools for Educators

10. Parlay : For those of us who like to discuss rich texts with students, Parlay lets us connect with students remotely, facilitate discussion, and track how the dialogue builds. As students respond, Parlay shows who is contributing and where the conversation is moving as the software visually tracks student responses in a discussion web. Teachers can use built-in tools to assess the frequency of students’ answers in real time even when students are not in the same room. Teachers can then spend their time on deeper level assessment of the depth of student responses, which can also be recorded.

9. Flip : One of the most popular tech tools in schools, Flip won praise from teachers across the country because of the flexibility it gives students to submit digital projects and how it effectively supports peer and teacher feedback.

8. Edpuzzle : I’ve used Edpuzzle for a while, but it became much more vital as more of my content shifted online. Teachers use Edpuzzle to make video clips interactive by requiring student responses, which are easy to collect and assess.

7. Pear Deck : Pear Deck does to Google Slides what Edpuzzle does to videos. Slides become interactive, and teachers are able to collect feedback immediately.

6. Prezi : I’ve used the virtual presentation software Prezi for years, and I appreciate a recent update that allows me to be on the same screen with the graphics, creating a more engaging presentation. Prezi offers teachers another tool to capture short lectures, explanations, or other content in a more visually appealing and personal way than as a disembodied voice or thumbnail in the bottom corner of a screen.

5. Screencastify : Every student can be Sal Khan working out problems with explanations. This tool was first recommended to me by a teacher in Kenya, who explained how Screencastify transformed her math assessments by allowing students to show what they’re thinking from wherever they happen to be working. Screencastify is also valuable for reducing cheating as teachers can observe students working and explaining problems instead of just recording answers.

4. Mural : This has been a lifesaver for virtual collaboration. Mural allows teachers, students, and other contributors to write on virtual sticky notes and then organize and reorganize them in real time. The best in-person meetings are always the ones where the collective expertise of the room can be captured visually, and if we can’t be in the same room with students—or colleagues—Mural is the next best thing. Even better, there’s no need to go back and summarize or clean up evidence from the meeting. The Mural is the artifact. Many teachers are now using Jamboard in a similar way.

3. Gimkit : Created by a high school student who thought he could improve upon Kahoot! , Gimkit allows teachers to create question sets that students can answer over and over again while competing against each other, which is great for surface learning and review. Because Gimkit allows for repetition of answers and has a variety of ways for students to earn points, students remain engaged as they work at their own pace.

2. Mentimeter and Slido : These are both excellent for collecting feedback from groups, so I’ve ranked them together. I use these almost weekly for professional learning and in my classes. Slido allows participants to ask questions and then upvote others. There are many similar tools, but Slido is easy and free. Mentimeter allows students and teachers to collect real-time data on questions they have, in the form of word clouds, rankings, and various scales. These are great discussion starters that allow everyone to contribute to the collective wisdom of the group.

1. Learning management system: A good LMS is key to reducing stress for teachers, students, and parents. A list like this one would be counterproductive if it left your educational delivery fragmented among disparate tools, and a good LMS helps you organize everything into a one-stop shop.

I personally love Canvas and Schoology , but I know many teachers have worked miracles with Google Classroom , which is “free.” I use those quotes for a reason: Google Classroom is only truly free if it is not requiring significantly more human capital in the form of time and energy than a fee-for-service platform like Canvas or Schoology. The most significant asset for managing learning in the chaos of this school year has been staying organized, and Canvas has been a lifesaver for me.

I have taught virtually, led professional learning across many time zones, delivered content asynchronously, and taught students in masks with others Zooming in due to quarantine, Covid, or personal preference. For me, nothing will ever replace in-person teaching and learning, but like many other teachers and administrators, I now know how to effectively facilitate learning in a previously unimaginable set of circumstances. This school year’s desperation has driven us to explore a wide range of tools, and we can be better because of the firehose learning we have done this year. We just need a few tools to create some space for us to breathe.

• Educational Assessment

The Importance of Educational Assessment: Tools and Techniques for Assessing Your Students

• June 23, 2022
• Faculty Focus

In this educational assessment guide, we’ll answer questions such as:

What is the purpose of educational assessment? What are the benefits of assessments? What are the main types of assessment? What is the difference between evaluation and assessment?

Formative assessment can be any assessment that first and foremost promotes students’ learning. Many refer to this type of assessment as assessment “for” learning. In contrast, summative assessment, or assessment “of” learning, looks at grades or scores that give a final judgment or evaluation of proficiency. Assessment “for” learning is usually more informal and includes aspects of teaching. It is formative because it gathers evidence that helps teachers better meet the learning needs of students as well as empowering students to be change agents in their achievement. A host of studies have shown that when formative assessment is implemented effectively, it can greatly enhance, or even double, the speed of student learning. This guide provides specific examples of educational assessment tools, why assessment is important in gauging student comprehension, and how you can implement assessment techniques into your own course.

The importance of educational assessment

Educational assessment is one of the most talked about topics in higher education today. Despite the admirable goal of improving student learning, the trend toward greater accountability through increased academic testing carries with it a diverse range of educational assessment tools, methodologies, perspectives, and stakeholders.

If today’s mandates for educational testing has you searching for answers, take a dive into the following articles and products that cover topics such as forms of assessment, cognitive demand levels, formative and summative assessments, alternative assessment methods, and evaluative assessment.

Free articles

• Assessment for Learning: It Just Makes Sense
• Four Reasons Assessment Doesn’t Work and What We Can Do About It
• Re-envisioning Online Course Revision
• Educational Assessment: A Different Kind of Feedback

Teaching Professor articles ( requires paid subscription )

• The Link Between Self-assessment and Examination Performance
• Does Self- and Peer Assessment Improve Learning in Groups?
• What Fitness Bands Can Teach Us about Classroom Assessment

Related Products

Each 20-Minute Mentor is $19 for individual, on-demand, one-week access, and each Magna Online Seminar is $39 for individual, on-demand, one-week access.

2 0-Minute Mentor: How Can I Build a Mindset for Assessment in an Academic Department?

20-Minute Mentor: How Can I Analyze Department Data and Move Towards Closing the Assessment Loop?

Types of educational assessment

While most faculty stick with the tried-and-true quiz and paper assessment strategies for their online courses, the wide range of technologies available today offers a variety of assessment options beyond the traditional forms. But what do students think of these different forms? And more importantly, what types of educational assessment work best for your course and your students?

A few educational assessments that we will cover:

• A learning assessment technique (LAT) is a three-part integrated structure that helps teachers to first identify significant learning goals, then to implement effectively the kinds of learning activities that help achieve those goals, and finally—and perhaps most importantly—to analyze and report on the learning outcomes that have been achieved from those learning activities.
• Word Clouds.  Word clouds are images composed of words associated with concepts, questions, or reactions sought by an instructor; they are fast, engaging, and can provide an emotional connection for students. Think of the powerful insights a facilitator gains by simply asking students to report a single word describing how they feel about their progress on a project? choice?”). Wordle and TagCloud are two popular choices for creating word clouds.
• Focused Listing.  Focused listing can be used before, during, or after a lesson. This method helps you to gauge student learning.
• Elevator Pitch.  As a review activity, ask students to summarize main ideas or key topics in fewer than 60 seconds. A fun variation of this approach is to have students present to a classmate acting as a well-known personality or theorist who works in your discipline.

Browse the articles and products below to find what educational assessments will work best for your course!

• This Semester, Don’t Forget Participation Feedback!
• Assessments by Design: Rethinking Assessment for Learner Variability
• Reimagining Classroom Community, Assessment, and Our Own Self-care
• Which Assessment Strategies Do Students Prefer?
• Using Interview to Assess and Mentor Students
• Using Screencasts for Formative and Summative Assessment
• Formative Assessment: The Secret Sauce of Blended Success
• Students as Formative Assessment Partners
• Formative Assessment Techniques for Online Learning
• Writing Questions about the Reading: A Formative Assessment Technique
• Three Learning Assessment Techniques to Gauge Student Learning
• Informal Assessment Activities for Blended and Online Courses
• Five Classroom Assessment Techniques for the Online Classroom

Using Authentic Assessment to Assess Student Learning

Magna Online Seminar: Incorporating 360-Degree Assessment into Your Classroom

Magna Online Seminar: Assessment Strategies for Mastery Learning in Large Session Classes

Magna Online Seminar: Assessment Strategies for the Flipped Classroom

Educational assessment tools

Most conventional assessment strategies provide limited opportunities for instructors to realign teaching methods and revisit topics that students have not understood well. Teachers can communicate with students individually, but time constraints may prevent multiple individual conversations. Some students in the classroom are reluctant to ask questions and admit confusion. Find out how to overcome these difficulties with specific educational assessment tools. Below are just a few assessment tools that you’ll find within the articles and products.

• Continuous and Rapid Testing (CaRT): Improves communication between teachers and students
• C.A.P Model : Offers students diverse possibilities to express their understandings of course content, though the explicit aim of the creative component was to center non-dominant cultural ways of knowing, being, and making sense of the world
• Pre-formative assessment: This refers to assessments given while students are learning new material independently, before any group interaction has taken place
• An Anti-racist Form of Assessment: The CAP Model: Creative. Academic. Practical.
• Grading Exams: How Gradescope Revealed Deeper Insights into Our Teaching
• Now More Than Ever: Why Collaborative Grading Works, Even Online
• Leveraging Bloom’s Taxonomy to Elevate Discussion Boards in Online Courses
• Keeping Students Engaged: How to Rethink Your Assessments Amidst the Shift to Online Learning
• But What If They Cheat? Giving Non-proctored Online Assessments

Teaching Professor articles ( paid subscription )

• Four Strategies for Effective Assessment in a Flipped Learning Environment
• Continuous and Rapid Testing (CaRT): A Simple Tool for Assessment and Communication
• Self-Grading: The Ultimate Self-Assessment

Magna Online Seminar: Effective Writing Assessment in the Online Classroom

20-Minute Mentor: How Can I Use Classroom Assessment Techniques ( CATs) Online?

Stay Updated with Faculty Focus!

Get exclusive access to programs, reports, podcast episodes, articles, and more!

• Opens in a new tab

Welcome Back

Username or Email

Remember Me

Already a subscriber? log in here.

Please open the page in your browser to use this feature. Close

Educational Tools: An educational tools centre for students and professionals

• 4 minutes of reading  •  11 January 2024

Brisk Teaching offers an AI-powered Chrome extension for educators that can help when creating curriculum, giving feedback, assessing student writing, adjusting reading levels and translating texts. It integrates with tools like Google Docs and Slides, to help teachers to generate quizzes, lesson plans, rubrics and presentations efficiently.

Alyssa Faubion, a teacher in El Paso, Texas, expressed her passion for using SchoolAI: "I love using SchoolAI with students because it offers personalized learning experiences, tailoring content to meet each of my student's unique needs and preferences. There are pre-made spaces that make it user-friendly for educators to explore without knowing how to prompt. It's a great replacement for ChatGPT for students and allows teachers to monitor through the mission control."

SchoolAI is an AI platform designed for K–12 classrooms, offering personalized tutoring, guidance, and support. Teachers can create custom AI bots to help students with their learning, monitor student progress and set controls to keep AI on task. The platform includes real-time insights to help teachers engage and support students.

Bonnie Nieves, a science teacher in Dudley, Massachusetts, explained that she uses Diffit because it “creates differentiated thinking and writing activities from any material, website or prompt.” She loves that it “Translates to multiple languages and reading levels.” and that the “Premade graphic organizer templates encourage logical thinking, collaboration and build student self-efficacy because of the scaffolded support.”

Diffit is a platform designed for teachers to create learning materials quickly and easily. It generates leveled content tailored to students' reading abilities. Users can input text or topics and Diffit produces customized versions that are accessible to various reading levels.

Aileen Wallace, a teacher in Scotland, is a big advocate for Curipod and posted: "It enables me to devise engaging lessons that boost the confidence of even the most reserved students. Utilizing AI-generated feedback with a customizable rubric has significantly improved the responses from my students. When they come into class keen to use it, you know it's effective."

Curipod is designed to enhance classroom engagement through interactive, AI-assisted lesson planning. Teachers can generate lessons on any topic using AI, which helps create polls, personalized feedback, word clouds and other interactive elements to spark curiosity and participation among students.

Christoffer Dithmer, a teacher in Denmark, loves using the Skybox image generation tool integrated into ThingLink: "Skybox in ThingLink allows students and educators to bring their thoughts and ideas to life with 360-degree AI universes. The tool is ideal for demonstrating scientific phenomena or visualizing stories and fairytales. The tool is accessible so that all students can succeed with their projects with multiple communication tools."

ThingLink is a platform that enables users to create interactive experiences with images, video, and 360° media. The integration of Skybox by Blockade Labs means that teachers using ThingLink can now use AI to generate bespoke 360° images. These images can be annotated and customized in various styles for interactive learning experiences, gamified lessons and immersive storytelling.

Matt Miller, a teacher and author from Indiana, shared his love for Ideogram: "Finding images to teach vocabulary has been a challenge. But when you can generate custom images based on specific vocabulary, it's a dream. In my Spanish classes, we were studying animal vocabulary and sports vocabulary. Custom images of animals playing sports were a hit, and really effective for vocabulary repetitions."

Although Ideogram is not specifically an education tool, it allows teachers to create and design visual content. It provides tools for generating images based on text descriptions, making it easier for users to visualize concepts and ideas.

The Potential To Transform The Classroom

These six tools are just a few of the many platforms that were shared by educators around the world. To search for other AI platforms being used in education, many educators use the AI Educator Tools repository.

Before integrating any digital platform into your classroom, it is crucial to follow your organization's procedures regarding data protection. Always seek guidance from the people responsible for this in your school, college or university.

The teachers I speak to are saving many hours per week using tools such as the ones above. When integrated in a safe way, AI has the potential to transform the practices of any teacher. This could be you.

• Editorial Standards
• Reprints & Permissions

Join The Conversation

One Community. Many Voices. Create a free account to share your thoughts. 

Forbes Community Guidelines

Our community is about connecting people through open and thoughtful conversations. We want our readers to share their views and exchange ideas and facts in a safe space.

In order to do so, please follow the posting rules in our site's  Terms of Service.   We've summarized some of those key rules below. Simply put, keep it civil.

Your post will be rejected if we notice that it seems to contain:

• False or intentionally out-of-context or misleading information
• Insults, profanity, incoherent, obscene or inflammatory language or threats of any kind
• Attacks on the identity of other commenters or the article's author
• Content that otherwise violates our site's  terms.

User accounts will be blocked if we notice or believe that users are engaged in:

• Continuous attempts to re-post comments that have been previously moderated/rejected
• Racist, sexist, homophobic or other discriminatory comments
• Attempts or tactics that put the site security at risk
• Actions that otherwise violate our site's  terms.

So, how can you be a power user?

• Stay on topic and share your insights
• Feel free to be clear and thoughtful to get your point across
• ‘Like’ or ‘Dislike’ to show your point of view.
• Protect your community.
• Use the report tool to alert us when someone breaks the rules.

Thanks for reading our community guidelines. Please read the full list of posting rules found in our site's  Terms of Service.

• Open access
• Published: 22 May 2024

Enhancing clinical skills in pediatric trainees: a comparative study of ChatGPT-assisted and traditional teaching methods

• Hongjun Ba 1 , 2 ,
• Lili zhang 1 &
• Zizheng Yi 1  

BMC Medical Education volume  24 , Article number:  558 ( 2024 ) Cite this article

201 Accesses

Metrics details

As artificial intelligence (AI) increasingly integrates into medical education, its specific impact on the development of clinical skills among pediatric trainees needs detailed investigation. Pediatric training presents unique challenges which AI tools like ChatGPT may be well-suited to address.

This study evaluates the effectiveness of ChatGPT-assisted instruction versus traditional teaching methods on pediatric trainees’ clinical skills performance.

A cohort of pediatric trainees ( n  = 77) was randomly assigned to two groups; one underwent ChatGPT-assisted training, while the other received conventional instruction over a period of two weeks. Performance was assessed using theoretical knowledge exams and Mini-Clinical Evaluation Exercises (Mini-CEX), with particular attention to professional conduct, clinical judgment, patient communication, and overall clinical skills. Trainees’ acceptance and satisfaction with the AI-assisted method were evaluated through a structured survey.

Both groups performed similarly in theoretical exams, indicating no significant difference ( p  > 0.05). However, the ChatGPT-assisted group showed a statistically significant improvement in Mini-CEX scores ( p  < 0.05), particularly in patient communication and clinical judgment. The AI-teaching approach received positive feedback from the majority of trainees, highlighting the perceived benefits in interactive learning and skill acquisition.

ChatGPT-assisted instruction did not affect theoretical knowledge acquisition but did enhance practical clinical skills among pediatric trainees. The positive reception of the AI-based method suggests that it has the potential to complement and augment traditional training approaches in pediatric education. These promising results warrant further exploration into the broader applications of AI in medical education scenarios.

Peer Review reports

Introduction

The introduction of ChatGPT by OpenAI in November 2022 marked a watershed moment in educational technology, heralded as the third major innovation following Web 2.0’s emergence over a decade earlier [ 1 ] and the rapid expansion of e-learning driven by the COVID-19 pandemic [ 2 ]. In medical education, the integration of state-of-the-art Artificial Intelligence (AI) has been particularly transformative for pediatric clinical skills training—a field where AI is now at the forefront.

Pediatric training, with its intricate blend of extensive medical knowledge and soft skills like empathetic patient interaction, is pivotal for effective child healthcare. The need for swift decision-making, especially in emergency care settings, underscores the specialty’s complexity. Traditional teaching methods often fall short, hindered by logistical challenges and difficulties in providing a standardized training experience. AI tools such as ChatGPT offer a promising solution, with their ability to simulate complex patient interactions and thus improve pediatric trainees’ communication, clinical reasoning, and decision-making skills across diverse scenarios [ 3 , 4 ].

ChatGPT’s consistent, repeatable, and scalable learning experiences represent a significant advancement over traditional constraints, such as resource limitations and standardization challenges [ 5 ], offering a new paradigm for medical training. Its proficiency in providing immediate, personalized feedback could revolutionize the educational journey of pediatric interns. Our study seeks to investigate the full extent of this potential revolution, employing a mixed-methods approach to quantitatively and qualitatively measure the impact of ChatGPT on pediatric trainees’ clinical competencies.

Despite AI’s recognized potential within the academic community, empirical evidence detailing its influence on clinical skills development is limited [ 6 ]. Addressing this gap, our research aims to contribute substantive insights into the efficacy of ChatGPT in enhancing the clinical capabilities of pediatric trainees, establishing a new benchmark for the intersection of AI and medical education.

Participants and methods

Participants.

Our study evaluated the impact of ChatGPT-assisted instruction on the clinical skills of 77 medical interns enrolled in Sun Yat-sen University’s five-year program in 2023. The cohort, consisting of 42 males and 35 females, was randomly allocated into four groups based on practicum rotation, using a computer-generated randomization list. Each group, composed of 3–4 students, was assigned to either the ChatGPT-assisted or traditional teaching group for a two-week pediatric internship rotation. Randomization was stratified by baseline clinical examination scores to ensure group comparability.

Study design

A controlled experimental design was implemented with blind assessment. The interns were randomly assigned to the ChatGPT-assisted group (39 students) or the traditional group (38 students), with no significant differences in gender, age, or baseline clinical examination scores ( p  > 0.05). The ChatGPT-assisted group received instruction supplemented with ChatGPT version 4.0, while the traditional group received standard bedside teaching (as depicted in Fig.  1 ). Both groups encountered identical clinical case scenarios involving common pediatric conditions: Kawasaki disease, gastroenteritis, congenital heart disease, nephrotic syndrome, bronchopneumonia, and febrile convulsion. All interns had equal access to the same teaching materials, instructors, and intensity of courses. The core textbook was the 9th edition of “Pediatrics” published by the People’s Medical Publishing House. Ethical approval was obtained from the institutional review board, and informed consent was secured, with special attention to privacy concerns due to the involvement of pediatric patient data.

Study design and flow chart

Instructional implementation

Traditional teaching group, pre-rotation preparation.

Instructors designed typical cases representing common pediatric diseases and updated knowledge on the latest diagnostic and therapeutic advancements. They developed multimedia presentations detailing the presentation, diagnostic criteria, and treatment plans for each condition.

Teaching process

The teaching method during the rotation was divided into three stages:

Case introduction and demonstration

Instructors began with a detailed introduction of clinical cases, explaining diagnostic reasoning and emphasizing key aspects of medical history-taking and physical examination techniques.

Student participation

Students then conducted patient interviews and physical assessments independently, with the instructor observing. For pediatric patients, particularly infants, history was provided by the guardians.

Feedback and discussion

At the end of each session, instructors provided personalized feedback on student performance and answered questions, fostering an interactive learning environment.

ChatGPT-assisted teaching group

Educators prepared structured teaching plans focusing on common pediatric diseases and representative cases. The preparation phase involved configuring ChatGPT (version 4.0) settings to align with the educational objectives of the rotation.

The rotation was executed in four consecutive steps:

ChatGPT orientation

Students were familiarized with the functionalities and potential educational applications of ChatGPT version 4.0.

ChatGPT-driven tasks

In our study, ChatGPT version 4.0 was used as a supplementary educational tool within the curriculum. Students engaged with the AI to interactively explore dynamically generated clinical case vignettes based on pediatric medicine. These vignettes encompassed clinical presentations, history taking, physical examinations, diagnostic strategies, differential diagnoses, and treatment protocols, allowing students to query the AI to enhance their understanding of various clinical scenarios.

Students accessed clinical vignettes in both text and video formats, with video particularly effective in demonstrating physical examination techniques and communication strategies with guardians, thereby facilitating a more interactive learning experience.

ChatGPT initially guided students in forming assessments, while educators critically reviewed their work, providing immediate, personalized feedback to ensure proper development of clinical reasoning and decision-making skills. This blend of AI and direct educator involvement aimed to improve learning outcomes by leveraging AI’s scalability alongside expert educators’ insights.

Bedside clinical practice

Students practiced history-taking and physical examinations at the patient’s bedside, with information about infants provided by their guardians.

Feedback and inquiry

Instructors offered feedback on performance and addressed student queries to reinforce learning outcomes.

Assessment methods

The methods used to evaluate the interns’ post-rotation performance included three assessment tools:

Theoretical knowledge exam

Both groups completed the same closed-book exam to test their pediatric theoretical knowledge, ensuring consistency in cognitive understanding assessment.

Mini-CEX assessment

The Mini-CEX has been widely recognized as an effective and reliable method for assessing clinical skills [ 7 , 8 ]. Practical skills were evaluated using the Mini-CEX, which involved students taking histories from parents of pediatric patients and conducting physical examinations on infants, supervised by an instructor. Mini-CEX scoring utilized a nine-point scale with seven criteria, assessing history-taking, physical examination, professionalism, clinical judgment, doctor-patient communication, organizational skills, and overall competence.

History taking

This assessment measures students’ ability to accurately collect patient histories, utilize effective questioning techniques, respond to non-verbal cues, and exhibit respect, empathy, and trust, while addressing patient comfort, dignity, and confidentiality.

Physical examination

This evaluates students on informing patients about examination procedures, conducting examinations in an orderly sequence, adjusting examinations based on patient condition, attending to patient discomfort, and ensuring privacy.

Professionalism

This assesses students’ demonstration of respect, compassion, and empathy, establishment of trust, attention to patient comfort, maintenance of confidentiality, adherence to ethical standards, understanding of legal aspects, and recognition of their professional limits.

Clinical judgment

This includes evaluating students’ selection and execution of appropriate diagnostic tests and their consideration of the risks and benefits of various treatment options.

Doctor-patient communication

This involves explaining test and treatment rationales, obtaining patient consent, educating on disease management, and discussing issues effectively and timely based on disease severity.

Organizational efficiency

This measures how students prioritize based on urgency, handle patient issues efficiently, demonstrate integrative skills, understand the healthcare system, and effectively use resources for optimal service.

Overall competence

This assesses students on judgment, integration, and effectiveness in patient care, evaluating their overall capabilities in caring and efficiency.

The scale ranged from below expectations (1–3 points), meeting expectations (4–6 points), to exceeding expectations (7–9 points). To maintain assessment consistency, all Mini-CEX evaluations were conducted by a single assessor.

ChatGPT method feedback survey

Only for the ChatGPT-assisted group, the educational impact of the ChatGPT teaching method was evaluated post-rotation through a questionnaire. This survey used a self-assessment scale with a Cronbach’s Alpha coefficient of 0.812, confirming its internal consistency and reliability. Assessment items involved active learning engagement, communication skills, empathy, retention of clinical knowledge, and improvement in diagnostic reasoning. Participant satisfaction was categorized as (1) very satisfied, (2) satisfied, (3) neutral, or (4) dissatisfied.

Statistical analysis

Data were analyzed using R software (version 4.2.2) and SPSS (version 26.0). Descriptive statistics were presented as mean ± standard deviation (x ± s), and independent t-tests were performed to compare groups. Categorical data were presented as frequency and percentage (n[%]), with chi-square tests applied where appropriate. A P -value of < 0.05 was considered statistically significant. All assessors of the Mini-CEX were blinded to the group assignments to minimize bias.

Theoretical knowledge exam scores for both groups of trainees

The theoretical knowledge exam revealed comparable results between the two groups, with the ChatGPT-assisted group achieving a mean score of 92.21 ± 2.37, and the traditional teaching group scoring slightly higher at 92.38 ± 2.68. Statistical analysis using an independent t-test showed no significant difference in the exam scores (t = 0.295, p  = 0.768), suggesting that both teaching methods similarly supported the trainees’ theoretical learning.

Mini-CEX evaluation results for both groups of trainees

All trainees completed the Mini-CEX evaluation in 38 ± 0.5 min on average, with immediate post-evaluation feedback averaging 5.8 ± 0.6 min per student. The ChatGPT group demonstrated statistically significant improvement in professional conduct, clinical judgment, patient communication, and overall clinical skills compared to the traditional group. A detailed comparison of the CEX scoring for both student groups is presented in Table  1 ; Fig.  2 .

The scale scores of Mini-CEX assessment between the two groups. A : ChatGPT-assisted group; B : Traditional teaching group

Satisfaction survey results of trainees in the ChatGPT-assisted teaching

Feedback from the trainees regarding the ChatGPT-assisted teaching method was overwhelmingly positive. High levels of satisfaction and interest were reported, with no instances of dissatisfaction noted. The summary of these findings, including specific aspects of the teaching method that were rated highly by the students, is detailed in Table  2 .

The integration of ChatGPT into pediatric medical education represents a significant stride in leveraging artificial intelligence (AI) to enhance the learning process. Our findings suggest that while AI does not substantially alter outcomes in theoretical knowledge assessments, it plays a pivotal role in the advancement of clinical competencies.

The parity in theoretical examination scores between the ChatGPT-assisted and traditionally taught groups indicates that foundational medical knowledge can still be effectively acquired through existing educational frameworks. This underscores the potential of ChatGPT as a complementary, rather than a substitutive, educational instrument [ 9 , 10 ].

Mini-CEX evaluations paint a different picture, revealing the ChatGPT group’s superior performance in clinical realms. These competencies are crucial for the comprehensive development of a pediatrician and highlight the value of an interactive learning environment in bridging the gap between theory and practice [ 11 , 12 ].

The unanimous satisfaction with ChatGPT-assisted learning points to AI’s capacity to enhance student engagement. This positive response could be attributed to the personalized and interactive nature of the AI experience, catering to diverse learning styles [ 13 , 14 ]. However, it is critical to consider the potential for overreliance on technology and the need for maintaining an appropriate balance between AI and human interaction in medical training.

The ChatGPT group’s ascendency in clinical skillfulness could be a testament to the repetitive, adaptive learning scenarios proffered by AI technology. ChatGPT’s proficiency in tailoring educational content to individual performance metrics propels a more incisive and efficacious learning journey. Furthermore, the on-site, real-time feedback from evaluators is likely instrumental in consolidating clinical skillsets, echoing findings on the potency of immediate feedback in clinical education [ 15 , 16 ].

The study’s strength lies in its pioneering exploration of ChatGPT in pediatric education and the structured use of Mini-CEX for appraising clinical competencies, but it is not without limitations. The ceiling effect may have masked subtle differences in theoretical knowledge, and our small, single-center cohort limits the generalizability of our findings. The transitory nature of the study precludes assessment of long-term retention, a factor that future research should aim to elucidate [ 17 , 18 ].

Moreover, the ongoing evolution of AI and medical curricula necessitates continuous reevaluation of ChatGPT’s role in education. Future studies should explore multicenter trials, long-term outcomes, and integration strategies within existing curricula to provide deeper insights into AI’s role in medical education. Ethical and practical considerations, including data privacy, resource allocation, and cost, must also be carefully navigated to ensure that AI tools like ChatGPT are implemented responsibly and sustainably.

In conclusion, ChatGPT’s incorporation into pediatric training did not significantly affect the acquisition of theoretical knowledge but did enhance clinical skill development. The high levels of trainee satisfaction suggest that ChatGPT is a valuable adjunct to traditional educational methods, warranting further investigation and thoughtful integration into medical curricula.

Availability of data and materials

All data sets generated for this study were included in the manuscript.

Hollinderbäumer A, Hartz T, Uckert F. Education 2.0 — how has social media and web 2.0 been integrated into medical education? A systematical literature review. GMS Z Med Ausbild. 2013;30(1):Doc14. https://doi.org/10.3205/zma000857 .

Article   Google Scholar  

Turnbull D, Chugh R, Luck J. Transitioning to E-Learning during the COVID-19 pandemic: how have higher education institutions responded to the challenge? Educ Inf Technol (Dordr). 2021;26(5):6401–19. https://doi.org/10.1007/s10639-021-10633-w .

Bright TJ, Wong A, Dhurjati R, Bristow E, Bastian L, Coeytaux RR, et al. Effect of clinical decision-support systems: a systematic review. Ann Intern Med. 2012;157(1):29–43. https://doi.org/10.7326/0003-4819-157-1-201207030-00450 .

Chan KS, Zary N. Applications and challenges of implementing artificial intelligence in medical education: integrative review. JMIR Med Educ. 2019;5(1):e13930. https://doi.org/10.2196/13930 .

Geoffrey Currie. A conversation with ChatGPT. J Nucl Med Technol. 2023;51(3):255–60. https://doi.org/10.2967/jnmt.123.265864 .

Hossain E, Rana R, Higgins N, Soar J, Barua PD, Pisani AR, et al. Natural language processing in electronic health records in relation to healthcare decision-making: a systematic review. Comput Biol Med. 2023;155:106649. https://doi.org/10.1016/j.compbiomed.2023.106649 .

Mortaz Hejri S, Jalili M, Masoomi R, Shirazi M, Nedjat S, Norcini J. The utility of mini-clinical evaluation Exercise in undergraduate and postgraduate medical education: a BEME review: BEME guide 59. Med Teach. 2020;42(2):125–42. https://doi.org/10.1080/0142159X.2019.1652732 .

Motefakker S, Shirinabadi Farahani A, Nourian M, Nasiri M, Heydari F. The impact of the evaluations made by Mini-CEX on the clinical competency of nursing students. BMC Med Educ. 2022;22(1):634. https://doi.org/10.1186/s12909-022-03667-2 .

Eysenbach G. The role of ChatGPT, generative language models, and artificial intelligence in medical education: a conversation with ChatGPT and a call for papers. JMIR Med Educ. 2023;9:e46885. https://doi.org/10.2196/46885 .

Guo J, Bai L, Yu Z, Zhao Z, Wan B. An AI-application-oriented In-Class teaching evaluation model by using statistical modeling and ensemble learning. Sens (Basel). 2021;21(1):241. https://doi.org/10.3390/s21010241 .

Vishwanathaiah S, Fageeh HN, Khanagar SB, Maganur PC. Artificial Intelligence its uses and application in pediatric dentistry: a review. Biomedicines. 2023;11(3):788. https://doi.org/10.3390/biomedicines11030788 .

Cooper A, Rodman A. AI and Medical Education - A 21st-Century Pandora’s Box. N Engl J Med. 2023;389(5):385–7. https://doi.org/10.1056/NEJMp2304993 .

Arif TB, Munaf U, Ul-Haque I. The future of medical education and research: is ChatGPT a blessing or blight in disguise? Med Educ Online. 2023;28(1):2181052. https://doi.org/10.1080/10872981.2023.2181052 .

Nagi F, Salih R, Alzubaidi M, Shah H, Alam T, Shah Z, et al. Applications of Artificial Intelligence (AI) in medical education: a scoping review. Stud Health Technol Inf. 2023;305:648–51. https://doi.org/10.3233/SHTI230581 .

van de Ridder JM, Stokking KM, McGaghie WC, ten Cate OT. What is feedback in clinical education? Med Educ. 2008;42(2):189–97. https://doi.org/10.1111/j.1365-2923.2007.02973.x .

Wilbur K, BenSmail N, Ahkter S. Student feedback experiences in a cross-border medical education curriculum. Int J Med Educ. 2019;10:98–105. https://doi.org/10.5116/ijme.5ce1.149f .

Morrow E, Zidaru T, Ross F, Mason C, Patel KD, Ream M, et al. Artificial intelligence technologies and compassion in healthcare: a systematic scoping review. Front Psychol. 2023;13:971044. https://doi.org/10.3389/fpsyg.2022.971044 .

Soong TK, Ho CM. Artificial intelligence in medical OSCEs: reflections and future developments. Adv Med Educ Pract. 2021;12:167–73. https://doi.org/10.2147/AMEP.S287926 .

Download references

Acknowledgements

Author information, authors and affiliations.

Department of Pediatric Cardiology, Heart Center, First Affiliated Hospital of Sun Yat-sen University, 58# Zhongshan Road 2, Guangzhou, 510080, China

Hongjun Ba, Lili zhang & Zizheng Yi

Key Laboratory on Assisted Circulation, Ministry of Health, 58# Zhongshan Road 2, Guangzhou, 510080, China

You can also search for this author in PubMed   Google Scholar

Contributions

H-JB conceived and designed the ideas for the manuscript. H-JB, L-LZ, and Z-ZY participated in all data collection and processing. H-JB was the major contributors in organizing records and drafting the manuscript. All authors proofread and approved the manuscript.

Corresponding author

Correspondence to Hongjun Ba .

Ethics declarations

Ethics approval and consent to participate.

Ethical approval was obtained from the the Ethics Committee of The First Affiliated Hospital, Sun Yat-sen University, and informed consent was secured, with special attention to privacy concerns due to the involvement of pediatric patient data.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Ba, H., zhang, L. & Yi, Z. Enhancing clinical skills in pediatric trainees: a comparative study of ChatGPT-assisted and traditional teaching methods. BMC Med Educ 24 , 558 (2024). https://doi.org/10.1186/s12909-024-05565-1

Download citation

Received : 01 March 2024

Accepted : 16 May 2024

Published : 22 May 2024

DOI : https://doi.org/10.1186/s12909-024-05565-1

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Pediatric training
• Clinical skills
• Medical education
• AI in healthcare
• Mini-clinical evaluation exercise
• Interactive learning

BMC Medical Education

ISSN: 1472-6920

• Submission enquiries: [email protected]
• General enquiries: [email protected]

Search Products & Information | Logitech United States

Free standard shipping on orders over $29.00

• Express Delivery
• Free Shipping
• Ukraine promo
• Carbon Neutral

Educational Tools

• Price - Low to High
• Price - High to Low

Logitech Crayon Digital Pencil in Silver

Logitech Rugged Folio - Protective Keyboard Case for iPad in Graphite

Rugged Folio for iPad® (7th and 8th generation) is a slim keyboard case with drop protection that exceeds military standard tests. It uses Smart Connector technology so no pairing or charging is required. High performance spill and dirt-resistant sealed keys make this the perfect tool for on-the-go work. Enjoy a full-size keyboard with convenient iOS shortcut keys, front and back protection for your iPad, and a convenient spot to stow Logitech Crayon or Apple Pencil® (1st gen). | Logitech Rugged Folio - Protective Keyboard Case for iPad in Graphite

Logitech Lift Vertical Ergonomic Mouse in Off-White

Logitech Craft Wireless Keyboard for Advanced Creativity & Productivity

Advanced Keyboard with Creative Input Dial | Logitech Craft Wireless Keyboard for Advanced Creativity & Productivity

Logitech Spotlight Advanced Wireless Presentation Remote in Slate

Logitech MX Ergo Advanced Wireless Trackball Mouse

Logitech's most advanced trackball for trackball enthusiasts and consumers searching for alternatives to mice and touchpads. Delivers 20% less muscular strain compared to a regular mouse. MX ERGO features a unique adjustable hinge for personalized comfort and the latest tracking, scrolling and power management technology. Logitech FLOW™ enables effortless cross computer control. | Logitech MX Ergo Advanced Wireless Trackball Mouse

Logitech Combo Touch Keyboard Case for iPad Pro in Sand

Logitech Keys-to-Go Portable Wireless Keyboard for Apple Devices in Classic Blue/Orange

Logitech Keys-to-Go Portable Wireless Keyboard for Apple Devices in Stone/White

Logitech Buy MX Brio UHD 4K Webcam in Black

Logitech M705 Marathon Wireless Mouse with 3Y Battery Life in Charcoal

Power on - a wireless mouse that runs for up to three years on one set of batteries. | Logitech M705 Marathon Wireless Mouse with 3Y Battery Life in Charcoal

Logitech Combo Touch Keyboard Case for iPad Pro in Oxford Grey

Logitech Litra Glow - Premium Streaming Light Creators in Graphite

Logitech Ergo M575 Wireless Trackball with Smooth Tracking in Graphite

Logitech Ergo M575 Wireless Trackball with Smooth Tracking in Black

Logitech Desk Mat - Studio Series in Mid Grey

Logitech M720 Triathlon Multi-Computer Wireless Mouse in Black

Succeed in a multi-device world with M720 Triathlon Mouse. You'll power through tasks by switching seamlessly between 3 computers, and fly through documents and web pages with hyper-fast scrolling. Boosted efficiency means this Bluetooth® and Unifying mouse lasts for 24 months on a single AA battery. All that in a full-size shape fitting comfortably under your hand. | Logitech M720 Triathlon Multi-Computer Wireless Mouse in Black

Logitech MX Master 3S Wireless Bluetooth Mouse for Mac in Space Grey

Logitech Travel Case for MX Series Mice in Grey Black

Logitech Combo Touch Keyboard Case for iPad in Oxford Grey

Logitech Lift Vertical Ergonomic Mouse in Graphite

Logitech Lift Vertical Ergonomic Mouse in Rose

End of results.

Logitech Mouse Pad - Studio Series in Blue

Logitech Mouse Pad - Studio Series in Dark Rose

Logitech Mouse Pad - Studio Series in Graphite

Logitech Desk Mat - Studio Series in Dark Rose

Academic tools.

"Easy to get in sync with my laptop and presentation tools and easy-to-use. Worth the purchase"

Online Learning Tools

"If that is you, a flat, backlit keyboard is better for that. If I do learn to type, this is more com..."

Educational Headsets

"This is an excellent headset for leading online meetings. You can hear others clearly and can hear a..."

Technology For Education

"My only problem with it was that it took 2 people with higher education degrees to figure out how to..."

Productivity Tools For Students

"This is not a gamer&#x27;s mouse...that being said it is a great mouse for work with the added scrol..."

FPS Gaming Gear

Logitech G is a pioneer in innovative FPS gaming gear. Decades of engineering & design have resulted in renowned speed, precision, and reliability.

G305 LIGHTSPEED Wireless Gaming Mouse Next-generation LIGHTSPEED wireless now available for all gamers

Shop G305 Wireless Gaming Mouse.Features LightSpeed technology for ultra-fast 1-ms response times, advanced HERO optical sensor, and 250 hour battery life.

G840 XL Star Wars Gaming Mouse Pad

Shop G840 XL Battle of Endor and Darth Vader Mouse Pad. Create the ideal space for gaming with a performance-tuned surface, sturdy rubber base, and lots of room to maneuver.

PRO X Superlight 2 Mouse

Shop PRO X Superlight 2 Gaming Mouse. Features LIGHTFORCE switches, HERO 2 sensor, up to 95 hr battery life, smooth glide mouse feet, USB-C charging, POWERPLAY compatibility, and more

G705 Wireless Gaming Mouse

Shop G705 Wireless Mouse. Features new compact sculpted design, pro-grade wireless, Bluetooth, 85g weight, gliding feet, 3 RGB zones, 6 PGM buttons, and more

G840 XL Gaming Mouse Pad

Logitech G840 XL gaming mouse pad creates the ideal space for gaming with a performance-tuned surface, sturdy rubber base, and lots of room to maneuver.

Gaming Mouse Pad Hard Surface G440

G440 Gaming Mouse Pad features ultra-low friction, making it an ideal surface for subtle, quick movements. Learn more about Logitech Gaming.

LIGHTSPEED Wireless Gaming Mouse

Shop G502 LIGHTSPEED Wireless Mouse. Features POWERPLAY wireless charging compatibility, HERO 25k sensor, LIGHTSYNC RGB lighting, PVC free, & more

LIGHTSPEED Wireless Gaming Mouse with HERO Sensor

Shop G703 Gaming Mouse. Features, HERO 25k sensor, LIGHTSPEED wireless, powerplay ready, supreme comfort, advanced button technology, LIGHTSYNC RGB & more

HERO Gaming Mouse

Shop G403 Gaming Mouse. Features HERO 25k sensor with 1ms response times, LIGHTSYNC RGB, high build quality with side grips, G Hub, programmable buttons & more

High Performance Gaming Mouse

Logitech G502 HERO high performance gaming mouse. Featuring HERO, our best sensor, 11 programmable buttons, LIGHTSYNC RGB technology & tunable weight.

Shop G903 Gaming Mouse. Features HERO 25k sensor, LIGHTSPEED wireless, powerplay ready, ambidextrous design, advanced button technology, LIGHTSYNC RGB & more

G600 MMO Gaming Mouse

With 20 buttons, the G600 MMO Gaming Mouse is the most customizable and easy to use mouse specifically designed for your favorite MMOs. Learn more.

Gaming Mouse Pads, Gaming Mouse Mats

Logitech G gaming mouse pads engineered for professional gamers. Hard or soft, extended or minimal-sized, all designed for consistency and stability.

Shop G740 Mouse Pad. Features a 5mm-thickness that is optimized for Logitech G gaming mouse sensors—and all the rapid movements of low-DPI gaming.

G640 Large Cloth Gaming Mouse Pad performance addition

The moderate surface friction and consistent surface texture of G640 are matched to Logitech G senors. The rubber base ensures it lays flat and states in place.

Shop G502 X Wireless Mouse. Features LIGHTFORCE hybrid optical-mechanical switch tech, POWERPLAY compatible, 68% faster response, HERO 25K sensor, and more

G502 X Plus Millennium Falcon

Shop G502 X Plus Millennium Falcon Mouse. A cutting-edge technological marvel from a distant galaxy, far, far away, inspired by the legendary Star Wars universe

G502 X Plus

Shop G502 X Plus Mouse. Features LIGHTFORCE hybrid optical-mechanical switch technology, power-saving play detection, 68% faster response, 8-LED RGB, and more

G502 X Wired

Shop G502 X Wired Mouse. Features LIGHTFORCE hybrid optical-mechanical switch tech, 89 grams exoskeleton design, lightened scroll wheel, HERO 25K, and more.

G303 Shroud

Shop G303 Wireless Mouse. Designed in collaboration with Shroud, it’s engineered for elevated performance, featuring responsive LIGHTSPEED and HERO 25K Sensor

PRO - Footloose

Shop Pro Wireless Gaming Mouse. Built with Esports pros, it features a HERO sensor, LIGHTSPEED wireless technology, ultra lightweight materials, & more.

G GAMING Headset + Mouse Carry Case

Mouse pad for g705.

Shop Mouse Pad for G705 Wireless Gaming Mouse. From the Aurora Collection - available in Pink Dawn, Green Flash and White Mist

Item Subtotal: (Not Including Tax or Shipping)

We'll email you when this product becomes available for purchase

Get the latest from Logitech

Yes, I want to receive news and products emails from Logitech.

Thank you for signing up.