research papers in language teaching and learning

Current Issue

Vol. 14, february 2024, part i, vol. 13, february 2023, vol. 12, february 2022, vol. 11, february 2021, vol. 10, february 2019, vol. 9, february 2018, vol. 8, july 2017, vol. 8, february 2017, vol. 7, february 2016, vol. 6, february 2015, vol. 5, february 2014, vol. 4, february 2013, vol. 3, february 2012, vol. 2, february 2011, vol. 1, february 2010, about rpltl, issn: 1792-1244.

This peer-reviewed electronic journal is dedicated to publishing research in the domains of TESOL (Teaching English to Speakers of Other Languages) and Applied Linguistics. Its mission is two-fold: (a) to promote efficient dissemination of the best of the research that is carried out by students and graduates of the M.Ed. in TESOL of the Hellenic Open University, and (b) to facilitate academic exchange between the students and faculty of that programme and members of the wider professional and academic community on a world-wide basis. The official languages of the journal are English and Greek.

ISSUES published

YEARS of operation

published papers

special issues

Announcements: RPLTL 14 (Special edition) - Call for papers

Editorial Board

editor in chief

assistant editors

int. advisory board

editorial board

• Open access
• Published: 13 May 2021

A meta-analysis on educational technology in English language teaching

• Jafar Rahmati 1 ,
• Siros Izadpanah   ORCID: orcid.org/0000-0002-2061-8110 1 &
• Ali Shahnavaz 2  

Language Testing in Asia volume  11 , Article number:  7 ( 2021 ) Cite this article

15k Accesses

8 Citations

6 Altmetric

Metrics details

As more various types of computer-assisted language learning (CALL) programs have been incorporated into language classrooms over the recent decades, it has become more important to uncover whether, to what extent, and under which moderator variables CALL can be yield more effective outcomes than traditional language instruction. The issue of education is one of the most important materials addressed by technology. Instead, meta-analysis is a statistical and quantitative method that leads us to a general conclusion by integrating the results of different researches. In this study, researchers worked on the impact of educational technology in English language teaching by studying 67 articles and theses (from 1000 studies that were relevant in title and abstract). All articles and theses were included from 2009 to 2020 and 7 articles were excluded from this study due to insufficient information. Furthermore, two instruments, SPSS (mainly its sub-branch Kruskal-Wallis test) and CMA were used to calculate and evaluate data in this research. The total effect size calculated for studies under both fixed and random models was statistically significant and also the study of effects by year of publication, instruments used in research and research methods showed that their effect size was significant. Teaching English with the help of technology has an effective effect size and has shown the success of this technology in language learning.

Introduction

Due to the rapid advances in Information and Communication Technology (ICT) in the world, there is growing attention to combine technologies into the classrooms to prepare learners to meet the needs of a progressively technological-dependent culture (Bond). The presence of technology and its constant advances have been disclosed into society by shifting the way how people cooperate with technology itself and through technology devices (Hollands & Escueta, 2020 ; Gonzalez-Acevedo, 2016 ). Warni, Aziz, and Febriawan ( 2018 ) believe that technology allows students to study independently and cooperate with their peers. This is possible because technology inspires students to reflect and analyze where these two capabilities are at the basics of developing autonomy. Since the 1960s, educational technologists have tried to make this image become a reality through emerging programs based on computer-assisted instruction (CAI) to drill, train, and test students (Andone & Frydenberg, 2019 ). According to Xiao ( 2019 ), every educator must utterly think about, update concepts, be courageous in innovation, let advanced science and technology assist college English education, and familiarize multimedia technology with a large amount of information, closeness and interactivity into college English teaching. Many educational researchers believe that computer-assisted language learning (CALL) would prove to be effective because it would decrease educational costs and increase learning outcomes in the long period (Atabek, 2020 ; Oz, Demirezen, & Pourfeiz, 2015 ).

Research in English language teaching sometimes contradicts differences in educational measures, situations, measurement instruments, and research methods that make it difficult for the researcher to easily compare the findings (Ozkale & Koc, 2020 ).

The disagreement between the results means that there is no acceptable answer to guide policymakers, and there is always an endless demand for re-research. There is a danger that the sponsors of social and educational research would conclude that this research is unproductive and unscientific. In addition, reviewing the sources of empirical research is usually not helpful, and because it depends so much on the opinion, judgment, preferences, and tendencies of the reviewers, conflicting interpretations of a piece of evidence are not uncommon.

However, examining the effectiveness of CALL is not easy for a number of reasons. First, the effect of any CALL program on learning outcomes is some way related to its uses. A specific CALL program may have great educational potential not revealed until it is used properly. Hence, evaluating the effectiveness of the CALL in language education is evaluating its uses rather than the CALL program itself. Second, the effectiveness of CALL is affected by some other moderator variables such as the learners, tasks, the educational setting conditions, and the assessment instruments. Third, CALL can be used either in isolation, as the sole instrument to convey language knowledge to the students, or in combination with traditional, face-to-face teaching methods (Sadeghi & Dousti, 2013 ). In addition to the above obstacles, no individual study by itself can show whether CALL programs are actually effective or not. In most countries, the use of educational technology is a headway and a national movement, and many organizations and educational institutions have been created in order to properly use educational technology and find better and more complete ways and techniques.

General objectives of the article: The role of educational technology in teaching English in Iran

Literature review

• Educational technology

An accurate understanding of the definition, subject, and scope of educational technology depends largely on the root meaning of the word technology. The word is derived from the word technology in Greek, meaning systematically performing art or profession. The first part of this word (technologia) is a combination of performing art and a technique involving knowledge of the meaning of the principles and the ability to achieve the desired results. In other words, logos mean practical things like knowing and doing. The word root means reasoning, explanation, principle, and ratiocination. So technology means the rational application of knowledge. The word consists of two words “technique” and “logic.” “Technique” means skillfully doing any work and “logic” is equivalent to the “knowing” suffix and means “science and knowledge.” Technology can, therefore, be regarded as methodology or knowledge and science in subtle ways of doing things. The second meaning is what the word “technology” is mostly used to express (Faradanesh, 2001 ).

Concept of educational technology

There are many definitions of educational technology, each referring to its various aspects. Before the application of technology in its new sense, planners helped improve the teaching and learning outcomes of audiovisual cases and devices. Thus, it can be concluded that the contributions of this branch are summarized from education to the use of purely educational items. But Brown ( 1972 ) has defined educational technology differently: Educational technology goes beyond the use of instruments. Educational technology is thus more than just a set of components (Ipek & Ziatdinov, 2018 ). It is a systematic way of designing, executing, and evaluating the whole process of research and learning to use specific goals, utilizing research findings in psychology and human communication, and employing a combination of human and non-human resources to create more effective learning, more reliable, and more deeply. In-depth attention to the above definition leads the reader to several basic conclusions:

The first conclusion to be drawn from the first part of the definition is that educational technology is not just about the use of educational instruments, but the broader scope of the use of educational instruments and the use of educational materials as part of it.

As educational technology is considered to be a systematic way or method, so it is more like an empire than its constituent parts because they are actions and reactions. Because the action, reaction, or interaction between its constituents lead to effects and results that are greater than the sum of its constituent elements.

Educational technology uses scientific findings such as psychology and the humanities to design and implement the whole process of teaching and learning.

Educational technology employs a good mix of human and non-human resources. In other words, unlike the use of educational materials or audio-visual training in which the use of material instruments is concerned, in technology, human resources are appropriately used.

The most recent definition agreed by educational technology experts, The American Association of Educational Communication & Technology (AECT) stated that educational technology is the theory and practice of designing, producing, using, and evaluating learning processes and resources (Spector & Yuen, 2016 ). Caffarela and Fly ( 1992 ) define this as considering that in any field of science philosophical issues such as epistemological epistemology and methodology are raised and that experts in different disciplines present theories in that field.

Application of educational technology in English language teaching

The application of educational technology in English language teaching includes any possible means and information that can be used in language teaching. It deals with language teaching instruments such as television, language labs, and a variety of designed media. In other words, the use of educational technology in language teaching is the same folk concept of educational technology as the use of audiovisual devices, monitors, and computer keyboards. The public domain of its audiovisual equipment consists of two distinct parts: the hardware and the software. The hardware talking part deals with physical and real equipment, such as projectors, sound recorders, TV sets, microcomputers, etc., and the software part includes many items used in connection with such equipment and devices like slide, audiotapes, videotapes of computer programs, written languages, and more (Ahdian, 2007 ); (Xu, Banerjee, Ramirez, Zhu, & Wijekumar, 2019 ).

Research in the field of education is sometimes contradictory. Differences in educational measures, situations, measuring tools, and research methods make it difficult for the researcher to compare the findings (Rai'i, Farzaneh, & Delavar, 2013 ). The contradiction between the results leads to no acceptable answer to be a guidance for policy makers (Talan, 2021 ). It also means that there is always an endless demand for research and re-research. There is a risk that research sponsors may conclude that this research is confusing, unproductive, and unscientific (Asgharpour, 2006 ).

Considering the research done in the field of language learning with the help of technology, it can be seen that there are a lot of disagreements about the success rate of teaching English with the help of technology. Some researchers such as Sung, Cheng and Liu ( 2016 ) and Lee ( 2010 ) are its defenders, while other researchers such as Lipsey and Wilson ( 2001 ), Norris and Ortega ( 2000 ), and Oswald and Plonsky ( 2010 ) have expressed doubts about its success.

Proponents of using technology for language learning, giving the learner freedom of action, access to a variety of language content, ease of access and its inherent attractiveness, opponents of factors such as lack of infrastructure, lack of software and hardware to especially in developing countries, students and teachers are not familiar with this technology, teachers do not master technology to produce appropriate educational content, superficial and unrealistic interactions in existing software, too much emphasis on the use of multimedia, heavy volume Content for language learners, lack of appropriate feedback and finally receiving superficial and unrealistic feedback as reasons for their opposition to using technology for language learning. These contradictory reasons led us to perform meta-analysis to determine the effectiveness of technology-assisted language learning and, in general, whether it was successful or not. These reasons became contradictory in order to determine the effectiveness of language learning through meta-analysis with the help of technology and to reach a general conclusion whether it was successful or not.

Research methods

Considering that the purpose of this research was to describe, analyze, and combine the studies presented in the field of educational technology in English language teaching based on research; the method of this research was a meta-analysis. Meta-analysis is a set of statistical methods that are performed in order to combine the results of independent experimental and correlation studies that have the same research questions on a single topic, and leads to a single estimate and result. Unlike traditional research methods, a meta-analysis uses statistical summaries of individual studies as research data.

Based on the main assumption of this method, each study provided different estimates of the underlying relationships in society. Therefore, by combining the results of these studies, a more accurate view of these relationships could be provided, which was provided by estimating individual studies. Meta-analysis research was applied type and was among the few pieces of research. The method used to collect data in this research was the library.

General objective subgroups: variables

Hypothesis 1 — There is a significant difference between the years of publication in research on the application of technology in English language teaching.

Hypothesis 2 — The research method used in the research has been used in the field of application of educational technology in English language teaching.

Hypothesis 3 — Research tools have been used in the application of educational technology in English language teaching.

Hypothesis 4 — There is a significant difference between the effect size of different statistical methods in research in the field of technology application in English language teaching.

Hypothesis 5 — There is a significant difference between the size of the work based on the gender of the sample in research on the application of technology through English language teaching.

Eligibility criteria and exclusion criteria

In this research, 67 articles or theses out of 1000 articles or theses which were related to technology in English language teaching in Iranian cites, like Tehran, Shahr-e- Qods, Yasouj, Shahrood, Mazandaran, Bandar Abbas, Alborz, Shahrekord, Ahvaz, Qeshm, Guilan, Semnan, and Chabahar, were randomly chosen from Iran Doc, google scholar, and science direct websites.

It is important to mention that 7 out of 67 articles and theses were ignored in this thesis based on the table that has been mentioned (Additional file 1 : Appendix A). Topics in selected articles were completely consistent and had a relatively high and appropriate subject similarity for meta-analysis were from the years between 2009 and 2020 (Additional file 1 : Appendix B). Conditions were detected and meta-analysis tests had been performed on them. It should be noted that in the meta-analysis method, there is no specific limit on the number of studies.

Method of data collection

To perform meta-analysis, the specifications of all theses in the field of educational technology in English language teaching, which are the year of publication, sampling method, statistical method, research method, and gender of the sample were studied. These data were then used in analysis, syntheses, and comparison.

Instruments

Meta-analysis is the statistical method which was used in this study. The SPSS (Sciences Statistical Package for the Social) software (SPSS Statistics 26) that researchers examined the frequency and statistical significance. The research hypotheses were also tested by SPSS software. CMA 2 software (Comprehensive Meta-Analysis version 2) was used to calculate the effect size for each study, the overall size effect, and the size of the discriminant effect to test their statistical significance in this research.

In addition, the research hypotheses were tested using SPSS software and the Kruskal-Wallis nonparametric test. The effect size in this r was calculated using the Hex method.

The Egger regression method has also been used to evaluate the homogeneity of the studies. The advantage of this method compared to other tests is that it is stronger. This method uses real effect size methods for prediction.

Data analysis

Part 1: descriptive analysis, description of general characteristics of the studied samples.

Descriptive information about the year of publication of the studies was examined in this study.

As can be seen in Table 1 , the highest percentage is related to research published during the years 2015 to 2017 with a rate of 51.7% and the lowest percentage is related to research published in the years 2018 to 2020 with a rate of 6.7 percentage.

Descriptive information about the research methods used in the studies reviewed in this study

As can be seen in Table 2 , the highest percentage is related to the quasi-experimental research method with 36.7% and the lowest percentage is related to the qualitative research method with 3.3%.

Descriptive information about the instruments used in the studies examined in this study

As can be seen in Table 3 , for the instruments used, the highest percentage is related to the Questionnaire instrument with 37% and the lowest percentage is related to the Observations instrument with a rate of 3.3%.

Descriptive information about the statistical method used in the studies examined in this study

As can be seen in Table 4 , the highest percentage is related to the method of using pre- and post-tests with 56.7% and the lowest percentage is related to the statistical method of ANOVA with 1.7%.

Descriptive information about the sex of the sample in the studies

As can be seen in Table 5 , for the sample gender, the highest percentage is related to mixed-gender with a rate of 48% and the lowest percentage is related to female gender with a rate of 20%.

Homogeneity of studies

In order to check the homogeneity of the studies, the Eger regression test is used and the results of this test are summarized in the following table:

As can be seen in Table 6 , due to the value of Sig, which is greater than 0.05, the assumption of homogeneity of studies at an error level of 0.05% is accepted.

The following Fig.  1 is used to determine whether the initial studies are biased and their impact on data analysis.

Funnel diagram shape the size of each study with the effect size accuracy

If the initial studies do not have a diffusion bias, they should be distributed symmetrically around the average effect size, as shown in the diagram above.

Overall effect size

Before examining the effect size separately for the variables in this study, the overall effect size is calculated in two modes: a model with random effects and a model with fixed effects, and the results are recorded in the table below.

It should be noted that due to the homogeneity of the initial studies in this study, the model with fixed effects is more efficient than the model with random effects.

As can be seen in Table 7 , considering that the sig value for both models is less than 0.01, it can be accepted that the total effect size in both models is significant with random effects and fixed effects at the error level of one percent.

Effect size by year of publication

The following table records the results related to the effect size by year of publication of studies in both model modes with random effects and fixed effects.

According to the Sig values obtained in Table 8 , the size of the effects in all the studied years is significant in both types of models with fixed effects and random effects.

The size of the work is used separately according to the research method

In the table below, the results related to the size of the effect are recorded separately by the research method used in the studies in both models with random effects and fixed effects.

According to the Sig values obtained in Table 9 , the size of the effects in all research methods used in the studies under study in both types of models with fixed effects and random effects are significant.

Effect size by the instrument used

In the table below, the results related to the size of the effect by instruments used in the studies are recorded in both model modes with random effects and with fixed effects.

According to the Sig values obtained in Table 10 , the size of the effects in all instruments used in the studies under study in both types of models with fixed effects and random effects are significant.

The size of the work is separated by statistical methods

In the table below, the results related to the effect size are recorded separately by the statistical methods used in the studies in both models with random effects and with fixed effects.

According to the Sig values obtained in Table 11 , the size of the effects in all statistical methods used in the studies in both types of models with fixed effects and random effects, except the random effects model in the case where the statistical instrument is used Qualitative Have been meaningful.

Effect size by sample gender

In the table below, the results related to the effect size by sample gender in both models with random effects and fixed effects are recorded.

According to the Sig values obtained in Table 12 , the size of the effects on the sex of the sample in both types of models with fixed effects and random effects are significant.

Part II: Inferential analysis

Hypothesis 1.

There is no significant difference between the size of the effect of years of the publication on research in the application of educational technology in English language teaching.

To test the above hypothesis, the Kruskal-Wallis test was used and the results of this test are recorded in the following tables:

As can be seen in Table 13 , considering the value of Sig = 0.151, which is greater than 0.05, the assumption of zero, i.e. the assumption that the size of the work is the same according to the year of publication is not rejected at the level of 5% error.

The effect of years of publication in research in the field of technology application in English language teaching is not significantly different.

Hypothesis 2

There is no significant difference between the size of the work and the research method used. Research conducted in the field of technology application in English language teaching.

To test the above hypothesis, the Kruskal-Wallis test was used and the results of this test are recorded in the following table:

As can be seen in Table 14 , considering the value of Sig = 0.302, which is greater than 0.05, the null hypothesis, i.e. the assumption that the size of the work is the same, is not rejected at the 5% error level, so researchers can say: There is no significant difference between the size of the work and the research method used in the field of technology application in English language teaching.

Hypothesis 3

There is no significant difference between the size of the work and the instruments used in research in the field of technology application in English language teaching.

To test the above hypothesis, Kruskal-Wallis test was used and the results of this test are recorded in the following table:

As can be seen in Table 15 , considering the value of Sig = 0.830, which is greater than 0.05, the null hypothesis, i.e. the assumption that the size of the work is the same, is not rejected at the 5% error level, so researchers can say: There is no significant difference between the size of the work and the instruments used in research in the field of technology application in English language teaching.

Hypothesis 4

There is no significant difference between the size of the work according to the statistical method used in research in the field of technology application in English language teaching.

To test the above hypothesis, Kruskal-Wallis test was used and the results of this test are recorded in the following tables:

As can be seen in Table 16 , considering the value of Sig = 0.814, which is greater than 0.05, the null hypothesis, i.e. the assumption that the size of the work is the same, is not rejected at the 5% error level, so researchers can say: There is no significant difference between the size of the work according to the statistical method used in research in the field of technology application in English language teaching.

Hypothesis 5

There is no significant difference between the size of the work by gender of the sample in research in the field of technology application through English language teaching.

As can be seen in Table 17 , considering the value of Sig = 0.819, which is greater than 0.05, the null hypothesis, i.e. the assumption which the size of the work is the same, is not rejected at the 5% error level, so researchers can say: There is no significant difference between the size of the work by gender of the sample in research in the field of technology application through English language teaching.

In this part, researchers describe the collected results in general and discuss the statistical results obtained. The present study includes 67 studies out of 1000 theses and articles which 7 of them were excluded from this study due to a lack of sufficient information ( Appendix A ).

The main purpose of this study was to investigate the impact of educational technology on English language teaching. The optimal research method to achieve this goal was meta-analysis. In this method, “each research” was a unit of study, furthermore, the amount of effect size was calculated for each research in order to obtain the effectiveness of each research.

Our results indicate that technology applications have a large effect (1.68 and 0.91, fixed effect model and random effect model respectively) on English language teaching. This proposes that the use of technology is more effective than traditional teaching methods without technology for English language teaching quality.

Overall effects of educational technology on English language teaching

The result of a medium-sized overall positive effect of educational technology on English language teaching confirmed that the use of a computer, telegram, mobile, laptop devices, and software could facilitate language learning. These results were consistent with other research findings regarding the effects of different devices and software on English language teaching.

Related to the first research question: Year of publication

This research question was in line with Sung, Yang, and Lee ( 2017 ) and Chauhan ( 2017 ), which both had the same experimental results show that their meta-analysis was not substantially affected by publication bias. The most obvious finding to emerge from this research question was that years of publication did not have a significant result in this research.

Based on the fact, the year of publication was selected for research as a variable; if years are considered differently, that is, for example, the year 2009 is assumed alone, they are meaningful. They also have the same feature for the year of publication until 2020. However, based on the research question of how much the effect of the year of publication affects educational technology, it should be noted that this variable is not recommended for future research because it changes every time based on advances in technology and different methods for research. Considering the year of publication, it will not have a significant effect as a whole on the effect size of the work.

Related to the second research question: research method

It is in line with Farzaneh Shakki ( 2015 ). There is no significant difference between the effect size by the research method used and the research conducted in the field of technology application in the English language teaching.

In fact, the research method as a whole depends on the researcher and the type of research that is being done. In this research, we conclude that if we want to consider the research methods one by one we can claim that they all have a significant effect but when we want to consider all of them relative to each other, they do not have a significant effect. Therefore, this research shows us that the required research methods or resources required as well as different goals can be variable, so it depends on the researcher in what circumstances, in what environment and with what tools they can choose the research method. Of course, a single research method may not be used in an article, and several types can be used.

Related to the third research question: instruments

It was in line with Fazeli ( 2016 ). It was not in line with Pourtayebi ( 2015 ), Alinejad ( 2015 ), Sadeqi ( 2015 ), Rastegar ( 2014 ), Shahkooei ( 2016 ), and Parinaz ( 2010 ). There is no significant difference between the effect size and the instruments used in research in the field of technology application in English language teaching

As we have seen, among the number of theses and articles we reviewed, a variety of instruments were used. In the meantime, the questionnaire was used more than other instruments, but this does not mean that this instrument is superior to others in research instruments. In this study, in each of the articles and theses, one or more instruments were used, which were significant, but in general, they were not significant in comparison to each other. This means that we cannot say which tool is better than other instruments so it depends on the researcher which instrument to choose over the research.

Related to the fourth research question: statistical method

It was not in line with Shahkooei ( 2016 ). There is no significant difference between the size of the work according to the statistical method used in research in the field of technology application in English language teaching. Although the number of statistical methods used in these studies was different, in the ranking, they did not have a significant difference.

To check the quantitative research data, the use of statistical tests is mandatory. A statistical method is necessary to use for each research. Reviewing all statistical tests can be a good guide for analyzing data in an article. Meanwhile, it may not always be enough to use a test.

Statistical methods are one of the practical ways to identify problems and provide solutions to managerial, social, and psychological problems, etc. that, if implemented correctly, can provide real data for our research.

In other words, there may be different ways of doing research or how we can collect our data to prove or answer questions. At this point, having high analytical power, problem-solving ability. And sufficient experience can help you to know the correct method of research. Because it directly uses people’s opinions, it can solve society’s problems, and these studies are often very practical and can be cited.

Related to the fifth research question: gender

It was in line with Alipour ( 2017 ), Sadeqi ( 2015 ), Nateghi ( 2018 ), Dayani ( 2014 ), Ghazavi ( 2017 ), and Aliakbari ( 2013 ). It was not in line with Mohammadi ( 2014 ), Alashti ( 2013 ), AsgharHeidari ( 2014 ), and Nakhaei ( 2017 ) found the result of the study that the attitudes of English teachers or students regarding their gender towards the use of the Internet, mobile or other devices were positive and high.

According to the statistical part of this study, the participants were mixed in most of the articles, but in some of them, exclusively female participants or in some other male participants were used to conduct the research. Based on the findings, we conclude that there is not much difference between men being superior to women or vice versa.

Based on the availability of technology in education, educational technology has caused many changes in the field that meet the needs of students in different ways. With the provision of software that teaches students with special needs, the appropriate educational equipment is designed to make learning easier for the individual.

With the use of technology, the concept of education is changing for both students and teachers to progress. Therefore, the introduction of technology in education is very important.

Research limitations

The present meta-analysis, like many others, has its limitations and forces the researcher to interpret the findings with extreme caution.

Lack of access to some articles and dissertations that did not receive a response from the authors despite sending an email.

Suggestion for further study

Due to the limitations that researchers applied in this research, 67 theses and articles were selected from different cities of Iran that had a topic related to the subject of this research, but it should be noted that due to development and progress in recent years, the importance of this thesis is observed. It is better to select researches that have been published in reputable publications all over the world, in addition to this, it is suggested to work on other various variables.

A meta-analysis of research on the application of technology in English language teaching, which was published in valid journals in this field and examined, showed that the application of information technology in this field has an acceptable impact factor.

In this section, the overall findings of the current study were presented. According to the studied variables, we conclude that the five variables studied and researched, according to their statistical information in this study, did not have a significant effect size. And in response to the overall purpose of the article, how much technology can affect English language teaching, it can be concluded that, initially, compared to the variable of the year, 2017 to 2020, the size of the work was more representative than previous years, so technology has been effective. Other variables, such as tools, research methods, statistical methods, and gender, have had a smaller effect than size that we can ignore.

The results showed that all chosen variables in this study, considering every unique thesis or article, were significant, but as the whole consideration of each variable to 60 theses and articles, they were not significant.

Availability of data and materials

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Abbreviations

Information and communication technology

Computer-assisted language learning

Abbassi, Z. (2015). A comparative study of applying websites and mobile phones on learning collocations among Iranian EFL learners . Thesis (Unpublished). Khatam University.

Ahdian, M. (2007). Introduction to Educational Technology, Tehran, Iran .

Alashti, L. A. (2013). Exploring and comparing the attitudes of Iranian English teachers and learners toward using the Internet for language learning . Thesis (Unpublished). Semnan University.

Aliakbari, M., (2013). Teachers’ perception of the barriers to critical thinking. Procedia-Social and Behavioral Sciences, 70 , 1–5. https://doi.org/10.1016/j.sbspro.2013.01.031 .

Alinejad, J. (2015). The impact of interactive whiteboard on developing vocabulary learning , Thesis (Unpublished) (). Shahrood: Islamic Azad University.

Google Scholar  

Alipour, M. (2017). The effect of using social networks on developing EFL learners’ pragmatic competence through speaking accuracy and fluency . Thesis (Unpublished). Islamic Azad University.

Andone, D., & Frydenberg, M. (2019). Creating virtual reality in a business and technology educational context. In Augmented reality and virtual reality (pp. 147-159). Springer.

Ardebili, M. (2013). Computer-assisted language learning: attitudes of Iranian EFL learners . Thesis (Unpublished). Urmia University.

AsgharHeidari, F. (2014). The impact of media analysis on the development of critical thinking and reading comprehension skills of the EFL learners . Thesis (Unpublished). University of Guilan.

Asgharpour, D. (2006). Bibliography of research methods and methodology of humanities. Seminary and University of Humanities Methodology , 12 (48), 154–167.

Atabek, O. (2020). Associations between emotional states, self-efficacy for and attitude towards using educational technology. International Journal of Progressive Education , 16 (2), 175–194. https://doi.org/10.29329/ijpe.2020.241.12 .

Article   Google Scholar  

Bozorgmanesh, B. (2013). Innovative versus traditional approaches to vocabulary teaching: examining effects of online quotations in teaching vocabulary in Iranian EFL context . Thesis (Unpublished). Ilam University.

Brown, J. W. (1972). Administering educational media: Instructional Technology and Library Services . USA: McGraw-Hill Companies.

Caffarella, E. P., & Fly, K. (1992). Developing a knowledge base and taxonomy in instructional technology . Retrieved April 22, 2021 from https://www.learntechlib.org/p/144856/.

Chauhan, S. (2017). A meta-analysis of the impact of technology on learning effectiveness of elementary students. Computers & Education , 105 , 14–30. https://doi.org/10.1016/j.compedu.2016.11.005 .

Dayani, A. (2014). The effect of using blogs on Iranian EFL learners’ critical thinking . thesis (unpublished). Shahid Chamran University.

Dousti, M. (2012). The effect of length of exposure to call technology on young Iranian EFL learners’ grammar and vocabulary gain . Thesis (Unpublished). Urmia University.

Ebrahimi, H. (2014). The impact of teaching- medium on EFL learners’ meta-pragmatic information: the case of compliment . Thesis (Unpublished). University of Guilan.

EbrahimiSeraji, N. (2016). Teacher’s attitudes toward educational technology in language institutes in Mazandaran . Thesis (Unpublished). Islamic Azad University.

Esferjani, R. (2017). Effect of internet-assisted pronunciation instruction on Iranian l2 learners’ production of primary stress in English compound nouns. Thesis (Unpublished). Shahrekord University, Shahrekord.

Esmaili, S. (2012). EFL writing and computer-assisted language learning: towards an “eclectic” view and far “beyond” that . Thesis (Unpublished). Chabahar Maritime University.

Ezzatian, S. (2013). The differential effects of debate and media analysis as relate to enhancing EFL learners’ critical thinking ability and writing performances . Thesis (Unpublished). Sheykhbahayee University.

Faradanesh, H (2001). Theoretical foundations of educational technology . Tehran: Organization for the Study and Compilation of Humanities Books.

Fard, M. K. (2016). A comparative study on the use of teaching aids in Iranian EFL institutes and schools . Thesis (Unpublished). Payame Noor University.

Farhesh, S. (2012). On the effect of using PPT slides on vocabulary and grammar acquisition, and students’ attitudes . Thesis (Unpublished). Allameh Tabataba’i University.

Farshadnia, S. (2010). Culture in online communication: netiquette applications and implementations in TEFL community . Thesis (Unpublished). Al-Zahra University.

Fazeli, S. A. (2016). An investigation into the effectiveness of call in the writing ability of young adolescent EFL learners: with some reference to the perceptions of call teachers and learners . Thesis (Unpublished). Yasouj University.

Ghazavi, A. Z. (2017). EFL student’s assumptions on using smartphone applications in learning English language: a case study of students at the Department of English, Razi University . Thesis (Unpublished). Razi University.

Ghaziyani, N. N. (2017). The effect of selective feedback on paragraph writing performance of Iranian intermediate EFL learners in telegram as a social network setting . Thesis (Unpublished). Payame Noor University.

Gonzalez-Acevedo, N. (2016). Technology-enhanced-gadgets in the teaching of English as a foreign language to very young learners. Ideas on implementation. Procedia-Social and Behavioral Sciences, 232 , 507-513. https://doi.org/10.1016/j.sbspro.2016.10.070 .

Hassanzadeh, M. (2010). A comparison and contrast of two distance education systems (e-learning vs. print-based learning) in the reading comprehension skill . Thesis (Unpublished). Payam-e-Noor University.

Hollands, F., & Escueta, M. (2020). How research informs educational technology decision-making in higher education: the role of external research versus internal research. Educational Technology Research and Development , 68 (1), 163–180. https://doi.org/10.1007/s11423-019-09678-z .

Inanloo, F. (2017). The impact of community-based web sites, online discussions forums and chat rooms on l2 learners’ expressive writing skill improvement . Thesis (Unpublished). Islamic Azad University.

Ipek, I., & Ziatdinov, R. (2018). New approaches and trends in the philosophy of educational technology for learning and teaching environments. arXiv preprint arXiv.

Javdani, M. (2017). Iranian EFL teachers’ attitudes toward blended learning approach . Thesis (Unpublished). Islamic Azad University.

Kashani, N. N. (2015). The impact of multimedia-supported metacognitive listening strategies instruction on Iranian EFL learners’ listening proficiency . Thesis (Unpublished). Shahid Rajaee Teacher Training University.

KhademianHashemi, H. (2014). The effect of video-mediated listening texts on EFL learners’ writing ability . Thesis (Unpublished). Islamic Azad University.

Khalilabad, M. H. (2016). The effect of multimedia texts presented on the interactive whiteboard (smart board) on high school EFL learners’ reading comprehension . Thesis (Unpublished). Hakim Sabzevari University.

Khalili, S. (2013). Vocabulary instruction through blended learning and multimedia softwares in Iranian EFL classes . Thesis (Unpublished). Sheikhbahaee University, Esfahan.

Khazaee, S. (2017). Comparing the efficacy of chatting as one of the call-based activities on children vs. adult EFL learners’ speaking ability . Thesis (Unpublished). Chabahar Maritime University.

Lee, I. (2010). Writing teacher education and teacher learning: testimonies of four EFL teachers. Journal of Second Language Writing, 19 (3), 143–157. https://doi.org/10.1016/j.jslw.2010.05.001.

Letafati, M. (2013). The effect of asynchronous forum discussions on the writing ability of Iranian EFL learners . Thesis (Unpublished). Payam-e-Noor University.

Li, S. (2010). The effectiveness of corrective feedback in SLA: a meta-analysis. Language Learning , 61 (2), 319–365.

Lipsey, M. W. ,& Wilson, D. B. (2001). Practical meta-analysis . Thousand Oaks: Sage.

Mardian, F. (2014). A socio-cultural study of the impact of computer-mediated corrective feedback on the development of EFL learners’ grammatical knowledge . Thesis (Unpublished). Alzahra University.

Mohammadi, M. (2014). The relationship between internet use and academic procrastination of EFL learners . Thesis (Unpublished). University of Guilan.

Najafi, S. (2013). The effects of call multimedia (still-picture and motion-picture multimedia) on the vocabulary learning of elementary EFL students . Thesis (Unpublished). Shahid Madani University.

Nakhaei, M. (2017). The effect of using telegram as a social network on vocabulary learning of iranian students . Thesis (Unpublished). Islamic Azad University.

Nami, F. (2020). Educational smartphone apps for language learning in higher education: students’ choices and perceptions. Australasian Journal of Educational Technology , 5 , 82–95.

Nateghi, M. (2018). The effect of mobile learning for collaborative dialogues on EFL Iranian learners’ speaking performance . Thesis (Unpublished). Islamic Azad University.

Noghani, F A. (2015). The effect of multimodal presentation on EFL learners’ listening comprehension and self-efficacy . Thesis (Unpublished). Hakim Sabzevari University.

Noori, L. (2014). The effect of wiki on complexity, accuracy and fluency of intermediate EFL learners’ writing performance . Thesis (Unpublished). University of Zanjan.

Norris, J. M., & Ortega, L. (2000). Effectiveness of L2 instruction: A research synthesis and a quantitative meta-analysis. Language Learning, 50, 417–528.

Oswald, F. L., & Plonsky, L. (2010). Meta-analysis in second language research: choices and challenges. Annual Review of Applied Linguistics , 31 , 85.

Oz, H., Demirezen, M., & Pourfeiz, J. (2015). Digital device ownership, computer literacy, and attitudes toward foreign and computer-assisted language learning. Procedia-Social and Behavioral Sciences , 186 , 359–366. https://doi.org/10.1016/j.sbspro.2015.04.028 .

Ozkale, A., & Koc, M. (2020). Investigating academicians’ use of tablet PC from the perspectives of human computer interaction and Technology Acceptance Model. International Journal of Technology in Education and Science , 4 (1), 37–52. https://doi.org/10.46328/ijtes.v4i1.36 .

Parinaz, I. (2010). The impact of web-based reading lessons on EFL students reading comprehension, motivation, and autonomy . Thesis (Unpublished). Al-Zahra University.

Parisa, P. (2017). On the relationship between media literacy and personality traits and Iranian EFL learners’ listening comprehension . Thesis (Unpublished). Imam Khomeini International University.

Paslar, Z. (2017). Politeness strategies used by intermediate Iranian EFL learners in online what’s app interactions . Thesis (Unpublished). Islamic Azad University.

Poorkhalil, M. R. (2016). The effect of using multimedia on vocabulary learning of basic and elementary Iranian EFL learners . Thesis (Unpublished). Semnan University.

Pour, A. M. (2015). The impact of synchronous computer-mediated communication on writing ability, patterns of collaboration and motivation of Iranian EFL learners . Thesis (Unpublished). University of Guilan.

Pourtayebi, H. (2015). The effect of mobile instant messaging on reading comprehension ability . Thesis (Unpublished). Hakim Sabzevari University.

Rafei, A. (2017). The relationship between willingness to communicate and internet use among Iranian EFL learners . Thesis (Unpublished). Shahid Rajaee Teacher Training University.

Rai'i, Farzaneh, & Delavar, A. (2013). Meta-analysis: the art of correcting the mistakes of others. Journal of Educational Measurement , 4 (13), 119–132.

Rajabi, S. (2016). Traditional class environment (TCE) vs. mobile assisted language learning (mall): the case of intermediate Iranian EFL learners English idioms achievement. Thesis (Unpublished). Alborz University of Higher Education.

Rastegar, S. (2014). The effect of interactive text-messaging on learning English phrasal verbs by Iranian EFL learners . Thesis (Unpublished). University of Mazandaran.

Raye-Ahmadi, M. V. (2014). Media literacy in the mediation of English language learning . Ph.D. DISSERTATION. Alzahra University.

Razavi, A. (2016). Constructing and validating a multimedia techniques (MTS) scale and examining the impact of using technology in teaching English in Iranian high schools on students’ attitudes, anxiety, and language proficiency . Thesis (Unpublished). Imam Reza International University.

Sadeqi, M. (2015). The effects of call and online resources on learning collocation of Iranian EFL learners . Thesis (Unpublished). Islamic Azad University.

Sadeghi, K., & Dousti, M. (2013). The effect of length of exposure to CALL technology on young Iranian EFL learners' grammar gain. English Language Teaching, 6 (2), 14–26. https://doi.org/10.5539/elt.v6n2p14 .

Salimi, H. (2016). The application of wiki-mediated collaborative writing as a pedagogical instrument to promote ESP learners’ writing performance, autonomy, and perceptions of uses of wikis . Thesis (Unpublished). Allameh Tabataba’i University.

Shafaghiha, G. (2016). Watching English language TV series and visual literacy of Iranian EFL learners: challenges, possibilities, reasons . Thesis (Unpublished). Alzahra University.

Shafati, M. (2013). The effects of etymological elaboration and multimedia instruction on EFL learners’ idiom retention . Thesis (Unpublished). Sheikhbahaee University.

Shahkooei, S. (2016). A comparison of traditional flash-card vs. mobile learning: Using mms for learning idioms by Iranian EFL learners (idiomobile) . Thesis (Unpublished). Allameh Mohaddes Nouri University.

Shakki, F. (2015). The effect of dynamic assessment on EFL learners’ listening comprehension through mediational strategies . Thesis (Unpublished). Islamic Azad University.

Shargh, S. N. (2010). The pedagogical effectiveness of multimedia call software on vocabulary recall and retention of Iranian pre-intermediate EFL learners . Thesis (Unpublished). Ferdowsi University.

Spector, J. M., & Yuen, A. H. (2016). Educational technology program and project evaluation . Routledge. https://doi.org/10.4324/9781315724140 .

Book   Google Scholar  

Sung, Y. T., Chang, K. E., & Liu, T. C. (2016). The effects of integrating mobile devices with teaching and learning on students’ learning performance: A meta analysis and research synthesis. Computers & Education, 94 , 252–275. https://doi.org/10.1016/j.compedu.2015.11.008 .

Sung, Y. T., Yang, J. M., & Lee, H. Y. (2017). The effects of mobile-computer-supported collaborative learning: meta-analysis and critical synthesis. Review of Educational Research , 87 (4), 768–805. https://doi.org/10.3102/0034654317704307 .

Talan, T. (2021). The effect of simulation technique on academic achievement: a meta-analysis study. International Journal of Technology in Education and Science, 5 (1), 17–36. https://doi.org/10.14686/buefad.706231 .

Warni, S., Aziz, T. A., & Febriawan, D. (2018). The use of technology in English as a foreign language learning outside the classroom: an insight into learner autonomy. LLT Journal: A Journal on Language and Language Teaching , 21 (2), 148–156.

Xiao, L. (2019). Application development of modern multimedia technology in English teaching. Frontiers in Educational Research , 2 (2), 12–39.

Xu, Z., Banerjee, M., Ramirez, G., Zhu, G., & Wijekumar, K. (2019). The effectiveness of educational technology applications on adult English language learners’ writing quality: a meta-analysis. Computer Assisted Language Learning , 32 (1-2), 132–162. https://doi.org/10.1080/09588221.2018.1501069 .

Zakeri, D. (2018). A study of the effects of using multimedia in teaching technical translation model for Iranian students . Thesis (Unpublished). Islamic Azad University.

Zanussi, M. P. (2015). The impact of watching captioned TV series on vocabulary development of EFL students . Thesis (Unpublished). Allameh Mohaddes Nouri University.

Zarat-ehsan, M. (2015). The impact of multimedia glosses on intermediate EFL learners’ vocabulary learning and retention . Thesis (Unpublished). University of Guilan.

Download references

Author information

Authors and affiliations.

Department of English Language Teaching, Zanjan Branch, Islamic Azad University, Zanjan, Iran

Jafar Rahmati & Siros Izadpanah

Department of Mathematics and Statistics, Zanjan Branch, Islamic Azad University, Zanjan, Iran

Ali Shahnavaz

You can also search for this author in PubMed   Google Scholar

Contributions

All authors of the research had more or less equal contributions in the process of conception, design, acquisition of data, analysis, and interpretation of data. They have all been involved in revising the manuscript critically to the same extent. All take public responsibility for the whole content. All are equally accountable for all aspects of the work. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Siros Izadpanah .

Ethics declarations

Competing interests.

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1. .

Abbassi, 2015 , Ardebili, 2013 , Bozorgmanesh, 2013 , Dousti, 2012 , Ebrahimi, 2014 , EbrahimiSeraji, 2016 , Esferjani, 2017 , Esmaili, 2012 , Ezzatian, 2013 , Fard, 2016 , Farhesh, 2012 , Farshadnia, 2010 , Ghaziyani, 2017 , Hassanzadeh, 2010 , Inanloo, 2017 , Javdani, 2017 , Kashani, 2015 , KhademianHashemi, 2014 , Khalilabad, 2016 , Khalili, 2013 , Khazaee, 2017 , Letafati, 2013 , Li, 2010 , Mardian, 2014 , Najafi, 2013 , Nami, 2020 , Noghani, 2015 , Noori, 2014 , Parisa, 2017 , Paslar, 2017 , Poorkhalil, 2016 , Pour, 2015 , Rafei, 2017 , Rajabi, 2016 , Raye-Ahmadi, 2014 , Razavi, 2016 , Salimi, 2016 , Shafaghiha, 2016 , Shafati, 2013 , Shargh, 2010 , Zakeri, 2018 , Zanussi, 2015 , Zarat-ehsan, 2015 , Zhang & Liu, 2019.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Rahmati, J., Izadpanah, S. & Shahnavaz, A. A meta-analysis on educational technology in English language teaching. Lang Test Asia 11 , 7 (2021). https://doi.org/10.1186/s40468-021-00121-w

Download citation

Received : 26 December 2020

Accepted : 13 April 2021

Published : 13 May 2021

DOI : https://doi.org/10.1186/s40468-021-00121-w

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• English language teaching
• Meta-analysis

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Front Psychol

A Review of Research on Technology-Supported Language Learning and 21st Century Skills

Associated data.

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Modern society needs people to be equipped with 21st century skills (e.g., critical thinking, creativity, communication, digital literacy, or collaboration skills). For this reason, teaching and learning nowadays should promote not only students' knowledge acquisition in various learning contexts but also their 21st century skills, and language learning context is no exception. This study reviewed research on technology-supported language learning and 21st century skills. The reason is that earlier studies reviewed only articles related to language learning supported by technology and mostly focused on languages, language skills and technologies used. That is to say, 21st century skills were not considered in earlier review studies. The present study selected and reviewed 34 articles published between 2011 and 2022 (February) and focused on the following dimensions: (1) research focus such as language skills and 21st century skills; (2) theoretical foundations; (3) technologies; (4) learning activities; (5) methodology; and (6) findings. The present research found that reviewed studies had focused most frequently on such language skills as speaking and writing and on such 21st century skills as communication and collaboration. The social constructivism theory was often used by scholars to base their studies on. Facebook, Google Docs, and Moodle were popular technologies in reviewed studies to facilitate language and 21st century skills. Scholars in reviewed studies reported that technology-supported language learning activities provided learners with good learning experiences and enhanced their learning motivation, engagement, and confidence. However, some challenges that learners faced during learning activities were also reported. Based on the results of the review, this study made several recommendations for stakeholders such as educators and researchers in the field.

Introduction

It is important that our students not only acquire new knowledge when they learn, but also develop skills, such as problem-solving, social cooperation, creativity, and so on, in order to apply newly learned knowledge to the real world. Such knowledge and skills will help them adapt to modern society and will enhance their competitiveness (Shadiev et al., 2022a , b ). Many countries have put forward the 21st century skills framework to carry out education reform (Lin et al., 2020 ), and one of them was proposed by the Partnership for 21st Century (P21). The P21 (Partnership for 21st Century Skills, 2008 ) provided a detailed conceptual framework and listed three types of skills: (1) learning and innovation skills (critical thinking and problem solving, creativity and innovation, and communication and collaboration), (2) digital literacy (information literacy, media literacy, and information and communication technologies (ICT) literacy), and (3) career and life skills (flexibility and adaptability, initiative and self-direction, social and cross-cultural interaction, productivity and accountability, and leadership and responsibility). The essence of these skills is that they are key skills that learners will need for their social and professional life in the future. These skills also emphasize the ability of learners to use and transfer knowledge and solve problems in complex situations, so they can achieve deep levels of individual learning as well as lifelong learning (Shadiev et al., 2022a ).

Developing students' 21st century skills needs to be implemented in all disciplines, and foreign language learning is no exception (Shadiev et al., In Press ). This matter has been addressed in the documents related to Asia Pacific Economic Cooperation ( 2004 ). Furthermore, researchers have carried out related studies, and pointed out the advantages of technology in developing both language skills and 21st century skills (Shadiev et al., In Press ). For example, Suzanne ( 2014 ) pointed out that when developing learners' reading skills, they deepened the learners' understanding of reading content, and also developed critical thinking skills. García-Sánchez and Burbules ( 2016 ) have found that students' skills such as problem solving, collaboration, listening and speaking improved after they completed online collaborative tasks. Srebnaja and Stavicka ( 2018 ) also pointed out that, in language learning projects supported by WebQuest, students' creativity, collaboration, and speaking skills have been developed. In the study by Chiang ( 2020 ), the digital storytelling activity was designed which promoted language learners' writing skills as well as their digital literacy skills.

A theoretical foundation to support technology supported language learning and development of 21st century skills can be built on various theories. The most relevant can be considered as second language acquisition theory, socio-cultural theory, and constructivism theory. For example, second language acquisition theory states that language acquisition is a process of input, absorption, and output. Language acquisition is acquired through exposure to contexts, understanding discourse, and then using language in natural communicative contexts (Krashen, 1985 ). According to socio-cultural theory, learning is a social phenomenon; it emphasizes the social nature of learning and argues that the development of learners' abilities arises from interpersonal interactions (Lantolf, 2000 ). Constructivism theory suggests that learning is a process in which a learner actively constructs meaning. That is, learners generate meaning and construct understanding based on prior knowledge and experience, often in the context of socio-cultural interactions. Constructivism theory emphasizes the social and contextual nature of learning (Vygotsky, 1978 ). Over the years, scholars have created technology-supported learning environments for language learning and 21st century skills development based on these theories. Such environments provide students with authentic learning materials, support social interaction, and facilitate their creative expression and construction of meaning actively using the target language.

Some related review studies already exist in the field. For example, Shadiev and Yang ( 2020 ) reviewed 398 articles related to technology-assisted language learning published in 10 Social Science Citation Index (SSCI) journals. The dimensions analyzed in their study included target language, language skills, technologies, and research findings. Shadiev and Yang ( 2020 ) found that the most commonly used language was English, followed by Chinese. The most targeted language skills were writing, speaking, and vocabulary acquisition. Digital games and online videos were the most commonly used technologies in these reviewed studies. In addition, most of the reviewed studies reported positive impacts of technology applications on language learning. Zhang and Zou ( 2020 ) reviewed 57 articles on technology applications for language learning that were published between 2016 and 2019 in 10 SSCI journals. The types of technology, the purpose of technology use, and the effectiveness of the technologies were reviewed by the authors. Zhang and Zou ( 2020 ) found that mobile learning, multimedia learning and socialization, voice to text recognition, text to speech recognition, and digital game-based learning were the most frequently investigated types of technology in the literature. The purposes for their use mainly covered four areas, including promoting practice, providing teaching content, promoting interaction, and reconstructing teaching methods. Scholars have claimed that technologies have positive effects on language learning. Goksu et al. ( 2020 ) reviewed 310 articles in 10 journals in the field of technology-assisted language learning. In addition, they evaluated a metadata set of 469 articles in the Web of Science database through bibliometric mapping. The review focused on the types, purposes, and effectiveness of the latest technologies on language learning. Goksu et al. ( 2020 ) found that most studies used quantitative research methods and were carried out with participants at higher academic levels. In addition, most studies focused on language skills as well as learning motivation and learner perceptions. Shadiev et al. ( 2017 ) studied 37 articles published in the top 10 SSCI journals related to educational technology from 2007 to 2016 (March). Scholars took mobile language learning in a real environment as the research object and summarized the results from four perspectives: journal publishing trends, language learning, research focus, and research methods. The results showed that the journal publishing trend was increasing. The most common research focus was cognition and language learner proficiency. The results also showed that mobile technology was positively perceived and accepted by students in most of these studies, and the technology was also found to have a positive impact on the students' language skills.

By exploring these review studies, the present review research found that 21st century skills were not considered in these earlier studies at all because scholars mainly focused on language skills. Therefore, several important aspects (e.g., theoretical foundations used to support the studies, methodology, and types of learning activities that promote language skills and 21st century skills) were ignored. These aspects are important for stakeholders in the design and implementation of language teaching and learning for 21st century skills development. In order to fill this gap in the literature, the present study was carried out, and the following research questions were addressed:

• What language skills and 21st century skills did the researchers focus on in the reviewed studies?
• What theories were used as a foundation in reviewed studies?
• What technologies were used to promote language skills and 21st century skills?
• What learning activities were used in the reviewed studies?
• What were the methodological characteristics of the reviewed studies?
• What research findings were obtained in the reviewed studies?

Research Method

The present study is a systematic review. The study used preferred reporting items for systematic reviews and meta-analyses (PRISMA) for the electronic search. PRISMA is considered as a set of programs that facilitates researchers to prepare and report various systematic evaluations and meta-analyses (Moher et al., 2009 ). According to scholars, PRISMA has been widely and successfully applied in educational research. In addition to PRISMA, this review followed the general guidelines for searching and selecting research articles proposed by Avgousti ( 2018 ), Shadiev and Yang ( 2020 ), and Shadiev and Yu ( In Press ). The search and selection processes are shown in Figure 1 . Articles were found through a search on the Web of Science database and Peer-Reviewed Instructional Materials Online Database (PRIMO). According to Kukulska-Hulme and Viberg ( 2018 ), PRIMO is a search tool and it contains several databases such as ERIC and Scopus. For this reason, PRIMO features a very comprehensive collection of full-text articles and bibliographic records, and it has been used by many researchers in their systematic reviews and meta-analyses to find relevant literature.

The search and selection process.

Based on general recommendations from previous review studies (Guan, 2014 ; Duman et al., 2015 ), this review used keywords such as 21st century skills, language learn * , and technology. 21st century skills were also included to widen the search results (e.g., creativity and innovation, critical thinking, problem solving, communication, collaboration, digital literacy, information literacy, media literacy, ICT literacy, flexibility and adaptability, initiative and self-direction, social and cross-cultural interaction, productivity and accountability, leadership and responsibility). This review used these terms in different combinations to search articles.

A total of 9,162 articles were found from the search. This review narrowed down the selection of research articles based on the following criteria (see Figure 1 ): articles that were (1) published during 2011–2022 (February); (2) published in English; and (3) focused on technology-supported language learning and 21st century skills. Two researchers screened each article individually and excluded articles from the study that did not focus on technology-supported language learning and 21st century skills. The researchers discussed any discrepancies in their selection results until an agreement was reached. At the end of the selection process, 34 empirical studies were chosen for the review.

This review proposed an analytical framework (see Figure 2 ) to answer the research questions of the study and to better understand the research design of the reviewed studies and findings. This review also used this framework to help us better review articles and regarded it as the basis for coding the content of reviewed studies. This review used the open coding method to carry out content analysis (Creswell, 2002 ) which can enable us to segment research content and to form categories relevant to the phenomenon of interest. The analytical dimensions included the following (see Figure 2 ): (1) language skills and 21st century skills—skills related to language learning and 21st century skills, (2) technology—the tools and devices participants used for language learning, (3) learning activities supported by technology to cultivate 21st century skills and language skills, (4) theoretical foundation—theories, models or hypothesis involved in research, (5) methodology—participants' academic level and major, research duration, sample size, data collection tools, and research design, and (6) findings—results reported in research.

Analytical framework.

Two researchers were involved in the coding process. They read articles and coded content according to the above coding scheme. After that, they categorized codes into categories and identified attributes for each category. If there were any differences in coding, the researchers re-examined an article to resolve differences, and then finally completed the coding phase. Interrater reliability was measured using Cohen's kappa coefficient and the result was high (k = 0.886).

The present study starts this section with the results related to publication year, languages, and participants. Figure 3 shows the distribution of articles published in the past 10 years. Most studies were published in 2019 ( n = 8), and no articles were published in 2012. From the figure, it can also be seen that the research trend in this field is on the rise. Figure 4 demonstrates the frequency at which different languages were the focus in the reviewed studies. 29 studies focused on English. There were also studies focused on Chinese ( n = 2), Ukrainian ( n = 1), Japanese ( n = 1), and Spanish ( n = 1). As shown in Figure 5 , undergraduates were the most common academic level ( n = 17), and there was a relatively low number of studies conducted on junior high school ( n = 5), senior high school ( n = 2), and primary school ( n = 1) academic levels. As shown in Figure 6 , researchers were more willing to involve students who were majoring in the fields of education ( n = 9), management ( n = 4), or engineering ( n = 4).

Distribution of research year.

Distribution of languages.

Distribution of educational level.

Distribution of participants' major.

Research Focus

This section presents the results related to research focus of reviewed articles. As can be seen from Figure 7 (and from Appendix 1 ), researchers carried out technology-assisted language learning studies and focused on the development of listening, speaking, reading, writing, grammar, and vocabulary skills. Among these skills, speaking skills ( n = 20) received considerable attention from researchers, followed by writing skills ( n = 19) and vocabulary ( n = 13). Reading ( n = 5) skills received less interest from researchers.

Distribution of language skills.

According to Figure 8 (and Appendix 1 ), researchers pointed out that technology-supported language learning can also promote 21st century skills. These skills relate to the following three categories: 4C (communication, collaboration, critical thinking, and creativity), digital literacy, and career and life skills. The most common skills that scholars targeted were communication ( n = 15) and collaboration ( n = 15), followed by critical thinking ( n = 10) and social and cross-cultural interaction ( n = 10). Problem solving ( n = 5) skills have received the least attention from researchers.

Distribution of 21st century skills.

Theoretical Foundation

This section focuses on theoretical foundation in the reviewed articles. As shown in Appendix 2 , a total of 16 theories were used. The most used theory was the social constructivism theory ( n = 9), followed by Byram's intercultural competence model ( n = 3), project-based learning ( n = 2), content based instruction ( n = 2), task based approach to language teaching ( n = 2), and sociocultural theory ( n = 2). The rest of theories were used only once.

As shown in Appendix 3 , a total of 52 technologies were used in reviewed studies. This review grouped them into eight categories: Social tools ( n = 20), Creative tools ( n = 19), Collaboration tools ( n = 13), Learning management system ( n = 9), Multimedia materials ( n = 5), Classroom interactive tools ( n = 4), Presentation tools ( n = 2), Wearable devices ( n = 1). Among the most commonly used technologies were Facebook ( n = 4), Google Docs ( n = 4), Moodle ( n = 4), followed sequentially by Skype ( n = 3), Padlet ( n = 3), WhatsApp ( n = 2), YouTube ( n = 2), Blogs ( n = 2), Google Drive ( n = 2), and Wiki ( n = 2). The other 40 technologies have only been used once, i.e., Windows Movie Maker, Live On, Edmodo, Kahoot, and Prezi. In addition, one study involved a virtual reality production tool (EduVenture) and a wearable device (Google Cardboard).

Learning Activity

As shown in Appendix 4 , in reviewed studies, scholars designed the following five main types of learning activities: (1) collaborative task-based language learning ( n = 9); (2) learning activities based on online communication ( n = 9); (3) creative work-based language learning ( n = 8); (4) adaptive learning activities ( n = 4); and (5) learning activities based on multimedia materials ( n = 4).

Methodology

This section presents methodological details of reviewed studies, such as sample size, research duration, data collection tools and research design.

As shown in Figure 9 , the most common sample size was from 11 to 30 participants ( n = 11), followed by sample sizes between 61 and 90 ( n = 8) and between 31 and 7 ( n = 7). Only two studies selected a sample size between 1 and 10. The sample size of two studies was >151. As shown in Figure 10 , most of research duration was between 3 and 6 months ( n = 10). There were 12 studies that did not state any research duration.

Sample size distribution.

Research duration distribution.

As shown in Appendix 5 , the most common data collection method was questionnaires ( n = 17), followed by tests ( n = 15) and interviews ( n = 13). Two data collection methods were used only 2 times, they were scales ( n = 2) and rubric ( n = 2).

As shown in Appendix 6 , research designs related to technology-supported language learning and 21st century skills were categorized into three main categories, namely quasi-experimental research ( n = 14), case studies ( n = 12), and action research ( n = 8).

As shown in Appendix 7 , various findings were reported in reviewed studies. In addition, that learners' language skills acquisition and 21st century skills, technology-supported language learning activities provided learners with good learning experiences, enhanced motivation and engagement, and improved self-confidence. In reviewed studies, some scholars reported about challenges faced by students during learning activities; they included challenges from technology, from their own competence, challenges of collaborating with others and self-attitude.

Language Skills

Regarding language skills, researchers have focused on improving learners' speaking, writing and vocabulary skills more. This shows that researchers are more concerned with the improvement of learners' skills related to language output. Researchers who reviewed studies on technology-supported language learning from 2014 to 2019 came to the same conclusion (Shadiev and Yang, 2020 ). However, the present study showed that reading skills received the least attention, while previous studies noted that grammar skills received less attention. This revealed that researchers are now beginning to pay more attention to previously neglected skills and are beginning to focus on the role of technology-supported language learning in facilitating learners' grammar skills. For example, Lai ( 2017 ) noted that grammar skills improved when learners completed activities to create vocabulary lists and greeting cards using multimedia resources. Jung et al. ( 2019 ) noted that students' grammar skills improved as they corrected each other's pronunciation and grammatical errors through video chat. Jamalai and Krish ( 2021 ) found that students' grammar skills improved through online forum discussions and knowledge sharing.

21st Century Skills

In terms of 21st century skills, communication and collaboration have received the most attention from researchers. It is probably because the 21st century society is more globalized and along with the increased complexity of related work, interpersonal communication and cooperation are being enhanced. The 21st century society emphasizes teamwork skills, and therefore scholars focus on collaborative and communication skills. Problem-solving skills have received little attention, and no researcher focused on career and life skills. In the face of the evolving and changing society of the future, problem-solving skills are among the core 21st century skills, emphasizing learners' ability to define problems, think critically, and solve problems. For example, scholars in reviewed studies have focused on learners' problem-solving skills in virtual technology-supported language learning (Chen et al., 2021 ).

Based on the results, this study has several recommendations for educators and researchers. First, input skills are an important component of language skills and an indispensable way for learners to develop output skills (Harmer, 2007 ). The present study suggests that researchers can focus on learners' input skills supported by technology, such as listening and reading. Second, problem-solving skills and career and life skills also deserve attention; therefore, future studies try to explore the effects of technology-supported language learning on these skills.

Theories Related to Instructional Design

The most commonly used instructional design theory in reviewed studies was social constructivism theory. The results of this research are consistent with those of previous review studies of technology-supported language learning (Parmaxi and Zaphiris, 2017 ). According to this theory (Vygotsky, 1978 ), knowledge is not a set of “facts” but rather a synthesis of information that is actively constructed and evolving in the learner's mind. The teacher does not “give” knowledge to the learner, but the learner should acquire knowledge actively. Learners' knowledge evolves as they process old and new information, as well as their experiences. The researchers designed collaborative, creative, and communicative activities based on a social constructivism perspective to encourage learners to input the target language and output the target language in a meaningful context. At the same time, researchers have used various learning and teaching activities to promote students' collaboration, communication, creativity, critical thinking and digital literacy skills (Yang et al., 2013 , 2014 , 2022 ; Lai, 2017 ; Sevilla-Pavón and Nicolaou, 2017 ; Huang, 2021 ).

Other researchers have also used theories based on learner-centered pedagogies such as problem-based or project-based theories. These pedagogies are all used to promote student-directed learning, adaptive learning, and personalized assessment. Learning theories were used to design activities that provided learners with opportunities for language input and output, e.g., to learn new knowledge and then apply it to the real world by creating own content. This allows learners to acquire language skills and develop 21st century skills such as communication, collaboration, and problem solving (Arnó-Macià and Rueda-Ramos, 2011 ; Yang et al., 2013 , 2014 ; Srebnaja and Stavicka, 2018 ).

Theories Related to Language Learning

Researchers have also designed learning activities based on theories related to foreign language learning, such as task-based language teaching, content-based instruction, and output-input theory. For example, digital story creation activities and integrated cross-cultural communication activities designed by the researcher are in line with these theoretical perspectives, in which learners have access to the target language through social tools and partner communication. The ability to use creative and collaborative tools to complete target-language based tasks also contributes to the acquisition of language skills and 21st century skills development, such as social and cross-cultural interaction, communication skills (Lewis and Schneider, 2015 ; Tseng, 2017 ).

Theories Related to Measuring Learning Outcomes

Since language learning is closely related to culture, scholars have designed foreign language courses based on cross-cultural communication, where learners acquired both language skills and cultural knowledge. Further, there are theories that have been used by scholars to assess and measure learners' outcomes. For example, researchers have focused on learners' intercultural competence along with their language skills and utilized the Byram' ICC model and the developmental model of intercultural sensitivity to measure their cross-cultural knowledge acquisition and skills development (Bennett, 1986 ; Byram, 1997 ). In addition, the Keller' ARCS motivational model (Keller, 1987 ) has been used by researchers to measure learners' perceived attention, relevance, confidence, and satisfaction in technology-supported language learning environments.

This review analyzed the theoretical foundation that was used by those few studies that focused on non-English languages such as Chinese, Ukrainian, Japanese, and Spanish. This review found that learning theories used by scholars in these studies were diverse. They were related to instructional design (e.g., social constructivism), language learning (e.g., language output and input), and cross-cultural learning (e.g., intercultural sensitivity).

Based on the findings, several suggestions for educators and researchers are proposed. First, the theories mentioned by researchers are instruction-related theories, language learning-related theories, and measurement-related theories; they were used to guide the design of technology-supported language learning activities that focus both on the acquisition of language skills and on the 21st century skills. These theories can be useful to inform the design of appropriate language learning activities for educators and researchers in the future. Second, this review found that many researchers did not indicate what theories were used in their studies. Theoretical foundations are important for the instructional design, language learning or measuring activities, so such information should be clearly indicated so that other researchers can gain a deeper understanding of them.

Eight Technologies With Different Functions

Based on the literature review, this study grouped technologies into eight categories based on their functions: (1) collaborative tools (e.g., Google Docs or Padlet) for supporting learners to collaborate on a task through co-editing and information sharing; (2) social tools (e.g., Facebook or Skype) for supporting learners to communicate and share content remotely or synchronously using text, audio and video; (3) creative tools (e.g., Photo Story or Adobe Spark) to support learners in creating work, such as digital stories or videos; (4) learning management system (e.g., Moodle) to integrate learning activities and learning resources for adaptive online learning; (5) classroom interaction tools (e.g., Quizlet or Kahoot) to support question-answering, polling, and other activities in the classroom; (6) multimedia materials are some audio and video resources on the web or multimedia textbooks; (7) presentation tools (e.g., PowerPoint) are used to support learners to present their learning content digitally; (8) wearable devices (e.g., Google Glass) to support learners to view or interact with content in virtual reality learning environments.

Most Commonly Used Technologies

Facebook (social tool), Google Docs (collaboration tool), and Moodle technologies (learning management system) were used the most in previous studies to facilitate language and 21st century skills. The study further analyzed which technologies are most often used by researchers to promote 21st century skills. Appendix 8 demonstrates these most commonly used tools. The study found that Facebook (social tool), Google Docs (collaboration tool) and Moodle (learning management system) were also the tools most often used by researchers to promote communication, collaboration and critical thinking, social and cross-cultural interaction skills. This indicates that scholars valued such 21st century skills as collaboration and communication among students in technology-supported language learning activities. For example, Sevy-Biloon and Chroman ( 2019 ) used social and collaborative tools (e.g., Google Docs, Facebook, etc.) to support communication between students from different cultural backgrounds and their results showed that students' speaking skills, social and cross-cultural interaction, and communication skills were promoted. Moodle is popular among researchers because this learning management system not only supports learners' adaptive and inquiry-based learning, but also helps teachers share learning resources with learners, design learning activities, and manage learners' learning progress (García-Sánchez and Burbules, 2016 ). For example, Yang et al. ( 2014 ) designed a language learning activity based on the Moodle platform that asked students to complete reading and writing tasks in the system to promote the development of reading, writing skills and critical thinking. In addition, researchers most often used Google Docs (collaboration tool), Prezi (presentation tool), Windows Movie Maker, Photo Story3 (creative tools) and Blogs (social tool) to support students' creativity and innovation skills, problem-solving skills, and ICT literacy. And only two studies have used films (multimedia materials) and blogs (social tool) to support students' media literacy.

Experienced Challenges of Using Technology

Scholars reported that technologies pose some challenges for learners. For example, students were not experienced to use technology and had no trainings before learning activities; then they complained about problems to use technology during learning (Lai, 2017 ). Students were also confused about the layout of the platform and noted that there were incompatibilities and connectivity issues with learning devices (Hosseinpour et al., 2019 ). When communicating remotely, students pointed out that there were problems with the network and they were not able to connect and participate in learning process (Mohamadi Zenouzagh, 2018 ; Jung et al., 2019 ).

The Distribution of Technology in non-English Language Studies and Different Theories

This review also analyzed technologies that were used by those few studies that focused on non-English languages. This review found that, in general, scholars in these studies used such technologies as creative tools (Adobe Spark), collaboration tools (Google Docs), and social tool (Facebook) to present multimedia content to learners and support collaborative, creative and communicative learning activities (Valdebenito and Chen, 2019 ).

With regard to the distribution of technology in theory. Social constructivism theory was the most commonly cited theory in reviewed research and scholars used various technologies such as learning management systems (e.g., Moodle), creative tools (e.g., iMovie) or social tools (e.g., Facebook) to support constructivism-based learning activities. That is, interactive and collaborative learning activities were designed for students to learn new knowledge and then apply it to construct meaning in authentic contexts.

Based on the results of this study, several recommendations for educators and researchers were proposed. First, it is recommended that learning activities supported by technologies are designed based on appropriate theoretical foundation. Second, teachers are encouraged to conduct appropriate technology training for students beforehand so that they become familiar with technology tools. Third, teachers and researchers should test learning tools with students in advance in order to identify any possible technical problems, and provide timely support during learning process.

Learning Activities Used to Promote Language Skills and 21st Century Skills

This section describes what technologies are used in each type of learning activity and how they contribute to the development of learners' language skills and 21st century skills. In addition, it offers relevant suggestions to researchers and educators.

Adaptive Language Learning Activities on Learning Platforms

As shown in Table 1 , in the reviewed study, researchers used the following tools: Moodle, Google classroom, Quantum leap, and WebQuest, to develop adaptive language learning activities on learning platforms. These tools are used to integrate different types of instructional resources and diverse language learning activities to provide learners with adaptive learning materials that meet their learning needs. Students can ask questions and receive feedback from other students or teachers, and take control of their own learning progress.

Adaptive language learning activities on learning platforms.

For example, Arnó-Macià and Rueda-Ramos ( 2011 ) designed tasks for reading, listening, and speaking practice in Quantum leap platform. Researchers have designed listening tasks in Moodle platform; students were required to analyze, evaluate, and summarize content after listening (Yang et al., 2013 , 2014 ). Srebnaja and Stavicka ( 2018 ) designed WebQuests-based speaking and writing tasks.

All of these studies noted that learners' performance in speaking, listening, reading, writing, and grammar improved after completing the computer-assisted adaptive language learning tasks. In addition, students' critical thinking skills were developed.

Collaborative Task-Based Language Learning Activities

As shown in Table 2 , the following tools were used by researchers for the development of collaborative-based language learning activities: (1) collaboration tools: Google Docs, Google Drive, Wiki, Edmodo, and E-writing forum. These collaborative tools have the following functions: sharing, collaborative editing, cloud storage, synchronized display, and help students freely share information in various formats (e.g., text, images, videos, web links, audio recordings, music, etc.) on the platform so that they can exchange ideas and collaborate on editing content; (2) creative tools: Adobe Spark, to support students' expression of ideas; (3) social tools: Blogs or WordPress, to support students in reading and commenting on each other's work.

Collaboration-based language learning activities.

Collaboration-based language learning activities are those in which students work in groups to solve problems and complete tasks proposed by the teacher, such as asking students to provide an essay or present their ideas in other ways (e.g., a solution, a report, and a performance). For example, Amir et al. ( 2011 ) asked students to work in groups to publish six articles based on different topics over the course of 14 weeks, and one of the tasks required students to find and discuss software about computer-assisted writing.

Mohamadi Zenouzagh ( 2018 ) designed a collaborative writing activity based on the E-writing platform. Valdebenito and Chen ( 2019 ) designed a collaborative activity on the theme of “food and culture” in which students first had to use Google Maps to identify geographic areas related to the content, then use a Google Doc to record their ideas, and finally use video production tools such as Adobe Spark to express their ideas and share them on the WordPress platform. Huh and Lee ( 2020 ) designed a creative learning English collaborative activity in which students first used a mobile app to learn how to spell words, then the group took the words they learned and expressed them through the role play and song. Lai ( 2017 ) designed different collaborative tasks, for example, students needed to use the ThingLink tool to create vocabulary lists and greeting cards related to the topic, which were then shared on the Padlet platform and discussed. In addition, students were required to use HomeStyler to collaboratively design a dream home and use some vocabulary related to “location” to describe the design of their home.

Girgin and Cabaroglu ( 2021 ) designed an English learning project that integrates Web 2.0 technology and flipped classroom, and students used Padlet to watch videos in class. In grammar classes, students used Kahoot, Quizlet, Quizizz, Animoto, Powtoon, and Poster MyWall to answer grammar questions. In vocabulary and reading classes, students used tools such as Mind Meister, Voki, Canva, Cram, Go Animate and Story-bird to create mind maps, as well as create digital stories, which can be presented and shared. Chen et al. ( 2021 ) used virtual reality technology to design language learning activities. Learners were required to first watch a virtual reality scene and think about how to solve the problem based on a series of guiding questions provided by the teacher. Then students role-played in English to create a virtual reality video of the problem being solved.

The results of the abovementioned studies showed that collaborative-based language learning activities facilitated the development of learners' language skills. The researchers noted that collaborative problem-solving language learning activities provided learners with a large number of writing tasks, such as writing reports, essays, or creating storylines and designing works. The process of sharing with each other enabled to point out grammatical errors (Amir et al., 2011 ; Mohamadi Zenouzagh, 2018 ; Hosseinpour et al., 2019 ). When learners used multimedia resources to create vocabulary lists and greeting cards, their vocabulary and grammar skills were also improved (Lai, 2017 ).

At the same time, students' critical thinking was developed as they gave each other's critical and constructive comments (Valdebenito and Chen, 2019 ; Zou and Xie, 2019 ; Girgin and Cabaroglu, 2021 ). In addition, students completed tasks in small groups which promoted the development of communication and collaboration skills during discussions with each other (Amir et al., 2011 ; García-Sánchez and Burbules, 2016 ; Lai, 2017 ; Mohamadi Zenouzagh, 2018 ; Hosseinpour et al., 2019 ; Zou and Xie, 2019 ; Girgin and Cabaroglu, 2021 ). The process of students voicing digital content promoted the development of speaking skills (Huh and Lee, 2020 ). In the process of creating digital works, digital literacy was developed (García-Sánchez and Burbules, 2016 ; Valdebenito and Chen, 2019 ). Chen et al. ( 2021 ) pointed out that learners learn contextually in an immersive learning environment, and solving real problems through virtual reality technology improved learners' vocabulary as well as promoted their problem-solving skills.

Creative Work-Based Language Learning Activities

As shown in Table 3 , in reviewed studies, language learning activities based on creative works consisted of two main categories: creating digital stories or videos. The main models for this type of learning activity were as follows: students communicated in groups about how to create a digital story or video, then collected and processed relevant information, after that created a digital story, and finally shared content and communicated with each other about it.

Creative work-based language learning activities.

The researchers chose different tools to support such learning process, e.g., (1) creating digital stories, i.e., Photo Story3, Windows Movie Maker, or iMovie; (2) creating video scripts in collaboration, i.e., Google Docs or Google Drive; (3) presenting digital stories, i.e., Prezi or PPT; (4) sharing digital stories and communicating, i.e., Google+ forums, Facebook, Instagram, WhatsApp, Google Classroom, and classroom management systems.

The researcher noted that digital storytelling promoted language skills, specifically, the process of writing story scripts promoted students' writing and vocabulary skills (Thang et al., 2014 ; Sevilla-Pavón and Nicolaou, 2017 ; Kulsiri, 2018 ; Yalçin and Öztürk, 2019 ; Chiang, 2020 ). It also promoted 21st century skills. Researchers mentioned three approaches for creating digital stories or videos such as free-writing, rewriting the ending of the story, and specifying the theme, and in this open-ended work creation process, students' sense of creativity, problem-solving skills, and digital literacy were developed (Thang et al., 2014 ; Sevilla-Pavón and Nicolaou, 2017 ; Kulsiri, 2018 ; Yalçin and Öztürk, 2019 ; Yang et al., 2022 ). Regarding the creation of digital stories on a specific theme, the researcher asked learners to design a new country, and students needed to understand a range of elements including different countries and cultures, such as national characteristics, language, national policies, climate and life. As a result, students' social and cross-cultural skills were improved. In addition, critical thinking was facilitated as students developed different ideas and perspectives as they evaluated each other's digital stories (Sevilla-Pavón and Nicolaou, 2017 ). Finally, students developed their communication and collaboration skills when working in groups (Thang et al., 2014 ; Sevilla-Pavón and Nicolaou, 2017 ; Kulsiri, 2018 ; Yalçin and Öztürk, 2019 ; Mirza, 2020 ; Huang, 2021 ).

Language Learning Activities Based on Multimedia Learning Materials

As shown in Table 4 , language learning activities based on multimedia materials involved such tools as (1) web-based learning management system, e.g., EDpuzzle; (2) social tool, e.g., YouTube; and (3) multimedia textbooks. All of them provided multimedia resources for students. There were also (4) collaboration tools, e.g., Padlet and Google docs, which supported learners to share ideas with each other.

Language learning activities based on learning multimedia materials.

Scholars have designed a variety of language learning activities based on multimedia materials, but the topics and learning tasks of the multimedia materials involved in these studies differed. For example, Tseng ( 2017 ) asked learners to watch a video on the topic of cultural differences, and then students gave oral presentations and reflections to present their views on cultural differences. Zou and Xie ( 2019 ) asked students to watch a video on EDpuzzle, then to discuss in groups, negotiate and compare answers, to share their output to the Padlet platform, and finally submit their reports in Google docs. Nikitova et al. ( 2020 ) asked students to watch videos from multimedia textbooks with different English contexts and then simulated learners' role play activities. Aristizábal-Jiménez ( 2020 ) asked learners to watch YouTube videos, analyze the structure and content of video content, and then create posters to present and share their ideas.

The researcher noted that language learning activities based on multimedia materials promoted learners' language skills and 21st century skills. Specifically, learners' listening skills were promoted after watching the videos (Tseng, 2017 ). Culturally relevant content in videos and culture-based communication among peers promoted students' social and cross-cultural interaction skills (Tseng, 2017 ). Learners actively used dictionaries and discussed grammar while completing tasks to make the information easier to understand, which also promoted students' vocabulary and grammar skills (Aristizábal-Jiménez, 2020 ). In addition, working in groups to complete tasks promoted speaking, writing, grammar, and vocabulary skills. This was also beneficial to develop students' problem solving, collaboration, critical thinking, and communication skills (Aristizábal-Jiménez, 2020 ; Nikitova et al., 2020 ).

Language Learning Activities Based on Online Communication

As shown in Table 5 , the researchers designed online communication-based language learning activities. Most of them were cross-cultural communication activities to support cross-cultural communication between students from different cultural backgrounds. In terms of technology, the researchers mainly used social tools to support textual or video communication, e.g., Facebook, Skype, and WhatsApp. In addition, researchers have utilized learning management systems to support students to view learning resources uploaded by teachers.

Language learning activities based on online communication.

The design of cross-cultural communication activities followed the same pattern—exposure to cross-cultural knowledge, reflection on cross-cultural differences, and cross-cultural exchange. For example, Calogerakou and Vlachos ( 2011 ) had students from two countries to watch movies and compare culture presented in movies with their own culture. Then students had to post comments on a blog and discuss their ideas. Chen and Yang ( 2016 ) asked students to share culturally specific folklore stories with their partners and to make videos of the stories to send to their partners. In addition, students were asked to perform a puppet show via videoconference. All of these were for students to learn about cultural similarities and differences. Chen and Yang ( 2014 ) designed a discussion activity based on cultural themes; for example, students discussed movies that involved culturally different content, and then students shared their opinions on Wiki. Lewis and Schneider ( 2015 ) asked learners to interact with native Spanish-speaking students and discuss cultural topics such as “local living conditions” and “how to celebrate holidays.” Learners were then asked to write a mini-biography or travel brochure for their study partner to demonstrate the cultural knowledge they gained during the exchange. Özdemir ( 2017 ) asked students to watch YouTube videos and discuss them based on cross-cultural questions prepared by the teacher. Sevy-Biloon and Chroman ( 2019 ) designed an intercultural exchange program in which students from Ecuador and the United States were randomly paired and then engaged in a cultural exchange based on the theme of the language course. Jung et al. ( 2019 ) asked students from different cultural backgrounds to discuss cultural topics, including “happiness factors, family, and food,” and finally, students reflected on the discussion, exchanged proverbs with each other, and then presented cultural differences. They reflected on their experiences in a reflective journal. Jamalai and Krish ( 2021 ) designed an online discussion activity, in which learners were required to engage in online discussions based on topics posted by teachers in a forum.

The results showed that students' speaking, vocabulary, writing, reading, and grammar skills improved when communicating through text and speech because students double-checked vocabulary spelling and grammar. Students identified errors they made when communicating using text and speech and corrected them to ensure that others understood their intended meaning (Calogerakou and Vlachos, 2011 ; Chen and Yang, 2014 , 2016 ; Lewis and Schneider, 2015 ; Özdemir, 2017 ; Hirotani and Fujii, 2019 ; Jung et al., 2019 ; Sevy-Biloon and Chroman, 2019 ; Jamalai and Krish, 2021 ). In addition, students' listening skills improved after watching YouTube videos (Özdemir, 2017 ).

At the same time, students' communication process using social tools developed the ability to use writing software, electronic dictionaries, and collect information on the Internet, and therefore media literacy was improved (Calogerakou and Vlachos, 2011 ). All studies point to the development of cultural interaction skills after students interacted and exchanged different cultural perspectives with partners (Calogerakou and Vlachos, 2011 ; Chen and Yang, 2014 , 2016 ; Lewis and Schneider, 2015 ; Özdemir, 2017 ; Hirotani and Fujii, 2019 ; Jung et al., 2019 ; Sevy-Biloon and Chroman, 2019 ). Communication (Chen and Yang, 2014 ; Lewis and Schneider, 2015 ; Hirotani and Fujii, 2019 ) and collaboration skills were also developed (Chen and Yang, 2014 ) in reviewed studies.

This review also analyzed learning activities that were used by those few studies that focused on non-English languages. This review found that most learning activities designed in these studies were online cross-cultural communicative activities. This shows that the primary goal of these learning projects was to develop students' foreign language and intercultural communication skills.

Based on the findings of the reviewed literature, the five types of language learning activities supported by technology had a positive impact on students' language skills as well as their 21st century skills development. Moreover, this review found that these learning activities followed similar pattern. The common pattern for language learning activities based on culture-related communication was exposure to cross-cultural knowledge, reflection on cross-cultural differences, and cross-cultural exchange. The common pattern of language learning activities for creative works was as follows: students communicated in groups about how to create a work (such as digital story or video), then collected and processed relevant information, created a work, and then shared content and communicated with each other about it. These patterns could provide suggestions for researchers and teachers to design similar instructional activities that target development of language skills and 21st century skills in the future.

Second, this review found that researchers designed similar instructional activities, but the research focus was different. For example, in the adaptive language learning activities on learning platforms, researchers focused on the development of students' speaking skills and lacked attention to reading skills. And in the collaborative task-based language learning activities, researchers have focused more on writing and vocabulary skills, collaboration, and communication skills, and lacked attention to listening skills. In creative writing-based language learning activities, researchers focused more on speaking and writing skills as well as creative and communication skills.

Research Duration, Participants' Academic Level, and Sample Size

The most common study samples were small ones with participants range from 11 to 30 ( n = 11) and medium samples with range between 61 and 90 ( n = 8) participants. Research durations were mostly between 3 and 6 months ( n = 10). Small sample size was acknowledged as a limitation in some studies (Hirotani and Fujii, 2019 ; Zou and Xie, 2019 ). The possible reason for this is that most of the studies were based on small classroom settings. In the reviewed studies, the most common academic level of participants was undergraduate level. There were 12 studies that did not specify research duration. Regarding this finding, there is a lack of attention in previous retrospective studies (Guan, 2014 ; Duman et al., 2015 ; Persson and Nouri, 2018 ).

Data Collection

Most of the studies collected both quantitative and qualitative data, which can help researchers to draw conclusions from different perspectives. Quantitative data included tests, scales, and rubrics; qualitative data included student's work, open-ended questions, student feedback, interviews, student chat transcripts, student reflections, teacher journals, and observations. One of the most common forms of quantitative data collection is a test ( n = 15), involving student language tests (tests of English speaking and listening) and tests of 21st century skills (critical thinking and creative thinking). The most common method of qualitative data collection was interview ( n = 13), where the researcher usually designed an interview outline and then asked learners questions to understand their learning experiences, attitudes, motivations, and challenges in the learning process. In addition, researchers have extensively used questionnaires ( n = 17), including both closed-ended and open-ended questions, to collect both quantitative and qualitative data. For example, the researchers used questionnaires to investigate learners' perceptions of technology-supported language learning, including effectiveness, usefulness, and students' perceptions of developing intercultural communicative competence and language skills through online discussions (Jung et al., 2019 ).

Based on the above findings, the recommendations of the present study for researchers and teachers are as follow. First, researchers could consider studies with longer time spans and collect data from bigger number of participants to investigate students' development over time and have generalizable conclusions. Second, researchers can collect multiple types of data, focus on students' learning processes and outcomes, and then interpret findings from different perspectives.

Research Design

There are a variety of research designs for reviewed studies on technology-supported language learning and 21st century skills. The most common are quasi-experimental studies. Such studies are characterized by using pre- and post-tests to measure changes in participants' language skills, 21st century skills and other learning outcomes and attitudes before and after participation in learning activities. In quasi-experimental studies, participants are not randomly assigned to an experimental or control group (Persson and Nouri, 2018 ; Huang, 2021 ). These findings are consistent with other reviews on technology-supported language learning (Persson and Nouri, 2018 ). The present study suggests that educators and researchers can use the three research methods mentioned above to validate their studies in future.

Positive Learning Experiences

In this section, the study discusses findings from reviewed studies and recommendations for educators and researchers. In reviewed studies, in addition to finding that technology-supported learning activities promoted learners' language skills and 21st century skills, researchers also found that these technologies led to positive learning experiences, which resulted in better learning outcomes. For example, learning through multimedia textbooks, collaborative blog-based writing activities, smartphone-based video filming activities and language learning projects based on intercultural exchange all increased students' motivation (Amir et al., 2011 ; García-Sánchez and Burbules, 2016 ; Sevy-Biloon and Chroman, 2019 ; Aristizábal-Jiménez, 2020 ; Huang, 2021 ). For example, Hosseinpour et al. ( 2019 ) noted that through collaborative writing activities, learners' motivation and self-confidence levels were increased. Mirza ( 2020 ) argued that through digital storytelling-based learning activities, students gained more confidence. Researchers have also looked at the different learning performance of students due to individual differences in abilities or their characteristics. Yang et al. ( 2014 ) found that in terms of writing, significant differences were found between “basic” and “low-intermediate” learners as a result of the difference in ability. Yalçin and Öztürk ( 2019 ) found that girls had a more negative attitude toward technology than boys.

Challenges Faced by Students

While many studies pointed to positive student attitudes toward technology-supported learning activities (Arnó-Macià and Rueda-Ramos, 2011 ; Girgin and Cabaroglu, 2021 ), several studies highlighted challenges that students faced when using technology for learning. Challenges from technology, with some learners finding it difficult to use in learning activities or being confused about the layout of mobile applications were mentioned. Students also noted problems with device incompatibility and poor network quality and speed when using technology. Self-competence challenges, with learners noting that learning tasks were difficult for them, for example, insufficient time to complete learning tasks, lack of research skills, or language skills needed to complete tasks, were reported. Difficulties in finding an interesting topic and choosing the right tools to create their work were also reported in reviewed studies. Challenges of collaborating with others, with some learners noting that they encounter uncoordinated teamwork, uneven distribution of work and unequal student contributions in collaborative tasks, were mentioned by scholars. Self-attitudes, as noted by learners who felt anxious about video chatting when they were communicating remotely, as well as fear of having their writing errors discovered by their partners when communicating in text, were reported in reviewed studies.

Based on the above findings, the present study recommends to educators and researchers, in addition to focusing on the impact of technology-supported learning activities on learners' language skills and 21st century skills, it is also important to focus on students' perceptions of technology, motivation, engagement, and confidence. This is because positive learning experiences can lead to better learning outcomes (Sevy-Biloon and Chroman, 2019 ; An et al., 2021 ). Regarding the technological challenges that students encounter in the learning process, it is recommended that they be addressed through advance trainings and through providing students with appropriate technological services during learning activities. Self-competence challenges can be addressed by designing collaborative tasks in which students with higher levels of competence can help students with lower levels of competence to complete the task. Regarding the challenges in collaborative activities, it is recommended that teachers and researchers design learning activities with clear rules for collaborative division of labor and rules regarding how learning performance of every learner will be evaluated. With regard to alleviating negative student attitudes, it is recommended that teachers design diverse teaching strategies and scaffolds to give students assistance during learning activities.

This study reviewed articles on technology-supported language learning and 21st century skills published from 2011 to 2022 (February) in terms of (a) research focus; (b) theoretical foundations; (c) technology; (d) learning activities; (e) methodology and (f) findings. The results indicate that research on technology-supported language learning and 21st century skills have shown an upward trend in the overall research in the covered time period, with most of the research focusing on English and the majority of participants in these studies majored in education.

Secondly, in terms of research focus, most of the researchers focused on learners' speaking skills (27.40%), followed by writing (26.03%) and vocabulary skills (17.81%). In terms of 21st century skills, most researchers focused on communication (20.83%), collaboration (20.83%), critical thinking (13.89%), and social and cross-cultural interaction skills (13.89%). In terms of theoretical foundations, social constructivist learning theory was most often adopted by researchers. In terms of technology, tools that support learners' creativity and socialization are often utilized by researchers, e.g., Facebook or Google Docs. In terms of learning activities, researchers have designed the following five types of learning activities to support learners' language learning and 21st century skills: (1) collaborative task-based language learning activities; (2) language learning activities based on online communication; (3) creative work-based language learning activities (4) adaptive language learning activities based on learning platforms; and (5) language learning activities based on multimedia learning materials. The results of reviewed studies indicate that these learning activities supported by technology are effective in promoting the development of learners' different language skills and 21st century skills. Finally, in terms of methodology, most of the studies had a sample of 11–30, the most common study period was 3–6 months, the data collection method often used by researchers was questionnaires, the most common method to collect quantitative data was tests, and the most common method to collect qualitative data was interviews.

In contrast to traditional paper and pencil-based learning, technologies used by researchers in reviewed studies allowed learners to improve language learning outcomes and 21st century skills through individual and collaborative learning activities. Some reported advantages are learning with technologies without the constraints of time and space, technologies enable personalized learning, technologies create authentic learning environments that provides adaptive learning content, helps create multimedia content actively, allows social interaction such as sharing, giving or receiving feedback, and reflecting on learning more efficiently.

Based on the above findings, recommendations for researchers and educators in this study include: (1) In terms of language skills, in addition to focusing on output skills, input skills (reading, listening) also deserve attention from researchers. In terms of 21st century skills, learners' problem-solving skills and career and life skills also need more attention from researchers in the future; (2) Advanced technology training for learners to familiarize them with technology and its effective usage as well as teachers need to check in advance for possible technology problems, such as network problems. These suggestions can help teachers address the technological barriers that learners encounter in the learning process; (3) The use of various theoretical approaches, such as instructional design-related theories and language learning-related theories, is important for the rational design of instructional activities that promote learners' language and 21st century skills; (4) Researchers and educators can follow the general model of conducting the five types of instructional activities summarized above to design instructional activities. In addition, it is recommended that researchers and educators use variety of technologies and design different instructional activities to promote learners' language and 21st century skills. It is also important to be aware of the challenges that students may encounter in terms of technology, learning activity tasks, peer collaboration and self-attitudes when implementing learning activities; (5) Teachers and educators could involve more participants and consider longer time spans in future studies to focus on more learners' development and to collect diverse quantitative and qualitative data to explain students' learning processes and outcomes.

There are few limitations to this study. Articles reviewed in this study were sourced from PRIMO and Web of Science databases, and some conference papers, books and dissertations were excluded. For this reason, this study reviewed smaller number of articles. Future studies may consider this limitation and address it by including more relevant sources.

Data Availability Statement

Author contributions.

RS and XW contributed to the conception, designed the work, collected the data, analyzed, and interpreted data. XW drafted the work and RS substantively revised it. RS was responsible for correspondence. All authors approved the submitted version and agreed both to be personally accountable for the author's own contributions and to the accuracy the work.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

1 Articles reviewed in this study.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpsyg.2022.897689/full#supplementary-material

• * . Amir Z., Ismail K., Hussin S. (2011). Blogs in language learning: maximizing students' collaborative writing . Procedia Soc. Behav. Sci. 18 , 537–543. 10.1016/j.sbspro.2011.05.079 [ CrossRef ] [ Google Scholar ]
• An Z., Wang C., Li S., Gan Z., Li H. (2021). Technology-assisted self-regulated English language learning: associations with English language self-efficacy, English enjoyment, and learning outcomes . Front. Psychol. 11 :558466. 10.3389/fpsyg.2020.558466 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• * . Aristizábal-Jiménez Y. (2020). Fostering talk as performance in an EFL class through the critical analysis of youtubers' content . Profile Issues Teach. Professional Dev. 22 , 181–195. 10.15446/profile.v22n2.82510 [ CrossRef ] [ Google Scholar ]
• * . Arnó-Macià E., Rueda-Ramos C. (2011). Promoting reflection on science, technology, and society among engineering students through an EAP online learning environment . J. English Acad. Purposes 10 , 19–31. 10.1016/j.jeap.2010.12.004 [ CrossRef ] [ Google Scholar ]
• Asia Pacific Economic Cooperation (2004). 2004 APEC Ministerial Meeting. Available online at: https://www.apec.org/Meeting-Papers/Annual-Ministerial-Meetings/2004/2004_amm (accessed June 11, 2022).
• Avgousti M. I. (2018). Intercultural communicative competence and online exchanges: a systematic review . Computer Assisted Lang. Learn. 31 , 819–853. 10.1080/09588221.2018.1455713 [ CrossRef ] [ Google Scholar ]
• Bennett M. (1986). A developmental approach to training for intercultural sensitivity . Int. J. Intercultural Relat. 10 , 179–196. 10.1016/0147-1767(86)90005-2 [ CrossRef ] [ Google Scholar ]
• Byram M. (1997). Teaching and Assessing Intercultural Communicative Competence . Clevedon: Multilingual Matters. [ Google Scholar ]
• * . Calogerakou C., Vlachos K. (2011). Films and Blogs: an authentic approach to improve the writing skill-an intercultural project-based framework in the Senior High State School . Res. Papers Lang. Teach. Learn. 2 , 98–110. [ Google Scholar ]
• * . Chen C. H., Hung H. T., Yeh H. C. (2021). Virtual reality in problem-based learning contexts: effects on the problem-solving performance, vocabulary acquisition and motivation of English language learners . J. Computer Assisted Learn. 37 , 851–860. 10.1111/jcal.12528 [ CrossRef ] [ Google Scholar ]
• * . Chen J. J., Yang S. C. (2014). Fostering foreign language learning through technology-enhanced intercultural projects . Lang. Learn. Technol. 18 , 57–75. [ Google Scholar ]
• * . Chen J. J., Yang S. C. (2016). Promoting cross-cultural understanding and language use in research-oriented Internet-mediated intercultural exchange . Computer Assisted Lang. Learn. 29 , 262–288. 10.1080/09588221.2014.937441 [ CrossRef ] [ Google Scholar ]
• * . Chiang M. H. (2020). Exploring the effects of digital storytelling: a case study of adult L2 writers . IAFOR J. Educ. 8 , 65–82. 10.22492/ije.8.1.04 [ CrossRef ] [ Google Scholar ]
• Creswell J. W. (2002). Educational Research: Planning, Conducting, and Evaluating Quantitative. Upper Saddle River, NJ: Prentice Hall. [ Google Scholar ]
• Duman G., Orhon G., Gedik N. (2015). Research trends in mobile assisted language learning from 2000 to 2012 . ReCALL 27 , 197–216. 10.1017/S0958344014000287 [ CrossRef ] [ Google Scholar ]
• * . García-Sánchez M. S., Burbules N. C. (2016). Learning technologies and EFL teamwork . Revista de Lenguas para Fines Específicos 22 , 100–115. 10.20420/rlfe.2016.0092 [ CrossRef ] [ Google Scholar ]
• * . Girgin P., Cabaroglu N. (2021). Web 2.0 supported flipped learning model: EFL students' perceptions and motivation . Cukurova Univ. Faculty Educ. J. 50 , 858–876. 10.14812/cuefd.944217 [ CrossRef ] [ Google Scholar ]
• Goksu I., Ozkaya E., Gunduz A. (2020). The content analysis and bibliometric mapping of CALL journal . Computer Assisted Lang. Learn. 1–31. 10.1080/09588221.2020.1857409 [ CrossRef ] [ Google Scholar ]
• Guan S. (2014). Internet-based technology use in second language learning: a systematic review . Int. J. Cyber Behav. Psychol. Learn. 4 , 69–81. 10.4018/ijcbpl.2014100106 [ CrossRef ] [ Google Scholar ]
• Harmer J. (2007). The Practice of English Language Teaching . London: Longman. [ Google Scholar ]
• * . Hirotani M., Fujii K. (2019). Learning proverbs through telecollaboration with Japanese native speakers: facilitating L2 learners' intercultural communicative competence . Asian Pacific J. Second Foreign Lang. Educ. 4 , 1–22. 10.1186/s40862-019-0067-5 [ CrossRef ] [ Google Scholar ]
• * . Hosseinpour N., Biria R., Rezvani E. (2019). Promoting academic writing proficiency of Iranian EFL learners through blended learning . Turkish Online J. Distance Educ. 20 , 99–116. 10.17718/tojde.640525 [ CrossRef ] [ Google Scholar ]
• * . Huang H. W. (2021). Effects of smartphone-based collaborative vlog projects on EFL learners' speaking performance and learning engagement . Austral. J. Educ. Technol. 37 , 18–40. 10.14742/ajet.6623 [ CrossRef ] [ Google Scholar ]
• * . Huh K., Lee J. (2020). Fostering creativity and language skills of foreign language learners through SMART learning environments: evidence from fifth-grade Korean EFL learners . TESOL J. 11 , e489. 10.1002/tesj.489 [ CrossRef ] [ Google Scholar ]
• * . Jamalai M., Krish P. (2021). Fostering 21st century skills using an online discussion forum in an English for specific purpose course . Malaysian J. Learn. Instruct. 18 , 219–240. 10.32890/mjli2021.18.1.9 [ CrossRef ] [ Google Scholar ]
• * . Jung Y., Kim Y., Lee H., Cathey R., Carver J., Skalicky S. (2019). Learner perception of multimodal synchronous computer-mediated communication in foreign language classrooms . Lang. Teach. Res. 23 , 287–309. 10.1177/1362168817731910 [ CrossRef ] [ Google Scholar ]
• Keller J. M. (1987). Development and use of the ARCS model of instructional design . J. Instruct. Dev. 10 , 2–10. 10.1007/BF02905780 [ CrossRef ] [ Google Scholar ]
• Krashen S. D. (1985). The Input Hypothesis: Issues and Implications . London: Longman. [ Google Scholar ]
• Kukulska-Hulme A., Viberg O. (2018). Mobile collaborative language learning: state of the art . Br. J. Educ. Technol. 49 , 207–218. 10.1111/bjet.12580 [ CrossRef ] [ Google Scholar ]
• * . Kulsiri S. (2018). Students' perceptions of a student-produced video project in the General English language course at Srinakharinwirot University, Thailand . Arab World Eng. J. 4 , 40–54. 10.24093/awej/call4.4 [ CrossRef ] [ Google Scholar ]
• * . Lai A. (2017). Implementing online platforms to promote collaborative learning in Chinese language classrooms . J. Technol. Chin. Lang. Teach. 8 , 39–52. [ Google Scholar ]
• Lantolf J. (2000). Sociocultural Theory and Language Learning . Oxford: OUP. [ Google Scholar ]
• * . Lewis T. N., Schneider H. (2015). Integrating international video chat into the foreign language curriculum . Int. J. Comput. Assisted Lang. Learn. Teach. 5 , 72–84. 10.4018/IJCALLT.2015040105 [ CrossRef ] [ Google Scholar ]
• Lin L., Shadiev R., Hwang W. Y., Shen S. S. (2020). From knowledge and skills to digital works: an application of design thinking in the information technology course . Think. Skill Creat. 36 , 100646. 10.1016/j.tsc.2020.100646 [ CrossRef ] [ Google Scholar ]
• * . Mirza H. S. (2020). Improving university students' english proficiency with digital storytelling . Int. Online J. Educ. Teach. 7 , 84–94. [ Google Scholar ]
• * . Mohamadi Zenouzagh Z. (2018). Multidimensional analysis of efficacy of multimedia learning in development and sustained development of textuality in EFL writing performances . Educ. Information Technol. 23 , 2969–2989. 10.1007/s10639-018-9754-y [ CrossRef ] [ Google Scholar ]
• Moher D., Liberati A., Tetzlaff J., Altman D. G., PRISMA Group * (2009). Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement . Ann. Internal Med. 151 , 264–269. 10.7326/0003-4819-151-4-200908180-00135 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• * . Nikitova I., Kutova S., Shvets T., Pasichnyk O., Matsko V. (2020). “Flipped learning” methodology in professional training of future language teachers . Euro. J. Educ. Res. 9 , 19–31. 10.12973/eu-jer.9.1.19 [ CrossRef ] [ Google Scholar ]
• * . Özdemir E. (2017). Promoting EFL learners' intercultural communication effectiveness: a focus on Facebook . Comput. Assisted Lang. Learn. 30 , 510–528. 10.1080/09588221.2017.1325907 [ CrossRef ] [ Google Scholar ]
• Parmaxi A., Zaphiris P. (2017). Web 2.0 in Computer-Assisted Language Learning: a research synthesis and implications for instructional design and educational practice . Interact. Learn. Environ. 25 , 704–716. 10.1080/10494820.2016.1172243 [ CrossRef ] [ Google Scholar ]
• Partnership for 21st Century Skills (2008). P21 Framework Definitions Document . Available online at: http://www.21stcenturyskills.org (accessed February 28, 2022).
• Persson V., Nouri J. (2018). A systematic review of second language learning with mobile technologies . Int. J. Emerg. Technol. Learn. 13 , 188–210. 10.3991/ijet.v13i02.8094 [ CrossRef ] [ Google Scholar ]
• * . Sevilla-Pavón A., Nicolaou A. (2017). Online intercultural exchanges through digital storytelling . Int. J. Comput. Assisted Lang. Learn. Teach. 7 , 44–58. 10.4018/IJCALLT.2017100104 [ CrossRef ] [ Google Scholar ]
• * . Sevy-Biloon J., Chroman T. (2019). Authentic use of technology to improve EFL communication and motivation through international language exchange video chat . Teach. English Technol. 19 , 44–58. [ Google Scholar ]
• Shadiev R., Hwang W.-Y., Ghinea G. (2022a). Guest editorial: Creative learning in authentic contexts with advanced educational technologies . Educ. Technol. Soc. 25 , 76–79. Available online at: https://www.jstor.org/stable/48660125 [ Google Scholar ]
• Shadiev R., Hwang W. Y., Huang Y. M. (2017). Review of research on mobile language learning in authentic environments . Comput. Assist. Lang. Learn. 30 , 284–303. 10.1080/09588221.2017.1308383 [ CrossRef ] [ Google Scholar ]
• Shadiev R., Wang X., Liu T.Y., Yang M. (In Press). Improving students' creativity in familiar versus unfamiliar mobile-assisted language learning environments. Interact. Learn. Environ. 10.1080/10494820.2021.2023891 [ CrossRef ] [ Google Scholar ]
• Shadiev R., Yang M. (2020). Review of studies on technology-enhanced language learning and teaching . Sustainability. 12 , 524. 10.3390/su12020524 [ CrossRef ] [ Google Scholar ]
• Shadiev R., Yi S., Dang C. Sintawati W. (2022b). Facilitating students' creativity, innovation and entrepreneurship in a telecollaborative project . Front. Psychol. 13 , 887620. 10.3389/fpsyg.2022.887620 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Shadiev R., Yu J. T. (In Press). Review of research on computer-assisted language learning with a focus on intercultural education. Comput. Assist. Lang. Learn. 10.1080/09588221.2022.2056616 [ CrossRef ] [ Google Scholar ]
• * . Srebnaja J., Stavicka A. (2018). Web-based projects to develop transversal skills in secondary school . Hum. Technol. Qual. Educ. 2018 , 25–34. 10.22364/htqe.2018.03 [ CrossRef ] [ Google Scholar ]
• Suzanne N. (2014). “Critical and effective reading to build the characters as active readers,” in International Conference on Languages and Arts (Padang: ), 47–352. [ Google Scholar ]
• * . Thang S. M., Sim L. Y., Mahmud N., Lin L. K., Zabidi N. A., Ismail K. (2014). Enhancing 21st century learning skills via digital storytelling: voices of Malaysian teachers and undergraduates . Procedia Soc. Behav. Sci. 118 , 489–494. 10.1016/j.sbspro.2014.02.067 [ CrossRef ] [ Google Scholar ]
• * . Tseng C. T. H. (2017). Teaching “Cross-Cultural Communication” through content based instruction: curriculum design and learning outcome from EFL learners' perspectives . English Lang. Teach. 10 , 22–34. 10.5539/elt.v10n4p22 [ CrossRef ] [ Google Scholar ]
• * . Valdebenito M., Chen Y. (2019). Technology as enabler of learner autonomy and authentic learning in chinese language acquisition: a case study in higher education . J. Technol. Chin. Lang. Teach. 10, 61. [ Google Scholar ]
• Vygotsky L. S. (1978). Mind in Society: The Development of Higher Psychological Processes . Cambridge, MA: Harvard University Press. [ Google Scholar ]
• * . Yalçin O. B., Öztürk E. (2019). “The effects of digital storytelling on the creative writing skills of literature students based on their gender,” in ICGR 2019 2nd International Conference on Gender Research . (Rome: Academic Conferences and Publishing Limited; ), 59. [ Google Scholar ]
• * . Yang Y. T. C., Chen Y. C., Hung H. T. (2022). Digital storytelling as an interdisciplinary project to improve students' English speaking and creative thinking . Comput. Assist. Lang. Learn. 35 , 840–862. 10.1080/09588221.2020.1750431 [ CrossRef ] [ Google Scholar ]
• * . Yang Y. T. C., Chuang Y. C., Li L. Y., Tseng S. S. (2013). A blended learning environment for individualized English listening and speaking integrating critical thinking . Comput. Educ. 63 , 285–305. 10.1016/j.compedu.2012.12.012 [ CrossRef ] [ Google Scholar ]
• * . Yang Y. T. C., Gamble J. H., Hung Y. W., Lin T. Y. (2014). An online adaptive learning environment for critical-thinking-infused English literacy instruction . Br. J. Educ. Technol. 45 , 723–747. 10.1111/bjet.12080 [ CrossRef ] [ Google Scholar ]
• Zhang R., Zou D. (2020). Types, purposes, and effectiveness of state-of-the-art technologies for second and foreign language learning . Comput. Assist. Lang. Learn. 35 , 696–742. 10.1080/09588221.2020.1744666 [ CrossRef ] [ Google Scholar ]
• * . Zou D., Xie H. (2019). Flipping an English writing class with technology-enhanced just-in-time teaching and peer instruction . Interact. Learn. Environ. 27 , 1127–1142. 10.1080/10494820.2018.1495654 [ CrossRef ] [ Google Scholar ]

129 List Of Research Topics In English Language Teaching [updated]

English Language Teaching (ELT) is a field dedicated to teaching English to non-native speakers. It’s important because English is a global language used for communication, business, and education worldwide. Research in ELT helps improve teaching methods, making it easier for students to learn English effectively. This blog will explore a list of research topics in English language teaching.

What Are The Areas Of Research In English Language Teaching?

Table of Contents

Research in English Language Teaching (ELT) encompasses a wide range of areas, including:

• Language Learning: Understanding how people learn English well, like when they learn a new language and if there’s a best time to do it.
• Teaching Ways: Looking into different ways teachers teach, like using conversations, tasks, or mixing language with other subjects.
• Curriculum Design and Syllabus Development: Designing and evaluating language curricula and syllabi to meet the needs of diverse learners and contexts.
• Assessment and Evaluation: Developing and validating assessment tools, exploring alternative assessment methods, and investigating the effectiveness of feedback and error correction strategies.
• Technology in ELT: Exploring the integration of technology in language teaching and learning, including computer-assisted language learning (CALL), mobile-assisted language learning (MALL), and online learning platforms.
• Teacher Education and Professional Development: Investigating pre-service and in-service teacher education programs, reflective practices, and challenges in teacher training.
• Cultural and Sociolinguistic Aspects: Examining the role of culture in language teaching and learning, sociolinguistic competence, and addressing cultural diversity in the classroom.
• Learner Diversity and Inclusive Practices: Researching teaching strategies for diverse learners, including young learners, learners with learning disabilities, and learners from diverse linguistic and cultural backgrounds.
• Policy and Planning in ELT: Analyzing language policies at national and international levels, exploring the implementation of ELT programs, and examining the role of ELT in national development.
• Research Methodologies in ELT: Investigating qualitative, quantitative, and mixed-methods research approaches in ELT research, including action research conducted by teachers in their own classrooms.
• Future Trends and Innovations: Exploring emerging trends and innovations in ELT, such as the impact of globalization, the use of artificial intelligence (AI) in language learning, and innovative teaching strategies.

129 List Of Research Topics In English Language Teaching: Category Wise

Language acquisition and development.

• Second Language Acquisition Theories: Explore different theories explaining how learners acquire a second language.
• Critical Period Hypothesis: Investigate the idea of an optimal age range for language acquisition.
• Multilingualism and Language Development: Study how knowing multiple languages affects language development.
• Cognitive and Affective Factors in Language Learning: Examine the role of cognitive abilities and emotions in language learning.
• Language Learning Strategies: Investigate the strategies learners use to acquire and develop language skills.
• Input Hypothesis: Explore the role of comprehensible input in language acquisition.
• Interaction Hypothesis: Examine the importance of interaction in language learning.
• Fossilization in Second Language Learning: Study why some learners reach a plateau in their language development.

Teaching Methodologies and Approaches

• Communicative Language Teaching (CLT): Analyze the effectiveness of CLT in promoting communication skills.
• Task-Based Language Teaching (TBLT): Explore the use of real-world tasks to teach language.
• Content and Language Integrated Learning (CLIL): Investigate teaching subject content through English.
• Blended Learning in ELT: Study the integration of traditional and online teaching methods.
• Audio-Lingual Method: Assess the effectiveness of drills and repetition in language teaching.
• Grammar-Translation Method: Compare traditional grammar-focused methods with communicative approaches.
• Lexical Approach: Explore teaching vocabulary as a key component of language proficiency.
• Suggestopedia: Investigate the use of relaxation techniques to enhance language learning.

Curriculum Design and Syllabus Development

• Needs Analysis in ELT: Identify the language needs of learners and design appropriate curricula.
• Integrating Language Skills in Curriculum: Examine strategies for integrating reading, writing, listening, and speaking skills.
• Syllabus Types: Compare different types of syllabi, such as structural and task-based.
• Task-Based Syllabus Design: Design syllabi based on real-world tasks to promote language acquisition.
• Content-Based Instruction (CBI): Integrate language learning with academic content in syllabus design.
• Needs Analysis in Specific Contexts: Conduct needs analyses for learners in specific professional or academic contexts.
• Cross-Cultural Communication in Curriculum Design: Incorporate intercultural communication skills into language curricula.

Assessment and Evaluation

• Standardized Testing in ELT: Evaluate the reliability and validity of standardized English language tests.
• Alternative Assessment Approaches: Explore non-traditional assessment methods like portfolios and self-assessment.
• Feedback Strategies in Language Learning: Investigate effective feedback techniques for improving language proficiency.
• Washback Effect of Testing: Study how assessment practices influence teaching and learning.
• Authentic Assessment in ELT: Develop assessment tasks that mirror real-life language use situations.
• Portfolio Assessment: Investigate the use of portfolios to track language learning progress over time.
• Computer Adaptive Testing (CAT): Evaluate the feasibility and effectiveness of adaptive testing methods in ELT.

Technology in ELT

• Computer-Assisted Language Learning (CALL): Assess the impact of computer-based language learning programs.
• Mobile-Assisted Language Learning (MALL): Study the effectiveness of mobile devices in language learning.
• Online Learning Platforms for ELT: Analyze the features and usability of online platforms for language education.
• Virtual Reality (VR) in Language Learning: Explore immersive VR environments for language practice and instruction.
• Artificial Intelligence (AI) Tutoring Systems: Assess the effectiveness of AI-based tutors in providing personalized language instruction.
• Social Media in Language Learning: Study the role of social media platforms in informal language learning contexts.
• Gamification in ELT: Investigate the use of game elements to enhance engagement and motivation in language learning.

Teacher Education and Professional Development

• Pre-service Teacher Education Programs: Evaluate the effectiveness of teacher training programs.
• Reflective Practice in Teaching: Investigate how teachers reflect on their practice to improve teaching.
• Challenges in Teacher Education: Identify challenges faced by educators in training and development.
• Teacher Beliefs and Practices: Examine how teachers’ beliefs about language learning influence their instructional practices.
• Peer Observation in Teacher Development: Explore the benefits of peer observation and feedback for teacher professional growth.
• Mentoring Programs for New Teachers: Evaluate the effectiveness of mentoring programs in supporting novice teachers.
• Continuing Professional Development (CPD) Models: Compare different models of CPD for language teachers and their impact on teaching quality.

Cultural and Sociolinguistic Aspects

• Language and Culture Interrelationship: Explore the relationship between language and culture in ELT.
• Sociolinguistic Competence and Pragmatics: Study how social context influences language use and understanding.
• Gender and Identity in Language Learning: Investigate how gender identity affects language learning experiences.
• Intercultural Competence in Language Teaching: Develop strategies for promoting intercultural communicative competence in language learners.
• Language Policy and Minority Language Education: Analyze the impact of language policies on the education of minority language speakers.
• Gender and Language Learning Strategies: Investigate gender differences in language learning strategies and their implications for instruction.
• Code-Switching in Multilingual Classrooms: Study the role of code-switching in language learning and classroom interaction.

Learner Diversity and Inclusive Practices

• Teaching English to Young Learners (TEYL): Examine effective teaching strategies for children learning English.
• Addressing Learning Disabilities in ELT: Investigate methods for supporting learners with disabilities in language learning.
• ELT for Specific Purposes (ESP): Explore specialized English language instruction for specific fields.
• Differentiated Instruction in Language Teaching: Develop strategies for addressing diverse learner needs in the language classroom.
• Inclusive Pedagogies for Learners with Special Educational Needs: Design instructional approaches that accommodate learners with disabilities in language learning.
• Language Learning Strategies of Autistic Learners: Investigate effective language learning strategies for individuals on the autism spectrum.
• Language Identity and Learner Motivation: Explore the relationship between language identity and motivation in language learning.

Policy and Planning in ELT

• National and International Language Policies: Analyze policies governing English language education at different levels.
• ELT Program Implementation Challenges: Identify challenges in implementing ELT programs in diverse contexts.
• Role of ELT in National Development: Examine the contribution of English language education to national development goals.
• English as a Medium of Instruction (EMI) Policies: Analyze the impact of EMI policies on educational equity and access.
• Language Teacher Recruitment and Deployment Policies: Evaluate policies related to the recruitment and deployment of language teachers in diverse contexts.
• Language Assessment Policy Reform: Propose reforms to language assessment policies to promote fairness and validity.
• Biliteracy Development Policies: Study policies aimed at promoting biliteracy development among bilingual learners.

Research Methodologies in ELT

• Qualitative Research Methods in ELT: Explore qualitative approaches like interviews and case studies in ELT research.
• Quantitative Research Methods in ELT: Investigate quantitative methods such as surveys and experiments in language education research.
• Mixed-Methods Approaches in ELT Research: Combine qualitative and quantitative methods to gain a comprehensive understanding of research questions.
• Ethnographic Approaches to ELT Research: Conduct ethnographic studies to explore language learning and teaching in naturalistic settings.
• Case Study Research in Language Education: Investigate specific language learning contexts or programs through in-depth case studies.
• Corpus Linguistics in ELT Research: Analyze language use patterns and learner language production using corpus linguistic methods.
• Longitudinal Studies of Language Learning: Follow language learners over an extended period to examine developmental trajectories and factors influencing language acquisition.

Future Trends and Innovations

• Emerging Technologies in ELT: Study the integration of technologies like AI and VR in language teaching.
• Innovations in Teaching Strategies: Explore new approaches to teaching language, such as flipped classrooms and gamification.
• Future Directions in ELT Research: Investigate potential areas for future research in English language teaching.
• Wearable Technology in Language Learning: Explore the potential of wearable devices for delivering personalized language instruction.
• Data Analytics for Adaptive Learning: Develop data-driven approaches to adaptive learning in language education.
• Augmented Reality (AR) Applications in ELT: Design AR-enhanced language learning experiences for immersive language practice.
• Global Citizenship Education and Language Learning: Investigate the role of language education in fostering global citizenship skills.
• Eco-Linguistics and Language Education: Explore the intersection of language education and environmental sustainability.
• Metacognition and Language Learning: Explore how learners’ awareness of their own learning processes affects language acquisition.
• Peer Interaction in Language Learning: Investigate the role of peer collaboration and discussion in promoting language development.
• Heritage Language Education: Study strategies for maintaining and revitalizing heritage languages among immigrant and minority communities.
• Language Learning Motivation in Adolescents: Examine factors influencing motivation and engagement in adolescent language learners.
• Phonological Awareness in Language Learning: Investigate the role of phonological awareness in literacy development for language learners.
• Pragmatic Development in Language Learners: Explore how learners acquire pragmatic competence and understanding of language use in context.
• Digital Literacies and Language Learning: Examine how digital literacy skills contribute to language proficiency and communication in the digital age.
• Critical Language Awareness: Investigate approaches to developing learners’ critical awareness of language use and power dynamics.
• Language Teacher Identity: Study how language teachers’ identities shape their beliefs, practices, and interactions in the classroom.
• Collaborative Learning in Language Education: Explore the benefits and challenges of collaborative learning environments for language learners.
• Motivational Strategies in Language Teaching: Develop and evaluate motivational techniques to enhance student engagement and persistence in language learning.
• Heritage Language Maintenance: Investigate factors influencing the maintenance and transmission of heritage languages across generations.
• Phonics Instruction in Language Learning: Examine the effectiveness of phonics-based approaches for teaching reading and pronunciation.
• Language Policy Implementation: Analyze the challenges and successes of implementing language policies at the institutional, regional, and national levels.
• Language Teacher Cognition: Explore language teachers’ beliefs, knowledge, and decision-making processes in the classroom.
• Intercultural Communicative Competence: Develop strategies for fostering learners’ ability to communicate effectively across cultures.
• Critical Pedagogy in Language Education: Explore approaches to teaching language that promote critical thinking, social justice, and equity.
• Language Learning Strategies for Autodidacts: Investigate effective self-directed learning strategies for language learners outside formal educational settings.
• Content and Language Integrated Learning (CLIL) in Higher Education: Examine the implementation and outcomes of CLIL programs in tertiary education.
• Sociocultural Theory and Language Learning: Explore how social and cultural factors influence language acquisition and development.
• Language Socialization: Investigate how individuals learn language within social and cultural contexts, including family, peer groups, and communities.
• Speech Perception and Language Learning: Examine the relationship between speech perception abilities and language proficiency in second language learners.
• Genre-Based Approaches to Language Teaching: Explore the use of genre analysis and genre-based pedagogy to teach language skills in context.
• Learner Autonomy in Language Learning: Investigate strategies for promoting learner autonomy and independence in language education.
• Multimodal Literacy in Language Learning: Examine the integration of multiple modes of communication, such as text, image, and sound, in language instruction.
• Community-Based Language Learning: Study language learning initiatives that engage learners with their local communities and resources.
• English as a Lingua Franca (ELF) Communication: Explore the use of English as a global means of communication among speakers from diverse linguistic backgrounds.

Research in English Language Teaching covers a wide range of topics, from language acquisition theories to the impact of technology on learning. By exploring these topics (from a list of research topics in english language teaching), we can improve how English is taught and learned, making it more effective and accessible for everyone.

Continuous research and collaboration among educators, researchers, and policymakers are essential for the ongoing development of ELT.

Related Posts

Step by Step Guide on The Best Way to Finance Car

The Best Way on How to Get Fund For Business to Grow it Efficiently

Leave a comment cancel reply.

Your email address will not be published. Required fields are marked *

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• My Account Login
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 24 May 2024

Detecting contract cheating through linguistic fingerprint

• Mohammed Kutbi   ORCID: orcid.org/0000-0002-3815-8028 1 ,
• Ali H. Al-Hoorie   ORCID: orcid.org/0000-0003-3810-5978 2 &
• Abbas H. Al-Shammari 3  

Humanities and Social Sciences Communications volume  11 , Article number:  664 ( 2024 ) Cite this article

168 Accesses

Metrics details

• Language and linguistics
• Science, technology and society

Contract cheating, the act of students enlisting others to complete academic assignments on their behalf, poses a significant challenge in academic settings, undermining the integrity of education and assessment. It involves submitting work that is falsely represented as the student’s own, thus violating academic standards and ethics. The advent of artificial intelligence-based language models, such as ChatGPT, has raised concerns about the potential impact of contract cheating. As these language models can generate human-like text with ease, there are concerns about their role in facilitating and increasing contract cheating incidents. Innovative approaches are thus needed to detect contract cheating and address its implications for academic integrity. This study introduces a machine learning (ML) model focused on identifying deviations from a learner’s unique writing style (or their linguistic fingerprint) to detect contract cheating, complementing traditional plagiarism detection methods. The study involved 150 learners majoring in engineering and business who were studying English as a foreign language at a college in Saudi Arabia. The participants were asked to produce descriptive essays in English within a consistent genre over one semester. The proposed approach involved data preprocessing, followed by transformation using Term Frequency-Inverse Document Frequency (TF-IDF). To address data imbalance, random oversampling was applied, and logistic regression (LR) was trained with optimal hyperparameters obtained through grid search. Performance evaluation was conducted using various metrics. The results showed that the ML model was effective in identifying non-consistent essays with improved accuracy after implementing random oversampling. The LR model achieved an accuracy of 98.03%, precision of 98.52%, recall of 98.03%, and F1-score of 98.24%. The proposed ML model shows promise as an indicator of contract cheating incidents, providing an additional tool for educators and institutions to uphold academic integrity. However, it is essential to interpret the model results cautiously, as they do not constitute unequivocal evidence of cheating but rather serve as grounds for further investigation. We also emphasize the ethical implications of such approaches and suggest avenues for future research to explore the model’s applicability among first-language writers and to conduct longitudinal studies on second-language learners’ language development over longer periods.

Similar content being viewed by others

Perception, performance, and detectability of conversational artificial intelligence across 32 university courses

The model student: GPT-4 performance on graduate biomedical science exams

ChatGPT-3.5 as writing assistance in students’ essays

Introduction.

Technology has become an integral part of modern life and has been extensively utilized in educational settings. Technology has greatly enhanced the convenience of accessing and transmitting information in educational settings. With the onset of COVID-19, technology proved indispensable for institutions worldwide as they switched to emergency remote teaching (Hodges et al. 2020 ). However, one major impediment to integrating technology with education is assessment (Al Shlowiy et al. 2021 ). When educational institutions turned to emergency remote learning, the scale of this problem became apparent. For example, reported cheating incidents at the University of Waterloo increased by 146%, jumped by 269% at the University of Calgary, doubled at the University of Houston, and quadrupled at Queensland University of Technology (Basken, 2020 ). Lancaster and Cotarlan ( 2021 ) analyzed student requests on one file-sharing website, Chegg, and found that exam-style requests increased by almost 200% during the pandemic. With the pandemic over, institutions became eager to go back to traditional face-to-face teaching and assessment in order to address the sudden grade inflation occurring during that period, thus reducing the potential role of technology in education.

A major part of the phenomenon described above is known as contract cheating (Lancaster and Clarke, 2016 ). Contract cheating occurs when “a student is requesting an original bespoke piece of work to be created for them” (Lancaster and Clarke, 2016 , p. 639). Traditional plagiarism software is unable to detect this type of academic integrity breach because the text in question is original and not copied from elsewhere. Students engaged in contract cheating may obtain assistance from commercial essay mills or private tutors for a fee. Other students may obtain such assistance for free from family, friends, and other students. Obtaining work from somebody else and submitting it as one’s own for course credit is a serious academic offense, whether payment was involved or not, as grades would no longer be a true reflection of the student’s ability.

In this study, we attempted to develop a machine-learning model that can detect the linguistic fingerprint of the foreign language learner. We analyzed essays written by students over a semester and trained the model to detect whether an essay was written by the same student (i.e., consistent) or by a different student (non-consistent). Thus, our primary point of reference in developing this model was not detecting the similarity of text submitted to texts written by others but detecting deviation from texts written by the same student. Foreign language learners tend to make different language mistakes consistently as they go through a series of developmental stages (Mitchell et al. 2019 ), and therefore, it may be possible to recognize their linguistic fingerprints (i.e., distinct style of writing).

Contract cheating

In a study by Bretag et al. ( 2019 ), the researchers identified three main factors associated with contract cheating. The first factor was that students were dissatisfied with the learning and teaching environment. Students are usually under tremendous pressure to obtain high GPAs in order to be able to compete in the job market after graduation. This grade-focused educational environment may tempt some students to find shortcuts to achieve higher GPAs. The second factor was the availability of opportunities for cheating. Indeed, a quick Google search shows thousands of commercial services promising assistance in term essays, grant applications, conference presentations, and journal manuscripts (Bretag, 2019 ). Students are bombarded with advertisements, both online and offline, about discreet coursework assistance. This is a largely unregulated market, and many countries do not have clear and effective laws addressing this problem. Finally, students who speak languages other than English (LOTEs; Dörnyei and Al-Hoorie, 2017 ) may feel disadvantaged, potentially making contract cheating an enticing option to improve their grades.

The prevalence of such dishonest practices constitutes a serious risk to the credibility of academic institutions. Grades and degrees students receive from their institutions will no longer be credible, and consequently education loses its value. The larger society also suffers from this situation when the proportion of the workforce (e.g., doctors, engineers) whose qualifications do not reflect their actual skill increases. When these students join academia, they might also be tempted to resort to dishonesty in a publish-or-perish environment (Bretag, 2019 ). Research suggests that engagement in academic dishonesty is related to involvement in dishonest behavior in other life contexts (Guerrero-Dib et al. 2020 ) and to corruption at the country level (Orosz et al. 2018 ).

Existing strategies to counteract academic dishonesty generally depend on technology. One strategy requires students to turn on their cameras during the exam to ensure that the student does not receive assistance. Another strategy is to adopt a specially designed browser (e.g., LockDown Browser) that limits access to unwanted websites and applications. These strategies are not too challenging to trick, considering the availability of smartphones. Besides, not all assessments require students to complete tasks within a specific session under invigilation. Students are frequently asked to write reports and extended essays and submit them at a later date during the semester. In these cases, institutions usually resort to plagiarism detection software (e.g., Turnitin, PlagScan, AntiPlag). This software compares the submitted text with text available online and within its database and produces what is called an “originality score” or a “PlagLevel.” A clear problem with this approach is that the software will miss plagiarism cases when it cannot reach the original text (e.g., behind a paywall or still not digitalized) if it is in a different language, or even when there are typos in either text (Weber-Wulff, 2019 ).

More importantly, for the present purposes, the traditional anti-plagiarism approach is not designed to detect contract cheating. Contract cheating, by definition, involves an original text that the student does not produce themselves but submit as their own. This could be through a professional agency doing it for a fee, a family member doing it as a favor, or a free online service that paraphrases the text until the plagiarism score is low enough. Even using one of the rapidly increasing numbers of free paraphrasing services is problematic, as there is no guarantee that the student has actually learned the material. The problem becomes more acute when the student is learning a foreign language. Foreign language learners are expected to demonstrate improvement in their language proficiency, and therefore, obtaining assistance from such automated programs defeats the purpose. In short, “the illegal services are developing at a faster pace than the systems required to curb them” (Hill et al. 2021 , p. 15).

Academic integrity in the era of ChatGPT

The emergence of advanced artificial intelligence (AI) language models, such as ChatGPT, has brought forth new challenges in maintaining academic integrity. One critical issue is that ChatGPT has the capability of producing novel texts that can evade traditional plagiarism detection software, making it increasingly difficult to identify instances of contract cheating. Existing approaches to detect ChatGPT-generated texts often rely on keyword-based matching and syntax analysis, but these methods are proving to be ineffective due to the model’s sophisticated language generation abilities. Moreover, ChatGPT can generate highly original text, complicating teachers’ ability to discern genuine student work from content generated by these models. Available tools designed to detect AI texts have shown poor accuracy and reliability as well as a bias toward classifying texts as human-written (Weber-Wulff et al. 2023 ).

In certain contexts, using ChatGPT and other AI tools may be acceptable, and even beneficial. Academics, for example, may leverage these tools to enhance their writing, improve productivity, and explore novel ideas (Kim, 2023 ). Guidelines for the responsible use of such tools have been developed within academic communities to ensure transparency and acknowledge the involvement of AI tools in content creation (Flanagin et al. 2023 ). For academic professionals, the goal is not to deceive but to augment the clarity and value of their intellectual contributions.

However, the situation differs significantly for language learners. The primary objective of language assessment is to evaluate a student’s language proficiency and linguistic development. In this context, using AI tools to produce responses for language tests defeats the purpose of accurately assessing learners’ abilities. Moreover, the impact of ChatGPT extends beyond direct student usage. Even if students do not utilize ChatGPT themselves, the existence of such tools empowers individuals who sell essays to students. Unscrupulous essay-writing services can exploit ChatGPT to mass-produce essays and sell them cheaply, posing a considerable threat to academic integrity and rendering traditional plagiarism detection measures less effective against this new form of contract cheating.

Analytical approaches to detect contract cheating

As contract cheating is a pressing concern for academic institutions, the development of effective analytical approaches to detect such misconduct has become vital. In this section, we review ML algorithms that might be useful in identifying contract cheating instances based on the linguistic patterns and features exhibited in student essays.

ML algorithms

Logistic regression (LR): A LR is a supervised learning algorithm that predicts the probability of class membership based on relationships with predictor variables. It utilizes statistical analysis to determine binary outcomes and is valued for its ability to handle varied data sources with minimal complexity. However, LR is sensitive to minor changes in input values, which can significantly bias probability predictions, as noted by Dreiseitl and Ohno-Machado ( 2002 ). Additionally, the model’s effectiveness is influenced by the dimensionality of the input vector; a higher number of predictors can increase the cost of training and risk overfitting, thereby reducing the model’s generalization capabilities across different datasets.

Light Gradient-Boosting Machine (LightGBM): LightGBM is a widely used gradient-boosting algorithm based on a decision tree (Friedman, 2001 ). It is often used in various tasks such as classification, sorting, and regression and supports efficient parallel training. LightGBM uses an approximate loss function with a piecewise constant tree and quadratic Taylor approximation at each step. Then, it trains the decision tree to reduce this quadratic approximation. LightGBM has proved to be an efficient algorithm and exhibited higher classification results with distributed systems.

Oversampling techniques

Random Oversampling: Random oversampling is a technique for handling imbalanced data. It generates new samples by randomly duplicating them from the minority class (Mohammed et al. 2020 ). Consequently, the minority class will be adjusted as the majority class and thereby have the same sample distribution. This technique is deemed to be the simplest and easiest way to cope with class imbalance issues, showing robust performance. The main benefit of this straightforward technique is that there is no information loss. However, it often increases the likelihood of overfitting since it copies the minority class instances.

Synthetic Minority Oversampling (SMOTE): SMOTE is a sophisticated data balance technique (Chawla et al. 2002 ) used to overcome the imbalance problem. SMOTE algorithms aim to generate a balanced class distribution by generating synthesized samples of the minority class via interpolation. SMOTE randomly selects a member of the minority class and then uses k -nearest neighbors (KNN) to synthesize new samples according to the line segment of the minority class member and its neighbor. This process is repeated many times until the minority class has the same sample distribution as the majority class does. SMOTE can contribute to alleviating the overfitting problem caused by random oversampling since this is a synthetic sample rather than a replication.

The present study

Based on the above, there is a need to expand the available tools to detect a wider range of academic dishonesty, including contract cheating. As there are a number of statistical techniques available, the aim of the present study was, therefore to test the efficacy of these techniques to detect the possibility of contract cheating. To assess the efficiency of the proposed models, we focused on the following research question:

RQ. Can we create an AI-based method that detects inconsistency in writing styles?

To answer our research question, we compared the performance of three ML algorithms and three balancing techniques by using four evaluation metrics (Tharwat, 2021 ). The experimentation phase incorporated two important steps: with and without the application of the balancing technique. This fundamental step can deeply investigate the vital role of balancing in enhancing the model’s performance. These metrics can be derived from the confusion matrix outlined in Table 1 .

Accuracy represents the percentage of correctly predicted consistent essays relative to the whole dataset. It is calculated as in Eq. 1 .

Precision is the exactness representing the number of optimistic class predictions that belong to the positive class. It measures the proportion of students predicted by the classifier to write consistent essays that are actually consistent. It is the ratio of the True Positive (TP) instances to the sum of True Positive (TP) and False Positive (FP) instances (Eq. 2 ).

Recall represents the percentage of consistent essays that have been correctly predicted. It is calculated as in Eq. 3 .

F1-score is a harmonic mean of recall and precision values. It strikes a balance between precision and recall, thereby providing a correct evaluation of the model’s performance in classifying the consistency of essays (Eq. 4 ).

Participants

The participants ( N  = 150) were freshmen taking English language courses as prerequisites for their degree plans at an engineering and business college in Saudi Arabia. Their language proficiency level was B1–B2, as they had to successfully complete a foundation year before starting their majors.

The participants were asked to write descriptive essays about familiar topics, such as describing their campus and what they do on the weekend. The genre was consistent in controlling for genre-related vocabulary and grammatical structures. See Data Analysis for more details.

The participants completed the assignments during class time. One of the researchers or their class teacher was present to answer any questions. The participants wrote one essay every 2 weeks, so that each participant produced a total of seven essays over the semester. The participants were asked not to resort to assistance from dictionaries or their smartphones. They were assured that this task was not a test, that it was voluntary, and that their grades would not be affected. Institutional ethical approval was obtained before the study commenced.

Data analysis

Proposed approach.

This section introduces our proposed approach to classifying students’ essays into consistent and non-consistent. Mainly, the approach was based on five steps, as depicted in Fig. 1 . The first step was data preprocessing, which involved cleaning the data by removing punctuation marks, numbers, special characters, extra spaces, empty lines, and stop words. The data transformation step was conducted to prepare the data to be fed into the classifier. Next, an oversampling step was undertaken to increase the number of samples in the minor class to address the unbalancing of samples in each class of our dataset. As a final step of this approach, we trained an ML model with its optimal settings, which were acquired empirically using the grid search method. We assessed the performance according to specific evaluation indicators discussed in detail below.

Approach overview.

Data preprocessing

Data preprocessing is considered a vital step when dealing with classification tasks; its role mainly involves making the raw data fitter to be fed into ML and deep learning algorithms. Python offers numerous libraries ideal for handling these tasks. Amongst the finest ones is the NLTK library (Hardeniya et al. 2016 ). First, the samples of students’ writings were cleaned by removing special characters, white spaces, and punctuation marks, followed by a simple and effective lowercase process to convert each word to its equivalent in lowercase. Subsequently, a stop-word removal process was conducted to omit all the English stop words according to a predefined list provided by the NLTK library. The preprocessing tasks are straightforward; however, their impact on the model’s performance is quite remarkable.

Data transformation

In this significant step, we used one of the most prevalent techniques in data transformation. The Term Frequency-Inverse Document Frequency (TF-IDF) (Qaiser and Ali, 2018 ) is used essentially in information retrieval and text mining. Indeed, this technique balances the samples because it studies how each sample is a weighted text sequence. The TF-IDF technique is divided into two sub-steps. Initially, the term frequency is counted by calculating the number of times a word appears in a document divided by the total number of words, as illustrated in Eq. 5 , followed by inverse document frequency, which aims to scale with the number of essays.

where W denotes the weight value for term t and document d . N represents the total number of documents in the corpus. tf and df define the term frequency; more specifically, it indicates the number of times and the number of document in a particular term.

Oversampling phase

The data imbalance issue is broadly dominant in the AI field due to a shortage of many data types. Thus, it can adversely affect the model’s performance because it lacks training in particular patterns, leading to poor model generalizability. To this end, miscellaneous oversampling techniques are reported in the literature, proving their mettle against this challenge. We used one of the popular oversampling techniques for our imbalance data, called random oversampling. Before delving into details, we should be clear that this technique is applied only to the training data. Therefore, we divided the whole dataset into two subsets: training data which represents 70% of data according to the hold-out methods, and the remaining data (30%) is devoted to testing the decisive model. The used method is considered the most straightforward oversampling technique to balance the dataset. It balances the data by simply duplicating the samples of the minority class. This process does not cause any harm to the dataset, such as noise. Nevertheless, the model is likely to overfit while training because it feeds on the copied information. Additionally, random oversampling can significantly enhance the efficiency per class and the model’s overall performance. As noticed, the number of samples in class non-consistent outnumbers the instances of class consistency. Table 2 details how the number of samples increased after random oversampling.

Model training

In this step, we trained the model using the new training data obtained after the application of oversampling using ML algorithms. The main aim of this model was to differentiate between students’ essays, whether consistent or non-consistent, based on ML algorithms. Recently, ML, a subfield of AI, has established its ability to solve many tasks related to text mining, computer vision, and pattern classification. ML algorithms focus on building an intelligent model based on the data they learn and process. We used one of the highly used ML classifiers called LR in addition to fine-tuning hyperparameters to ensure the maximization of results and usage of the optimal set of parameters. LR model is a supervised learning algorithm that is virtually used to forecast the class of a given variable’s probability based on its relationship with the main class. Any slight alteration in the input data can excessively impact the prediction probability. Furthermore, the input vector’s dimension should be low enough in order not to affect the cost of the training model, leading to overfitting of the model and poor generalization. Nonetheless, LR is considered one of the prominent ML algorithms in classification tasks with low complexity. This training step was implemented using Python programming language and conducted via Google Collab.

Basically, some components must be considered to develop an effective classification model, such as the hyperparameter configuration, which is fundamental to building an accurate and optimal model (Elshawi et al. 2019 ). Nevertheless, the search space of parameter value combinations is likely to be countless, and thus, tuning manually becomes impractical, ineffective, time-consuming, and often needs deep knowledge of models. To this end, automatic hyperparameter optimization is of critical importance. Several techniques exist in this context, with each method’s strengths and drawbacks (Yang and Shami, 2020 ). We adopted the grid search function to find the optimal parameters belonging to a particular ML algorithm, which is the most straightforward hyperparameter method. Indeed, it generates a Cartesian product of all possible combinations of hyperparameters. This technique trains LR with all hybrids developed. It usually needs to be accompanied by a performance metric that is often the accuracy metric—which is the same as our case calculated using the “cross-validation” technique on the training set. In fact, this validation guarantees that the model is exposed to many samples of data. Grid search uses space of parameter values, calculates the score of each building model, and then selects the optimal model that provides the best results. Finally, the grid search algorithm outputs the best settings that give the uppermost results, which are used later in the actual model. To achieve the highest results with LR, we defined a search space of possible values of parameters, as illustrated in Table 3 .

After training the model with these values, the grid search algorithm outputs the following combination as the optimal one (max_iter = 200, penalty = ‘l1’, solver = ‘liblinear’).

Model evaluation

This step tackles the model’s performance using the testing data (30% of the whole dataset), which encompasses 8491 samples, including 167 from the consistent class and 8324 from the non-consistent category. Finally, the overall performance analysis for the studied classifiers was evaluated based on the metrics described below.

Experimental setup

Due to a lack of data in specific fields, we strived to build our dataset based on students’ essays for English courses taught at a college in Saudi Arabia. The reports were written manually by students, and then we transferred them into an electronic version for processing. Dealing with unlabeled data is a challenging issue primarily because it often affects the model’s efficiency in case of wrongly labeling the data. In fact, the information used was unlabeled, which incorporated 1050 essays for 150 students. In this case, we managed to label the dataset according to the following. Firstly, each essay by Student X was paired with another essay by Student Y, so that this pairing belonged to the non-consistent class label. Through this pairing, we guaranteed the non-consistency of different students’ essays. Moreover, each essay by Student X was split into two subparagraphs and maintained as two essays, and we labeled it as a consistent class. The reason behind this split is to ensure that the two essays belong to the same student in case the assignment submitted by the same person is obtained from elsewhere. Accordingly, the dataset contains 28302 essays in total, 27710 of which belong to the non-consistent class, and the remaining 592 are consistent samples.

As shown in Fig. 2 , the dataset is highly unbalanced, highlighting the pressing need for applying data imbalance techniques.

Imbalance between consistent and non-consistent samples.

We implemented three experiments using three ML algorithms: naive Bayes, LR, and LightGBM; and three balancing techniques: ADASYN, SMOTE, and random oversampling. We first conducted the experiments with the whole dataset without the oversampling phase to distinguish its positive impact on the ultimate results.

Experimental results without the oversampling phase

In the present step, we introduced an empirical evaluation of the studied classifiers without the phase of oversampling. Furthermore, we relied primarily on the data preprocessed after the preprocessing stage, and then we performed data transformation using two effective methods: TF-IDF (1-gram) and CountVectorize (bag of words). The results are found in Table 4 .

Generally, the obtained results are fairly similar in terms of performance. Logistic regression combined with a bag of words outperformed the other ML classifiers with an accuracy of 98.06%, a precision value of 99.86%, a recall value of 98.06%, and an F1-score of 98.92%.

As reported in Table 5 , the model poorly predicted the consistent samples while highly accurately predicting the non-consistent samples. This poor prediction unveils the model’s lack of learning from the non-consistent class. Due to the unbalanced data, the overall accuracy calculation is high.

Basically, classification accuracy defines the model’s performance by dividing the number of correct predictions by the total number of predictions. Nevertheless, accuracy would have some issues when the classes are not balanced, as in our case study, prompting low accuracy in one of the deployed classes. The lower the class accuracy is, the higher the increase in the rate of the model failure to correctly predict the samples of that class. Thus, poor class accuracy can become an unreliable model evaluation metric due to unequal distributions of examples in the training set. This disadvantage highlights the pressing need to balance the sample to the effective classification model.

Experimental results with oversampling phase

In this part, we discuss the results we obtained after applying oversampling. Indeed, this step substantially enhanced the results by providing the model with enough information to be trained on. We used random oversampling to raise the number of samples in the non-consistent class. In this way, the model gained a chance to be fed on more data to become well-trained.

To investigate which oversampling sampling technique was suitable for our ultimate model, we studied three oversampling techniques, including SMOTE, random oversampling, and ADASYN. Each oversampling procedure was executed using data transformation and ML models. Table 6 shows that most models generate significant differences compared to results without applying oversampling.

The main objective of applying oversampling was to boost the performance of the non-consistent class. Table 6 shows that random oversampling provides the best results with logistic regression. It yielded a slight difference compared to what was previously performed. Accuracy improved and reached 98.03%, with a precision of 98.52%, a recall of 98.03%, and an F1-score of 98.24%.

Table 7 reports the accuracy results per class for the model that achieved the highest accuracy of 53.88% per the consistent class. For the consistent class, the precision was also enhanced radically to reach 55.62% instead of 3.59%, a recall of 46.11%, and an F1-score of 50%. The significant impact of random oversampling on the model performance was quite noticeable.

To maximize results, we applied one of the automatic hyperparameter techniques to tune the model’s parameters: grid search. Such a step seems very influential when dealing with ML classifiers. As shown in Table 8 , the performance results recorded a slight increase: an accuracy of 54.63%, a precision of 56.37% (compared to the previous accuracy of 55.62%), a recall of 46.11%, and an F1-score of 50.42% instead of 49.12%. This method was beneficial in enhancing the results.

In summary, the logistic regression model combined with random oversampling achieved better results than the other studied models and demonstrated its ability to investigate the consistency of student essays with an overall accuracy of 98.03%.

Contract cheating, wherein students submit work not authored by them, has emerged as a significant challenge in academic institutions worldwide. To combat this misconduct, it is crucial to employ effective and innovative approaches to detect instances of contract cheating and uphold academic integrity. In this study, we developed a ML model that aims to tackle contract cheating by detecting deviations in the writing styles of individual students rather than relying solely on text similarity. In this section, we discuss the implications of our approach and explore the broader context of academic integrity in the era of ChatGPT and other AI tools, the effectiveness of existing approaches to detecting AI texts, and the ethical considerations surrounding their use.

The advent of advanced language models, such as ChatGPT, has transformed the landscape of academic integrity. These AI-powered models are capable of producing novel and coherent texts, rendering them virtually undetectable by traditional plagiarism detection software. As a result, students are increasingly tempted to use such tools to generate essays, reports, and assignments, leading to a rise in contract cheating incidents. The ease of access and the ability to produce custom-written content with minimal effort has made AI tools a potential enabler of academic misconduct. Educational institutions must be proactive in adopting strategies to identify and prevent contract cheating facilitated by AI technologies.

In this study, we proposed an analytical approach based on ML algorithms to detect contract cheating instances. Unlike traditional methods that rely on text similarity comparisons, our model focuses on identifying inconsistencies in a student’s writing style, which can indicate potential contract cheating. By using ML, we aim to learn the distinctive patterns of each student’s writing and establish a profile that allows recognition of sudden deviations from their own norm. Our results demonstrated promising accuracy in detecting discrepancies, offering valuable insights into the potential of data-driven methodologies for contract cheating detection.

However, we acknowledge that analytical approaches alone cannot completely eradicate contract cheating. As academic integrity violations continue to evolve, it is crucial to adopt a multi-pronged approach that combines analytical methods with other preventive measures, such as educational interventions, promoting a culture of academic honesty, and establishing strong policies against contract cheating. While these tools can be useful for academic writing, allowing academics to enhance their research and communication, their application in educational settings is a double-edged sword. Educators may utilize AI tools to improve their writing or brainstorm ideas for lectures and assignments. However, for language learners, the use of AI tools can defeat the very purpose of testing their language proficiency. If students resort to AI-generated content, they do not genuinely showcase their language skills, and educational assessments lose their accuracy and fairness.

Detecting AI-produced texts using traditional plagiarism detection methods is challenging (Weber-Wulff et al. 2023 ). Since the content generated by AI models is not sourced from existing online repositories, these texts remain undetected by similarity-based plagiarism detection systems. Existing approaches, such as rule-based detectors or keyword matching, are inadequate for identifying AI-generated texts, as these methods do not account for the linguistic nuances and coherence achieved by advanced large language models. Traditional plagiarism detection methods may not be adept at identifying AI-generated texts due to the unique features and language patterns characteristic of AI language models. Consequently, academics and educators face an uphill battle in staying ahead of contract cheating attempts utilizing AI technology.

Second language learners undergo a series of developmental stages as they acquire the target language. Different learners tend to make different mistakes consistently, which facilitates the creation of individualized learner profiles. Our model exhibited a high overall level of accuracy in detecting whether a piece of writing was written by the same individual or a different one. For example, second language learners’ reliance on readily available automated paraphrasing services or online translators can detrimentally affect their learning. The text they produce does not reflect their language proficiency, which is an essential evaluation criterion in second language learning courses. The purpose of language proficiency tests is to assess the learner’s ability to comprehend, express, and articulate ideas in the target language. Utilizing AI language models to produce texts can undermine the accuracy of language assessments, as the generated content may not genuinely reflect the student’s linguistic ability.

The accessibility and ease of use of AI language models also empower individuals involved in selling illegal essays to students. AI-generated content can be mass-produced with minimal effort, making it an attractive option for essay mills and unscrupulous individuals seeking to profit from academic misconduct. This exacerbates the problem of contract cheating, as it not only facilitates students’ evading detection but also enables the unethical essay-writing industry to flourish. As educators strive to combat contract cheating, it is essential to address the root causes and disrupt the supply chain of such illegitimate services.

Despite these promising results, we emphasize that the results from this model should not be implemented mechanistically. When the model flags a discrepancy, this can serve as an indicator of a potential cheating incident and mark a particular essay for further investigation, but it does not constitute unequivocal evidence of a cheating incident. When language teachers become familiar with a student’s writing style, they can become suspicious when that student submits work that greatly exceeds his/her current language proficiency. Our model formalizes this intuition and provides a probability figure that calls for further investigation. This is also helpful for teachers responsible for large classes, as they may not be able to become familiar with each student’s individual style. This may also, hopefully, deter potential cheaters from engaging in dishonest endeavors in the first place.

For any proposed approach, it is important to consider its ethical implications. Some approaches developed to enhance the security of online exams have been criticized for violating privacy. Examples include eye movement, biometric data, and keystroke tracking (see Hill et al. 2021 ). There are also concerns about the consequences of security breaches of the stored data. In our case, the ethical issues seem less severe. The model we developed is not very different from existing anti-plagiarism software, which collects texts from the internet and from other users to build its database. Nevertheless, we encourage further examination of the ethical consequences of implementing this approach and other similar ones in educational settings.

While ML algorithms can be powerful tools for preserving academic integrity, their implementation must be done ethically and responsibly. As with any technology that deals with student data and performance evaluation, concerns about privacy and fairness are paramount. Educational institutions and researchers need to ensure that the data used to train these models are collected and used in accordance with established ethical guidelines. Additionally, the interpretability of ML models is essential to build trust and transparency. Understanding how the model arrives at its decisions can help educators and administrators assess its reliability and applicability.

The fight against contract cheating requires a multi-pronged approach that leverages technology, educational interventions, and policy changes. As AI language models continue to evolve, one future research direction is improving detection models through advancing ML algorithms and natural language processing techniques to better identify AI-generated content and develop tailored approaches for detecting contract cheating. Another future direction is conducting longitudinal studies to examine the evolution of language proficiency among learners over time, while considering the influence of AI tools on language development and its ability to anticipate their varying developmental trajectories. Researchers should also collaborate with educational institutions to develop robust policies that address contract cheating and implement appropriate consequences for offenders while simultaneously promoting a supportive learning environment. Finally, future research should examine the possibility of developing a model to detect contract cheating when an individual writes in their native language. Native speakers range from school students to graduate students, and our model may not apply to all these groups.

In this study, we attempted to tackle the pressing issue of contract cheating in academic institutions by exploring the possibility of developing a statistical model to detect contract cheating instances, specifically in the context of student essays. Our approach departed from conventional plagiarism detection methods in favor of identifying deviations from an individual’s unique writing style, rather than relying solely on textual similarities. We acknowledge the growing impact of AI language models like ChatGPT on contract cheating, with their potential to generate undetectable novel texts that elude traditional detection mechanisms. We also acknowledge that it might be possible for AI tools to mimic an individual’s writing style, though this still requires feeding the model with various examples of one’s writing, making the process more burdensome and not merely a few clicks of a button. Similarly, this will make it harder for essay mills to mass-produce essays.

The advent of AI language models has indeed ushered in a new era of academic integrity, prompting educators and institutions to adapt their approaches to safeguard the credibility of education and assessment. The challenge lies in devising effective detection models that can distinguish AI-generated content from genuine human-authored work while also maintaining student privacy and data security. Our findings underscore the need for continuous research and development of ML algorithms that can better identify AI-generated texts and provide nuanced approaches to combatting contract cheating. Moreover, it is imperative to foster a culture of academic honesty through educational interventions, policies, and collaborative efforts that emphasize the ethical use of AI tools and promote responsible academic conduct.

As the academic community confronts the evolving landscape of academic misconduct, a multi-faceted approach is essential. This involves leveraging technological advancements to enhance detection capabilities, encouraging students to embrace the value of authentic learning experiences, and enacting institutional policies that deter and penalize contract cheating. While AI language models hold immense potential to revolutionize education positively, their misuse for the purpose of contract cheating threatens the very foundations of academic integrity. By remaining proactive in our strategies and engaging all stakeholders, we can collectively uphold the principles of academic integrity, preserve the credibility of educational assessments, and foster an environment where learning and knowledge flourish ethically.

Data availability

Data may be accessed at https://osf.io/m3zax/ .

Al Shlowiy A, Al-Hoorie AH, Alharbi M (2021) Discrepancy between language learners and teachers concerns about emergency remote teaching. J Comput Assist Learn 37(6):1528–1538. https://doi.org/10.1111/jcal.12543

Article   Google Scholar  

Basken P (2020) Universities say student cheating exploding in Covid era. In: Times Higher Education. https://www.timeshighereducation.com/news/universities-say-student-cheating-exploding-covid-era

Bretag T (2019) Contract cheating will erode trust in science. Nature 574(7780):599. https://doi.org/10.1038/d41586-019-03265-1

Article   ADS   CAS   PubMed   Google Scholar  

Bretag T, Harper R, Burton M, Ellis C, Newton P, Rozenberg P, Saddiqui S, van Haeringen K (2019) Contract cheating: a survey of Australian university students. Stud. High. Educ. 44(11):1837–1856. https://doi.org/10.1080/03075079.2018.1462788

Chawla NV, Bowyer KW, Hall LO, Kegelmeyer WP (2002) SMOTE: synthetic minority over-sampling technique. J Artif Intell Res 16:321–357. https://doi.org/10.1613/jair.953

Dörnyei Z, Al-Hoorie AH (2017) The motivational foundation of learning languages other than Global English. Mod Lang J 101(3):455–468. https://doi.org/10.1111/modl.12408

Dreiseitl S, Ohno-Machado L (2002) Logistic regression and artificial neural network classification models: a methodology review. J Biomed Inform 35(5):352–359. https://doi.org/10.1016/s1532-0464(03)00034-0

Article   PubMed   Google Scholar  

Elshawi R, Maher M, Sakr S (2019) Automated machine learning: State-of-the-art and open challenges. arXiv . https://doi.org/10.48550/arXiv.1906.02287

Flanagin A, Kendall-Taylor J, Bibbins-Domingo K (2023) Guidance for authors, peer reviewers, and editors on use of AI, language models, and chatbots. JAMA, Advance online publication. https://doi.org/10.1001/jama.2023.12500

Friedman JH (2001) Greedy function approximation: a gradient boosting machine. Ann Stat 29(5):1189–1232. https://doi.org/10.1214/aos/1013203451

Article   MathSciNet   Google Scholar  

Guerrero-Dib JG, Portales L, Heredia-Escorza Y (2020) Impact of academic integrity on workplace ethical behaviour. Int J Educ Integr. 16(1):2. https://doi.org/10.1007/s40979-020-0051-3

Hardeniya N, Perkins J, Chopra D, Joshi N, Mathur I (2016) Natural language processing: python and NLTK. Packt Publishing

Hill G, Mason J, Dunn A (2021) Contract cheating: an increasing challenge for global academic community arising from COVID-19. Res Pract Technol Enhanc Learn 16(1):24. https://doi.org/10.1186/s41039-021-00166-8

Article   PubMed   PubMed Central   Google Scholar  

Hodges C, Moore SL, Lockee B, Trust T, Bond A (2020) The difference between emergency remote teaching and online learning. EDUCAUSE Rev. https://er.educause.edu/articles/2020/3/the-difference-between-emergency-remote-teaching-and-online-learning

Kim S-G (2023) Using ChatGPT for language editing in scientific articles. Maxillofac Plast Reconstr Surg 45(1):13. https://doi.org/10.1186/s40902-023-00381-x . Article

Article   ADS   PubMed   PubMed Central   Google Scholar  

Lancaster T, Clarke R (2016) Contract cheating: the outsourcing of assessed student work. In: T Bretag (Ed.) Handbook of academic integrity (pp. 639–654) Springer

Lancaster T, Cotarlan C (2021) Contract cheating by STEM students through a file sharing website: a Covid-19 pandemic perspective. Int J Educ Integr 17(1):3. https://doi.org/10.1007/s40979-021-00070-0

Mitchell R, Myles F, Marsden E (2019) Second language learning theories (4th ed.). Routledge

Mohammed R, Rawashdeh J, Abdullah M (2020, April). Machine learning with oversampling and undersampling techniques: overview study and experimental results. paper presented at the 11th international conference on information and communication systems (ICICS), Irbid, Jordan

Orosz G, Tóth-Király I, Bőthe B, Paskuj B, Berkics M, Fülöp M, Roland-Lévy C (2018) Linking cheating in school and corruption. Eur Rev Appl Psychol 68(2):89–97. https://doi.org/10.1016/j.erap.2018.02.001

Qaiser S, Ali R (2018) Text mining: Use of TF-IDF to examine the relevance of words to documents. Int J Comput Appl 181(1):25–29. https://doi.org/10.5120/ijca2018917395

Tharwat A (2021) Classification assessment methods. Appl Comput Inform 17(1):168–192. https://doi.org/10.1016/j.aci.2018.08.003

Weber-Wulff D (2019) Plagiarism detectors are a crutch, and a problem. Nature 567(7749):435. https://doi.org/10.1038/d41586-019-00893-5

Weber-Wulff D, Anohina-Naumeca A, Bjelobaba S, Foltýnek T, Guerrero-Dib J, Popoola O, Šigut P, Waddington L (2023) Testing of detection tools for AI-generated text. Int J Educ Integr 19(1):26. https://doi.org/10.1007/s40979-023-00146-z

Yang L, Shami A (2020) On hyperparameter optimization of machine learning algorithms: theory and practice. Neurocomputing 415:295–316. https://doi.org/10.1016/j.neucom.2020.07.061

Download references

Author information

Authors and affiliations.

College of Computing and Informatics, Saudi Electronic University, Jeddah, Saudi Arabia

Mohammed Kutbi

Royal Commission for Jubail and Yanbu, Jubail, Saudi Arabia

Ali H. Al-Hoorie

Faculty of Graduate Studies, Kuwait University, Kuwait City, Kuwait

Abbas H. Al-Shammari

You can also search for this author in PubMed   Google Scholar

Contributions

Mohammed Kutbi: conceptualization, data curation, formal analysis, investigation, methodology, software, visualization, writing: original draft, and writing: review & editing. Ali H. Al-Hoorie: conceptualization, methodology, project administration, resources, writing: original draft, and writing: review & editing. Abbas Al-Shammari: conceptualization, data curation, methodology, writing: original draft, and writing: review & editing.

Corresponding author

Correspondence to Ali H. Al-Hoorie .

Ethics declarations

Ethical approval.

The authors sought and obtained IRB approval from the ethics team at the English Language and Preparatory Year Institute, Royal Commission for Jubail and Yanbu (dated 23 November 2020) with no number attached to it. All procedures implemented in this study adhered to the ethical standards of the granting institution and to the tenets of the Declaration of Helsinki.

Informed consent

Informed consent was obtained from all participants in this study to ensure adherence to ethical guidelines. Detailed consent forms explaining the scope of the study and the participants’ rights were signed by each participant before the commencement of the study. The consent forms also assured each participant of the confidentiality of their responses and their right to withdraw at any time without any consequences. The consent forms also explained that the data will be used for research purposes after anonymization including further exploratory research.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Kutbi, M., Al-Hoorie, A.H. & Al-Shammari, A.H. Detecting contract cheating through linguistic fingerprint. Humanit Soc Sci Commun 11 , 664 (2024). https://doi.org/10.1057/s41599-024-03160-9

Download citation

Received : 31 July 2023

Accepted : 13 May 2024

Published : 24 May 2024

DOI : https://doi.org/10.1057/s41599-024-03160-9

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

A systematic literature review of empirical research on ChatGPT in education

• Open access
• Published: 26 May 2024
• Volume 3 , article number  60 , ( 2024 )

Cite this article

You have full access to this open access article

• Yazid Albadarin   ORCID: orcid.org/0009-0005-8068-8902 1 ,
• Mohammed Saqr 1 ,
• Nicolas Pope 1 &
• Markku Tukiainen 1  

Over the last four decades, studies have investigated the incorporation of Artificial Intelligence (AI) into education. A recent prominent AI-powered technology that has impacted the education sector is ChatGPT. This article provides a systematic review of 14 empirical studies incorporating ChatGPT into various educational settings, published in 2022 and before the 10th of April 2023—the date of conducting the search process. It carefully followed the essential steps outlined in the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA 2020) guidelines, as well as Okoli’s (Okoli in Commun Assoc Inf Syst, 2015) steps for conducting a rigorous and transparent systematic review. In this review, we aimed to explore how students and teachers have utilized ChatGPT in various educational settings, as well as the primary findings of those studies. By employing Creswell’s (Creswell in Educational research: planning, conducting, and evaluating quantitative and qualitative research [Ebook], Pearson Education, London, 2015) coding techniques for data extraction and interpretation, we sought to gain insight into their initial attempts at ChatGPT incorporation into education. This approach also enabled us to extract insights and considerations that can facilitate its effective and responsible use in future educational contexts. The results of this review show that learners have utilized ChatGPT as a virtual intelligent assistant, where it offered instant feedback, on-demand answers, and explanations of complex topics. Additionally, learners have used it to enhance their writing and language skills by generating ideas, composing essays, summarizing, translating, paraphrasing texts, or checking grammar. Moreover, learners turned to it as an aiding tool to facilitate their directed and personalized learning by assisting in understanding concepts and homework, providing structured learning plans, and clarifying assignments and tasks. However, the results of specific studies (n = 3, 21.4%) show that overuse of ChatGPT may negatively impact innovative capacities and collaborative learning competencies among learners. Educators, on the other hand, have utilized ChatGPT to create lesson plans, generate quizzes, and provide additional resources, which helped them enhance their productivity and efficiency and promote different teaching methodologies. Despite these benefits, the majority of the reviewed studies recommend the importance of conducting structured training, support, and clear guidelines for both learners and educators to mitigate the drawbacks. This includes developing critical evaluation skills to assess the accuracy and relevance of information provided by ChatGPT, as well as strategies for integrating human interaction and collaboration into learning activities that involve AI tools. Furthermore, they also recommend ongoing research and proactive dialogue with policymakers, stakeholders, and educational practitioners to refine and enhance the use of AI in learning environments. This review could serve as an insightful resource for practitioners who seek to integrate ChatGPT into education and stimulate further research in the field.

Avoid common mistakes on your manuscript.

1 Introduction

Educational technology, a rapidly evolving field, plays a crucial role in reshaping the landscape of teaching and learning [ 82 ]. One of the most transformative technological innovations of our era that has influenced the field of education is Artificial Intelligence (AI) [ 50 ]. Over the last four decades, AI in education (AIEd) has gained remarkable attention for its potential to make significant advancements in learning, instructional methods, and administrative tasks within educational settings [ 11 ]. In particular, a large language model (LLM), a type of AI algorithm that applies artificial neural networks (ANNs) and uses massively large data sets to understand, summarize, generate, and predict new content that is almost difficult to differentiate from human creations [ 79 ], has opened up novel possibilities for enhancing various aspects of education, from content creation to personalized instruction [ 35 ]. Chatbots that leverage the capabilities of LLMs to understand and generate human-like responses have also presented the capacity to enhance student learning and educational outcomes by engaging students, offering timely support, and fostering interactive learning experiences [ 46 ].

The ongoing and remarkable technological advancements in chatbots have made their use more convenient, increasingly natural and effortless, and have expanded their potential for deployment across various domains [ 70 ]. One prominent example of chatbot applications is the Chat Generative Pre-Trained Transformer, known as ChatGPT, which was introduced by OpenAI, a leading AI research lab, on November 30th, 2022. ChatGPT employs a variety of deep learning techniques to generate human-like text, with a particular focus on recurrent neural networks (RNNs). Long short-term memory (LSTM) allows it to grasp the context of the text being processed and retain information from previous inputs. Also, the transformer architecture, a neural network architecture based on the self-attention mechanism, allows it to analyze specific parts of the input, thereby enabling it to produce more natural-sounding and coherent output. Additionally, the unsupervised generative pre-training and the fine-tuning methods allow ChatGPT to generate more relevant and accurate text for specific tasks [ 31 , 62 ]. Furthermore, reinforcement learning from human feedback (RLHF), a machine learning approach that combines reinforcement learning techniques with human-provided feedback, has helped improve ChatGPT’s model by accelerating the learning process and making it significantly more efficient.

This cutting-edge natural language processing (NLP) tool is widely recognized as one of today's most advanced LLMs-based chatbots [ 70 ], allowing users to ask questions and receive detailed, coherent, systematic, personalized, convincing, and informative human-like responses [ 55 ], even within complex and ambiguous contexts [ 63 , 77 ]. ChatGPT is considered the fastest-growing technology in history: in just three months following its public launch, it amassed an estimated 120 million monthly active users [ 16 ] with an estimated 13 million daily queries [ 49 ], surpassing all other applications [ 64 ]. This remarkable growth can be attributed to the unique features and user-friendly interface that ChatGPT offers. Its intuitive design allows users to interact seamlessly with the technology, making it accessible to a diverse range of individuals, regardless of their technical expertise [ 78 ]. Additionally, its exceptional performance results from a combination of advanced algorithms, continuous enhancements, and extensive training on a diverse dataset that includes various text sources such as books, articles, websites, and online forums [ 63 ], have contributed to a more engaging and satisfying user experience [ 62 ]. These factors collectively explain its remarkable global growth and set it apart from predecessors like Bard, Bing Chat, ERNIE, and others.

In this context, several studies have explored the technological advancements of chatbots. One noteworthy recent research effort, conducted by Schöbel et al. [ 70 ], stands out for its comprehensive analysis of more than 5,000 studies on communication agents. This study offered a comprehensive overview of the historical progression and future prospects of communication agents, including ChatGPT. Moreover, other studies have focused on making comparisons, particularly between ChatGPT and alternative chatbots like Bard, Bing Chat, ERNIE, LaMDA, BlenderBot, and various others. For example, O’Leary [ 53 ] compared two chatbots, LaMDA and BlenderBot, with ChatGPT and revealed that ChatGPT outperformed both. This superiority arises from ChatGPT’s capacity to handle a wider range of questions and generate slightly varied perspectives within specific contexts. Similarly, ChatGPT exhibited an impressive ability to formulate interpretable responses that were easily understood when compared with Google's feature snippet [ 34 ]. Additionally, ChatGPT was compared to other LLMs-based chatbots, including Bard and BERT, as well as ERNIE. The findings indicated that ChatGPT exhibited strong performance in the given tasks, often outperforming the other models [ 59 ].

Furthermore, in the education context, a comprehensive study systematically compared a range of the most promising chatbots, including Bard, Bing Chat, ChatGPT, and Ernie across a multidisciplinary test that required higher-order thinking. The study revealed that ChatGPT achieved the highest score, surpassing Bing Chat and Bard [ 64 ]. Similarly, a comparative analysis was conducted to compare ChatGPT with Bard in answering a set of 30 mathematical questions and logic problems, grouped into two question sets. Set (A) is unavailable online, while Set (B) is available online. The results revealed ChatGPT's superiority in Set (A) over Bard. Nevertheless, Bard's advantage emerged in Set (B) due to its capacity to access the internet directly and retrieve answers, a capability that ChatGPT does not possess [ 57 ]. However, through these varied assessments, ChatGPT consistently highlights its exceptional prowess compared to various alternatives in the ever-evolving chatbot technology.

The widespread adoption of chatbots, especially ChatGPT, by millions of students and educators, has sparked extensive discussions regarding its incorporation into the education sector [ 64 ]. Accordingly, many scholars have contributed to the discourse, expressing both optimism and pessimism regarding the incorporation of ChatGPT into education. For example, ChatGPT has been highlighted for its capabilities in enriching the learning and teaching experience through its ability to support different learning approaches, including adaptive learning, personalized learning, and self-directed learning [ 58 , 60 , 91 ]), deliver summative and formative feedback to students and provide real-time responses to questions, increase the accessibility of information [ 22 , 40 , 43 ], foster students’ performance, engagement and motivation [ 14 , 44 , 58 ], and enhance teaching practices [ 17 , 18 , 64 , 74 ].

On the other hand, concerns have been also raised regarding its potential negative effects on learning and teaching. These include the dissemination of false information and references [ 12 , 23 , 61 , 85 ], biased reinforcement [ 47 , 50 ], compromised academic integrity [ 18 , 40 , 66 , 74 ], and the potential decline in students' skills [ 43 , 61 , 64 , 74 ]. As a result, ChatGPT has been banned in multiple countries, including Russia, China, Venezuela, Belarus, and Iran, as well as in various educational institutions in India, Italy, Western Australia, France, and the United States [ 52 , 90 ].

Clearly, the advent of chatbots, especially ChatGPT, has provoked significant controversy due to their potential impact on learning and teaching. This indicates the necessity for further exploration to gain a deeper understanding of this technology and carefully evaluate its potential benefits, limitations, challenges, and threats to education [ 79 ]. Therefore, conducting a systematic literature review will provide valuable insights into the potential prospects and obstacles linked to its incorporation into education. This systematic literature review will primarily focus on ChatGPT, driven by the aforementioned key factors outlined above.

However, the existing literature lacks a systematic literature review of empirical studies. Thus, this systematic literature review aims to address this gap by synthesizing the existing empirical studies conducted on chatbots, particularly ChatGPT, in the field of education, highlighting how ChatGPT has been utilized in educational settings, and identifying any existing gaps. This review may be particularly useful for researchers in the field and educators who are contemplating the integration of ChatGPT or any chatbot into education. The following research questions will guide this study:

What are students' and teachers' initial attempts at utilizing ChatGPT in education?

What are the main findings derived from empirical studies that have incorporated ChatGPT into learning and teaching?

2 Methodology

To conduct this study, the authors followed the essential steps of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA 2020) and Okoli’s [ 54 ] steps for conducting a systematic review. These included identifying the study’s purpose, drafting a protocol, applying a practical screening process, searching the literature, extracting relevant data, evaluating the quality of the included studies, synthesizing the studies, and ultimately writing the review. The subsequent section provides an extensive explanation of how these steps were carried out in this study.

2.1 Identify the purpose

Given the widespread adoption of ChatGPT by students and teachers for various educational purposes, often without a thorough understanding of responsible and effective use or a clear recognition of its potential impact on learning and teaching, the authors recognized the need for further exploration of ChatGPT's impact on education in this early stage. Therefore, they have chosen to conduct a systematic literature review of existing empirical studies that incorporate ChatGPT into educational settings. Despite the limited number of empirical studies due to the novelty of the topic, their goal is to gain a deeper understanding of this technology and proactively evaluate its potential benefits, limitations, challenges, and threats to education. This effort could help to understand initial reactions and attempts at incorporating ChatGPT into education and bring out insights and considerations that can inform the future development of education.

2.2 Draft the protocol

The next step is formulating the protocol. This protocol serves to outline the study process in a rigorous and transparent manner, mitigating researcher bias in study selection and data extraction [ 88 ]. The protocol will include the following steps: generating the research question, predefining a literature search strategy, identifying search locations, establishing selection criteria, assessing the studies, developing a data extraction strategy, and creating a timeline.

2.3 Apply practical screen

The screening step aims to accurately filter the articles resulting from the searching step and select the empirical studies that have incorporated ChatGPT into educational contexts, which will guide us in answering the research questions and achieving the objectives of this study. To ensure the rigorous execution of this step, our inclusion and exclusion criteria were determined based on the authors' experience and informed by previous successful systematic reviews [ 21 ]. Table 1 summarizes the inclusion and exclusion criteria for study selection.

2.4 Literature search

We conducted a thorough literature search to identify articles that explored, examined, and addressed the use of ChatGPT in Educational contexts. We utilized two research databases: Dimensions.ai, which provides access to a large number of research publications, and lens.org, which offers access to over 300 million articles, patents, and other research outputs from diverse sources. Additionally, we included three databases, Scopus, Web of Knowledge, and ERIC, which contain relevant research on the topic that addresses our research questions. To browse and identify relevant articles, we used the following search formula: ("ChatGPT" AND "Education"), which included the Boolean operator "AND" to get more specific results. The subject area in the Scopus and ERIC databases were narrowed to "ChatGPT" and "Education" keywords, and in the WoS database was limited to the "Education" category. The search was conducted between the 3rd and 10th of April 2023, which resulted in 276 articles from all selected databases (111 articles from Dimensions.ai, 65 from Scopus, 28 from Web of Science, 14 from ERIC, and 58 from Lens.org). These articles were imported into the Rayyan web-based system for analysis. The duplicates were identified automatically by the system. Subsequently, the first author manually reviewed the duplicated articles ensured that they had the same content, and then removed them, leaving us with 135 unique articles. Afterward, the titles, abstracts, and keywords of the first 40 manuscripts were scanned and reviewed by the first author and were discussed with the second and third authors to resolve any disagreements. Subsequently, the first author proceeded with the filtering process for all articles and carefully applied the inclusion and exclusion criteria as presented in Table  1 . Articles that met any one of the exclusion criteria were eliminated, resulting in 26 articles. Afterward, the authors met to carefully scan and discuss them. The authors agreed to eliminate any empirical studies solely focused on checking ChatGPT capabilities, as these studies do not guide us in addressing the research questions and achieving the study's objectives. This resulted in 14 articles eligible for analysis.

2.5 Quality appraisal

The examination and evaluation of the quality of the extracted articles is a vital step [ 9 ]. Therefore, the extracted articles were carefully evaluated for quality using Fink’s [ 24 ] standards, which emphasize the necessity for detailed descriptions of methodology, results, conclusions, strengths, and limitations. The process began with a thorough assessment of each study's design, data collection, and analysis methods to ensure their appropriateness and comprehensive execution. The clarity, consistency, and logical progression from data to results and conclusions were also critically examined. Potential biases and recognized limitations within the studies were also scrutinized. Ultimately, two articles were excluded for failing to meet Fink’s criteria, particularly in providing sufficient detail on methodology, results, conclusions, strengths, or limitations. The review process is illustrated in Fig.  1 .

The study selection process

2.6 Data extraction

The next step is data extraction, the process of capturing the key information and categories from the included studies. To improve efficiency, reduce variation among authors, and minimize errors in data analysis, the coding categories were constructed using Creswell's [ 15 ] coding techniques for data extraction and interpretation. The coding process involves three sequential steps. The initial stage encompasses open coding , where the researcher examines the data, generates codes to describe and categorize it, and gains a deeper understanding without preconceived ideas. Following open coding is axial coding , where the interrelationships between codes from open coding are analyzed to establish more comprehensive categories or themes. The process concludes with selective coding , refining and integrating categories or themes to identify core concepts emerging from the data. The first coder performed the coding process, then engaged in discussions with the second and third authors to finalize the coding categories for the first five articles. The first coder then proceeded to code all studies and engaged again in discussions with the other authors to ensure the finalization of the coding process. After a comprehensive analysis and capturing of the key information from the included studies, the data extraction and interpretation process yielded several themes. These themes have been categorized and are presented in Table  2 . It is important to note that open coding results were removed from Table  2 for aesthetic reasons, as it included many generic aspects, such as words, short phrases, or sentences mentioned in the studies.

2.7 Synthesize studies

In this stage, we will gather, discuss, and analyze the key findings that emerged from the selected studies. The synthesis stage is considered a transition from an author-centric to a concept-centric focus, enabling us to map all the provided information to achieve the most effective evaluation of the data [ 87 ]. Initially, the authors extracted data that included general information about the selected studies, including the author(s)' names, study titles, years of publication, educational levels, research methodologies, sample sizes, participants, main aims or objectives, raw data sources, and analysis methods. Following that, all key information and significant results from the selected studies were compiled using Creswell’s [ 15 ] coding techniques for data extraction and interpretation to identify core concepts and themes emerging from the data, focusing on those that directly contributed to our research questions and objectives, such as the initial utilization of ChatGPT in learning and teaching, learners' and educators' familiarity with ChatGPT, and the main findings of each study. Finally, the data related to each selected study were extracted into an Excel spreadsheet for data processing. The Excel spreadsheet was reviewed by the authors, including a series of discussions to ensure the finalization of this process and prepare it for further analysis. Afterward, the final result being analyzed and presented in various types of charts and graphs. Table 4 presents the extracted data from the selected studies, with each study labeled with a capital 'S' followed by a number.

This section consists of two main parts. The first part provides a descriptive analysis of the data compiled from the reviewed studies. The second part presents the answers to the research questions and the main findings of these studies.

3.1 Part 1: descriptive analysis

This section will provide a descriptive analysis of the reviewed studies, including educational levels and fields, participants distribution, country contribution, research methodologies, study sample size, study population, publication year, list of journals, familiarity with ChatGPT, source of data, and the main aims and objectives of the studies. Table 4 presents a comprehensive overview of the extracted data from the selected studies.

3.1.1 The number of the reviewed studies and publication years

The total number of the reviewed studies was 14. All studies were empirical studies and published in different journals focusing on Education and Technology. One study was published in 2022 [S1], while the remaining were published in 2023 [S2]-[S14]. Table 3 illustrates the year of publication, the names of the journals, and the number of reviewed studies published in each journal for the studies reviewed.

3.1.2 Educational levels and fields

The majority of the reviewed studies, 11 studies, were conducted in higher education institutions [S1]-[S10] and [S13]. Two studies did not specify the educational level of the population [S12] and [S14], while one study focused on elementary education [S11]. However, the reviewed studies covered various fields of education. Three studies focused on Arts and Humanities Education [S8], [S11], and [S14], specifically English Education. Two studies focused on Engineering Education, with one in Computer Engineering [S2] and the other in Construction Education [S3]. Two studies focused on Mathematics Education [S5] and [S12]. One study focused on Social Science Education [S13]. One study focused on Early Education [S4]. One study focused on Journalism Education [S9]. Finally, three studies did not specify the field of education [S1], [S6], and [S7]. Figure  2 represents the educational levels in the reviewed studies, while Fig.  3 represents the context of the reviewed studies.

Educational levels in the reviewed studies

Context of the reviewed studies

3.1.3 Participants distribution and countries contribution

The reviewed studies have been conducted across different geographic regions, providing a diverse representation of the studies. The majority of the studies, 10 in total, [S1]-[S3], [S5]-[S9], [S11], and [S14], primarily focused on participants from single countries such as Pakistan, the United Arab Emirates, China, Indonesia, Poland, Saudi Arabia, South Korea, Spain, Tajikistan, and the United States. In contrast, four studies, [S4], [S10], [S12], and [S13], involved participants from multiple countries, including China and the United States [S4], China, the United Kingdom, and the United States [S10], the United Arab Emirates, Oman, Saudi Arabia, and Jordan [S12], Turkey, Sweden, Canada, and Australia [ 13 ]. Figures  4 and 5 illustrate the distribution of participants, whether from single or multiple countries, and the contribution of each country in the reviewed studies, respectively.

The reviewed studies conducted in single or multiple countries

The Contribution of each country in the studies

3.1.4 Study population and sample size

Four study populations were included: university students, university teachers, university teachers and students, and elementary school teachers. Six studies involved university students [S2], [S3], [S5] and [S6]-[S8]. Three studies focused on university teachers [S1], [S4], and [S6], while one study specifically targeted elementary school teachers [S11]. Additionally, four studies included both university teachers and students [S10] and [ 12 , 13 , 14 ], and among them, study [S13] specifically included postgraduate students. In terms of the sample size of the reviewed studies, nine studies included a small sample size of less than 50 participants [S1], [S3], [S6], [S8], and [S10]-[S13]. Three studies had 50–100 participants [S2], [S9], and [S14]. Only one study had more than 100 participants [S7]. It is worth mentioning that study [S4] adopted a mixed methods approach, including 10 participants for qualitative analysis and 110 participants for quantitative analysis.

3.1.5 Participants’ familiarity with using ChatGPT

The reviewed studies recruited a diverse range of participants with varying levels of familiarity with ChatGPT. Five studies [S2], [S4], [S6], [S8], and [S12] involved participants already familiar with ChatGPT, while eight studies [S1], [S3], [S5], [S7], [S9], [S10], [S13] and [S14] included individuals with differing levels of familiarity. Notably, one study [S11] had participants who were entirely unfamiliar with ChatGPT. It is important to note that four studies [S3], [S5], [S9], and [S11] provided training or guidance to their participants before conducting their studies, while ten studies [S1], [S2], [S4], [S6]-[S8], [S10], and [S12]-[S14] did not provide training due to the participants' existing familiarity with ChatGPT.

3.1.6 Research methodology approaches and source(S) of data

The reviewed studies adopted various research methodology approaches. Seven studies adopted qualitative research methodology [S1], [S4], [S6], [S8], [S10], [S11], and [S12], while three studies adopted quantitative research methodology [S3], [S7], and [S14], and four studies employed mixed-methods, which involved a combination of both the strengths of qualitative and quantitative methods [S2], [S5], [S9], and [S13].

In terms of the source(s) of data, the reviewed studies obtained their data from various sources, such as interviews, questionnaires, and pre-and post-tests. Six studies relied on interviews as their primary source of data collection [S1], [S4], [S6], [S10], [S11], and [S12], four studies relied on questionnaires [S2], [S7], [S13], and [S14], two studies combined the use of pre-and post-tests and questionnaires for data collection [S3] and [S9], while two studies combined the use of questionnaires and interviews to obtain the data [S5] and [S8]. It is important to note that six of the reviewed studies were quasi-experimental [S3], [S5], [S8], [S9], [S12], and [S14], while the remaining ones were experimental studies [S1], [S2], [S4], [S6], [S7], [S10], [S11], and [S13]. Figures  6 and 7 illustrate the research methodologies and the source (s) of data used in the reviewed studies, respectively.

Research methodologies in the reviewed studies

Source of data in the reviewed studies

3.1.7 The aim and objectives of the studies

The reviewed studies encompassed a diverse set of aims, with several of them incorporating multiple primary objectives. Six studies [S3], [S6], [S7], [S8], [S11], and [S12] examined the integration of ChatGPT in educational contexts, and four studies [S4], [S5], [S13], and [S14] investigated the various implications of its use in education, while three studies [S2], [S9], and [S10] aimed to explore both its integration and implications in education. Additionally, seven studies explicitly explored attitudes and perceptions of students [S2] and [S3], educators [S1] and [S6], or both [S10], [S12], and [S13] regarding the utilization of ChatGPT in educational settings.

3.2 Part 2: research questions and main findings of the reviewed studies

This part will present the answers to the research questions and the main findings of the reviewed studies, classified into two main categories (learning and teaching) according to AI Education classification by [ 36 ]. Figure  8 summarizes the main findings of the reviewed studies in a visually informative diagram. Table 4 provides a detailed list of the key information extracted from the selected studies that led to generating these themes.

The main findings in the reviewed studies

4 Students' initial attempts at utilizing ChatGPT in learning and main findings from students' perspective

4.1 virtual intelligent assistant.

Nine studies demonstrated that ChatGPT has been utilized by students as an intelligent assistant to enhance and support their learning. Students employed it for various purposes, such as answering on-demand questions [S2]-[S5], [S8], [S10], and [S12], providing valuable information and learning resources [S2]-[S5], [S6], and [S8], as well as receiving immediate feedback [S2], [S4], [S9], [S10], and [S12]. In this regard, students generally were confident in the accuracy of ChatGPT's responses, considering them relevant, reliable, and detailed [S3], [S4], [S5], and [S8]. However, some students indicated the need for improvement, as they found that answers are not always accurate [S2], and that misleading information may have been provided or that it may not always align with their expectations [S6] and [S10]. It was also observed by the students that the accuracy of ChatGPT is dependent on several factors, including the quality and specificity of the user's input, the complexity of the question or topic, and the scope and relevance of its training data [S12]. Many students felt that ChatGPT's answers were not always accurate and most of them believed that it requires good background knowledge to work with.

4.2 Writing and language proficiency assistant

Six of the reviewed studies highlighted that ChatGPT has been utilized by students as a valuable assistant tool to improve their academic writing skills and language proficiency. Among these studies, three mainly focused on English education, demonstrating that students showed sufficient mastery in using ChatGPT for generating ideas, summarizing, paraphrasing texts, and completing writing essays [S8], [S11], and [S14]. Furthermore, ChatGPT helped them in writing by making students active investigators rather than passive knowledge recipients and facilitated the development of their writing skills [S11] and [S14]. Similarly, ChatGPT allowed students to generate unique ideas and perspectives, leading to deeper analysis and reflection on their journalism writing [S9]. In terms of language proficiency, ChatGPT allowed participants to translate content into their home languages, making it more accessible and relevant to their context [S4]. It also enabled them to request changes in linguistic tones or flavors [S8]. Moreover, participants used it to check grammar or as a dictionary [S11].

4.3 Valuable resource for learning approaches

Five studies demonstrated that students used ChatGPT as a valuable complementary resource for self-directed learning. It provided learning resources and guidance on diverse educational topics and created a supportive home learning environment [S2] and [S4]. Moreover, it offered step-by-step guidance to grasp concepts at their own pace and enhance their understanding [S5], streamlined task and project completion carried out independently [S7], provided comprehensive and easy-to-understand explanations on various subjects [S10], and assisted in studying geometry operations, thereby empowering them to explore geometry operations at their own pace [S12]. Three studies showed that students used ChatGPT as a valuable learning resource for personalized learning. It delivered age-appropriate conversations and tailored teaching based on a child's interests [S4], acted as a personalized learning assistant, adapted to their needs and pace, which assisted them in understanding mathematical concepts [S12], and enabled personalized learning experiences in social sciences by adapting to students' needs and learning styles [S13]. On the other hand, it is important to note that, according to one study [S5], students suggested that using ChatGPT may negatively affect collaborative learning competencies between students.

4.4 Enhancing students' competencies

Six of the reviewed studies have shown that ChatGPT is a valuable tool for improving a wide range of skills among students. Two studies have provided evidence that ChatGPT led to improvements in students' critical thinking, reasoning skills, and hazard recognition competencies through engaging them in interactive conversations or activities and providing responses related to their disciplines in journalism [S5] and construction education [S9]. Furthermore, two studies focused on mathematical education have shown the positive impact of ChatGPT on students' problem-solving abilities in unraveling problem-solving questions [S12] and enhancing the students' understanding of the problem-solving process [S5]. Lastly, one study indicated that ChatGPT effectively contributed to the enhancement of conversational social skills [S4].

4.5 Supporting students' academic success

Seven of the reviewed studies highlighted that students found ChatGPT to be beneficial for learning as it enhanced learning efficiency and improved the learning experience. It has been observed to improve students' efficiency in computer engineering studies by providing well-structured responses and good explanations [S2]. Additionally, students found it extremely useful for hazard reporting [S3], and it also enhanced their efficiency in solving mathematics problems and capabilities [S5] and [S12]. Furthermore, by finding information, generating ideas, translating texts, and providing alternative questions, ChatGPT aided students in deepening their understanding of various subjects [S6]. It contributed to an increase in students' overall productivity [S7] and improved efficiency in composing written tasks [S8]. Regarding learning experiences, ChatGPT was instrumental in assisting students in identifying hazards that they might have otherwise overlooked [S3]. It also improved students' learning experiences in solving mathematics problems and developing abilities [S5] and [S12]. Moreover, it increased students' successful completion of important tasks in their studies [S7], particularly those involving average difficulty writing tasks [S8]. Additionally, ChatGPT increased the chances of educational success by providing students with baseline knowledge on various topics [S10].

5 Teachers' initial attempts at utilizing ChatGPT in teaching and main findings from teachers' perspective

5.1 valuable resource for teaching.

The reviewed studies showed that teachers have employed ChatGPT to recommend, modify, and generate diverse, creative, organized, and engaging educational contents, teaching materials, and testing resources more rapidly [S4], [S6], [S10] and [S11]. Additionally, teachers experienced increased productivity as ChatGPT facilitated quick and accurate responses to questions, fact-checking, and information searches [S1]. It also proved valuable in constructing new knowledge [S6] and providing timely answers to students' questions in classrooms [S11]. Moreover, ChatGPT enhanced teachers' efficiency by generating new ideas for activities and preplanning activities for their students [S4] and [S6], including interactive language game partners [S11].

5.2 Improving productivity and efficiency

The reviewed studies showed that participants' productivity and work efficiency have been significantly enhanced by using ChatGPT as it enabled them to allocate more time to other tasks and reduce their overall workloads [S6], [S10], [S11], [S13], and [S14]. However, three studies [S1], [S4], and [S11], indicated a negative perception and attitude among teachers toward using ChatGPT. This negativity stemmed from a lack of necessary skills to use it effectively [S1], a limited familiarity with it [S4], and occasional inaccuracies in the content provided by it [S10].

5.3 Catalyzing new teaching methodologies

Five of the reviewed studies highlighted that educators found the necessity of redefining their teaching profession with the assistance of ChatGPT [S11], developing new effective learning strategies [S4], and adapting teaching strategies and methodologies to ensure the development of essential skills for future engineers [S5]. They also emphasized the importance of adopting new educational philosophies and approaches that can evolve with the introduction of ChatGPT into the classroom [S12]. Furthermore, updating curricula to focus on improving human-specific features, such as emotional intelligence, creativity, and philosophical perspectives [S13], was found to be essential.

5.4 Effective utilization of CHATGPT in teaching

According to the reviewed studies, effective utilization of ChatGPT in education requires providing teachers with well-structured training, support, and adequate background on how to use ChatGPT responsibly [S1], [S3], [S11], and [S12]. Establishing clear rules and regulations regarding its usage is essential to ensure it positively impacts the teaching and learning processes, including students' skills [S1], [S4], [S5], [S8], [S9], and [S11]-[S14]. Moreover, conducting further research and engaging in discussions with policymakers and stakeholders is indeed crucial for the successful integration of ChatGPT in education and to maximize the benefits for both educators and students [S1], [S6]-[S10], and [S12]-[S14].

6 Discussion

The purpose of this review is to conduct a systematic review of empirical studies that have explored the utilization of ChatGPT, one of today’s most advanced LLM-based chatbots, in education. The findings of the reviewed studies showed several ways of ChatGPT utilization in different learning and teaching practices as well as it provided insights and considerations that can facilitate its effective and responsible use in future educational contexts. The results of the reviewed studies came from diverse fields of education, which helped us avoid a biased review that is limited to a specific field. Similarly, the reviewed studies have been conducted across different geographic regions. This kind of variety in geographic representation enriched the findings of this review.

In response to RQ1 , "What are students' and teachers' initial attempts at utilizing ChatGPT in education?", the findings from this review provide comprehensive insights. Chatbots, including ChatGPT, play a crucial role in supporting student learning, enhancing their learning experiences, and facilitating diverse learning approaches [ 42 , 43 ]. This review found that this tool, ChatGPT, has been instrumental in enhancing students' learning experiences by serving as a virtual intelligent assistant, providing immediate feedback, on-demand answers, and engaging in educational conversations. Additionally, students have benefited from ChatGPT’s ability to generate ideas, compose essays, and perform tasks like summarizing, translating, paraphrasing texts, or checking grammar, thereby enhancing their writing and language competencies. Furthermore, students have turned to ChatGPT for assistance in understanding concepts and homework, providing structured learning plans, and clarifying assignments and tasks, which fosters a supportive home learning environment, allowing them to take responsibility for their own learning and cultivate the skills and approaches essential for supportive home learning environment [ 26 , 27 , 28 ]. This finding aligns with the study of Saqr et al. [ 68 , 69 ] who highlighted that, when students actively engage in their own learning process, it yields additional advantages, such as heightened motivation, enhanced achievement, and the cultivation of enthusiasm, turning them into advocates for their own learning.

Moreover, students have utilized ChatGPT for tailored teaching and step-by-step guidance on diverse educational topics, streamlining task and project completion, and generating and recommending educational content. This personalization enhances the learning environment, leading to increased academic success. This finding aligns with other recent studies [ 26 , 27 , 28 , 60 , 66 ] which revealed that ChatGPT has the potential to offer personalized learning experiences and support an effective learning process by providing students with customized feedback and explanations tailored to their needs and abilities. Ultimately, fostering students' performance, engagement, and motivation, leading to increase students' academic success [ 14 , 44 , 58 ]. This ultimate outcome is in line with the findings of Saqr et al. [ 68 , 69 ], which emphasized that learning strategies are important catalysts of students' learning, as students who utilize effective learning strategies are more likely to have better academic achievement.

Teachers, too, have capitalized on ChatGPT's capabilities to enhance productivity and efficiency, using it for creating lesson plans, generating quizzes, providing additional resources, generating and preplanning new ideas for activities, and aiding in answering students’ questions. This adoption of technology introduces new opportunities to support teaching and learning practices, enhancing teacher productivity. This finding aligns with those of Day [ 17 ], De Castro [ 18 ], and Su and Yang [ 74 ] as well as with those of Valtonen et al. [ 82 ], who revealed that emerging technological advancements have opened up novel opportunities and means to support teaching and learning practices, and enhance teachers’ productivity.

In response to RQ2 , "What are the main findings derived from empirical studies that have incorporated ChatGPT into learning and teaching?", the findings from this review provide profound insights and raise significant concerns. Starting with the insights, chatbots, including ChatGPT, have demonstrated the potential to reshape and revolutionize education, creating new, novel opportunities for enhancing the learning process and outcomes [ 83 ], facilitating different learning approaches, and offering a range of pedagogical benefits [ 19 , 43 , 72 ]. In this context, this review found that ChatGPT could open avenues for educators to adopt or develop new effective learning and teaching strategies that can evolve with the introduction of ChatGPT into the classroom. Nonetheless, there is an evident lack of research understanding regarding the potential impact of generative machine learning models within diverse educational settings [ 83 ]. This necessitates teachers to attain a high level of proficiency in incorporating chatbots, such as ChatGPT, into their classrooms to create inventive, well-structured, and captivating learning strategies. In the same vein, the review also found that teachers without the requisite skills to utilize ChatGPT realized that it did not contribute positively to their work and could potentially have adverse effects [ 37 ]. This concern could lead to inequity of access to the benefits of chatbots, including ChatGPT, as individuals who lack the necessary expertise may not be able to harness their full potential, resulting in disparities in educational outcomes and opportunities. Therefore, immediate action is needed to address these potential issues. A potential solution is offering training, support, and competency development for teachers to ensure that all of them can leverage chatbots, including ChatGPT, effectively and equitably in their educational practices [ 5 , 28 , 80 ], which could enhance accessibility and inclusivity, and potentially result in innovative outcomes [ 82 , 83 ].

Additionally, chatbots, including ChatGPT, have the potential to significantly impact students' thinking abilities, including retention, reasoning, analysis skills [ 19 , 45 ], and foster innovation and creativity capabilities [ 83 ]. This review found that ChatGPT could contribute to improving a wide range of skills among students. However, it found that frequent use of ChatGPT may result in a decrease in innovative capacities, collaborative skills and cognitive capacities, and students' motivation to attend classes, as well as could lead to reduced higher-order thinking skills among students [ 22 , 29 ]. Therefore, immediate action is needed to carefully examine the long-term impact of chatbots such as ChatGPT, on learning outcomes as well as to explore its incorporation into educational settings as a supportive tool without compromising students' cognitive development and critical thinking abilities. In the same vein, the review also found that it is challenging to draw a consistent conclusion regarding the potential of ChatGPT to aid self-directed learning approach. This finding aligns with the recent study of Baskara [ 8 ]. Therefore, further research is needed to explore the potential of ChatGPT for self-directed learning. One potential solution involves utilizing learning analytics as a novel approach to examine various aspects of students' learning and support them in their individual endeavors [ 32 ]. This approach can bridge this gap by facilitating an in-depth analysis of how learners engage with ChatGPT, identifying trends in self-directed learning behavior, and assessing its influence on their outcomes.

Turning to the significant concerns, on the other hand, a fundamental challenge with LLM-based chatbots, including ChatGPT, is the accuracy and quality of the provided information and responses, as they provide false information as truth—a phenomenon often referred to as "hallucination" [ 3 , 49 ]. In this context, this review found that the provided information was not entirely satisfactory. Consequently, the utilization of chatbots presents potential concerns, such as generating and providing inaccurate or misleading information, especially for students who utilize it to support their learning. This finding aligns with other findings [ 6 , 30 , 35 , 40 ] which revealed that incorporating chatbots such as ChatGPT, into education presents challenges related to its accuracy and reliability due to its training on a large corpus of data, which may contain inaccuracies and the way users formulate or ask ChatGPT. Therefore, immediate action is needed to address these potential issues. One possible solution is to equip students with the necessary skills and competencies, which include a background understanding of how to use it effectively and the ability to assess and evaluate the information it generates, as the accuracy and the quality of the provided information depend on the input, its complexity, the topic, and the relevance of its training data [ 28 , 49 , 86 ]. However, it's also essential to examine how learners can be educated about how these models operate, the data used in their training, and how to recognize their limitations, challenges, and issues [ 79 ].

Furthermore, chatbots present a substantial challenge concerning maintaining academic integrity [ 20 , 56 ] and copyright violations [ 83 ], which are significant concerns in education. The review found that the potential misuse of ChatGPT might foster cheating, facilitate plagiarism, and threaten academic integrity. This issue is also affirmed by the research conducted by Basic et al. [ 7 ], who presented evidence that students who utilized ChatGPT in their writing assignments had more plagiarism cases than those who did not. These findings align with the conclusions drawn by Cotton et al. [ 13 ], Hisan and Amri [ 33 ] and Sullivan et al. [ 75 ], who revealed that the integration of chatbots such as ChatGPT into education poses a significant challenge to the preservation of academic integrity. Moreover, chatbots, including ChatGPT, have increased the difficulty in identifying plagiarism [ 47 , 67 , 76 ]. The findings from previous studies [ 1 , 84 ] indicate that AI-generated text often went undetected by plagiarism software, such as Turnitin. However, Turnitin and other similar plagiarism detection tools, such as ZeroGPT, GPTZero, and Copyleaks, have since evolved, incorporating enhanced techniques to detect AI-generated text, despite the possibility of false positives, as noted in different studies that have found these tools still not yet fully ready to accurately and reliably identify AI-generated text [ 10 , 51 ], and new novel detection methods may need to be created and implemented for AI-generated text detection [ 4 ]. This potential issue could lead to another concern, which is the difficulty of accurately evaluating student performance when they utilize chatbots such as ChatGPT assistance in their assignments. Consequently, the most LLM-driven chatbots present a substantial challenge to traditional assessments [ 64 ]. The findings from previous studies indicate the importance of rethinking, improving, and redesigning innovative assessment methods in the era of chatbots [ 14 , 20 , 64 , 75 ]. These methods should prioritize the process of evaluating students' ability to apply knowledge to complex cases and demonstrate comprehension, rather than solely focusing on the final product for assessment. Therefore, immediate action is needed to address these potential issues. One possible solution would be the development of clear guidelines, regulatory policies, and pedagogical guidance. These measures would help regulate the proper and ethical utilization of chatbots, such as ChatGPT, and must be established before their introduction to students [ 35 , 38 , 39 , 41 , 89 ].

In summary, our review has delved into the utilization of ChatGPT, a prominent example of chatbots, in education, addressing the question of how ChatGPT has been utilized in education. However, there remain significant gaps, which necessitate further research to shed light on this area.

7 Conclusions

This systematic review has shed light on the varied initial attempts at incorporating ChatGPT into education by both learners and educators, while also offering insights and considerations that can facilitate its effective and responsible use in future educational contexts. From the analysis of 14 selected studies, the review revealed the dual-edged impact of ChatGPT in educational settings. On the positive side, ChatGPT significantly aided the learning process in various ways. Learners have used it as a virtual intelligent assistant, benefiting from its ability to provide immediate feedback, on-demand answers, and easy access to educational resources. Additionally, it was clear that learners have used it to enhance their writing and language skills, engaging in practices such as generating ideas, composing essays, and performing tasks like summarizing, translating, paraphrasing texts, or checking grammar. Importantly, other learners have utilized it in supporting and facilitating their directed and personalized learning on a broad range of educational topics, assisting in understanding concepts and homework, providing structured learning plans, and clarifying assignments and tasks. Educators, on the other hand, found ChatGPT beneficial for enhancing productivity and efficiency. They used it for creating lesson plans, generating quizzes, providing additional resources, and answers learners' questions, which saved time and allowed for more dynamic and engaging teaching strategies and methodologies.

However, the review also pointed out negative impacts. The results revealed that overuse of ChatGPT could decrease innovative capacities and collaborative learning among learners. Specifically, relying too much on ChatGPT for quick answers can inhibit learners' critical thinking and problem-solving skills. Learners might not engage deeply with the material or consider multiple solutions to a problem. This tendency was particularly evident in group projects, where learners preferred consulting ChatGPT individually for solutions over brainstorming and collaborating with peers, which negatively affected their teamwork abilities. On a broader level, integrating ChatGPT into education has also raised several concerns, including the potential for providing inaccurate or misleading information, issues of inequity in access, challenges related to academic integrity, and the possibility of misusing the technology.

Accordingly, this review emphasizes the urgency of developing clear rules, policies, and regulations to ensure ChatGPT's effective and responsible use in educational settings, alongside other chatbots, by both learners and educators. This requires providing well-structured training to educate them on responsible usage and understanding its limitations, along with offering sufficient background information. Moreover, it highlights the importance of rethinking, improving, and redesigning innovative teaching and assessment methods in the era of ChatGPT. Furthermore, conducting further research and engaging in discussions with policymakers and stakeholders are essential steps to maximize the benefits for both educators and learners and ensure academic integrity.

It is important to acknowledge that this review has certain limitations. Firstly, the limited inclusion of reviewed studies can be attributed to several reasons, including the novelty of the technology, as new technologies often face initial skepticism and cautious adoption; the lack of clear guidelines or best practices for leveraging this technology for educational purposes; and institutional or governmental policies affecting the utilization of this technology in educational contexts. These factors, in turn, have affected the number of studies available for review. Secondly, the utilization of the original version of ChatGPT, based on GPT-3 or GPT-3.5, implies that new studies utilizing the updated version, GPT-4 may lead to different findings. Therefore, conducting follow-up systematic reviews is essential once more empirical studies on ChatGPT are published. Additionally, long-term studies are necessary to thoroughly examine and assess the impact of ChatGPT on various educational practices.

Despite these limitations, this systematic review has highlighted the transformative potential of ChatGPT in education, revealing its diverse utilization by learners and educators alike and summarized the benefits of incorporating it into education, as well as the forefront critical concerns and challenges that must be addressed to facilitate its effective and responsible use in future educational contexts. This review could serve as an insightful resource for practitioners who seek to integrate ChatGPT into education and stimulate further research in the field.

Data availability

The data supporting our findings are available upon request.

Abbreviations

• Artificial intelligence

AI in education

Large language model

Artificial neural networks

Chat Generative Pre-Trained Transformer

Recurrent neural networks

Long short-term memory

Reinforcement learning from human feedback

Natural language processing

Preferred Reporting Items for Systematic Reviews and Meta-Analyses

AlAfnan MA, Dishari S, Jovic M, Lomidze K. ChatGPT as an educational tool: opportunities, challenges, and recommendations for communication, business writing, and composition courses. J Artif Intell Technol. 2023. https://doi.org/10.37965/jait.2023.0184 .

Article   Google Scholar  

Ali JKM, Shamsan MAA, Hezam TA, Mohammed AAQ. Impact of ChatGPT on learning motivation. J Engl Stud Arabia Felix. 2023;2(1):41–9. https://doi.org/10.56540/jesaf.v2i1.51 .

Alkaissi H, McFarlane SI. Artificial hallucinations in ChatGPT: implications in scientific writing. Cureus. 2023. https://doi.org/10.7759/cureus.35179 .

Anderson N, Belavý DL, Perle SM, Hendricks S, Hespanhol L, Verhagen E, Memon AR. AI did not write this manuscript, or did it? Can we trick the AI text detector into generated texts? The potential future of ChatGPT and AI in sports & exercise medicine manuscript generation. BMJ Open Sport Exerc Med. 2023;9(1): e001568. https://doi.org/10.1136/bmjsem-2023-001568 .

Ausat AMA, Massang B, Efendi M, Nofirman N, Riady Y. Can chat GPT replace the role of the teacher in the classroom: a fundamental analysis. J Educ. 2023;5(4):16100–6.

Google Scholar  

Baidoo-Anu D, Ansah L. Education in the Era of generative artificial intelligence (AI): understanding the potential benefits of ChatGPT in promoting teaching and learning. Soc Sci Res Netw. 2023. https://doi.org/10.2139/ssrn.4337484 .

Basic Z, Banovac A, Kruzic I, Jerkovic I. Better by you, better than me, chatgpt3 as writing assistance in students essays. 2023. arXiv preprint arXiv:2302.04536 .‏

Baskara FR. The promises and pitfalls of using chat GPT for self-determined learning in higher education: an argumentative review. Prosiding Seminar Nasional Fakultas Tarbiyah dan Ilmu Keguruan IAIM Sinjai. 2023;2:95–101. https://doi.org/10.47435/sentikjar.v2i0.1825 .

Behera RK, Bala PK, Dhir A. The emerging role of cognitive computing in healthcare: a systematic literature review. Int J Med Inform. 2019;129:154–66. https://doi.org/10.1016/j.ijmedinf.2019.04.024 .

Chaka C. Detecting AI content in responses generated by ChatGPT, YouChat, and Chatsonic: the case of five AI content detection tools. J Appl Learn Teach. 2023. https://doi.org/10.37074/jalt.2023.6.2.12 .

Chiu TKF, Xia Q, Zhou X, Chai CS, Cheng M. Systematic literature review on opportunities, challenges, and future research recommendations of artificial intelligence in education. Comput Educ Artif Intell. 2023;4:100118. https://doi.org/10.1016/j.caeai.2022.100118 .

Choi EPH, Lee JJ, Ho M, Kwok JYY, Lok KYW. Chatting or cheating? The impacts of ChatGPT and other artificial intelligence language models on nurse education. Nurse Educ Today. 2023;125:105796. https://doi.org/10.1016/j.nedt.2023.105796 .

Cotton D, Cotton PA, Shipway JR. Chatting and cheating: ensuring academic integrity in the era of ChatGPT. Innov Educ Teach Int. 2023. https://doi.org/10.1080/14703297.2023.2190148 .

Crawford J, Cowling M, Allen K. Leadership is needed for ethical ChatGPT: Character, assessment, and learning using artificial intelligence (AI). J Univ Teach Learn Pract. 2023. https://doi.org/10.53761/1.20.3.02 .

Creswell JW. Educational research: planning, conducting, and evaluating quantitative and qualitative research [Ebook]. 4th ed. London: Pearson Education; 2015.

Curry D. ChatGPT Revenue and Usage Statistics (2023)—Business of Apps. 2023. https://www.businessofapps.com/data/chatgpt-statistics/

Day T. A preliminary investigation of fake peer-reviewed citations and references generated by ChatGPT. Prof Geogr. 2023. https://doi.org/10.1080/00330124.2023.2190373 .

De Castro CA. A Discussion about the Impact of ChatGPT in education: benefits and concerns. J Bus Theor Pract. 2023;11(2):p28. https://doi.org/10.22158/jbtp.v11n2p28 .

Deng X, Yu Z. A meta-analysis and systematic review of the effect of Chatbot technology use in sustainable education. Sustainability. 2023;15(4):2940. https://doi.org/10.3390/su15042940 .

Eke DO. ChatGPT and the rise of generative AI: threat to academic integrity? J Responsib Technol. 2023;13:100060. https://doi.org/10.1016/j.jrt.2023.100060 .

Elmoazen R, Saqr M, Tedre M, Hirsto L. A systematic literature review of empirical research on epistemic network analysis in education. IEEE Access. 2022;10:17330–48. https://doi.org/10.1109/access.2022.3149812 .

Farrokhnia M, Banihashem SK, Noroozi O, Wals AEJ. A SWOT analysis of ChatGPT: implications for educational practice and research. Innov Educ Teach Int. 2023. https://doi.org/10.1080/14703297.2023.2195846 .

Fergus S, Botha M, Ostovar M. Evaluating academic answers generated using ChatGPT. J Chem Educ. 2023;100(4):1672–5. https://doi.org/10.1021/acs.jchemed.3c00087 .

Fink A. Conducting research literature reviews: from the Internet to Paper. Incorporated: SAGE Publications; 2010.

Firaina R, Sulisworo D. Exploring the usage of ChatGPT in higher education: frequency and impact on productivity. Buletin Edukasi Indonesia (BEI). 2023;2(01):39–46. https://doi.org/10.56741/bei.v2i01.310 .

Firat, M. (2023). How chat GPT can transform autodidactic experiences and open education.  Department of Distance Education, Open Education Faculty, Anadolu Unive .‏ https://orcid.org/0000-0001-8707-5918

Firat M. What ChatGPT means for universities: perceptions of scholars and students. J Appl Learn Teach. 2023. https://doi.org/10.37074/jalt.2023.6.1.22 .

Fuchs K. Exploring the opportunities and challenges of NLP models in higher education: is Chat GPT a blessing or a curse? Front Educ. 2023. https://doi.org/10.3389/feduc.2023.1166682 .

García-Peñalvo FJ. La percepción de la inteligencia artificial en contextos educativos tras el lanzamiento de ChatGPT: disrupción o pánico. Educ Knowl Soc. 2023;24: e31279. https://doi.org/10.14201/eks.31279 .

Gilson A, Safranek CW, Huang T, Socrates V, Chi L, Taylor A, Chartash D. How does ChatGPT perform on the United States medical Licensing examination? The implications of large language models for medical education and knowledge assessment. JMIR Med Educ. 2023;9: e45312. https://doi.org/10.2196/45312 .

Hashana AJ, Brundha P, Ayoobkhan MUA, Fazila S. Deep Learning in ChatGPT—A Survey. In   2023 7th international conference on trends in electronics and informatics (ICOEI) . 2023. (pp. 1001–1005). IEEE. https://doi.org/10.1109/icoei56765.2023.10125852

Hirsto L, Saqr M, López-Pernas S, Valtonen T. (2022). A systematic narrative review of learning analytics research in K-12 and schools.  Proceedings . https://ceur-ws.org/Vol-3383/FLAIEC22_paper_9536.pdf

Hisan UK, Amri MM. ChatGPT and medical education: a double-edged sword. J Pedag Educ Sci. 2023;2(01):71–89. https://doi.org/10.13140/RG.2.2.31280.23043/1 .

Hopkins AM, Logan JM, Kichenadasse G, Sorich MJ. Artificial intelligence chatbots will revolutionize how cancer patients access information: ChatGPT represents a paradigm-shift. JNCI Cancer Spectr. 2023. https://doi.org/10.1093/jncics/pkad010 .

Househ M, AlSaad R, Alhuwail D, Ahmed A, Healy MG, Latifi S, Sheikh J. Large Language models in medical education: opportunities, challenges, and future directions. JMIR Med Educ. 2023;9: e48291. https://doi.org/10.2196/48291 .

Ilkka T. The impact of artificial intelligence on learning, teaching, and education. Minist de Educ. 2018. https://doi.org/10.2760/12297 .

Iqbal N, Ahmed H, Azhar KA. Exploring teachers’ attitudes towards using CHATGPT. Globa J Manag Adm Sci. 2022;3(4):97–111. https://doi.org/10.46568/gjmas.v3i4.163 .

Irfan M, Murray L, Ali S. Integration of Artificial intelligence in academia: a case study of critical teaching and learning in Higher education. Globa Soc Sci Rev. 2023;8(1):352–64. https://doi.org/10.31703/gssr.2023(viii-i).32 .

Jeon JH, Lee S. Large language models in education: a focus on the complementary relationship between human teachers and ChatGPT. Educ Inf Technol. 2023. https://doi.org/10.1007/s10639-023-11834-1 .

Khan RA, Jawaid M, Khan AR, Sajjad M. ChatGPT—Reshaping medical education and clinical management. Pak J Med Sci. 2023. https://doi.org/10.12669/pjms.39.2.7653 .

King MR. A conversation on artificial intelligence, Chatbots, and plagiarism in higher education. Cell Mol Bioeng. 2023;16(1):1–2. https://doi.org/10.1007/s12195-022-00754-8 .

Kooli C. Chatbots in education and research: a critical examination of ethical implications and solutions. Sustainability. 2023;15(7):5614. https://doi.org/10.3390/su15075614 .

Kuhail MA, Alturki N, Alramlawi S, Alhejori K. Interacting with educational chatbots: a systematic review. Educ Inf Technol. 2022;28(1):973–1018. https://doi.org/10.1007/s10639-022-11177-3 .

Lee H. The rise of ChatGPT: exploring its potential in medical education. Anat Sci Educ. 2023. https://doi.org/10.1002/ase.2270 .

Li L, Subbareddy R, Raghavendra CG. AI intelligence Chatbot to improve students learning in the higher education platform. J Interconnect Netw. 2022. https://doi.org/10.1142/s0219265921430325 .

Limna P. A Review of Artificial Intelligence (AI) in Education during the Digital Era. 2022. https://ssrn.com/abstract=4160798

Lo CK. What is the impact of ChatGPT on education? A rapid review of the literature. Educ Sci. 2023;13(4):410. https://doi.org/10.3390/educsci13040410 .

Luo W, He H, Liu J, Berson IR, Berson MJ, Zhou Y, Li H. Aladdin’s genie or pandora’s box For early childhood education? Experts chat on the roles, challenges, and developments of ChatGPT. Early Educ Dev. 2023. https://doi.org/10.1080/10409289.2023.2214181 .

Meyer JG, Urbanowicz RJ, Martin P, O’Connor K, Li R, Peng P, Moore JH. ChatGPT and large language models in academia: opportunities and challenges. Biodata Min. 2023. https://doi.org/10.1186/s13040-023-00339-9 .

Mhlanga D. Open AI in education, the responsible and ethical use of ChatGPT towards lifelong learning. Soc Sci Res Netw. 2023. https://doi.org/10.2139/ssrn.4354422 .

Neumann, M., Rauschenberger, M., & Schön, E. M. (2023). “We Need To Talk About ChatGPT”: The Future of AI and Higher Education.‏ https://doi.org/10.1109/seeng59157.2023.00010

Nolan B. Here are the schools and colleges that have banned the use of ChatGPT over plagiarism and misinformation fears. Business Insider . 2023. https://www.businessinsider.com

O’Leary DE. An analysis of three chatbots: BlenderBot, ChatGPT and LaMDA. Int J Intell Syst Account, Financ Manag. 2023;30(1):41–54. https://doi.org/10.1002/isaf.1531 .

Okoli C. A guide to conducting a standalone systematic literature review. Commun Assoc Inf Syst. 2015. https://doi.org/10.17705/1cais.03743 .

OpenAI. (2023). https://openai.com/blog/chatgpt

Perkins M. Academic integrity considerations of AI large language models in the post-pandemic era: ChatGPT and beyond. J Univ Teach Learn Pract. 2023. https://doi.org/10.53761/1.20.02.07 .

Plevris V, Papazafeiropoulos G, Rios AJ. Chatbots put to the test in math and logic problems: A preliminary comparison and assessment of ChatGPT-3.5, ChatGPT-4, and Google Bard. arXiv (Cornell University) . 2023. https://doi.org/10.48550/arxiv.2305.18618

Rahman MM, Watanobe Y (2023) ChatGPT for education and research: opportunities, threats, and strategies. Appl Sci 13(9):5783. https://doi.org/10.3390/app13095783

Ram B, Verma P. Artificial intelligence AI-based Chatbot study of ChatGPT, google AI bard and baidu AI. World J Adv Eng Technol Sci. 2023;8(1):258–61. https://doi.org/10.30574/wjaets.2023.8.1.0045 .

Rasul T, Nair S, Kalendra D, Robin M, de Oliveira Santini F, Ladeira WJ, Heathcote L. The role of ChatGPT in higher education: benefits, challenges, and future research directions. J Appl Learn Teach. 2023. https://doi.org/10.37074/jalt.2023.6.1.29 .

Ratnam M, Sharm B, Tomer A. ChatGPT: educational artificial intelligence. Int J Adv Trends Comput Sci Eng. 2023;12(2):84–91. https://doi.org/10.30534/ijatcse/2023/091222023 .

Ray PP. ChatGPT: a comprehensive review on background, applications, key challenges, bias, ethics, limitations and future scope. Internet Things Cyber-Phys Syst. 2023;3:121–54. https://doi.org/10.1016/j.iotcps.2023.04.003 .

Roumeliotis KI, Tselikas ND. ChatGPT and Open-AI models: a preliminary review. Future Internet. 2023;15(6):192. https://doi.org/10.3390/fi15060192 .

Rudolph J, Tan S, Tan S. War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education. J Appl Learn Teach. 2023. https://doi.org/10.37074/jalt.2023.6.1.23 .

Ruiz LMS, Moll-López S, Nuñez-Pérez A, Moraño J, Vega-Fleitas E. ChatGPT challenges blended learning methodologies in engineering education: a case study in mathematics. Appl Sci. 2023;13(10):6039. https://doi.org/10.3390/app13106039 .

Sallam M, Salim NA, Barakat M, Al-Tammemi AB. ChatGPT applications in medical, dental, pharmacy, and public health education: a descriptive study highlighting the advantages and limitations. Narra J. 2023;3(1): e103. https://doi.org/10.52225/narra.v3i1.103 .

Salvagno M, Taccone FS, Gerli AG. Can artificial intelligence help for scientific writing? Crit Care. 2023. https://doi.org/10.1186/s13054-023-04380-2 .

Saqr M, López-Pernas S, Helske S, Hrastinski S. The longitudinal association between engagement and achievement varies by time, students’ profiles, and achievement state: a full program study. Comput Educ. 2023;199:104787. https://doi.org/10.1016/j.compedu.2023.104787 .

Saqr M, Matcha W, Uzir N, Jovanović J, Gašević D, López-Pernas S. Transferring effective learning strategies across learning contexts matters: a study in problem-based learning. Australas J Educ Technol. 2023;39(3):9.

Schöbel S, Schmitt A, Benner D, Saqr M, Janson A, Leimeister JM. Charting the evolution and future of conversational agents: a research agenda along five waves and new frontiers. Inf Syst Front. 2023. https://doi.org/10.1007/s10796-023-10375-9 .

Shoufan A. Exploring students’ perceptions of CHATGPT: thematic analysis and follow-up survey. IEEE Access. 2023. https://doi.org/10.1109/access.2023.3268224 .

Sonderegger S, Seufert S. Chatbot-mediated learning: conceptual framework for the design of Chatbot use cases in education. Gallen: Institute for Educational Management and Technologies, University of St; 2022. https://doi.org/10.5220/0010999200003182 .

Book   Google Scholar  

Strzelecki A. To use or not to use ChatGPT in higher education? A study of students’ acceptance and use of technology. Interact Learn Environ. 2023. https://doi.org/10.1080/10494820.2023.2209881 .

Su J, Yang W. Unlocking the power of ChatGPT: a framework for applying generative AI in education. ECNU Rev Educ. 2023. https://doi.org/10.1177/20965311231168423 .

Sullivan M, Kelly A, McLaughlan P. ChatGPT in higher education: Considerations for academic integrity and student learning. J ApplLearn Teach. 2023;6(1):1–10. https://doi.org/10.37074/jalt.2023.6.1.17 .

Szabo A. ChatGPT is a breakthrough in science and education but fails a test in sports and exercise psychology. Balt J Sport Health Sci. 2023;1(128):25–40. https://doi.org/10.33607/bjshs.v127i4.1233 .

Taecharungroj V. “What can ChatGPT do?” analyzing early reactions to the innovative AI chatbot on Twitter. Big Data Cognit Comput. 2023;7(1):35. https://doi.org/10.3390/bdcc7010035 .

Tam S, Said RB. User preferences for ChatGPT-powered conversational interfaces versus traditional methods. Biomed Eng Soc. 2023. https://doi.org/10.58496/mjcsc/2023/004 .

Tedre M, Kahila J, Vartiainen H. (2023). Exploration on how co-designing with AI facilitates critical evaluation of ethics of AI in craft education. In: Langran E, Christensen P, Sanson J (Eds).  Proceedings of Society for Information Technology and Teacher Education International Conference . 2023. pp. 2289–2296.

Tlili A, Shehata B, Adarkwah MA, Bozkurt A, Hickey DT, Huang R, Agyemang B. What if the devil is my guardian angel: ChatGPT as a case study of using chatbots in education. Smart Learn Environ. 2023. https://doi.org/10.1186/s40561-023-00237-x .

Uddin SMJ, Albert A, Ovid A, Alsharef A. Leveraging CHATGPT to aid construction hazard recognition and support safety education and training. Sustainability. 2023;15(9):7121. https://doi.org/10.3390/su15097121 .

Valtonen T, López-Pernas S, Saqr M, Vartiainen H, Sointu E, Tedre M. The nature and building blocks of educational technology research. Comput Hum Behav. 2022;128:107123. https://doi.org/10.1016/j.chb.2021.107123 .

Vartiainen H, Tedre M. Using artificial intelligence in craft education: crafting with text-to-image generative models. Digit Creat. 2023;34(1):1–21. https://doi.org/10.1080/14626268.2023.2174557 .

Ventayen RJM. OpenAI ChatGPT generated results: similarity index of artificial intelligence-based contents. Soc Sci Res Netw. 2023. https://doi.org/10.2139/ssrn.4332664 .

Wagner MW, Ertl-Wagner BB. Accuracy of information and references using ChatGPT-3 for retrieval of clinical radiological information. Can Assoc Radiol J. 2023. https://doi.org/10.1177/08465371231171125 .

Wardat Y, Tashtoush MA, AlAli R, Jarrah AM. ChatGPT: a revolutionary tool for teaching and learning mathematics. Eurasia J Math, Sci Technol Educ. 2023;19(7):em2286. https://doi.org/10.29333/ejmste/13272 .

Webster J, Watson RT. Analyzing the past to prepare for the future: writing a literature review. Manag Inf Syst Quart. 2002;26(2):3.

Xiao Y, Watson ME. Guidance on conducting a systematic literature review. J Plan Educ Res. 2017;39(1):93–112. https://doi.org/10.1177/0739456x17723971 .

Yan D. Impact of ChatGPT on learners in a L2 writing practicum: an exploratory investigation. Educ Inf Technol. 2023. https://doi.org/10.1007/s10639-023-11742-4 .

Yu H. Reflection on whether Chat GPT should be banned by academia from the perspective of education and teaching. Front Psychol. 2023;14:1181712. https://doi.org/10.3389/fpsyg.2023.1181712 .

Zhu C, Sun M, Luo J, Li T, Wang M. How to harness the potential of ChatGPT in education? Knowl Manag ELearn. 2023;15(2):133–52. https://doi.org/10.34105/j.kmel.2023.15.008 .

Download references

The paper is co-funded by the Academy of Finland (Suomen Akatemia) Research Council for Natural Sciences and Engineering for the project Towards precision education: Idiographic learning analytics (TOPEILA), Decision Number 350560.

Author information

Authors and affiliations.

School of Computing, University of Eastern Finland, 80100, Joensuu, Finland

Yazid Albadarin, Mohammed Saqr, Nicolas Pope & Markku Tukiainen

You can also search for this author in PubMed   Google Scholar

Contributions

YA contributed to the literature search, data analysis, discussion, and conclusion. Additionally, YA contributed to the manuscript’s writing, editing, and finalization. MS contributed to the study’s design, conceptualization, acquisition of funding, project administration, allocation of resources, supervision, validation, literature search, and analysis of results. Furthermore, MS contributed to the manuscript's writing, revising, and approving it in its finalized state. NP contributed to the results, and discussions, and provided supervision. NP also contributed to the writing process, revisions, and the final approval of the manuscript in its finalized state. MT contributed to the study's conceptualization, resource management, supervision, writing, revising the manuscript, and approving it.

Corresponding author

Correspondence to Yazid Albadarin .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

See Table  4

The process of synthesizing the data presented in Table  4 involved identifying the relevant studies through a search process of databases (ERIC, Scopus, Web of Knowledge, Dimensions.ai, and lens.org) using specific keywords "ChatGPT" and "education". Following this, inclusion/exclusion criteria were applied, and data extraction was performed using Creswell's [ 15 ] coding techniques to capture key information and identify common themes across the included studies.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Albadarin, Y., Saqr, M., Pope, N. et al. A systematic literature review of empirical research on ChatGPT in education. Discov Educ 3 , 60 (2024). https://doi.org/10.1007/s44217-024-00138-2

Download citation

Received : 22 October 2023

Accepted : 10 May 2024

Published : 26 May 2024

DOI : https://doi.org/10.1007/s44217-024-00138-2

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Large language models
• Educational technology
• Systematic review

Advertisement

• Find a journal
• Publish with us
• Track your research