impact of digital technology on education research paper

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Springer Nature - PMC COVID-19 Collection

Impacts of digital technologies on education and factors influencing schools' digital capacity and transformation: A literature review

Stella timotheou.

1 CYENS Center of Excellence & Cyprus University of Technology (Cyprus Interaction Lab), Cyprus, CYENS Center of Excellence & Cyprus University of Technology, Nicosia-Limassol, Cyprus

Ourania Miliou

Yiannis dimitriadis.

2 Universidad de Valladolid (UVA), Spain, Valladolid, Spain

Sara Villagrá Sobrino

Nikoleta giannoutsou, romina cachia.

3 JRC - Joint Research Centre of the European Commission, Seville, Spain

Alejandra Martínez Monés

Andri ioannou, associated data.

Data sharing not applicable to this article as no datasets were generated or analysed during the current study.

Digital technologies have brought changes to the nature and scope of education and led education systems worldwide to adopt strategies and policies for ICT integration. The latter brought about issues regarding the quality of teaching and learning with ICTs, especially concerning the understanding, adaptation, and design of the education systems in accordance with current technological trends. These issues were emphasized during the recent COVID-19 pandemic that accelerated the use of digital technologies in education, generating questions regarding digitalization in schools. Specifically, many schools demonstrated a lack of experience and low digital capacity, which resulted in widening gaps, inequalities, and learning losses. Such results have engendered the need for schools to learn and build upon the experience to enhance their digital capacity and preparedness, increase their digitalization levels, and achieve a successful digital transformation. Given that the integration of digital technologies is a complex and continuous process that impacts different actors within the school ecosystem, there is a need to show how these impacts are interconnected and identify the factors that can encourage an effective and efficient change in the school environments. For this purpose, we conducted a non-systematic literature review. The results of the literature review were organized thematically based on the evidence presented about the impact of digital technology on education and the factors that affect the schools’ digital capacity and digital transformation. The findings suggest that ICT integration in schools impacts more than just students’ performance; it affects several other school-related aspects and stakeholders, too. Furthermore, various factors affect the impact of digital technologies on education. These factors are interconnected and play a vital role in the digital transformation process. The study results shed light on how ICTs can positively contribute to the digital transformation of schools and which factors should be considered for schools to achieve effective and efficient change.

Introduction

Digital technologies have brought changes to the nature and scope of education. Versatile and disruptive technological innovations, such as smart devices, the Internet of Things (IoT), artificial intelligence (AI), augmented reality (AR) and virtual reality (VR), blockchain, and software applications have opened up new opportunities for advancing teaching and learning (Gaol & Prasolova-Førland, 2021 ; OECD, 2021 ). Hence, in recent years, education systems worldwide have increased their investment in the integration of information and communication technology (ICT) (Fernández-Gutiérrez et al., 2020 ; Lawrence & Tar, 2018 ) and prioritized their educational agendas to adapt strategies or policies around ICT integration (European Commission, 2019 ). The latter brought about issues regarding the quality of teaching and learning with ICTs (Bates, 2015 ), especially concerning the understanding, adaptation, and design of education systems in accordance with current technological trends (Balyer & Öz, 2018 ). Studies have shown that despite the investment made in the integration of technology in schools, the results have not been promising, and the intended outcomes have not yet been achieved (Delgado et al., 2015 ; Lawrence & Tar, 2018 ). These issues were exacerbated during the COVID-19 pandemic, which forced teaching across education levels to move online (Daniel, 2020 ). Online teaching accelerated the use of digital technologies generating questions regarding the process, the nature, the extent, and the effectiveness of digitalization in schools (Cachia et al., 2021 ; König et al., 2020 ). Specifically, many schools demonstrated a lack of experience and low digital capacity, which resulted in widening gaps, inequalities, and learning losses (Blaskó et al., 2021 ; Di Pietro et al, 2020 ). Such results have engendered the need for schools to learn and build upon the experience in order to enhance their digital capacity (European Commission, 2020 ) and increase their digitalization levels (Costa et al., 2021 ). Digitalization offers possibilities for fundamental improvement in schools (OECD, 2021 ; Rott & Marouane, 2018 ) and touches many aspects of a school’s development (Delcker & Ifenthaler, 2021 ) . However, it is a complex process that requires large-scale transformative changes beyond the technical aspects of technology and infrastructure (Pettersson, 2021 ). Namely, digitalization refers to “ a series of deep and coordinated culture, workforce, and technology shifts and operating models ” (Brooks & McCormack, 2020 , p. 3) that brings cultural, organizational, and operational change through the integration of digital technologies (JISC, 2020 ). A successful digital transformation requires that schools increase their digital capacity levels, establishing the necessary “ culture, policies, infrastructure as well as digital competence of students and staff to support the effective integration of technology in teaching and learning practices ” (Costa et al, 2021 , p.163).

Given that the integration of digital technologies is a complex and continuous process that impacts different actors within the school ecosystem (Eng, 2005 ), there is a need to show how the different elements of the impact are interconnected and to identify the factors that can encourage an effective and efficient change in the school environment. To address the issues outlined above, we formulated the following research questions:

a) What is the impact of digital technologies on education?

b) Which factors might affect a school’s digital capacity and transformation?

In the present investigation, we conducted a non-systematic literature review of publications pertaining to the impact of digital technologies on education and the factors that affect a school’s digital capacity and transformation. The results of the literature review were organized thematically based on the evidence presented about the impact of digital technology on education and the factors which affect the schools’ digital capacity and digital transformation.

Methodology

The non-systematic literature review presented herein covers the main theories and research published over the past 17 years on the topic. It is based on meta-analyses and review papers found in scholarly, peer-reviewed content databases and other key studies and reports related to the concepts studied (e.g., digitalization, digital capacity) from professional and international bodies (e.g., the OECD). We searched the Scopus database, which indexes various online journals in the education sector with an international scope, to collect peer-reviewed academic papers. Furthermore, we used an all-inclusive Google Scholar search to include relevant key terms or to include studies found in the reference list of the peer-reviewed papers, and other key studies and reports related to the concepts studied by professional and international bodies. Lastly, we gathered sources from the Publications Office of the European Union ( https://op.europa.eu/en/home ); namely, documents that refer to policies related to digital transformation in education.

Regarding search terms, we first searched resources on the impact of digital technologies on education by performing the following search queries: “impact” OR “effects” AND “digital technologies” AND “education”, “impact” OR “effects” AND “ICT” AND “education”. We further refined our results by adding the terms “meta-analysis” and “review” or by adjusting the search options based on the features of each database to avoid collecting individual studies that would provide limited contributions to a particular domain. We relied on meta-analyses and review studies as these consider the findings of multiple studies to offer a more comprehensive view of the research in a given area (Schuele & Justice, 2006 ). Specifically, meta-analysis studies provided quantitative evidence based on statistically verifiable results regarding the impact of educational interventions that integrate digital technologies in school classrooms (Higgins et al., 2012 ; Tolani-Brown et al., 2011 ).

However, quantitative data does not offer explanations for the challenges or difficulties experienced during ICT integration in learning and teaching (Tolani-Brown et al., 2011 ). To fill this gap, we analyzed literature reviews and gathered in-depth qualitative evidence of the benefits and implications of technology integration in schools. In the analysis presented herein, we also included policy documents and reports from professional and international bodies and governmental reports, which offered useful explanations of the key concepts of this study and provided recent evidence on digital capacity and transformation in education along with policy recommendations. The inclusion and exclusion criteria that were considered in this study are presented in Table ​ Table1 1 .

Inclusion and exclusion criteria for the selection of resources on the impact of digital technologies on education

To ensure a reliable extraction of information from each study and assist the research synthesis we selected the study characteristics of interest (impact) and constructed coding forms. First, an overview of the synthesis was provided by the principal investigator who described the processes of coding, data entry, and data management. The coders followed the same set of instructions but worked independently. To ensure a common understanding of the process between coders, a sample of ten studies was tested. The results were compared, and the discrepancies were identified and resolved. Additionally, to ensure an efficient coding process, all coders participated in group meetings to discuss additions, deletions, and modifications (Stock, 1994 ). Due to the methodological diversity of the studied documents we began to synthesize the literature review findings based on similar study designs. Specifically, most of the meta-analysis studies were grouped in one category due to the quantitative nature of the measured impact. These studies tended to refer to student achievement (Hattie et al., 2014 ). Then, we organized the themes of the qualitative studies in several impact categories. Lastly, we synthesized both review and meta-analysis data across the categories. In order to establish a collective understanding of the concept of impact, we referred to a previous impact study by Balanskat ( 2009 ) which investigated the impact of technology in primary schools. In this context, the impact had a more specific ICT-related meaning and was described as “ a significant influence or effect of ICT on the measured or perceived quality of (parts of) education ” (Balanskat, 2009 , p. 9). In the study presented herein, the main impacts are in relation to learning and learners, teaching, and teachers, as well as other key stakeholders who are directly or indirectly connected to the school unit.

The study’s results identified multiple dimensions of the impact of digital technologies on students’ knowledge, skills, and attitudes; on equality, inclusion, and social integration; on teachers’ professional and teaching practices; and on other school-related aspects and stakeholders. The data analysis indicated various factors that might affect the schools’ digital capacity and transformation, such as digital competencies, the teachers’ personal characteristics and professional development, as well as the school’s leadership and management, administration, infrastructure, etc. The impacts and factors found in the literature review are presented below.

Impacts of digital technologies on students’ knowledge, skills, attitudes, and emotions

The impact of ICT use on students’ knowledge, skills, and attitudes has been investigated early in the literature. Eng ( 2005 ) found a small positive effect between ICT use and students' learning. Specifically, the author reported that access to computer-assisted instruction (CAI) programs in simulation or tutorial modes—used to supplement rather than substitute instruction – could enhance student learning. The author reported studies showing that teachers acknowledged the benefits of ICT on pupils with special educational needs; however, the impact of ICT on students' attainment was unclear. Balanskat et al. ( 2006 ) found a statistically significant positive association between ICT use and higher student achievement in primary and secondary education. The authors also reported improvements in the performance of low-achieving pupils. The use of ICT resulted in further positive gains for students, namely increased attention, engagement, motivation, communication and process skills, teamwork, and gains related to their behaviour towards learning. Evidence from qualitative studies showed that teachers, students, and parents recognized the positive impact of ICT on students' learning regardless of their competence level (strong/weak students). Punie et al. ( 2006 ) documented studies that showed positive results of ICT-based learning for supporting low-achieving pupils and young people with complex lives outside the education system. Liao et al. ( 2007 ) reported moderate positive effects of computer application instruction (CAI, computer simulations, and web-based learning) over traditional instruction on primary school student's achievement. Similarly, Tamim et al. ( 2011 ) reported small to moderate positive effects between the use of computer technology (CAI, ICT, simulations, computer-based instruction, digital and hypermedia) and student achievement in formal face-to-face classrooms compared to classrooms that did not use technology. Jewitt et al., ( 2011 ) found that the use of learning platforms (LPs) (virtual learning environments, management information systems, communication technologies, and information- and resource-sharing technologies) in schools allowed primary and secondary students to access a wider variety of quality learning resources, engage in independent and personalized learning, and conduct self- and peer-review; LPs also provide opportunities for teacher assessment and feedback. Similar findings were reported by Fu ( 2013 ), who documented a list of benefits and opportunities of ICT use. According to the author, the use of ICTs helps students access digital information and course content effectively and efficiently, supports student-centered and self-directed learning, as well as the development of a creative learning environment where more opportunities for critical thinking skills are offered, and promotes collaborative learning in a distance-learning environment. Higgins et al. ( 2012 ) found consistent but small positive associations between the use of technology and learning outcomes of school-age learners (5–18-year-olds) in studies linking the provision and use of technology with attainment. Additionally, Chauhan ( 2017 ) reported a medium positive effect of technology on the learning effectiveness of primary school students compared to students who followed traditional learning instruction.

The rise of mobile technologies and hardware devices instigated investigations into their impact on teaching and learning. Sung et al. ( 2016 ) reported a moderate effect on students' performance from the use of mobile devices in the classroom compared to the use of desktop computers or the non-use of mobile devices. Schmid et al. ( 2014 ) reported medium–low to low positive effects of technology integration (e.g., CAI, ICTs) in the classroom on students' achievement and attitude compared to not using technology or using technology to varying degrees. Tamim et al. ( 2015 ) found a low statistically significant effect of the use of tablets and other smart devices in educational contexts on students' achievement outcomes. The authors suggested that tablets offered additional advantages to students; namely, they reported improvements in students’ notetaking, organizational and communication skills, and creativity. Zheng et al. ( 2016 ) reported a small positive effect of one-to-one laptop programs on students’ academic achievement across subject areas. Additional reported benefits included student-centered, individualized, and project-based learning enhanced learner engagement and enthusiasm. Additionally, the authors found that students using one-to-one laptop programs tended to use technology more frequently than in non-laptop classrooms, and as a result, they developed a range of skills (e.g., information skills, media skills, technology skills, organizational skills). Haßler et al. ( 2016 ) found that most interventions that included the use of tablets across the curriculum reported positive learning outcomes. However, from 23 studies, five reported no differences, and two reported a negative effect on students' learning outcomes. Similar results were indicated by Kalati and Kim ( 2022 ) who investigated the effect of touchscreen technologies on young students’ learning. Specifically, from 53 studies, 34 advocated positive effects of touchscreen devices on children’s learning, 17 obtained mixed findings and two studies reported negative effects.

More recently, approaches that refer to the impact of gamification with the use of digital technologies on teaching and learning were also explored. A review by Pan et al. ( 2022 ) that examined the role of learning games in fostering mathematics education in K-12 settings, reported that gameplay improved students’ performance. Integration of digital games in teaching was also found as a promising pedagogical practice in STEM education that could lead to increased learning gains (Martinez et al., 2022 ; Wang et al., 2022 ). However, although Talan et al. ( 2020 ) reported a medium effect of the use of educational games (both digital and non-digital) on academic achievement, the effect of non-digital games was higher.

Over the last two years, the effects of more advanced technologies on teaching and learning were also investigated. Garzón and Acevedo ( 2019 ) found that AR applications had a medium effect on students' learning outcomes compared to traditional lectures. Similarly, Garzón et al. ( 2020 ) showed that AR had a medium impact on students' learning gains. VR applications integrated into various subjects were also found to have a moderate effect on students’ learning compared to control conditions (traditional classes, e.g., lectures, textbooks, and multimedia use, e.g., images, videos, animation, CAI) (Chen et al., 2022b ). Villena-Taranilla et al. ( 2022 ) noted the moderate effect of VR technologies on students’ learning when these were applied in STEM disciplines. In the same meta-analysis, Villena-Taranilla et al. ( 2022 ) highlighted the role of immersive VR, since its effect on students’ learning was greater (at a high level) across educational levels (K-6) compared to semi-immersive and non-immersive integrations. In another meta-analysis study, the effect size of the immersive VR was small and significantly differentiated across educational levels (Coban et al., 2022 ). The impact of AI on education was investigated by Su and Yang ( 2022 ) and Su et al. ( 2022 ), who showed that this technology significantly improved students’ understanding of AI computer science and machine learning concepts.

It is worth noting that the vast majority of studies referred to learning gains in specific subjects. Specifically, several studies examined the impact of digital technologies on students’ literacy skills and reported positive effects on language learning (Balanskat et al., 2006 ; Grgurović et al., 2013 ; Friedel et al., 2013 ; Zheng et al., 2016 ; Chen et al., 2022b ; Savva et al., 2022 ). Also, several studies documented positive effects on specific language learning areas, namely foreign language learning (Kao, 2014 ), writing (Higgins et al., 2012 ; Wen & Walters, 2022 ; Zheng et al., 2016 ), as well as reading and comprehension (Cheung & Slavin, 2011 ; Liao et al., 2007 ; Schwabe et al., 2022 ). ICTs were also found to have a positive impact on students' performance in STEM (science, technology, engineering, and mathematics) disciplines (Arztmann et al., 2022 ; Bado, 2022 ; Villena-Taranilla et al., 2022 ; Wang et al., 2022 ). Specifically, a number of studies reported positive impacts on students’ achievement in mathematics (Balanskat et al., 2006 ; Hillmayr et al., 2020 ; Li & Ma, 2010 ; Pan et al., 2022 ; Ran et al., 2022 ; Verschaffel et al., 2019 ; Zheng et al., 2016 ). Furthermore, studies documented positive effects of ICTs on science learning (Balanskat et al., 2006 ; Liao et al., 2007 ; Zheng et al., 2016 ; Hillmayr et al., 2020 ; Kalemkuş & Kalemkuş, 2022 ; Lei et al., 2022a ). Çelik ( 2022 ) also noted that computer simulations can help students understand learning concepts related to science. Furthermore, some studies documented that the use of ICTs had a positive impact on students’ achievement in other subjects, such as geography, history, music, and arts (Chauhan, 2017 ; Condie & Munro, 2007 ), and design and technology (Balanskat et al., 2006 ).

More specific positive learning gains were reported in a number of skills, e.g., problem-solving skills and pattern exploration skills (Higgins et al., 2012 ), metacognitive learning outcomes (Verschaffel et al., 2019 ), literacy skills, computational thinking skills, emotion control skills, and collaborative inquiry skills (Lu et al., 2022 ; Su & Yang, 2022 ; Su et al., 2022 ). Additionally, several investigations have reported benefits from the use of ICT on students’ creativity (Fielding & Murcia, 2022 ; Liu et al., 2022 ; Quah & Ng, 2022 ). Lastly, digital technologies were also found to be beneficial for enhancing students’ lifelong learning skills (Haleem et al., 2022 ).

Apart from gaining knowledge and skills, studies also reported improvement in motivation and interest in mathematics (Higgins et. al., 2019 ; Fadda et al., 2022 ) and increased positive achievement emotions towards several subjects during interventions using educational games (Lei et al., 2022a ). Chen et al. ( 2022a ) also reported a small but positive effect of digital health approaches in bullying and cyberbullying interventions with K-12 students, demonstrating that technology-based approaches can help reduce bullying and related consequences by providing emotional support, empowerment, and change of attitude. In their meta-review study, Su et al. ( 2022 ) also documented that AI technologies effectively strengthened students’ attitudes towards learning. In another meta-analysis, Arztmann et al. ( 2022 ) reported positive effects of digital games on motivation and behaviour towards STEM subjects.

Impacts of digital technologies on equality, inclusion and social integration

Although most of the reviewed studies focused on the impact of ICTs on students’ knowledge, skills, and attitudes, reports were also made on other aspects in the school context, such as equality, inclusion, and social integration. Condie and Munro ( 2007 ) documented research interventions investigating how ICT can support pupils with additional or special educational needs. While those interventions were relatively small scale and mostly based on qualitative data, their findings indicated that the use of ICTs enabled the development of communication, participation, and self-esteem. A recent meta-analysis (Baragash et al., 2022 ) with 119 participants with different disabilities, reported a significant overall effect size of AR on their functional skills acquisition. Koh’s meta-analysis ( 2022 ) also revealed that students with intellectual and developmental disabilities improved their competence and performance when they used digital games in the lessons.

Istenic Starcic and Bagon ( 2014 ) found that the role of ICT in inclusion and the design of pedagogical and technological interventions was not sufficiently explored in educational interventions with people with special needs; however, some benefits of ICT use were found in students’ social integration. The issue of gender and technology use was mentioned in a small number of studies. Zheng et al. ( 2016 ) reported a statistically significant positive interaction between one-to-one laptop programs and gender. Specifically, the results showed that girls and boys alike benefitted from the laptop program, but the effect on girls’ achievement was smaller than that on boys’. Along the same lines, Arztmann et al. ( 2022 ) reported no difference in the impact of game-based learning between boys and girls, arguing that boys and girls equally benefited from game-based interventions in STEM domains. However, results from a systematic review by Cussó-Calabuig et al. ( 2018 ) found limited and low-quality evidence on the effects of intensive use of computers on gender differences in computer anxiety, self-efficacy, and self-confidence. Based on their view, intensive use of computers can reduce gender differences in some areas and not in others, depending on contextual and implementation factors.

Impacts of digital technologies on teachers’ professional and teaching practices

Various research studies have explored the impact of ICT on teachers’ instructional practices and student assessment. Friedel et al. ( 2013 ) found that the use of mobile devices by students enabled teachers to successfully deliver content (e.g., mobile serious games), provide scaffolding, and facilitate synchronous collaborative learning. The integration of digital games in teaching and learning activities also gave teachers the opportunity to study and apply various pedagogical practices (Bado, 2022 ). Specifically, Bado ( 2022 ) found that teachers who implemented instructional activities in three stages (pre-game, game, and post-game) maximized students’ learning outcomes and engagement. For instance, during the pre-game stage, teachers focused on lectures and gameplay training, at the game stage teachers provided scaffolding on content, addressed technical issues, and managed the classroom activities. During the post-game stage, teachers organized activities for debriefing to ensure that the gameplay had indeed enhanced students’ learning outcomes.

Furthermore, ICT can increase efficiency in lesson planning and preparation by offering possibilities for a more collaborative approach among teachers. The sharing of curriculum plans and the analysis of students’ data led to clearer target settings and improvements in reporting to parents (Balanskat et al., 2006 ).

Additionally, the use and application of digital technologies in teaching and learning were found to enhance teachers’ digital competence. Balanskat et al. ( 2006 ) documented studies that revealed that the use of digital technologies in education had a positive effect on teachers’ basic ICT skills. The greatest impact was found on teachers with enough experience in integrating ICTs in their teaching and/or who had recently participated in development courses for the pedagogical use of technologies in teaching. Punie et al. ( 2006 ) reported that the provision of fully equipped multimedia portable computers and the development of online teacher communities had positive impacts on teachers’ confidence and competence in the use of ICTs.

Moreover, online assessment via ICTs benefits instruction. In particular, online assessments support the digitalization of students’ work and related logistics, allow teachers to gather immediate feedback and readjust to new objectives, and support the improvement of the technical quality of tests by providing more accurate results. Additionally, the capabilities of ICTs (e.g., interactive media, simulations) create new potential methods of testing specific skills, such as problem-solving and problem-processing skills, meta-cognitive skills, creativity and communication skills, and the ability to work productively in groups (Punie et al., 2006 ).

Impacts of digital technologies on other school-related aspects and stakeholders

There is evidence that the effective use of ICTs and the data transmission offered by broadband connections help improve administration (Balanskat et al., 2006 ). Specifically, ICTs have been found to provide better management systems to schools that have data gathering procedures in place. Condie and Munro ( 2007 ) reported impacts from the use of ICTs in schools in the following areas: attendance monitoring, assessment records, reporting to parents, financial management, creation of repositories for learning resources, and sharing of information amongst staff. Such data can be used strategically for self-evaluation and monitoring purposes which in turn can result in school improvements. Additionally, they reported that online access to other people with similar roles helped to reduce headteachers’ isolation by offering them opportunities to share insights into the use of ICT in learning and teaching and how it could be used to support school improvement. Furthermore, ICTs provided more efficient and successful examination management procedures, namely less time-consuming reporting processes compared to paper-based examinations and smooth communications between schools and examination authorities through electronic data exchange (Punie et al., 2006 ).

Zheng et al. ( 2016 ) reported that the use of ICTs improved home-school relationships. Additionally, Escueta et al. ( 2017 ) reported several ICT programs that had improved the flow of information from the school to parents. Particularly, they documented that the use of ICTs (learning management systems, emails, dedicated websites, mobile phones) allowed for personalized and customized information exchange between schools and parents, such as attendance records, upcoming class assignments, school events, and students’ grades, which generated positive results on students’ learning outcomes and attainment. Such information exchange between schools and families prompted parents to encourage their children to put more effort into their schoolwork.

The above findings suggest that the impact of ICT integration in schools goes beyond students’ performance in school subjects. Specifically, it affects a number of school-related aspects, such as equality and social integration, professional and teaching practices, and diverse stakeholders. In Table ​ Table2, 2 , we summarize the different impacts of digital technologies on school stakeholders based on the literature review, while in Table ​ Table3 3 we organized the tools/platforms and practices/policies addressed in the meta-analyses, literature reviews, EU reports, and international bodies included in the manuscript.

The impact of digital technologies on schools’ stakeholders based on the literature review

Tools/platforms and practices/policies addressed in the meta-analyses, literature reviews, EU reports, and international bodies included in the manuscript

Additionally, based on the results of the literature review, there are many types of digital technologies with different affordances (see, for example, studies on VR vs Immersive VR), which evolve over time (e.g. starting from CAIs in 2005 to Augmented and Virtual reality 2020). Furthermore, these technologies are linked to different pedagogies and policy initiatives, which are critical factors in the study of impact. Table ​ Table3 3 summarizes the different tools and practices that have been used to examine the impact of digital technologies on education since 2005 based on the review results.

Factors that affect the integration of digital technologies

Although the analysis of the literature review demonstrated different impacts of the use of digital technology on education, several authors highlighted the importance of various factors, besides the technology itself, that affect this impact. For example, Liao et al. ( 2007 ) suggested that future studies should carefully investigate which factors contribute to positive outcomes by clarifying the exact relationship between computer applications and learning. Additionally, Haßler et al., ( 2016 ) suggested that the neutral findings regarding the impact of tablets on students learning outcomes in some of the studies included in their review should encourage educators, school leaders, and school officials to further investigate the potential of such devices in teaching and learning. Several other researchers suggested that a number of variables play a significant role in the impact of ICTs on students’ learning that could be attributed to the school context, teaching practices and professional development, the curriculum, and learners’ characteristics (Underwood, 2009 ; Tamim et al., 2011 ; Higgins et al., 2012 ; Archer et al., 2014 ; Sung et al., 2016 ; Haßler et al., 2016 ; Chauhan, 2017 ; Lee et al., 2020 ; Tang et al., 2022 ).

Digital competencies

One of the most common challenges reported in studies that utilized digital tools in the classroom was the lack of students’ skills on how to use them. Fu ( 2013 ) found that students’ lack of technical skills is a barrier to the effective use of ICT in the classroom. Tamim et al. ( 2015 ) reported that students faced challenges when using tablets and smart mobile devices, associated with the technical issues or expertise needed for their use and the distracting nature of the devices and highlighted the need for teachers’ professional development. Higgins et al. ( 2012 ) reported that skills training about the use of digital technologies is essential for learners to fully exploit the benefits of instruction.

Delgado et al. ( 2015 ), meanwhile, reported studies that showed a strong positive association between teachers’ computer skills and students’ use of computers. Teachers’ lack of ICT skills and familiarization with technologies can become a constraint to the effective use of technology in the classroom (Balanskat et al., 2006 ; Delgado et al., 2015 ).

It is worth noting that the way teachers are introduced to ICTs affects the impact of digital technologies on education. Previous studies have shown that teachers may avoid using digital technologies due to limited digital skills (Balanskat, 2006 ), or they prefer applying “safe” technologies, namely technologies that their own teachers used and with which they are familiar (Condie & Munro, 2007 ). In this regard, the provision of digital skills training and exposure to new digital tools might encourage teachers to apply various technologies in their lessons (Condie & Munro, 2007 ). Apart from digital competence, technical support in the school setting has also been shown to affect teachers’ use of technology in their classrooms (Delgado et al., 2015 ). Ferrari et al. ( 2011 ) found that while teachers’ use of ICT is high, 75% stated that they needed more institutional support and a shift in the mindset of educational actors to achieve more innovative teaching practices. The provision of support can reduce time and effort as well as cognitive constraints, which could cause limited ICT integration in the school lessons by teachers (Escueta et al., 2017 ).

Teachers’ personal characteristics, training approaches, and professional development

Teachers’ personal characteristics and professional development affect the impact of digital technologies on education. Specifically, Cheok and Wong ( 2015 ) found that teachers’ personal characteristics (e.g., anxiety, self-efficacy) are associated with their satisfaction and engagement with technology. Bingimlas ( 2009 ) reported that lack of confidence, resistance to change, and negative attitudes in using new technologies in teaching are significant determinants of teachers’ levels of engagement in ICT. The same author reported that the provision of technical support, motivation support (e.g., awards, sufficient time for planning), and training on how technologies can benefit teaching and learning can eliminate the above barriers to ICT integration. Archer et al. ( 2014 ) found that comfort levels in using technology are an important predictor of technology integration and argued that it is essential to provide teachers with appropriate training and ongoing support until they are comfortable with using ICTs in the classroom. Hillmayr et al. ( 2020 ) documented that training teachers on ICT had an important effecton students’ learning.

According to Balanskat et al. ( 2006 ), the impact of ICTs on students’ learning is highly dependent on the teachers’ capacity to efficiently exploit their application for pedagogical purposes. Results obtained from the Teaching and Learning International Survey (TALIS) (OECD, 2021 ) revealed that although schools are open to innovative practices and have the capacity to adopt them, only 39% of teachers in the European Union reported that they are well or very well prepared to use digital technologies for teaching. Li and Ma ( 2010 ) and Hardman ( 2019 ) showed that the positive effect of technology on students’ achievement depends on the pedagogical practices used by teachers. Schmid et al. ( 2014 ) reported that learning was best supported when students were engaged in active, meaningful activities with the use of technological tools that provided cognitive support. Tamim et al. ( 2015 ) compared two different pedagogical uses of tablets and found a significant moderate effect when the devices were used in a student-centered context and approach rather than within teacher-led environments. Similarly, Garzón and Acevedo ( 2019 ) and Garzón et al. ( 2020 ) reported that the positive results from the integration of AR applications could be attributed to the existence of different variables which could influence AR interventions (e.g., pedagogical approach, learning environment, and duration of the intervention). Additionally, Garzón et al. ( 2020 ) suggested that the pedagogical resources that teachers used to complement their lectures and the pedagogical approaches they applied were crucial to the effective integration of AR on students’ learning gains. Garzón and Acevedo ( 2019 ) also emphasized that the success of a technology-enhanced intervention is based on both the technology per se and its characteristics and on the pedagogical strategies teachers choose to implement. For instance, their results indicated that the collaborative learning approach had the highest impact on students’ learning gains among other approaches (e.g., inquiry-based learning, situated learning, or project-based learning). Ran et al. ( 2022 ) also found that the use of technology to design collaborative and communicative environments showed the largest moderator effects among the other approaches.

Hattie ( 2008 ) reported that the effective use of computers is associated with training teachers in using computers as a teaching and learning tool. Zheng et al. ( 2016 ) noted that in addition to the strategies teachers adopt in teaching, ongoing professional development is also vital in ensuring the success of technology implementation programs. Sung et al. ( 2016 ) found that research on the use of mobile devices to support learning tends to report that the insufficient preparation of teachers is a major obstacle in implementing effective mobile learning programs in schools. Friedel et al. ( 2013 ) found that providing training and support to teachers increased the positive impact of the interventions on students’ learning gains. Trucano ( 2005 ) argued that positive impacts occur when digital technologies are used to enhance teachers’ existing pedagogical philosophies. Higgins et al. ( 2012 ) found that the types of technologies used and how they are used could also affect students’ learning. The authors suggested that training and professional development of teachers that focuses on the effective pedagogical use of technology to support teaching and learning is an important component of successful instructional approaches (Higgins et al., 2012 ). Archer et al. ( 2014 ) found that studies that reported ICT interventions during which teachers received training and support had moderate positive effects on students’ learning outcomes, which were significantly higher than studies where little or no detail about training and support was mentioned. Fu ( 2013 ) reported that the lack of teachers’ knowledge and skills on the technical and instructional aspects of ICT use in the classroom, in-service training, pedagogy support, technical and financial support, as well as the lack of teachers’ motivation and encouragement to integrate ICT on their teaching were significant barriers to the integration of ICT in education.

School leadership and management

Management and leadership are important cornerstones in the digital transformation process (Pihir et al., 2018 ). Zheng et al. ( 2016 ) documented leadership among the factors positively affecting the successful implementation of technology integration in schools. Strong leadership, strategic planning, and systematic integration of digital technologies are prerequisites for the digital transformation of education systems (Ređep, 2021 ). Management and leadership play a significant role in formulating policies that are translated into practice and ensure that developments in ICT become embedded into the life of the school and in the experiences of staff and pupils (Condie & Munro, 2007 ). Policy support and leadership must include the provision of an overall vision for the use of digital technologies in education, guidance for students and parents, logistical support, as well as teacher training (Conrads et al., 2017 ). Unless there is a commitment throughout the school, with accountability for progress at key points, it is unlikely for ICT integration to be sustained or become part of the culture (Condie & Munro, 2007 ). To achieve this, principals need to adopt and promote a whole-institution strategy and build a strong mutual support system that enables the school’s technological maturity (European Commission, 2019 ). In this context, school culture plays an essential role in shaping the mindsets and beliefs of school actors towards successful technology integration. Condie and Munro ( 2007 ) emphasized the importance of the principal’s enthusiasm and work as a source of inspiration for the school staff and the students to cultivate a culture of innovation and establish sustainable digital change. Specifically, school leaders need to create conditions in which the school staff is empowered to experiment and take risks with technology (Elkordy & Lovinelli, 2020 ).

In order for leaders to achieve the above, it is important to develop capacities for learning and leading, advocating professional learning, and creating support systems and structures (European Commission, 2019 ). Digital technology integration in education systems can be challenging and leadership needs guidance to achieve it. Such guidance can be introduced through the adoption of new methods and techniques in strategic planning for the integration of digital technologies (Ređep, 2021 ). Even though the role of leaders is vital, the relevant training offered to them has so far been inadequate. Specifically, only a third of the education systems in Europe have put in place national strategies that explicitly refer to the training of school principals (European Commission, 2019 , p. 16).

Connectivity, infrastructure, and government and other support

The effective integration of digital technologies across levels of education presupposes the development of infrastructure, the provision of digital content, and the selection of proper resources (Voogt et al., 2013 ). Particularly, a high-quality broadband connection in the school increases the quality and quantity of educational activities. There is evidence that ICT increases and formalizes cooperative planning between teachers and cooperation with managers, which in turn has a positive impact on teaching practices (Balanskat et al., 2006 ). Additionally, ICT resources, including software and hardware, increase the likelihood of teachers integrating technology into the curriculum to enhance their teaching practices (Delgado et al., 2015 ). For example, Zheng et al. ( 2016 ) found that the use of one-on-one laptop programs resulted in positive changes in teaching and learning, which would not have been accomplished without the infrastructure and technical support provided to teachers. Delgado et al. ( 2015 ) reported that limited access to technology (insufficient computers, peripherals, and software) and lack of technical support are important barriers to ICT integration. Access to infrastructure refers not only to the availability of technology in a school but also to the provision of a proper amount and the right types of technology in locations where teachers and students can use them. Effective technical support is a central element of the whole-school strategy for ICT (Underwood, 2009 ). Bingimlas ( 2009 ) reported that lack of technical support in the classroom and whole-school resources (e.g., failing to connect to the Internet, printers not printing, malfunctioning computers, and working on old computers) are significant barriers that discourage the use of ICT by teachers. Moreover, poor quality and inadequate hardware maintenance, and unsuitable educational software may discourage teachers from using ICTs (Balanskat et al., 2006 ; Bingimlas, 2009 ).

Government support can also impact the integration of ICTs in teaching. Specifically, Balanskat et al. ( 2006 ) reported that government interventions and training programs increased teachers’ enthusiasm and positive attitudes towards ICT and led to the routine use of embedded ICT.

Lastly, another important factor affecting digital transformation is the development and quality assurance of digital learning resources. Such resources can be support textbooks and related materials or resources that focus on specific subjects or parts of the curriculum. Policies on the provision of digital learning resources are essential for schools and can be achieved through various actions. For example, some countries are financing web portals that become repositories, enabling teachers to share resources or create their own. Additionally, they may offer e-learning opportunities or other services linked to digital education. In other cases, specific agencies of projects have also been set up to develop digital resources (Eurydice, 2019 ).

Administration and digital data management

The digital transformation of schools involves organizational improvements at the level of internal workflows, communication between the different stakeholders, and potential for collaboration. Vuorikari et al. ( 2020 ) presented evidence that digital technologies supported the automation of administrative practices in schools and reduced the administration’s workload. There is evidence that digital data affects the production of knowledge about schools and has the power to transform how schooling takes place. Specifically, Sellar ( 2015 ) reported that data infrastructure in education is developing due to the demand for “ information about student outcomes, teacher quality, school performance, and adult skills, associated with policy efforts to increase human capital and productivity practices ” (p. 771). In this regard, practices, such as datafication which refers to the “ translation of information about all kinds of things and processes into quantified formats” have become essential for decision-making based on accountability reports about the school’s quality. The data could be turned into deep insights about education or training incorporating ICTs. For example, measuring students’ online engagement with the learning material and drawing meaningful conclusions can allow teachers to improve their educational interventions (Vuorikari et al., 2020 ).

Students’ socioeconomic background and family support

Research show that the active engagement of parents in the school and their support for the school’s work can make a difference to their children’s attitudes towards learning and, as a result, their achievement (Hattie, 2008 ). In recent years, digital technologies have been used for more effective communication between school and family (Escueta et al., 2017 ). The European Commission ( 2020 ) presented data from a Eurostat survey regarding the use of computers by students during the pandemic. The data showed that younger pupils needed additional support and guidance from parents and the challenges were greater for families in which parents had lower levels of education and little to no digital skills.

In this regard, the socio-economic background of the learners and their socio-cultural environment also affect educational achievements (Punie et al., 2006 ). Trucano documented that the use of computers at home positively influenced students’ confidence and resulted in more frequent use at school, compared to students who had no home access (Trucano, 2005 ). In this sense, the socio-economic background affects the access to computers at home (OECD, 2015 ) which in turn influences the experience of ICT, an important factor for school achievement (Punie et al., 2006 ; Underwood, 2009 ). Furthermore, parents from different socio-economic backgrounds may have different abilities and availability to support their children in their learning process (Di Pietro et al., 2020 ).

Schools’ socioeconomic context and emergency situations

The socio-economic context of the school is closely related to a school’s digital transformation. For example, schools in disadvantaged, rural, or deprived areas are likely to lack the digital capacity and infrastructure required to adapt to the use of digital technologies during emergency periods, such as the COVID-19 pandemic (Di Pietro et al., 2020 ). Data collected from school principals confirmed that in several countries, there is a rural/urban divide in connectivity (OECD, 2015 ).

Emergency periods also affect the digitalization of schools. The COVID-19 pandemic led to the closure of schools and forced them to seek appropriate and connective ways to keep working on the curriculum (Di Pietro et al., 2020 ). The sudden large-scale shift to distance and online teaching and learning also presented challenges around quality and equity in education, such as the risk of increased inequalities in learning, digital, and social, as well as teachers facing difficulties coping with this demanding situation (European Commission, 2020 ).

Looking at the findings of the above studies, we can conclude that the impact of digital technologies on education is influenced by various actors and touches many aspects of the school ecosystem. Figure  1 summarizes the factors affecting the digital technologies’ impact on school stakeholders based on the findings from the literature review.

Factors that affect the impact of ICTs on education

The findings revealed that the use of digital technologies in education affects a variety of actors within a school’s ecosystem. First, we observed that as technologies evolve, so does the interest of the research community to apply them to school settings. Figure  2 summarizes the trends identified in current research around the impact of digital technologies on schools’ digital capacity and transformation as found in the present study. Starting as early as 2005, when computers, simulations, and interactive boards were the most commonly applied tools in school interventions (e.g., Eng, 2005 ; Liao et al., 2007 ; Moran et al., 2008 ; Tamim et al., 2011 ), moving towards the use of learning platforms (Jewitt et al., 2011 ), then to the use of mobile devices and digital games (e.g., Tamim et al., 2015 ; Sung et al., 2016 ; Talan et al., 2020 ), as well as e-books (e.g., Savva et al., 2022 ), to the more recent advanced technologies, such as AR and VR applications (e.g., Garzón & Acevedo, 2019 ; Garzón et al., 2020 ; Kalemkuş & Kalemkuş, 2022 ), or robotics and AI (e.g., Su & Yang, 2022 ; Su et al., 2022 ). As this evolution shows, digital technologies are a concept in flux with different affordances and characteristics. Additionally, from an instructional perspective, there has been a growing interest in different modes and models of content delivery such as online, blended, and hybrid modes (e.g., Cheok & Wong, 2015 ; Kazu & Yalçin, 2022 ; Ulum, 2022 ). This is an indication that the value of technologies to support teaching and learning as well as other school-related practices is increasingly recognized by the research and school community. The impact results from the literature review indicate that ICT integration on students’ learning outcomes has effects that are small (Coban et al., 2022 ; Eng, 2005 ; Higgins et al., 2012 ; Schmid et al., 2014 ; Tamim et al., 2015 ; Zheng et al., 2016 ) to moderate (Garzón & Acevedo, 2019 ; Garzón et al., 2020 ; Liao et al., 2007 ; Sung et al., 2016 ; Talan et al., 2020 ; Wen & Walters, 2022 ). That said, a number of recent studies have reported high effect sizes (e.g., Kazu & Yalçin, 2022 ).

Current work and trends in the study of the impact of digital technologies on schools’ digital capacity

Based on these findings, several authors have suggested that the impact of technology on education depends on several variables and not on the technology per se (Tamim et al., 2011 ; Higgins et al., 2012 ; Archer et al., 2014 ; Sung et al., 2016 ; Haßler et al., 2016 ; Chauhan, 2017 ; Lee et al., 2020 ; Lei et al., 2022a ). While the impact of ICTs on student achievement has been thoroughly investigated by researchers, other aspects related to school life that are also affected by ICTs, such as equality, inclusion, and social integration have received less attention. Further analysis of the literature review has revealed a greater investment in ICT interventions to support learning and teaching in the core subjects of literacy and STEM disciplines, especially mathematics, and science. These were the most common subjects studied in the reviewed papers often drawing on national testing results, while studies that investigated other subject areas, such as social studies, were limited (Chauhan, 2017 ; Condie & Munro, 2007 ). As such, research is still lacking impact studies that focus on the effects of ICTs on a range of curriculum subjects.

The qualitative research provided additional information about the impact of digital technologies on education, documenting positive effects and giving more details about implications, recommendations, and future research directions. Specifically, the findings regarding the role of ICTs in supporting learning highlight the importance of teachers’ instructional practice and the learning context in the use of technologies and consequently their impact on instruction (Çelik, 2022 ; Schmid et al., 2014 ; Tamim et al., 2015 ). The review also provided useful insights regarding the various factors that affect the impact of digital technologies on education. These factors are interconnected and play a vital role in the transformation process. Specifically, these factors include a) digital competencies; b) teachers’ personal characteristics and professional development; c) school leadership and management; d) connectivity, infrastructure, and government support; e) administration and data management practices; f) students’ socio-economic background and family support and g) the socioeconomic context of the school and emergency situations. It is worth noting that we observed factors that affect the integration of ICTs in education but may also be affected by it. For example, the frequent use of ICTs and the use of laptops by students for instructional purposes positively affect the development of digital competencies (Zheng et al., 2016 ) and at the same time, the digital competencies affect the use of ICTs (Fu, 2013 ; Higgins et al., 2012 ). As a result, the impact of digital technologies should be explored more as an enabler of desirable and new practices and not merely as a catalyst that improves the output of the education process i.e. namely student attainment.

Conclusions

Digital technologies offer immense potential for fundamental improvement in schools. However, investment in ICT infrastructure and professional development to improve school education are yet to provide fruitful results. Digital transformation is a complex process that requires large-scale transformative changes that presuppose digital capacity and preparedness. To achieve such changes, all actors within the school’s ecosystem need to share a common vision regarding the integration of ICTs in education and work towards achieving this goal. Our literature review, which synthesized quantitative and qualitative data from a list of meta-analyses and review studies, provided useful insights into the impact of ICTs on different school stakeholders and showed that the impact of digital technologies touches upon many different aspects of school life, which are often overlooked when the focus is on student achievement as the final output of education. Furthermore, the concept of digital technologies is a concept in flux as technologies are not only different among them calling for different uses in the educational practice but they also change through time. Additionally, we opened a forum for discussion regarding the factors that affect a school’s digital capacity and transformation. We hope that our study will inform policy, practice, and research and result in a paradigm shift towards more holistic approaches in impact and assessment studies.

Study limitations and future directions

We presented a review of the study of digital technologies' impact on education and factors influencing schools’ digital capacity and transformation. The study results were based on a non-systematic literature review grounded on the acquisition of documentation in specific databases. Future studies should investigate more databases to corroborate and enhance our results. Moreover, search queries could be enhanced with key terms that could provide additional insights about the integration of ICTs in education, such as “policies and strategies for ICT integration in education”. Also, the study drew information from meta-analyses and literature reviews to acquire evidence about the effects of ICT integration in schools. Such evidence was mostly based on the general conclusions of the studies. It is worth mentioning that, we located individual studies which showed different, such as negative or neutral results. Thus, further insights are needed about the impact of ICTs on education and the factors influencing the impact. Furthermore, the nature of the studies included in meta-analyses and reviews is different as they are based on different research methodologies and data gathering processes. For instance, in a meta-analysis, the impact among the studies investigated is measured in a particular way, depending on policy or research targets (e.g., results from national examinations, pre-/post-tests). Meanwhile, in literature reviews, qualitative studies offer additional insights and detail based on self-reports and research opinions on several different aspects and stakeholders who could affect and be affected by ICT integration. As a result, it was challenging to draw causal relationships between so many interrelating variables.

Despite the challenges mentioned above, this study envisaged examining school units as ecosystems that consist of several actors by bringing together several variables from different research epistemologies to provide an understanding of the integration of ICTs. However, the use of other tools and methodologies and models for evaluation of the impact of digital technologies on education could give more detailed data and more accurate results. For instance, self-reflection tools, like SELFIE—developed on the DigCompOrg framework- (Kampylis et al., 2015 ; Bocconi & Lightfoot, 2021 ) can help capture a school’s digital capacity and better assess the impact of ICTs on education. Furthermore, the development of a theory of change could be a good approach for documenting the impact of digital technologies on education. Specifically, theories of change are models used for the evaluation of interventions and their impact; they are developed to describe how interventions will work and give the desired outcomes (Mayne, 2015 ). Theory of change as a methodological approach has also been used by researchers to develop models for evaluation in the field of education (e.g., Aromatario et al., 2019 ; Chapman & Sammons, 2013 ; De Silva et al., 2014 ).

We also propose that future studies aim at similar investigations by applying more holistic approaches for impact assessment that can provide in-depth data about the impact of digital technologies on education. For instance, future studies could focus on different research questions about the technologies that are used during the interventions or the way the implementation takes place (e.g., What methodologies are used for documenting impact? How are experimental studies implemented? How can teachers be taken into account and trained on the technology and its functions? What are the elements of an appropriate and successful implementation? How is the whole intervention designed? On which learning theories is the technology implementation based?).

Future research could also focus on assessing the impact of digital technologies on various other subjects since there is a scarcity of research related to particular subjects, such as geography, history, arts, music, and design and technology. More research should also be done about the impact of ICTs on skills, emotions, and attitudes, and on equality, inclusion, social interaction, and special needs education. There is also a need for more research about the impact of ICTs on administration, management, digitalization, and home-school relationships. Additionally, although new forms of teaching and learning with the use of ICTs (e.g., blended, hybrid, and online learning) have initiated several investigations in mainstream classrooms, only a few studies have measured their impact on students’ learning. Additionally, our review did not document any study about the impact of flipped classrooms on K-12 education. Regarding teaching and learning approaches, it is worth noting that studies referred to STEM or STEAM did not investigate the impact of STEM/STEAM as an interdisciplinary approach to learning but only investigated the impact of ICTs on learning in each domain as a separate subject (science, technology, engineering, arts, mathematics). Hence, we propose future research to also investigate the impact of the STEM/STEAM approach on education. The impact of emerging technologies on education, such as AR, VR, robotics, and AI has also been investigated recently, but more work needs to be done.

Finally, we propose that future studies could focus on the way in which specific factors, e.g., infrastructure and government support, school leadership and management, students’ and teachers’ digital competencies, approaches teachers utilize in the teaching and learning (e.g., blended, online and hybrid learning, flipped classrooms, STEM/STEAM approach, project-based learning, inquiry-based learning), affect the impact of digital technologies on education. We hope that future studies will give detailed insights into the concept of schools’ digital transformation through further investigation of impacts and factors which influence digital capacity and transformation based on the results and the recommendations of the present study.

Acknowledgements

This project has received funding under Grant Agreement No Ref Ares (2021) 339036 7483039 as well as funding from the European Union’s Horizon 2020 Research and Innovation Program under Grant Agreement No 739578 and the Government of the Republic of Cyprus through the Deputy Ministry of Research, Innovation and Digital Policy. The UVa co-authors would like also to acknowledge funding from the European Regional Development Fund and the National Research Agency of the Spanish Ministry of Science and Innovation, under project grant PID2020-112584RB-C32.

Data availability statement

Declarations.

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

• Archer K, Savage R, Sanghera-Sidhu S, Wood E, Gottardo A, Chen V. Examining the effectiveness of technology use in classrooms: A tertiary meta-analysis. Computers & Education. 2014; 78 :140–149. doi: 10.1016/j.compedu.2014.06.001. [ CrossRef ] [ Google Scholar ]
• Aromatario O, Van Hoye A, Vuillemin A, Foucaut AM, Pommier J, Cambon L. Using theory of change to develop an intervention theory for designing and evaluating behavior change SDApps for healthy eating and physical exercise: The OCAPREV theory. BMC Public Health. 2019; 19 (1):1–12. doi: 10.1186/s12889-019-7828-4. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Arztmann, M., Hornstra, L., Jeuring, J., & Kester, L. (2022). Effects of games in STEM education: A meta-analysis on the moderating role of student background characteristics. Studies in Science Education , 1-37. 10.1080/03057267.2022.2057732
• Bado N. Game-based learning pedagogy: A review of the literature. Interactive Learning Environments. 2022; 30 (5):936–948. doi: 10.1080/10494820.2019.1683587. [ CrossRef ] [ Google Scholar ]
• Balanskat, A. (2009). Study of the impact of technology in primary schools – Synthesis Report. Empirica and European Schoolnet. Retrieved 30 June 2022 from: https://erte.dge.mec.pt/sites/default/files/Recursos/Estudos/synthesis_report_steps_en.pdf
• Balanskat, A. (2006). The ICT Impact Report: A review of studies of ICT impact on schools in Europe, European Schoolnet. Retrieved 30 June 2022 from:  https://en.unesco.org/icted/content/ict-impact-report-review-studies-ict-impact-schools-europe
• Balanskat, A., Blamire, R., & Kefala, S. (2006). The ICT impact report.  European Schoolnet . Retrieved from: http://colccti.colfinder.org/sites/default/files/ict_impact_report_0.pdf
• Balyer, A., & Öz, Ö. (2018). Academicians’ views on digital transformation in education. International Online Journal of Education and Teaching (IOJET), 5 (4), 809–830. Retrieved 30 June 2022 from  http://iojet.org/index.php/IOJET/article/view/441/295
• Baragash RS, Al-Samarraie H, Moody L, Zaqout F. Augmented reality and functional skills acquisition among individuals with special needs: A meta-analysis of group design studies. Journal of Special Education Technology. 2022; 37 (1):74–81. doi: 10.1177/0162643420910413. [ CrossRef ] [ Google Scholar ]
• Bates, A. W. (2015). Teaching in a digital age: Guidelines for designing teaching and learning . Open Educational Resources Collection . 6. Retrieved 30 June 2022 from: https://irl.umsl.edu/oer/6
• Bingimlas KA. Barriers to the successful integration of ICT in teaching and learning environments: A review of the literature. Eurasia Journal of Mathematics, Science and Technology Education. 2009; 5 (3):235–245. doi: 10.12973/ejmste/75275. [ CrossRef ] [ Google Scholar ]
• Blaskó Z, Costa PD, Schnepf SV. Learning losses and educational inequalities in Europe: Mapping the potential consequences of the COVID-19 crisis. Journal of European Social Policy. 2022; 32 (4):361–375. doi: 10.1177/09589287221091687. [ CrossRef ] [ Google Scholar ]
• Bocconi S, Lightfoot M. Scaling up and integrating the selfie tool for schools' digital capacity in education and training systems: Methodology and lessons learnt. European Training Foundation. 2021 doi: 10.2816/907029,JRC123936. [ CrossRef ] [ Google Scholar ]
• Brooks, D. C., & McCormack, M. (2020). Driving Digital Transformation in Higher Education . Retrieved 30 June 2022 from: https://library.educause.edu/-/media/files/library/2020/6/dx2020.pdf?la=en&hash=28FB8C377B59AFB1855C225BBA8E3CFBB0A271DA
• Cachia, R., Chaudron, S., Di Gioia, R., Velicu, A., & Vuorikari, R. (2021). Emergency remote schooling during COVID-19, a closer look at European families. Retrieved 30 June 2022 from  https://publications.jrc.ec.europa.eu/repository/handle/JRC125787
• Çelik B. The effects of computer simulations on students’ science process skills: Literature review. Canadian Journal of Educational and Social Studies. 2022; 2 (1):16–28. doi: 10.53103/cjess.v2i1.17. [ CrossRef ] [ Google Scholar ]
• Chapman, C., & Sammons, P. (2013). School Self-Evaluation for School Improvement: What Works and Why? . CfBT Education Trust. 60 Queens Road, Reading, RG1 4BS, England.
• Chauhan S. A meta-analysis of the impact of technology on learning effectiveness of elementary students. Computers & Education. 2017; 105 :14–30. doi: 10.1016/j.compedu.2016.11.005. [ CrossRef ] [ Google Scholar ]
• Chen, Q., Chan, K. L., Guo, S., Chen, M., Lo, C. K. M., & Ip, P. (2022a). Effectiveness of digital health interventions in reducing bullying and cyberbullying: a meta-analysis. Trauma, Violence, & Abuse , 15248380221082090. 10.1177/15248380221082090 [ PubMed ]
• Chen B, Wang Y, Wang L. The effects of virtual reality-assisted language learning: A meta-analysis. Sustainability. 2022; 14 (6):3147. doi: 10.3390/su14063147. [ CrossRef ] [ Google Scholar ]
• Cheok ML, Wong SL. Predictors of e-learning satisfaction in teaching and learning for school teachers: A literature review. International Journal of Instruction. 2015; 8 (1):75–90. doi: 10.12973/iji.2015.816a. [ CrossRef ] [ Google Scholar ]
• Cheung, A. C., & Slavin, R. E. (2011). The Effectiveness of Education Technology for Enhancing Reading Achievement: A Meta-Analysis. Center for Research and reform in Education .
• Coban, M., Bolat, Y. I., & Goksu, I. (2022). The potential of immersive virtual reality to enhance learning: A meta-analysis. Educational Research Review , 100452. 10.1016/j.edurev.2022.100452
• Condie, R., & Munro, R. K. (2007). The impact of ICT in schools-a landscape review. Retrieved 30 June 2022 from: https://oei.org.ar/ibertic/evaluacion/sites/default/files/biblioteca/33_impact_ict_in_schools.pdf
• Conrads, J., Rasmussen, M., Winters, N., Geniet, A., Langer, L., (2017). Digital Education Policies in Europe and Beyond: Key Design Principles for More Effective Policies. Redecker, C., P. Kampylis, M. Bacigalupo, Y. Punie (ed.), EUR 29000 EN, Publications Office of the European Union, Luxembourg, 10.2760/462941
• Costa P, Castaño-Muñoz J, Kampylis P. Capturing schools’ digital capacity: Psychometric analyses of the SELFIE self-reflection tool. Computers & Education. 2021; 162 :104080. doi: 10.1016/j.compedu.2020.104080. [ CrossRef ] [ Google Scholar ]
• Cussó-Calabuig R, Farran XC, Bosch-Capblanch X. Effects of intensive use of computers in secondary school on gender differences in attitudes towards ICT: A systematic review. Education and Information Technologies. 2018; 23 (5):2111–2139. doi: 10.1007/s10639-018-9706-6. [ CrossRef ] [ Google Scholar ]
• Daniel SJ. Education and the COVID-19 pandemic. Prospects. 2020; 49 (1):91–96. doi: 10.1007/s11125-020-09464-3. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Delcker J, Ifenthaler D. Teachers’ perspective on school development at German vocational schools during the Covid-19 pandemic. Technology, Pedagogy and Education. 2021; 30 (1):125–139. doi: 10.1080/1475939X.2020.1857826. [ CrossRef ] [ Google Scholar ]
• Delgado, A., Wardlow, L., O’Malley, K., & McKnight, K. (2015). Educational technology: A review of the integration, resources, and effectiveness of technology in K-12 classrooms. Journal of Information Technology Education Research , 14, 397. Retrieved 30 June 2022 from  http://www.jite.org/documents/Vol14/JITEv14ResearchP397-416Delgado1829.pdf
• De Silva MJ, Breuer E, Lee L, Asher L, Chowdhary N, Lund C, Patel V. Theory of change: A theory-driven approach to enhance the Medical Research Council's framework for complex interventions. Trials. 2014; 15 (1):1–13. doi: 10.1186/1745-6215-15-267. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Di Pietro G, Biagi F, Costa P, Karpiński Z, Mazza J. The likely impact of COVID-19 on education: Reflections based on the existing literature and recent international datasets. Publications Office of the European Union; 2020. [ Google Scholar ]
• Elkordy A, Lovinelli J. Competencies, Culture, and Change: A Model for Digital Transformation in K12 Educational Contexts. In: Ifenthaler D, Hofhues S, Egloffstein M, Helbig C, editors. Digital Transformation of Learning Organizations. Springer; 2020. pp. 203–219. [ Google Scholar ]
• Eng TS. The impact of ICT on learning: A review of research. International Education Journal. 2005; 6 (5):635–650. [ Google Scholar ]
• European Commission. (2020). Digital Education Action Plan 2021 – 2027. Resetting education and training for the digital age. Retrieved 30 June 2022 from  https://ec.europa.eu/education/sites/default/files/document-library-docs/deap-communication-sept2020_en.pdf
• European Commission. (2019). 2 nd survey of schools: ICT in education. Objective 1: Benchmark progress in ICT in schools . Retrieved 30 June 2022 from: https://data.europa.eu/euodp/data/storage/f/2019-03-19T084831/FinalreportObjective1-BenchmarkprogressinICTinschools.pdf
• Eurydice. (2019). Digital Education at School in Europe , Luxembourg: Publications Office of the European Union. Retrieved 30 June 2022 from: https://eacea.ec.europa.eu/national-policies/eurydice/content/digital-education-school-europe_en
• Escueta, M., Quan, V., Nickow, A. J., & Oreopoulos, P. (2017). Education technology: An evidence-based review. Retrieved 30 June 2022 from  https://ssrn.com/abstract=3031695
• Fadda D, Pellegrini M, Vivanet G, Zandonella Callegher C. Effects of digital games on student motivation in mathematics: A meta-analysis in K-12. Journal of Computer Assisted Learning. 2022; 38 (1):304–325. doi: 10.1111/jcal.12618. [ CrossRef ] [ Google Scholar ]
• Fernández-Gutiérrez M, Gimenez G, Calero J. Is the use of ICT in education leading to higher student outcomes? Analysis from the Spanish Autonomous Communities. Computers & Education. 2020; 157 :103969. doi: 10.1016/j.compedu.2020.103969. [ CrossRef ] [ Google Scholar ]
• Ferrari, A., Cachia, R., & Punie, Y. (2011). Educational change through technology: A challenge for obligatory schooling in Europe. Lecture Notes in Computer Science , 6964 , 97–110. Retrieved 30 June 2022  https://link.springer.com/content/pdf/10.1007/978-3-642-23985-4.pdf
• Fielding, K., & Murcia, K. (2022). Research linking digital technologies to young children’s creativity: An interpretive framework and systematic review. Issues in Educational Research , 32 (1), 105–125. Retrieved 30 June 2022 from  http://www.iier.org.au/iier32/fielding-abs.html
• Friedel, H., Bos, B., Lee, K., & Smith, S. (2013). The impact of mobile handheld digital devices on student learning: A literature review with meta-analysis. In Society for Information Technology & Teacher Education International Conference (pp. 3708–3717). Association for the Advancement of Computing in Education (AACE).
• Fu JS. ICT in education: A critical literature review and its implications. International Journal of Education and Development Using Information and Communication Technology (IJEDICT) 2013; 9 (1):112–125. [ Google Scholar ]
• Gaol FL, Prasolova-Førland E. Special section editorial: The frontiers of augmented and mixed reality in all levels of education. Education and Information Technologies. 2022; 27 (1):611–623. doi: 10.1007/s10639-021-10746-2. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Garzón J, Acevedo J. Meta-analysis of the impact of Augmented Reality on students’ learning gains. Educational Research Review. 2019; 27 :244–260. doi: 10.1016/j.edurev.2019.04.001. [ CrossRef ] [ Google Scholar ]
• Garzón, J., Baldiris, S., Gutiérrez, J., & Pavón, J. (2020). How do pedagogical approaches affect the impact of augmented reality on education? A meta-analysis and research synthesis. Educational Research Review , 100334. 10.1016/j.edurev.2020.100334
• Grgurović M, Chapelle CA, Shelley MC. A meta-analysis of effectiveness studies on computer technology-supported language learning. ReCALL. 2013; 25 (2):165–198. doi: 10.1017/S0958344013000013. [ CrossRef ] [ Google Scholar ]
• Haßler B, Major L, Hennessy S. Tablet use in schools: A critical review of the evidence for learning outcomes. Journal of Computer Assisted Learning. 2016; 32 (2):139–156. doi: 10.1111/jcal.12123. [ CrossRef ] [ Google Scholar ]
• Haleem A, Javaid M, Qadri MA, Suman R. Understanding the role of digital technologies in education: A review. Sustainable Operations and Computers. 2022; 3 :275–285. doi: 10.1016/j.susoc.2022.05.004. [ CrossRef ] [ Google Scholar ]
• Hardman J. Towards a pedagogical model of teaching with ICTs for mathematics attainment in primary school: A review of studies 2008–2018. Heliyon. 2019; 5 (5):e01726. doi: 10.1016/j.heliyon.2019.e01726. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Hattie J, Rogers HJ, Swaminathan H. The role of meta-analysis in educational research. In: Reid AD, Hart P, Peters MA, editors. A companion to research in education. Springer; 2014. pp. 197–207. [ Google Scholar ]
• Hattie J. Visible learning: A synthesis of over 800 meta-analyses relating to achievement. Routledge. 2008 doi: 10.4324/9780203887332. [ CrossRef ] [ Google Scholar ]
• Higgins S, Xiao Z, Katsipataki M. The impact of digital technology on learning: A summary for the education endowment foundation. Education Endowment Foundation and Durham University; 2012. [ Google Scholar ]
• Higgins, K., Huscroft-D’Angelo, J., & Crawford, L. (2019). Effects of technology in mathematics on achievement, motivation, and attitude: A meta-analysis. Journal of Educational Computing Research , 57(2), 283-319.
• Hillmayr D, Ziernwald L, Reinhold F, Hofer SI, Reiss KM. The potential of digital tools to enhance mathematics and science learning in secondary schools: A context-specific meta-analysis. Computers & Education. 2020; 153 (1038):97. doi: 10.1016/j.compedu.2020.103897. [ CrossRef ] [ Google Scholar ]
• Istenic Starcic A, Bagon S. ICT-supported learning for inclusion of people with special needs: Review of seven educational technology journals, 1970–2011. British Journal of Educational Technology. 2014; 45 (2):202–230. doi: 10.1111/bjet.12086. [ CrossRef ] [ Google Scholar ]
• Jewitt C, Clark W, Hadjithoma-Garstka C. The use of learning platforms to organise learning in English primary and secondary schools. Learning, Media and Technology. 2011; 36 (4):335–348. doi: 10.1080/17439884.2011.621955. [ CrossRef ] [ Google Scholar ]
• JISC. (2020). What is digital transformation?.  Retrieved 30 June 2022 from: https://www.jisc.ac.uk/guides/digital-strategy-framework-for-university-leaders/what-is-digital-transformation
• Kalati, A. T., & Kim, M. S. (2022). What is the effect of touchscreen technology on young children’s learning?: A systematic review. Education and Information Technologies , 1-19. 10.1007/s10639-021-10816-5
• Kalemkuş, J., & Kalemkuş, F. (2022). Effect of the use of augmented reality applications on academic achievement of student in science education: Meta-analysis review. Interactive Learning Environments , 1-18. 10.1080/10494820.2022.2027458
• Kao C-W. The effects of digital game-based learning task in English as a foreign language contexts: A meta-analysis. Education Journal. 2014; 42 (2):113–141. [ Google Scholar ]
• Kampylis P, Punie Y, Devine J. Promoting effective digital-age learning - a European framework for digitally competent educational organisations. JRC Technical Reports. 2015 doi: 10.2791/54070. [ CrossRef ] [ Google Scholar ]
• Kazu IY, Yalçin CK. Investigation of the effectiveness of hybrid learning on academic achievement: A meta-analysis study. International Journal of Progressive Education. 2022; 18 (1):249–265. doi: 10.29329/ijpe.2022.426.14. [ CrossRef ] [ Google Scholar ]
• Koh C. A qualitative meta-analysis on the use of serious games to support learners with intellectual and developmental disabilities: What we know, what we need to know and what we can do. International Journal of Disability, Development and Education. 2022; 69 (3):919–950. doi: 10.1080/1034912X.2020.1746245. [ CrossRef ] [ Google Scholar ]
• König J, Jäger-Biela DJ, Glutsch N. Adapting to online teaching during COVID-19 school closure: Teacher education and teacher competence effects among early career teachers in Germany. European Journal of Teacher Education. 2020; 43 (4):608–622. doi: 10.1080/02619768.2020.1809650. [ CrossRef ] [ Google Scholar ]
• Lawrence JE, Tar UA. Factors that influence teachers’ adoption and integration of ICT in teaching/learning process. Educational Media International. 2018; 55 (1):79–105. doi: 10.1080/09523987.2018.1439712. [ CrossRef ] [ Google Scholar ]
• Lee, S., Kuo, L. J., Xu, Z., & Hu, X. (2020). The effects of technology-integrated classroom instruction on K-12 English language learners’ literacy development: A meta-analysis. Computer Assisted Language Learning , 1-32. 10.1080/09588221.2020.1774612
• Lei, H., Chiu, M. M., Wang, D., Wang, C., & Xie, T. (2022a). Effects of game-based learning on students’ achievement in science: a meta-analysis. Journal of Educational Computing Research . 10.1177/07356331211064543
• Lei H, Wang C, Chiu MM, Chen S. Do educational games affect students' achievement emotions? Evidence from a meta-analysis. Journal of Computer Assisted Learning. 2022; 38 (4):946–959. doi: 10.1111/jcal.12664. [ CrossRef ] [ Google Scholar ]
• Liao YKC, Chang HW, Chen YW. Effects of computer application on elementary school student's achievement: A meta-analysis of students in Taiwan. Computers in the Schools. 2007; 24 (3–4):43–64. doi: 10.1300/J025v24n03_04. [ CrossRef ] [ Google Scholar ]
• Li Q, Ma X. A meta-analysis of the effects of computer technology on school students’ mathematics learning. Educational Psychology Review. 2010; 22 (3):215–243. doi: 10.1007/s10648-010-9125-8. [ CrossRef ] [ Google Scholar ]
• Liu, M., Pang, W., Guo, J., & Zhang, Y. (2022). A meta-analysis of the effect of multimedia technology on creative performance. Education and Information Technologies , 1-28. 10.1007/s10639-022-10981-1
• Lu Z, Chiu MM, Cui Y, Mao W, Lei H. Effects of game-based learning on students’ computational thinking: A meta-analysis. Journal of Educational Computing Research. 2022 doi: 10.1177/07356331221100740. [ CrossRef ] [ Google Scholar ]
• Martinez L, Gimenes M, Lambert E. Entertainment video games for academic learning: A systematic review. Journal of Educational Computing Research. 2022 doi: 10.1177/07356331211053848. [ CrossRef ] [ Google Scholar ]
• Mayne J. Useful theory of change models. Canadian Journal of Program Evaluation. 2015; 30 (2):119–142. doi: 10.3138/cjpe.230. [ CrossRef ] [ Google Scholar ]
• Moran J, Ferdig RE, Pearson PD, Wardrop J, Blomeyer RL., Jr Technology and reading performance in the middle-school grades: A meta-analysis with recommendations for policy and practice. Journal of Literacy Research. 2008; 40 (1):6–58. doi: 10.1080/10862960802070483. [ CrossRef ] [ Google Scholar ]
• OECD. (2015). Students, Computers and Learning: Making the Connection . PISA, OECD Publishing, Paris. Retrieved from: 10.1787/9789264239555-en
• OECD. (2021). OECD Digital Education Outlook 2021: Pushing the Frontiers with Artificial Intelligence, Blockchain and Robots. Retrieved from: https://www.oecd-ilibrary.org/education/oecd-digital-education-outlook-2021_589b283f-en
• Pan Y, Ke F, Xu X. A systematic review of the role of learning games in fostering mathematics education in K-12 settings. Educational Research Review. 2022; 36 :100448. doi: 10.1016/j.edurev.2022.100448. [ CrossRef ] [ Google Scholar ]
• Pettersson F. Understanding digitalization and educational change in school by means of activity theory and the levels of learning concept. Education and Information Technologies. 2021; 26 (1):187–204. doi: 10.1007/s10639-020-10239-8. [ CrossRef ] [ Google Scholar ]
• Pihir, I., Tomičić-Pupek, K., & Furjan, M. T. (2018). Digital transformation insights and trends. In Central European Conference on Information and Intelligent Systems (pp. 141–149). Faculty of Organization and Informatics Varazdin. Retrieved 30 June 2022 from https://www.proquest.com/conference-papers-proceedings/digital-transformation-insights-trends/docview/2125639934/se-2
• Punie, Y., Zinnbauer, D., & Cabrera, M. (2006). A review of the impact of ICT on learning. Working Paper prepared for DG EAC. Retrieved 30 June 2022 from: http://www.eurosfaire.prd.fr/7pc/doc/1224678677_jrc47246n.pdf
• Quah CY, Ng KH. A systematic literature review on digital storytelling authoring tool in education: January 2010 to January 2020. International Journal of Human-Computer Interaction. 2022; 38 (9):851–867. doi: 10.1080/10447318.2021.1972608. [ CrossRef ] [ Google Scholar ]
• Ran H, Kim NJ, Secada WG. A meta-analysis on the effects of technology's functions and roles on students' mathematics achievement in K-12 classrooms. Journal of computer assisted learning. 2022; 38 (1):258–284. doi: 10.1111/jcal.12611. [ CrossRef ] [ Google Scholar ]
• Ređep, N. B. (2021). Comparative overview of the digital preparedness of education systems in selected CEE countries. Center for Policy Studies. CEU Democracy Institute .
• Rott, B., & Marouane, C. (2018). Digitalization in schools–organization, collaboration and communication. In Digital Marketplaces Unleashed (pp. 113–124). Springer, Berlin, Heidelberg.
• Savva M, Higgins S, Beckmann N. Meta-analysis examining the effects of electronic storybooks on language and literacy outcomes for children in grades Pre-K to grade 2. Journal of Computer Assisted Learning. 2022; 38 (2):526–564. doi: 10.1111/jcal.12623. [ CrossRef ] [ Google Scholar ]
• Schmid RF, Bernard RM, Borokhovski E, Tamim RM, Abrami PC, Surkes MA, Wade CA, Woods J. The effects of technology use in postsecondary education: A meta-analysis of classroom applications. Computers & Education. 2014; 72 :271–291. doi: 10.1016/j.compedu.2013.11.002. [ CrossRef ] [ Google Scholar ]
• Schuele CM, Justice LM. The importance of effect sizes in the interpretation of research: Primer on research: Part 3. The ASHA Leader. 2006; 11 (10):14–27. doi: 10.1044/leader.FTR4.11102006.14. [ CrossRef ] [ Google Scholar ]
• Schwabe, A., Lind, F., Kosch, L., & Boomgaarden, H. G. (2022). No negative effects of reading on screen on comprehension of narrative texts compared to print: A meta-analysis. Media Psychology , 1-18. 10.1080/15213269.2022.2070216
• Sellar S. Data infrastructure: a review of expanding accountability systems and large-scale assessments in education. Discourse: Studies in the Cultural Politics of Education. 2015; 36 (5):765–777. doi: 10.1080/01596306.2014.931117. [ CrossRef ] [ Google Scholar ]
• Stock WA. Systematic coding for research synthesis. In: Cooper H, Hedges LV, editors. The handbook of research synthesis, 236. Russel Sage; 1994. pp. 125–138. [ Google Scholar ]
• Su, J., Zhong, Y., & Ng, D. T. K. (2022). A meta-review of literature on educational approaches for teaching AI at the K-12 levels in the Asia-Pacific region. Computers and Education: Artificial Intelligence , 100065. 10.1016/j.caeai.2022.100065
• Su J, Yang W. Artificial intelligence in early childhood education: A scoping review. Computers and Education: Artificial Intelligence. 2022; 3 :100049. doi: 10.1016/j.caeai.2022.100049. [ CrossRef ] [ Google Scholar ]
• Sung YT, Chang KE, Liu TC. The effects of integrating mobile devices with teaching and learning on students' learning performance: A meta-analysis and research synthesis. Computers & Education. 2016; 94 :252–275. doi: 10.1016/j.compedu.2015.11.008. [ CrossRef ] [ Google Scholar ]
• Talan T, Doğan Y, Batdı V. Efficiency of digital and non-digital educational games: A comparative meta-analysis and a meta-thematic analysis. Journal of Research on Technology in Education. 2020; 52 (4):474–514. doi: 10.1080/15391523.2020.1743798. [ CrossRef ] [ Google Scholar ]
• Tamim, R. M., Bernard, R. M., Borokhovski, E., Abrami, P. C., & Schmid, R. F. (2011). What forty years of research says about the impact of technology on learning: A second-order meta-analysis and validation study. Review of Educational research, 81 (1), 4–28. Retrieved 30 June 2022 from 10.3102/0034654310393361
• Tamim, R. M., Borokhovski, E., Pickup, D., Bernard, R. M., & El Saadi, L. (2015). Tablets for teaching and learning: A systematic review and meta-analysis. Commonwealth of Learning. Retrieved from: http://oasis.col.org/bitstream/handle/11599/1012/2015_Tamim-et-al_Tablets-for-Teaching-and-Learning.pdf
• Tang C, Mao S, Xing Z, Naumann S. Improving student creativity through digital technology products: A literature review. Thinking Skills and Creativity. 2022; 44 :101032. doi: 10.1016/j.tsc.2022.101032. [ CrossRef ] [ Google Scholar ]
• Tolani-Brown, N., McCormac, M., & Zimmermann, R. (2011). An analysis of the research and impact of ICT in education in developing country contexts. In ICTs and sustainable solutions for the digital divide: Theory and perspectives (pp. 218–242). IGI Global.
• Trucano, M. (2005). Knowledge Maps: ICTs in Education. Washington, DC: info Dev / World Bank. Retrieved 30 June 2022 from  https://files.eric.ed.gov/fulltext/ED496513.pdf
• Ulum H. The effects of online education on academic success: A meta-analysis study. Education and Information Technologies. 2022; 27 (1):429–450. doi: 10.1007/s10639-021-10740-8. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Underwood, J. D. (2009). The impact of digital technology: A review of the evidence of the impact of digital technologies on formal education. Retrieved 30 June 2022 from: http://dera.ioe.ac.uk/id/eprint/10491
• Verschaffel, L., Depaepe, F., & Mevarech, Z. (2019). Learning Mathematics in metacognitively oriented ICT-Based learning environments: A systematic review of the literature. Education Research International , 2019 . 10.1155/2019/3402035
• Villena-Taranilla R, Tirado-Olivares S, Cózar-Gutiérrez R, González-Calero JA. Effects of virtual reality on learning outcomes in K-6 education: A meta-analysis. Educational Research Review. 2022; 35 :100434. doi: 10.1016/j.edurev.2022.100434. [ CrossRef ] [ Google Scholar ]
• Voogt J, Knezek G, Cox M, Knezek D, ten Brummelhuis A. Under which conditions does ICT have a positive effect on teaching and learning? A call to action. Journal of Computer Assisted Learning. 2013; 29 (1):4–14. doi: 10.1111/j.1365-2729.2011.00453.x. [ CrossRef ] [ Google Scholar ]
• Vuorikari, R., Punie, Y., & Cabrera, M. (2020). Emerging technologies and the teaching profession: Ethical and pedagogical considerations based on near-future scenarios  (No. JRC120183). Joint Research Centre. Retrieved 30 June 2022 from: https://publications.jrc.ec.europa.eu/repository/handle/JRC120183
• Wang LH, Chen B, Hwang GJ, Guan JQ, Wang YQ. Effects of digital game-based STEM education on students’ learning achievement: A meta-analysis. International Journal of STEM Education. 2022; 9 (1):1–13. doi: 10.1186/s40594-022-00344-0. [ CrossRef ] [ Google Scholar ]
• Wen X, Walters SM. The impact of technology on students’ writing performances in elementary classrooms: A meta-analysis. Computers and Education Open. 2022; 3 :100082. doi: 10.1016/j.caeo.2022.100082. [ CrossRef ] [ Google Scholar ]
• Zheng B, Warschauer M, Lin CH, Chang C. Learning in one-to-one laptop environments: A meta-analysis and research synthesis. Review of Educational Research. 2016; 86 (4):1052–1084. doi: 10.3102/0034654316628645. [ CrossRef ] [ Google Scholar ]

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• My Account Login
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Review Article
• Open access
• Published: 12 February 2024

Education reform and change driven by digital technology: a bibliometric study from a global perspective

• Chengliang Wang 1 ,
• Xiaojiao Chen 1 ,
• Teng Yu   ORCID: orcid.org/0000-0001-5198-7261 2 , 3 ,
• Yidan Liu 1 , 4 &
• Yuhui Jing 1  

Humanities and Social Sciences Communications volume  11 , Article number:  256 ( 2024 ) Cite this article

7521 Accesses

1 Citations

1 Altmetric

Metrics details

• Development studies
• Science, technology and society

Amidst the global digital transformation of educational institutions, digital technology has emerged as a significant area of interest among scholars. Such technologies have played an instrumental role in enhancing learner performance and improving the effectiveness of teaching and learning. These digital technologies also ensure the sustainability and stability of education during the epidemic. Despite this, a dearth of systematic reviews exists regarding the current state of digital technology application in education. To address this gap, this study utilized the Web of Science Core Collection as a data source (specifically selecting the high-quality SSCI and SCIE) and implemented a topic search by setting keywords, yielding 1849 initial publications. Furthermore, following the PRISMA guidelines, we refined the selection to 588 high-quality articles. Using software tools such as CiteSpace, VOSviewer, and Charticulator, we reviewed these 588 publications to identify core authors (such as Selwyn, Henderson, Edwards), highly productive countries/regions (England, Australia, USA), key institutions (Monash University, Australian Catholic University), and crucial journals in the field ( Education and Information Technologies , Computers & Education , British Journal of Educational Technology ). Evolutionary analysis reveals four developmental periods in the research field of digital technology education application: the embryonic period, the preliminary development period, the key exploration, and the acceleration period of change. The study highlights the dual influence of technological factors and historical context on the research topic. Technology is a key factor in enabling education to transform and upgrade, and the context of the times is an important driving force in promoting the adoption of new technologies in the education system and the transformation and upgrading of education. Additionally, the study identifies three frontier hotspots in the field: physical education, digital transformation, and professional development under the promotion of digital technology. This study presents a clear framework for digital technology application in education, which can serve as a valuable reference for researchers and educational practitioners concerned with digital technology education application in theory and practice.

Similar content being viewed by others

A bibliometric analysis of knowledge mapping in Chinese education digitalization research from 2012 to 2022

Digital transformation and digital literacy in the context of complexity within higher education institutions: a systematic literature review

Education big data and learning analytics: a bibliometric analysis

Introduction.

Digital technology has become an essential component of modern education, facilitating the extension of temporal and spatial boundaries and enriching the pedagogical contexts (Selwyn and Facer, 2014 ). The advent of mobile communication technology has enabled learning through social media platforms (Szeto et al. 2015 ; Pires et al. 2022 ), while the advancement of augmented reality technology has disrupted traditional conceptions of learning environments and spaces (Perez-Sanagustin et al., 2014 ; Kyza and Georgiou, 2018 ). A wide range of digital technologies has enabled learning to become a norm in various settings, including the workplace (Sjöberg and Holmgren, 2021 ), home (Nazare et al. 2022 ), and online communities (Tang and Lam, 2014 ). Education is no longer limited to fixed locations and schedules, but has permeated all aspects of life, allowing learning to continue at any time and any place (Camilleri and Camilleri, 2016 ; Selwyn and Facer, 2014 ).

The advent of digital technology has led to the creation of several informal learning environments (Greenhow and Lewin, 2015 ) that exhibit divergent form, function, features, and patterns in comparison to conventional learning environments (Nygren et al. 2019 ). Consequently, the associated teaching and learning processes, as well as the strategies for the creation, dissemination, and acquisition of learning resources, have undergone a complete overhaul. The ensuing transformations have posed a myriad of novel issues, such as the optimal structuring of teaching methods by instructors and the adoption of appropriate learning strategies by students in the new digital technology environment. Consequently, an examination of the principles that underpin effective teaching and learning in this environment is a topic of significant interest to numerous scholars engaged in digital technology education research.

Over the course of the last two decades, digital technology has made significant strides in the field of education, notably in extending education time and space and creating novel educational contexts with sustainability. Despite research attempts to consolidate the application of digital technology in education, previous studies have only focused on specific aspects of digital technology, such as Pinto and Leite’s ( 2020 ) investigation into digital technology in higher education and Mustapha et al.’s ( 2021 ) examination of the role and value of digital technology in education during the pandemic. While these studies have provided valuable insights into the practical applications of digital technology in particular educational domains, they have not comprehensively explored the macro-mechanisms and internal logic of digital technology implementation in education. Additionally, these studies were conducted over a relatively brief period, making it challenging to gain a comprehensive understanding of the macro-dynamics and evolutionary process of digital technology in education. Some studies have provided an overview of digital education from an educational perspective but lack a precise understanding of technological advancement and change (Yang et al. 2022 ). Therefore, this study seeks to employ a systematic scientific approach to collate relevant research from 2000 to 2022, comprehend the internal logic and development trends of digital technology in education, and grasp the outstanding contribution of digital technology in promoting the sustainability of education in time and space. In summary, this study aims to address the following questions:

RQ1: Since the turn of the century, what is the productivity distribution of the field of digital technology education application research in terms of authorship, country/region, institutional and journal level?

RQ2: What is the development trend of research on the application of digital technology in education in the past two decades?

RQ3: What are the current frontiers of research on the application of digital technology in education?

Literature review

Although the term “digital technology” has become ubiquitous, a unified definition has yet to be agreed upon by scholars. Because the meaning of the word digital technology is closely related to the specific context. Within the educational research domain, Selwyn’s ( 2016 ) definition is widely favored by scholars (Pinto and Leite, 2020 ). Selwyn ( 2016 ) provides a comprehensive view of various concrete digital technologies and their applications in education through ten specific cases, such as immediate feedback in classes, orchestrating teaching, and community learning. Through these specific application scenarios, Selwyn ( 2016 ) argues that digital technology encompasses technologies associated with digital devices, including but not limited to tablets, smartphones, computers, and social media platforms (such as Facebook and YouTube). Furthermore, Further, the behavior of accessing the internet at any location through portable devices can be taken as an extension of the behavior of applying digital technology.

The evolving nature of digital technology has significant implications in the field of education. In the 1890s, the focus of digital technology in education was on comprehending the nuances of digital space, digital culture, and educational methodologies, with its connotations aligned more towards the idea of e-learning. The advent and subsequent widespread usage of mobile devices since the dawn of the new millennium have been instrumental in the rapid expansion of the concept of digital technology. Notably, mobile learning devices such as smartphones and tablets, along with social media platforms, have become integral components of digital technology (Conole and Alevizou, 2010 ; Batista et al. 2016 ). In recent times, the burgeoning application of AI technology in the education sector has played a vital role in enriching the digital technology lexicon (Banerjee et al. 2021 ). ChatGPT, for instance, is identified as a novel educational technology that has immense potential to revolutionize future education (Rospigliosi, 2023 ; Arif, Munaf and Ul-Haque, 2023 ).

Pinto and Leite ( 2020 ) conducted a comprehensive macroscopic survey of the use of digital technologies in the education sector and identified three distinct categories, namely technologies for assessment and feedback, mobile technologies, and Information Communication Technologies (ICT). This classification criterion is both macroscopic and highly condensed. In light of the established concept definitions of digital technology in the educational research literature, this study has adopted the characterizations of digital technology proposed by Selwyn ( 2016 ) and Pinto and Leite ( 2020 ) as crucial criteria for analysis and research inclusion. Specifically, this criterion encompasses several distinct types of digital technologies, including Information and Communication Technologies (ICT), Mobile tools, eXtended Reality (XR) Technologies, Assessment and Feedback systems, Learning Management Systems (LMS), Publish and Share tools, Collaborative systems, Social media, Interpersonal Communication tools, and Content Aggregation tools.

Methodology and materials

Research method: bibliometric.

The research on econometric properties has been present in various aspects of human production and life, yet systematic scientific theoretical guidance has been lacking, resulting in disorganization. In 1969, British scholar Pritchard ( 1969 ) proposed “bibliometrics,” which subsequently emerged as an independent discipline in scientific quantification research. Initially, Pritchard defined bibliometrics as “the application of mathematical and statistical methods to books and other media of communication,” however, the definition was not entirely rigorous. To remedy this, Hawkins ( 2001 ) expanded Pritchard’s definition to “the quantitative analysis of the bibliographic features of a body of literature.” De Bellis further clarified the objectives of bibliometrics, stating that it aims to analyze and identify patterns in literature, such as the most productive authors, institutions, countries, and journals in scientific disciplines, trends in literary production over time, and collaboration networks (De Bellis, 2009 ). According to Garfield ( 2006 ), bibliometric research enables the examination of the history and structure of a field, the flow of information within the field, the impact of journals, and the citation status of publications over a longer time scale. All of these definitions illustrate the unique role of bibliometrics as a research method for evaluating specific research fields.

This study uses CiteSpace, VOSviewer, and Charticulator to analyze data and create visualizations. Each of these three tools has its own strengths and can complement each other. CiteSpace and VOSviewer use set theory and probability theory to provide various visualization views in fields such as keywords, co-occurrence, and co-authors. They are easy to use and produce visually appealing graphics (Chen, 2006 ; van Eck and Waltman, 2009 ) and are currently the two most widely used bibliometric tools in the field of visualization (Pan et al. 2018 ). In this study, VOSviewer provided the data necessary for the Performance Analysis; Charticulator was then used to redraw using the tabular data exported from VOSviewer (for creating the chord diagram of country collaboration); this was to complement the mapping process, while CiteSpace was primarily utilized to generate keyword maps and conduct burst word analysis.

Data retrieval

This study selected documents from the Science Citation Index Expanded (SCIE) and Social Science Citation Index (SSCI) in the Web of Science Core Collection as the data source, for the following reasons:

(1) The Web of Science Core Collection, as a high-quality digital literature resource database, has been widely accepted by many researchers and is currently considered the most suitable database for bibliometric analysis (Jing et al. 2023a ). Compared to other databases, Web of Science provides more comprehensive data information (Chen et al. 2022a ), and also provides data formats suitable for analysis using VOSviewer and CiteSpace (Gaviria-Marin et al. 2019 ).

(2) The application of digital technology in the field of education is an interdisciplinary research topic, involving technical knowledge literature belonging to the natural sciences and education-related literature belonging to the social sciences. Therefore, it is necessary to select Science Citation Index Expanded (SCIE) and Social Science Citation Index (SSCI) as the sources of research data, ensuring the comprehensiveness of data while ensuring the reliability and persuasiveness of bibliometric research (Hwang and Tsai, 2011 ; Wang et al. 2022 ).

After establishing the source of research data, it is necessary to determine a retrieval strategy (Jing et al. 2023b ). The choice of a retrieval strategy should consider a balance between the breadth and precision of the search formula. That is to say, it should encompass all the literature pertaining to the research topic while excluding irrelevant documents as much as possible. In light of this, this study has set a retrieval strategy informed by multiple related papers (Mustapha et al. 2021 ; Luo et al. 2021 ). The research by Mustapha et al. ( 2021 ) guided us in selecting keywords (“digital” AND “technolog*”) to target digital technology, while Luo et al. ( 2021 ) informed the selection of terms (such as “instruct*,” “teach*,” and “education”) to establish links with the field of education. Then, based on the current application of digital technology in the educational domain and the scope of selection criteria, we constructed the final retrieval strategy. Following the general patterns of past research (Jing et al. 2023a , 2023b ), we conducted a specific screening using the topic search (Topics, TS) function in Web of Science. For the specific criteria used in the screening for this study, please refer to Table 1 .

Literature screening

Literature acquired through keyword searches may contain ostensibly related yet actually unrelated works. Therefore, to ensure the close relevance of literature included in the analysis to the research topic, it is often necessary to perform a manual screening process to identify the final literature to be analyzed, subsequent to completing the initial literature search.

The manual screening process consists of two steps. Initially, irrelevant literature is weeded out based on the title and abstract, with two members of the research team involved in this phase. This stage lasted about one week, resulting in 1106 articles being retained. Subsequently, a comprehensive review of the full text is conducted to accurately identify the literature required for the study. To carry out the second phase of manual screening effectively and scientifically, and to minimize the potential for researcher bias, the research team established the inclusion criteria presented in Table 2 . Three members were engaged in this phase, which took approximately 2 weeks, culminating in the retention of 588 articles after meticulous screening. The entire screening process is depicted in Fig. 1 , adhering to the PRISMA guidelines (Page et al. 2021 ).

The process of obtaining and filtering the necessary literature data for research.

Data standardization

Nguyen and Hallinger ( 2020 ) pointed out that raw data extracted from scientific databases often contains multiple expressions of the same term, and not addressing these synonymous expressions could affect research results in bibliometric analysis. For instance, in the original data, the author list may include “Tsai, C. C.” and “Tsai, C.-C.”, while the keyword list may include “professional-development” and “professional development,” which often require merging. Therefore, before analyzing the selected literature, a data disambiguation process is necessary to standardize the data (Strotmann and Zhao, 2012 ; Van Eck and Waltman, 2019 ). This study adopted the data standardization process proposed by Taskin and Al ( 2019 ), mainly including the following standardization operations:

Firstly, the author and source fields in the data are corrected and standardized to differentiate authors with similar names.

Secondly, the study checks whether the journals to which the literature belongs have been renamed in the past over 20 years, so as to avoid the influence of periodical name change on the analysis results.

Finally, the keyword field is standardized by unifying parts of speech and singular/plural forms of keywords, which can help eliminate redundant entries in the knowledge graph.

Performance analysis (RQ1)

This section offers a thorough and detailed analysis of the state of research in the field of digital technology education. By utilizing descriptive statistics and visual maps, it provides a comprehensive overview of the development trends, authors, countries, institutions, and journal distribution within the field. The insights presented in this section are of great significance in advancing our understanding of the current state of research in this field and identifying areas for further investigation. The use of visual aids to display inter-country cooperation and the evolution of the field adds to the clarity and coherence of the analysis.

Time trend of the publications

To understand a research field, it is first necessary to understand the most basic quantitative information, among which the change in the number of publications per year best reflects the development trend of a research field. Figure 2 shows the distribution of publication dates.

Time trend of the publications on application of digital technology in education.

From the Fig. 2 , it can be seen that the development of this field over the past over 20 years can be roughly divided into three stages. The first stage was from 2000 to 2007, during which the number of publications was relatively low. Due to various factors such as technological maturity, the academic community did not pay widespread attention to the role of digital technology in expanding the scope of teaching and learning. The second stage was from 2008 to 2019, during which the overall number of publications showed an upward trend, and the development of the field entered an accelerated period, attracting more and more scholars’ attention. The third stage was from 2020 to 2022, during which the number of publications stabilized at around 100. During this period, the impact of the pandemic led to a large number of scholars focusing on the role of digital technology in education during the pandemic, and research on the application of digital technology in education became a core topic in social science research.

Analysis of authors

An analysis of the author’s publication volume provides information about the representative scholars and core research strengths of a research area. Table 3 presents information on the core authors in adaptive learning research, including name, publication number, and average number of citations per article (based on the analysis and statistics from VOSviewer).

Variations in research foci among scholars abound. Within the field of digital technology education application research over the past two decades, Neil Selwyn stands as the most productive author, having published 15 papers garnering a total of 1027 citations, resulting in an average of 68.47 citations per paper. As a Professor at the Faculty of Education at Monash University, Selwyn concentrates on exploring the application of digital technology in higher education contexts (Selwyn et al. 2021 ), as well as related products in higher education such as Coursera, edX, and Udacity MOOC platforms (Bulfin et al. 2014 ). Selwyn’s contributions to the educational sociology perspective include extensive research on the impact of digital technology on education, highlighting the spatiotemporal extension of educational processes and practices through technological means as the greatest value of educational technology (Selwyn, 2012 ; Selwyn and Facer, 2014 ). In addition, he provides a blueprint for the development of future schools in 2030 based on the present impact of digital technology on education (Selwyn et al. 2019 ). The second most productive author in this field, Henderson, also offers significant contributions to the understanding of the important value of digital technology in education, specifically in the higher education setting, with a focus on the impact of the pandemic (Henderson et al. 2015 ; Cohen et al. 2022 ). In contrast, Edwards’ research interests focus on early childhood education, particularly the application of digital technology in this context (Edwards, 2013 ; Bird and Edwards, 2015 ). Additionally, on the technical level, Edwards also mainly prefers digital game technology, because it is a digital technology that children are relatively easy to accept (Edwards, 2015 ).

Analysis of countries/regions and organization

The present study aimed to ascertain the leading countries in digital technology education application research by analyzing 75 countries related to 558 works of literature. Table 4 depicts the top ten countries that have contributed significantly to this field in terms of publication count (based on the analysis and statistics from VOSviewer). Our analysis of Table 4 data shows that England emerged as the most influential country/region, with 92 published papers and 2401 citations. Australia and the United States secured the second and third ranks, respectively, with 90 papers (2187 citations) and 70 papers (1331 citations) published. Geographically, most of the countries featured in the top ten publication volumes are situated in Australia, North America, and Europe, with China being the only exception. Notably, all these countries, except China, belong to the group of developed nations, suggesting that economic strength is a prerequisite for fostering research in the digital technology education application field.

This study presents a visual representation of the publication output and cooperation relationships among different countries in the field of digital technology education application research. Specifically, a chord diagram is employed to display the top 30 countries in terms of publication output, as depicted in Fig. 3 . The chord diagram is composed of nodes and chords, where the nodes are positioned as scattered points along the circumference, and the length of each node corresponds to the publication output, with longer lengths indicating higher publication output. The chords, on the other hand, represent the cooperation relationships between any two countries, and are weighted based on the degree of closeness of the cooperation, with wider chords indicating closer cooperation. Through the analysis of the cooperation relationships, the findings suggest that the main publishing countries in this field are engaged in cooperative relationships with each other, indicating a relatively high level of international academic exchange and research internationalization.

In the diagram, nodes are scattered along the circumference of a circle, with the length of each node representing the volume of publications. The weighted arcs connecting any two points on the circle are known as chords, representing the collaborative relationship between the two, with the width of the arc indicating the closeness of the collaboration.

Further analyzing Fig. 3 , we can extract more valuable information, enabling a deeper understanding of the connections between countries in the research field of digital technology in educational applications. It is evident that certain countries, such as the United States, China, and England, display thicker connections, indicating robust collaborative relationships in terms of productivity. These thicker lines signify substantial mutual contributions and shared objectives in certain sectors or fields, highlighting the interconnectedness and global integration in these areas. By delving deeper, we can also explore potential future collaboration opportunities through the chord diagram, identifying possible partners to propel research and development in this field. In essence, the chord diagram successfully encapsulates and conveys the multi-dimensionality of global productivity and cooperation, allowing for a comprehensive understanding of the intricate inter-country relationships and networks in a global context, providing valuable guidance and insights for future research and collaborations.

An in-depth examination of the publishing institutions is provided in Table 5 , showcasing the foremost 10 institutions ranked by their publication volume. Notably, Monash University and Australian Catholic University, situated in Australia, have recorded the most prolific publications within the digital technology education application realm, with 22 and 10 publications respectively. Moreover, the University of Oslo from Norway is featured among the top 10 publishing institutions, with an impressive average citation count of 64 per publication. It is worth highlighting that six institutions based in the United Kingdom were also ranked within the top 10 publishing institutions, signifying their leading position in this area of research.

Analysis of journals

Journals are the main carriers for publishing high-quality papers. Some scholars point out that the two key factors to measure the influence of journals in the specified field are the number of articles published and the number of citations. The more papers published in a magazine and the more citations, the greater its influence (Dzikowski, 2018 ). Therefore, this study utilized VOSviewer to statistically analyze the top 10 journals with the most publications in the field of digital technology in education and calculated the average citations per article (see Table 6 ).

Based on Table 6 , it is apparent that the highest number of articles in the domain of digital technology in education research were published in Education and Information Technologies (47 articles), Computers & Education (34 articles), and British Journal of Educational Technology (32 articles), indicating a higher article output compared to other journals. This underscores the fact that these three journals concentrate more on the application of digital technology in education. Furthermore, several other journals, such as Technology Pedagogy and Education and Sustainability, have published more than 15 articles in this domain. Sustainability represents the open access movement, which has notably facilitated research progress in this field, indicating that the development of open access journals in recent years has had a significant impact. Although there is still considerable disagreement among scholars on the optimal approach to achieve open access, the notion that research outcomes should be accessible to all is widely recognized (Huang et al. 2020 ). On further analysis of the research fields to which these journals belong, except for Sustainability, it is evident that they all pertain to educational technology, thus providing a qualitative definition of the research area of digital technology education from the perspective of journals.

Temporal keyword analysis: thematic evolution (RQ2)

The evolution of research themes is a dynamic process, and previous studies have attempted to present the developmental trajectory of fields by drawing keyword networks in phases (Kumar et al. 2021 ; Chen et al. 2022b ). To understand the shifts in research topics across different periods, this study follows past research and, based on the significant changes in the research field and corresponding technological advancements during the outlined periods, divides the timeline into four stages (the first stage from January 2000 to December 2005, the second stage from January 2006 to December 2011, the third stage from January 2012 to December 2017; and the fourth stage from January 2018 to December 2022). The division into these four stages was determined through a combination of bibliometric analysis and literature review, which presented a clear trajectory of the field’s development. The research analyzes the keyword networks for each time period (as there are only three articles in the first stage, it was not possible to generate an appropriate keyword co-occurrence map, hence only the keyword co-occurrence maps from the second to the fourth stages are provided), to understand the evolutionary track of the digital technology education application research field over time.

2000.1–2005.12: germination period

From January 2000 to December 2005, digital technology education application research was in its infancy. Only three studies focused on digital technology, all of which were related to computers. Due to the popularity of computers, the home became a new learning environment, highlighting the important role of digital technology in expanding the scope of learning spaces (Sutherland et al. 2000 ). In specific disciplines and contexts, digital technology was first favored in medical clinical practice, becoming an important tool for supporting the learning of clinical knowledge and practice (Tegtmeyer et al. 2001 ; Durfee et al. 2003 ).

2006.1–2011.12: initial development period

Between January 2006 and December 2011, it was the initial development period of digital technology education research. Significant growth was observed in research related to digital technology, and discussions and theoretical analyses about “digital natives” emerged. During this phase, scholars focused on the debate about “how to use digital technology reasonably” and “whether current educational models and school curriculum design need to be adjusted on a large scale” (Bennett and Maton, 2010 ; Selwyn, 2009 ; Margaryan et al. 2011 ). These theoretical and speculative arguments provided a unique perspective on the impact of cognitive digital technology on education and teaching. As can be seen from the vocabulary such as “rethinking”, “disruptive pedagogy”, and “attitude” in Fig. 4 , many scholars joined the calm reflection and analysis under the trend of digital technology (Laurillard, 2008 ; Vratulis et al. 2011 ). During this phase, technology was still undergoing dramatic changes. The development of mobile technology had already caught the attention of many scholars (Wong et al. 2011 ), but digital technology represented by computers was still very active (Selwyn et al. 2011 ). The change in technological form would inevitably lead to educational transformation. Collins and Halverson ( 2010 ) summarized the prospects and challenges of using digital technology for learning and educational practices, believing that digital technology would bring a disruptive revolution to the education field and bring about a new educational system. In addition, the term “teacher education” in Fig. 4 reflects the impact of digital technology development on teachers. The rapid development of technology has widened the generation gap between teachers and students. To ensure smooth communication between teachers and students, teachers must keep up with the trend of technological development and establish a lifelong learning concept (Donnison, 2009 ).

In the diagram, each node represents a keyword, with the size of the node indicating the frequency of occurrence of the keyword. The connections represent the co-occurrence relationships between keywords, with a higher frequency of co-occurrence resulting in tighter connections.

2012.1–2017.12: critical exploration period

During the period spanning January 2012 to December 2017, the application of digital technology in education research underwent a significant exploration phase. As can be seen from Fig. 5 , different from the previous stage, the specific elements of specific digital technology have started to increase significantly, including the enrichment of technological contexts, the greater variety of research methods, and the diversification of learning modes. Moreover, the temporal and spatial dimensions of the learning environment were further de-emphasized, as noted in previous literature (Za et al. 2014 ). Given the rapidly accelerating pace of technological development, the education system in the digital era is in urgent need of collaborative evolution and reconstruction, as argued by Davis, Eickelmann, and Zaka ( 2013 ).

In the domain of digital technology, social media has garnered substantial scholarly attention as a promising avenue for learning, as noted by Pasquini and Evangelopoulos ( 2016 ). The implementation of social media in education presents several benefits, including the liberation of education from the restrictions of physical distance and time, as well as the erasure of conventional educational boundaries. The user-generated content (UGC) model in social media has emerged as a crucial source for knowledge creation and distribution, with the widespread adoption of mobile devices. Moreover, social networks have become an integral component of ubiquitous learning environments (Hwang et al. 2013 ). The utilization of social media allows individuals to function as both knowledge producers and recipients, which leads to a blurring of the conventional roles of learners and teachers. On mobile platforms, the roles of learners and teachers are not fixed, but instead interchangeable.

In terms of research methodology, the prevalence of empirical studies with survey designs in the field of educational technology during this period is evident from the vocabulary used, such as “achievement,” “acceptance,” “attitude,” and “ict.” in Fig. 5 . These studies aim to understand learners’ willingness to adopt and attitudes towards new technologies, and some seek to investigate the impact of digital technologies on learning outcomes through quasi-experimental designs (Domínguez et al. 2013 ). Among these empirical studies, mobile learning emerged as a hot topic, and this is not surprising. First, the advantages of mobile learning environments over traditional ones have been empirically demonstrated (Hwang et al. 2013 ). Second, learners born around the turn of the century have been heavily influenced by digital technologies and have developed their own learning styles that are more open to mobile devices as a means of learning. Consequently, analyzing mobile learning as a relatively novel mode of learning has become an important issue for scholars in the field of educational technology.

The intervention of technology has led to the emergence of several novel learning modes, with the blended learning model being the most representative one in the current phase. Blended learning, a novel concept introduced in the information age, emphasizes the integration of the benefits of traditional learning methods and online learning. This learning mode not only highlights the prominent role of teachers in guiding, inspiring, and monitoring the learning process but also underlines the importance of learners’ initiative, enthusiasm, and creativity in the learning process. Despite being an early conceptualization, blended learning’s meaning has been expanded by the widespread use of mobile technology and social media in education. The implementation of new technologies, particularly mobile devices, has resulted in the transformation of curriculum design and increased flexibility and autonomy in students’ learning processes (Trujillo Maza et al. 2016 ), rekindling scholarly attention to this learning mode. However, some scholars have raised concerns about the potential drawbacks of the blended learning model, such as its significant impact on the traditional teaching system, the lack of systematic coping strategies and relevant policies in several schools and regions (Moskal et al. 2013 ).

2018.1–2022.12: accelerated transformation period

The period spanning from January 2018 to December 2022 witnessed a rapid transformation in the application of digital technology in education research. The field of digital technology education research reached a peak period of publication, largely influenced by factors such as the COVID-19 pandemic (Yu et al. 2023 ). Research during this period was built upon the achievements, attitudes, and social media of the previous phase, and included more elements that reflect the characteristics of this research field, such as digital literacy, digital competence, and professional development, as depicted in Fig. 6 . Alongside this, scholars’ expectations for the value of digital technology have expanded, and the pursuit of improving learning efficiency and performance is no longer the sole focus. Some research now aims to cultivate learners’ motivation and enhance their self-efficacy by applying digital technology in a reasonable manner, as demonstrated by recent studies (Beardsley et al. 2021 ; Creely et al. 2021 ).

The COVID-19 pandemic has emerged as a crucial backdrop for the digital technology’s role in sustaining global education, as highlighted by recent scholarly research (Zhou et al. 2022 ; Pan and Zhang, 2020 ; Mo et al. 2022 ). The online learning environment, which is supported by digital technology, has become the primary battleground for global education (Yu, 2022 ). This social context has led to various studies being conducted, with some scholars positing that the pandemic has impacted the traditional teaching order while also expanding learning possibilities in terms of patterns and forms (Alabdulaziz, 2021 ). Furthermore, the pandemic has acted as a catalyst for teacher teaching and technological innovation, and this viewpoint has been empirically substantiated (Moorhouse and Wong, 2021 ). Additionally, some scholars believe that the pandemic’s push is a crucial driving force for the digital transformation of the education system, serving as an essential mechanism for overcoming the system’s inertia (Romero et al. 2021 ).

The rapid outbreak of the pandemic posed a challenge to the large-scale implementation of digital technologies, which was influenced by a complex interplay of subjective and objective factors. Objective constraints included the lack of infrastructure in some regions to support digital technologies, while subjective obstacles included psychological resistance among certain students and teachers (Moorhouse, 2021 ). These factors greatly impacted the progress of online learning during the pandemic. Additionally, Timotheou et al. ( 2023 ) conducted a comprehensive systematic review of existing research on digital technology use during the pandemic, highlighting the critical role played by various factors such as learners’ and teachers’ digital skills, teachers’ personal attributes and professional development, school leadership and management, and administration in facilitating the digitalization and transformation of schools.

The current stage of research is characterized by the pivotal term “digital literacy,” denoting a growing interest in learners’ attitudes and adoption of emerging technologies. Initially, the term “literacy” was restricted to fundamental abilities and knowledge associated with books and print materials (McMillan, 1996 ). However, with the swift advancement of computers and digital technology, there have been various attempts to broaden the scope of literacy beyond its traditional meaning, including game literacy (Buckingham and Burn, 2007 ), information literacy (Eisenberg, 2008 ), and media literacy (Turin and Friesem, 2020 ). Similarly, digital literacy has emerged as a crucial concept, and Gilster and Glister ( 1997 ) were the first to introduce this concept, referring to the proficiency in utilizing technology and processing digital information in academic, professional, and daily life settings. In practical educational settings, learners who possess higher digital literacy often exhibit an aptitude for quickly mastering digital devices and applying them intelligently to education and teaching (Yu, 2022 ).

The utilization of digital technology in education has undergone significant changes over the past two decades, and has been a crucial driver of educational reform with each new technological revolution. The impact of these changes on the underlying logic of digital technology education applications has been noticeable. From computer technology to more recent developments such as virtual reality (VR), augmented reality (AR), and artificial intelligence (AI), the acceleration in digital technology development has been ongoing. Educational reforms spurred by digital technology development continue to be dynamic, as each new digital innovation presents new possibilities and models for teaching practice. This is especially relevant in the post-pandemic era, where the importance of technological progress in supporting teaching cannot be overstated (Mughal et al. 2022 ). Existing digital technologies have already greatly expanded the dimensions of education in both time and space, while future digital technologies aim to expand learners’ perceptions. Researchers have highlighted the potential of integrated technology and immersive technology in the development of the educational metaverse, which is highly anticipated to create a new dimension for the teaching and learning environment, foster a new value system for the discipline of educational technology, and more effectively and efficiently achieve the grand educational blueprint of the United Nations’ Sustainable Development Goals (Zhang et al. 2022 ; Li and Yu, 2023 ).

Hotspot evolution analysis (RQ3)

The examination of keyword evolution reveals a consistent trend in the advancement of digital technology education application research. The emergence and transformation of keywords serve as indicators of the varying research interests in this field. Thus, the utilization of the burst detection function available in CiteSpace allowed for the identification of the top 10 burst words that exhibited a high level of burst strength. This outcome is illustrated in Table 7 .

According to the results presented in Table 7 , the explosive terminology within the realm of digital technology education research has exhibited a concentration mainly between the years 2018 and 2022. Prior to this time frame, the emerging keywords were limited to “information technology” and “computer”. Notably, among them, computer, as an emergent keyword, has always had a high explosive intensity from 2008 to 2018, which reflects the important position of computer in digital technology and is the main carrier of many digital technologies such as Learning Management Systems (LMS) and Assessment and Feedback systems (Barlovits et al. 2022 ).

Since 2018, an increasing number of research studies have focused on evaluating the capabilities of learners to accept, apply, and comprehend digital technologies. As indicated by the use of terms such as “digital literacy” and “digital skill,” the assessment of learners’ digital literacy has become a critical task. Scholarly efforts have been directed towards the development of literacy assessment tools and the implementation of empirical assessments. Furthermore, enhancing the digital literacy of both learners and educators has garnered significant attention. (Nagle, 2018 ; Yu, 2022 ). Simultaneously, given the widespread use of various digital technologies in different formal and informal learning settings, promoting learners’ digital skills has become a crucial objective for contemporary schools (Nygren et al. 2019 ; Forde and OBrien, 2022 ).

Since 2020, the field of applied research on digital technology education has witnessed the emergence of three new hotspots, all of which have been affected to some extent by the pandemic. Firstly, digital technology has been widely applied in physical education, which is one of the subjects that has been severely affected by the pandemic (Parris et al. 2022 ; Jiang and Ning, 2022 ). Secondly, digital transformation has become an important measure for most schools, especially higher education institutions, to cope with the impact of the pandemic globally (García-Morales et al. 2021 ). Although the concept of digital transformation was proposed earlier, the COVID-19 pandemic has greatly accelerated this transformation process. Educational institutions must carefully redesign their educational products to face this new situation, providing timely digital learning methods, environments, tools, and support systems that have far-reaching impacts on modern society (Krishnamurthy, 2020 ; Salas-Pilco et al. 2022 ). Moreover, the professional development of teachers has become a key mission of educational institutions in the post-pandemic era. Teachers need to have a certain level of digital literacy and be familiar with the tools and online teaching resources used in online teaching, which has become a research hotspot today. Organizing digital skills training for teachers to cope with the application of emerging technologies in education is an important issue for teacher professional development and lifelong learning (Garzón-Artacho et al. 2021 ). As the main organizers and practitioners of emergency remote teaching (ERT) during the pandemic, teachers must put cognitive effort into their professional development to ensure effective implementation of ERT (Romero-Hall and Jaramillo Cherrez, 2022 ).

The burst word “digital transformation” reveals that we are in the midst of an ongoing digital technology revolution. With the emergence of innovative digital technologies such as ChatGPT and Microsoft 365 Copilot, technology trends will continue to evolve, albeit unpredictably. While the impact of these advancements on school education remains uncertain, it is anticipated that the widespread integration of technology will significantly affect the current education system. Rejecting emerging technologies without careful consideration is unwise. Like any revolution, the technological revolution in the education field has both positive and negative aspects. Detractors argue that digital technology disrupts learning and memory (Baron, 2021 ) or causes learners to become addicted and distracted from learning (Selwyn and Aagaard, 2020 ). On the other hand, the prudent use of digital technology in education offers a glimpse of a golden age of open learning. Educational leaders and practitioners have the opportunity to leverage cutting-edge digital technologies to address current educational challenges and develop a rational path for the sustainable and healthy growth of education.

Discussion on performance analysis (RQ1)

The field of digital technology education application research has experienced substantial growth since the turn of the century, a phenomenon that is quantifiably apparent through an analysis of authorship, country/region contributions, and institutional engagement. This expansion reflects the increased integration of digital technologies in educational settings and the heightened scholarly interest in understanding and optimizing their use.

Discussion on authorship productivity in digital technology education research

The authorship distribution within digital technology education research is indicative of the field’s intellectual structure and depth. A primary figure in this domain is Neil Selwyn, whose substantial citation rate underscores the profound impact of his work. His focus on the implications of digital technology in higher education and educational sociology has proven to be seminal. Selwyn’s research trajectory, especially the exploration of spatiotemporal extensions of education through technology, provides valuable insights into the multifaceted role of digital tools in learning processes (Selwyn et al. 2019 ).

Other notable contributors, like Henderson and Edwards, present diversified research interests, such as the impact of digital technologies during the pandemic and their application in early childhood education, respectively. Their varied focuses highlight the breadth of digital technology education research, encompassing pedagogical innovation, technological adaptation, and policy development.

Discussion on country/region-level productivity and collaboration

At the country/region level, the United Kingdom, specifically England, emerges as a leading contributor with 92 published papers and a significant citation count. This is closely followed by Australia and the United States, indicating a strong English-speaking research axis. Such geographical concentration of scholarly output often correlates with investment in research and development, technological infrastructure, and the prevalence of higher education institutions engaging in cutting-edge research.

China’s notable inclusion as the only non-Western country among the top contributors to the field suggests a growing research capacity and interest in digital technology in education. However, the lower average citation per paper for China could reflect emerging engagement or different research focuses that may not yet have achieved the same international recognition as Western counterparts.

The chord diagram analysis furthers this understanding, revealing dense interconnections between countries like the United States, China, and England, which indicates robust collaborations. Such collaborations are fundamental in addressing global educational challenges and shaping international research agendas.

Discussion on institutional-level contributions to digital technology education

Institutional productivity in digital technology education research reveals a constellation of universities driving the field forward. Monash University and the Australian Catholic University have the highest publication output, signaling Australia’s significant role in advancing digital education research. The University of Oslo’s remarkable average citation count per publication indicates influential research contributions, potentially reflecting high-quality studies that resonate with the broader academic community.

The strong showing of UK institutions, including the University of London, The Open University, and the University of Cambridge, reinforces the UK’s prominence in this research field. Such institutions are often at the forefront of pedagogical innovation, benefiting from established research cultures and funding mechanisms that support sustained inquiry into digital education.

Discussion on journal publication analysis

An examination of journal outputs offers a lens into the communicative channels of the field’s knowledge base. Journals such as Education and Information Technologies , Computers & Education , and the British Journal of Educational Technology not only serve as the primary disseminators of research findings but also as indicators of research quality and relevance. The impact factor (IF) serves as a proxy for the quality and influence of these journals within the academic community.

The high citation counts for articles published in Computers & Education suggest that research disseminated through this medium has a wide-reaching impact and is of particular interest to the field. This is further evidenced by its significant IF of 11.182, indicating that the journal is a pivotal platform for seminal work in the application of digital technology in education.

The authorship, regional, and institutional productivity in the field of digital technology education application research collectively narrate the evolution of this domain since the turn of the century. The prominence of certain authors and countries underscores the importance of socioeconomic factors and existing academic infrastructure in fostering research productivity. Meanwhile, the centrality of specific journals as outlets for high-impact research emphasizes the role of academic publishing in shaping the research landscape.

As the field continues to grow, future research may benefit from leveraging the collaborative networks that have been elucidated through this analysis, perhaps focusing on underrepresented regions to broaden the scope and diversity of research. Furthermore, the stabilization of publication numbers in recent years invites a deeper exploration into potential plateaus in research trends or saturation in certain sub-fields, signaling an opportunity for novel inquiries and methodological innovations.

Discussion on the evolutionary trends (RQ2)

The evolution of the research field concerning the application of digital technology in education over the past two decades is a story of convergence, diversification, and transformation, shaped by rapid technological advancements and shifting educational paradigms.

At the turn of the century, the inception of digital technology in education was largely exploratory, with a focus on how emerging computer technologies could be harnessed to enhance traditional learning environments. Research from this early period was primarily descriptive, reflecting on the potential and challenges of incorporating digital tools into the educational setting. This phase was critical in establishing the fundamental discourse that would guide subsequent research, as it set the stage for understanding the scope and impact of digital technology in learning spaces (Wang et al. 2023 ).

As the first decade progressed, the narrative expanded to encompass the pedagogical implications of digital technologies. This was a period of conceptual debates, where terms like “digital natives” and “disruptive pedagogy” entered the academic lexicon, underscoring the growing acknowledgment of digital technology as a transformative force within education (Bennett and Maton, 2010 ). During this time, the research began to reflect a more nuanced understanding of the integration of technology, considering not only its potential to change where and how learning occurred but also its implications for educational equity and access.

In the second decade, with the maturation of internet connectivity and mobile technology, the focus of research shifted from theoretical speculations to empirical investigations. The proliferation of digital devices and the ubiquity of social media influenced how learners interacted with information and each other, prompting a surge in studies that sought to measure the impact of these tools on learning outcomes. The digital divide and issues related to digital literacy became central concerns, as scholars explored the varying capacities of students and educators to engage with technology effectively.

Throughout this period, there was an increasing emphasis on the individualization of learning experiences, facilitated by adaptive technologies that could cater to the unique needs and pacing of learners (Jing et al. 2023a ). This individualization was coupled with a growing recognition of the importance of collaborative learning, both online and offline, and the role of digital tools in supporting these processes. Blended learning models, which combined face-to-face instruction with online resources, emerged as a significant trend, advocating for a balance between traditional pedagogies and innovative digital strategies.

The later years, particularly marked by the COVID-19 pandemic, accelerated the necessity for digital technology in education, transforming it from a supplementary tool to an essential platform for delivering education globally (Mo et al. 2022 ; Mustapha et al. 2021 ). This era brought about an unprecedented focus on online learning environments, distance education, and virtual classrooms. Research became more granular, examining not just the pedagogical effectiveness of digital tools, but also their role in maintaining continuity of education during crises, their impact on teacher and student well-being, and their implications for the future of educational policy and infrastructure.

Across these two decades, the research field has seen a shift from examining digital technology as an external addition to the educational process, to viewing it as an integral component of curriculum design, instructional strategies, and even assessment methods. The emergent themes have broadened from a narrow focus on specific tools or platforms to include wider considerations such as data privacy, ethical use of technology, and the environmental impact of digital tools.

Moreover, the field has moved from considering the application of digital technology in education as a primarily cognitive endeavor to recognizing its role in facilitating socio-emotional learning, digital citizenship, and global competencies. Researchers have increasingly turned their attention to the ways in which technology can support collaborative skills, cultural understanding, and ethical reasoning within diverse student populations.

In summary, the past over twenty years in the research field of digital technology applications in education have been characterized by a progression from foundational inquiries to complex analyses of digital integration. This evolution has mirrored the trajectory of technology itself, from a facilitative tool to a pervasive ecosystem defining contemporary educational experiences. As we look to the future, the field is poised to delve into the implications of emerging technologies like AI, AR, and VR, and their potential to redefine the educational landscape even further. This ongoing metamorphosis suggests that the application of digital technology in education will continue to be a rich area of inquiry, demanding continual adaptation and forward-thinking from educators and researchers alike.

Discussion on the study of research hotspots (RQ3)

The analysis of keyword evolution in digital technology education application research elucidates the current frontiers in the field, reflecting a trajectory that is in tandem with the rapidly advancing digital age. This landscape is sculpted by emergent technological innovations and shaped by the demands of an increasingly digital society.

Interdisciplinary integration and pedagogical transformation

One of the frontiers identified from recent keyword bursts includes the integration of digital technology into diverse educational contexts, particularly noted with the keyword “physical education.” The digitalization of disciplines traditionally characterized by physical presence illustrates the pervasive reach of technology and signifies a push towards interdisciplinary integration where technology is not only a facilitator but also a transformative agent. This integration challenges educators to reconceptualize curriculum delivery to accommodate digital tools that can enhance or simulate the physical aspects of learning.

Digital literacy and skills acquisition

Another pivotal frontier is the focus on “digital literacy” and “digital skill”, which has intensified in recent years. This suggests a shift from mere access to technology towards a comprehensive understanding and utilization of digital tools. In this realm, the emphasis is not only on the ability to use technology but also on critical thinking, problem-solving, and the ethical use of digital resources (Yu, 2022 ). The acquisition of digital literacy is no longer an additive skill but a fundamental aspect of modern education, essential for navigating and contributing to the digital world.

Educational digital transformation

The keyword “digital transformation” marks a significant research frontier, emphasizing the systemic changes that education institutions must undergo to align with the digital era (Romero et al. 2021 ). This transformation includes the redesigning of learning environments, pedagogical strategies, and assessment methods to harness digital technology’s full potential. Research in this area explores the complexity of institutional change, addressing the infrastructural, cultural, and policy adjustments needed for a seamless digital transition.

Engagement and participation

Further exploration into “engagement” and “participation” underscores the importance of student-centered learning environments that are mediated by technology. The current frontiers examine how digital platforms can foster collaboration, inclusivity, and active learning, potentially leading to more meaningful and personalized educational experiences. Here, the use of technology seeks to support the emotional and cognitive aspects of learning, moving beyond the transactional view of education to one that is relational and interactive.

Professional development and teacher readiness

As the field evolves, “professional development” emerges as a crucial area, particularly in light of the pandemic which necessitated emergency remote teaching. The need for teacher readiness in a digital age is a pressing frontier, with research focusing on the competencies required for educators to effectively integrate technology into their teaching practices. This includes familiarity with digital tools, pedagogical innovation, and an ongoing commitment to personal and professional growth in the digital domain.

Pandemic as a catalyst

The recent pandemic has acted as a catalyst for accelerated research and application in this field, particularly in the domains of “digital transformation,” “professional development,” and “physical education.” This period has been a litmus test for the resilience and adaptability of educational systems to continue their operations in an emergency. Research has thus been directed at understanding how digital technologies can support not only continuity but also enhance the quality and reach of education in such contexts.

Ethical and societal considerations

The frontier of digital technology in education is also expanding to consider broader ethical and societal implications. This includes issues of digital equity, data privacy, and the sociocultural impact of technology on learning communities. The research explores how educational technology can be leveraged to address inequities and create more equitable learning opportunities for all students, regardless of their socioeconomic background.

Innovation and emerging technologies

Looking forward, the frontiers are set to be influenced by ongoing and future technological innovations, such as artificial intelligence (AI) (Wu and Yu, 2023 ; Chen et al. 2022a ). The exploration into how these technologies can be integrated into educational practices to create immersive and adaptive learning experiences represents a bold new chapter for the field.

In conclusion, the current frontiers of research on the application of digital technology in education are multifaceted and dynamic. They reflect an overarching movement towards deeper integration of technology in educational systems and pedagogical practices, where the goals are not only to facilitate learning but to redefine it. As these frontiers continue to expand and evolve, they will shape the educational landscape, requiring a concerted effort from researchers, educators, policymakers, and technologists to navigate the challenges and harness the opportunities presented by the digital revolution in education.

Conclusions and future research

Conclusions.

The utilization of digital technology in education is a research area that cuts across multiple technical and educational domains and continues to experience dynamic growth due to the continuous progress of technology. In this study, a systematic review of this field was conducted through bibliometric techniques to examine its development trajectory. The primary focus of the review was to investigate the leading contributors, productive national institutions, significant publications, and evolving development patterns. The study’s quantitative analysis resulted in several key conclusions that shed light on this research field’s current state and future prospects.

(1) The research field of digital technology education applications has entered a stage of rapid development, particularly in recent years due to the impact of the pandemic, resulting in a peak of publications. Within this field, several key authors (Selwyn, Henderson, Edwards, etc.) and countries/regions (England, Australia, USA, etc.) have emerged, who have made significant contributions. International exchanges in this field have become frequent, with a high degree of internationalization in academic research. Higher education institutions in the UK and Australia are the core productive forces in this field at the institutional level.

(2) Education and Information Technologies , Computers & Education , and the British Journal of Educational Technology are notable journals that publish research related to digital technology education applications. These journals are affiliated with the research field of educational technology and provide effective communication platforms for sharing digital technology education applications.

(3) Over the past two decades, research on digital technology education applications has progressed from its early stages of budding, initial development, and critical exploration to accelerated transformation, and it is currently approaching maturity. Technological progress and changes in the times have been key driving forces for educational transformation and innovation, and both have played important roles in promoting the continuous development of education.

(4) Influenced by the pandemic, three emerging frontiers have emerged in current research on digital technology education applications, which are physical education, digital transformation, and professional development under the promotion of digital technology. These frontier research hotspots reflect the core issues that the education system faces when encountering new technologies. The evolution of research hotspots shows that technology breakthroughs in education’s original boundaries of time and space create new challenges. The continuous self-renewal of education is achieved by solving one hotspot problem after another.

The present study offers significant practical implications for scholars and practitioners in the field of digital technology education applications. Firstly, it presents a well-defined framework of the existing research in this area, serving as a comprehensive guide for new entrants to the field and shedding light on the developmental trajectory of this research domain. Secondly, the study identifies several contemporary research hotspots, thus offering a valuable decision-making resource for scholars aiming to explore potential research directions. Thirdly, the study undertakes an exhaustive analysis of published literature to identify core journals in the field of digital technology education applications, with Sustainability being identified as a promising open access journal that publishes extensively on this topic. This finding can potentially facilitate scholars in selecting appropriate journals for their research outputs.

Limitation and future research

Influenced by some objective factors, this study also has some limitations. First of all, the bibliometrics analysis software has high standards for data. In order to ensure the quality and integrity of the collected data, the research only selects the periodical papers in SCIE and SSCI indexes, which are the core collection of Web of Science database, and excludes other databases, conference papers, editorials and other publications, which may ignore some scientific research and original opinions in the field of digital technology education and application research. In addition, although this study used professional software to carry out bibliometric analysis and obtained more objective quantitative data, the analysis and interpretation of data will inevitably have a certain subjective color, and the influence of subjectivity on data analysis cannot be completely avoided. As such, future research endeavors will broaden the scope of literature screening and proactively engage scholars in the field to gain objective and state-of-the-art insights, while minimizing the adverse impact of personal subjectivity on research analysis.

Data availability

The datasets analyzed during the current study are available in the Dataverse repository: https://doi.org/10.7910/DVN/F9QMHY

Alabdulaziz MS (2021) COVID-19 and the use of digital technology in mathematics education. Educ Inf Technol 26(6):7609–7633. https://doi.org/10.1007/s10639-021-10602-3

Arif TB, Munaf U, Ul-Haque I (2023) The future of medical education and research: is ChatGPT a blessing or blight in disguise? Med Educ Online 28. https://doi.org/10.1080/10872981.2023.2181052

Banerjee M, Chiew D, Patel KT, Johns I, Chappell D, Linton N, Cole GD, Francis DP, Szram J, Ross J, Zaman S (2021) The impact of artificial intelligence on clinical education: perceptions of postgraduate trainee doctors in London (UK) and recommendations for trainers. BMC Med Educ 21. https://doi.org/10.1186/s12909-021-02870-x

Barlovits S, Caldeira A, Fesakis G, Jablonski S, Koutsomanoli Filippaki D, Lázaro C, Ludwig M, Mammana MF, Moura A, Oehler DXK, Recio T, Taranto E, Volika S(2022) Adaptive, synchronous, and mobile online education: developing the ASYMPTOTE learning environment. Mathematics 10:1628. https://doi.org/10.3390/math10101628

Article   Google Scholar  

Baron NS(2021) Know what? How digital technologies undermine learning and remembering J Pragmat 175:27–37. https://doi.org/10.1016/j.pragma.2021.01.011

Batista J, Morais NS, Ramos F (2016) Researching the use of communication technologies in higher education institutions in Portugal. https://doi.org/10.4018/978-1-5225-0571-6.ch057

Beardsley M, Albó L, Aragón P, Hernández-Leo D (2021) Emergency education effects on teacher abilities and motivation to use digital technologies. Br J Educ Technol 52. https://doi.org/10.1111/bjet.13101

Bennett S, Maton K(2010) Beyond the “digital natives” debate: towards a more nuanced understanding of students’ technology experiences J Comput Assist Learn 26:321–331. https://doi.org/10.1111/j.1365-2729.2010.00360.x

Buckingham D, Burn A (2007) Game literacy in theory and practice 16:323–349

Google Scholar  

Bulfin S, Pangrazio L, Selwyn N (2014) Making “MOOCs”: the construction of a new digital higher education within news media discourse. In: The International Review of Research in Open and Distributed Learning 15. https://doi.org/10.19173/irrodl.v15i5.1856

Camilleri MA, Camilleri AC(2016) Digital learning resources and ubiquitous technologies in education Technol Knowl Learn 22:65–82. https://doi.org/10.1007/s10758-016-9287-7

Chen C(2006) CiteSpace II: detecting and visualizing emerging trends and transient patterns in scientific literature J Am Soc Inf Sci Technol 57:359–377. https://doi.org/10.1002/asi.20317

Chen J, Dai J, Zhu K, Xu L(2022) Effects of extended reality on language learning: a meta-analysis Front Psychol 13:1016519. https://doi.org/10.3389/fpsyg.2022.1016519

Article   PubMed   PubMed Central   Google Scholar  

Chen J, Wang CL, Tang Y (2022b) Knowledge mapping of volunteer motivation: a bibliometric analysis and cross-cultural comparative study. Front Psychol 13. https://doi.org/10.3389/fpsyg.2022.883150

Cohen A, Soffer T, Henderson M(2022) Students’ use of technology and their perceptions of its usefulness in higher education: International comparison J Comput Assist Learn 38(5):1321–1331. https://doi.org/10.1111/jcal.12678

Collins A, Halverson R(2010) The second educational revolution: rethinking education in the age of technology J Comput Assist Learn 26:18–27. https://doi.org/10.1111/j.1365-2729.2009.00339.x

Conole G, Alevizou P (2010) A literature review of the use of Web 2.0 tools in higher education. Walton Hall, Milton Keynes, UK: the Open University, retrieved 17 February

Creely E, Henriksen D, Crawford R, Henderson M(2021) Exploring creative risk-taking and productive failure in classroom practice. A case study of the perceived self-efficacy and agency of teachers at one school Think Ski Creat 42:100951. https://doi.org/10.1016/j.tsc.2021.100951

Davis N, Eickelmann B, Zaka P(2013) Restructuring of educational systems in the digital age from a co-evolutionary perspective J Comput Assist Learn 29:438–450. https://doi.org/10.1111/jcal.12032

De Belli N (2009) Bibliometrics and citation analysis: from the science citation index to cybermetrics, Scarecrow Press. https://doi.org/10.1111/jcal.12032

Domínguez A, Saenz-de-Navarrete J, de-Marcos L, Fernández-Sanz L, Pagés C, Martínez-Herráiz JJ(2013) Gamifying learning experiences: practical implications and outcomes Comput Educ 63:380–392. https://doi.org/10.1016/j.compedu.2012.12.020

Donnison S (2009) Discourses in conflict: the relationship between Gen Y pre-service teachers, digital technologies and lifelong learning. Australasian J Educ Technol 25. https://doi.org/10.14742/ajet.1138

Durfee SM, Jain S, Shaffer K (2003) Incorporating electronic media into medical student education. Acad Radiol 10:205–210. https://doi.org/10.1016/s1076-6332(03)80046-6

Dzikowski P(2018) A bibliometric analysis of born global firms J Bus Res 85:281–294. https://doi.org/10.1016/j.jbusres.2017.12.054

van Eck NJ, Waltman L(2009) Software survey: VOSviewer, a computer program for bibliometric mapping Scientometrics 84:523–538 https://doi.org/10.1007/s11192-009-0146-3

Edwards S(2013) Digital play in the early years: a contextual response to the problem of integrating technologies and play-based pedagogies in the early childhood curriculum Eur Early Child Educ Res J 21:199–212. https://doi.org/10.1080/1350293x.2013.789190

Edwards S(2015) New concepts of play and the problem of technology, digital media and popular-culture integration with play-based learning in early childhood education Technol Pedagogy Educ 25:513–532 https://doi.org/10.1080/1475939x.2015.1108929

Article   MathSciNet   Google Scholar  

Eisenberg MB(2008) Information literacy: essential skills for the information age DESIDOC J Libr Inf Technol 28:39–47. https://doi.org/10.14429/djlit.28.2.166

Forde C, OBrien A (2022) A literature review of barriers and opportunities presented by digitally enhanced practical skill teaching and learning in health science education. Med Educ Online 27. https://doi.org/10.1080/10872981.2022.2068210

García-Morales VJ, Garrido-Moreno A, Martín-Rojas R (2021) The transformation of higher education after the COVID disruption: emerging challenges in an online learning scenario. Front Psychol 12. https://doi.org/10.3389/fpsyg.2021.616059

Garfield E(2006) The history and meaning of the journal impact factor JAMA 295:90. https://doi.org/10.1001/jama.295.1.90

Article   PubMed   Google Scholar  

Garzón-Artacho E, Sola-Martínez T, Romero-Rodríguez JM, Gómez-García G(2021) Teachers’ perceptions of digital competence at the lifelong learning stage Heliyon 7:e07513. https://doi.org/10.1016/j.heliyon.2021.e07513

Gaviria-Marin M, Merigó JM, Baier-Fuentes H(2019) Knowledge management: a global examination based on bibliometric analysis Technol Forecast Soc Change 140:194–220. https://doi.org/10.1016/j.techfore.2018.07.006

Gilster P, Glister P (1997) Digital literacy. Wiley Computer Pub, New York

Greenhow C, Lewin C(2015) Social media and education: reconceptualizing the boundaries of formal and informal learning Learn Media Technol 41:6–30. https://doi.org/10.1080/17439884.2015.1064954

Hawkins DT(2001) Bibliometrics of electronic journals in information science Infor Res 7(1):7–1. http://informationr.net/ir/7-1/paper120.html

Henderson M, Selwyn N, Finger G, Aston R(2015) Students’ everyday engagement with digital technology in university: exploring patterns of use and “usefulness J High Educ Policy Manag 37:308–319 https://doi.org/10.1080/1360080x.2015.1034424

Huang CK, Neylon C, Hosking R, Montgomery L, Wilson KS, Ozaygen A, Brookes-Kenworthy C (2020) Evaluating the impact of open access policies on research institutions. eLife 9. https://doi.org/10.7554/elife.57067

Hwang GJ, Tsai CC(2011) Research trends in mobile and ubiquitous learning: a review of publications in selected journals from 2001 to 2010 Br J Educ Technol 42:E65–E70. https://doi.org/10.1111/j.1467-8535.2011.01183.x

Hwang GJ, Wu PH, Zhuang YY, Huang YM(2013) Effects of the inquiry-based mobile learning model on the cognitive load and learning achievement of students Interact Learn Environ 21:338–354. https://doi.org/10.1080/10494820.2011.575789

Jiang S, Ning CF (2022) Interactive communication in the process of physical education: are social media contributing to the improvement of physical training performance. Universal Access Inf Soc, 1–10. https://doi.org/10.1007/s10209-022-00911-w

Jing Y, Zhao L, Zhu KK, Wang H, Wang CL, Xia Q(2023) Research landscape of adaptive learning in education: a bibliometric study on research publications from 2000 to 2022 Sustainability 15:3115–3115. https://doi.org/10.3390/su15043115

Jing Y, Wang CL, Chen Y, Wang H, Yu T, Shadiev R (2023b) Bibliometric mapping techniques in educational technology research: a systematic literature review. Educ Inf Technol 1–29. https://doi.org/10.1007/s10639-023-12178-6

Krishnamurthy S (2020) The future of business education: a commentary in the shadow of the Covid-19 pandemic. J Bus Res. https://doi.org/10.1016/j.jbusres.2020.05.034

Kumar S, Lim WM, Pandey N, Christopher Westland J (2021) 20 years of electronic commerce research. Electron Commer Res 21:1–40

Kyza EA, Georgiou Y(2018) Scaffolding augmented reality inquiry learning: the design and investigation of the TraceReaders location-based, augmented reality platform Interact Learn Environ 27:211–225. https://doi.org/10.1080/10494820.2018.1458039

Laurillard D(2008) Technology enhanced learning as a tool for pedagogical innovation J Philos Educ 42:521–533. https://doi.org/10.1111/j.1467-9752.2008.00658.x

Li M, Yu Z (2023) A systematic review on the metaverse-based blended English learning. Front Psychol 13. https://doi.org/10.3389/fpsyg.2022.1087508

Luo H, Li G, Feng Q, Yang Y, Zuo M (2021) Virtual reality in K-12 and higher education: a systematic review of the literature from 2000 to 2019. J Comput Assist Learn. https://doi.org/10.1111/jcal.12538

Margaryan A, Littlejohn A, Vojt G(2011) Are digital natives a myth or reality? University students’ use of digital technologies Comput Educ 56:429–440. https://doi.org/10.1016/j.compedu.2010.09.004

McMillan S(1996) Literacy and computer literacy: definitions and comparisons Comput Educ 27:161–170. https://doi.org/10.1016/s0360-1315(96)00026-7

Mo CY, Wang CL, Dai J, Jin P (2022) Video playback speed influence on learning effect from the perspective of personalized adaptive learning: a study based on cognitive load theory. Front Psychology 13. https://doi.org/10.3389/fpsyg.2022.839982

Moorhouse BL (2021) Beginning teaching during COVID-19: newly qualified Hong Kong teachers’ preparedness for online teaching. Educ Stud 1–17. https://doi.org/10.1080/03055698.2021.1964939

Moorhouse BL, Wong KM (2021) The COVID-19 Pandemic as a catalyst for teacher pedagogical and technological innovation and development: teachers’ perspectives. Asia Pac J Educ 1–16. https://doi.org/10.1080/02188791.2021.1988511

Moskal P, Dziuban C, Hartman J (2013) Blended learning: a dangerous idea? Internet High Educ 18:15–23

Mughal MY, Andleeb N, Khurram AFA, Ali MY, Aslam MS, Saleem MN (2022) Perceptions of teaching-learning force about Metaverse for education: a qualitative study. J. Positive School Psychol 6:1738–1745

Mustapha I, Thuy Van N, Shahverdi M, Qureshi MI, Khan N (2021) Effectiveness of digital technology in education during COVID-19 pandemic. a bibliometric analysis. Int J Interact Mob Technol 15:136

Nagle J (2018) Twitter, cyber-violence, and the need for a critical social media literacy in teacher education: a review of the literature. Teach Teach Education 76:86–94

Nazare J, Woolf A, Sysoev I, Ballinger S, Saveski M, Walker M, Roy D (2022) Technology-assisted coaching can increase engagement with learning technology at home and caregivers’ awareness of it. Comput Educ 188:104565

Nguyen UP, Hallinger P (2020) Assessing the distinctive contributions of simulation & gaming to the literature, 1970-2019: a bibliometric review. Simul Gaming 104687812094156. https://doi.org/10.1177/1046878120941569

Nygren H, Nissinen K, Hämäläinen R, Wever B(2019) Lifelong learning: formal, non-formal and informal learning in the context of the use of problem-solving skills in technology-rich environments Br J Educ Technol 50:1759–1770. https://doi.org/10.1111/bjet.12807

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, Moher D (2021) The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. Int J Surg 88:105906

Pan SL, Zhang S(2020) From fighting COVID-19 pandemic to tackling sustainable development goals: an opportunity for responsible information systems research Int J Inf Manage 55:102196. https://doi.org/10.1016/j.ijinfomgt.2020.102196

Pan X, Yan E, Cui M, Hua W(2018) Examining the usage, citation, and diffusion patterns of bibliometric mapping software: a comparative study of three tools J Informetr 12:481–493. https://doi.org/10.1016/j.joi.2018.03.005

Parris Z, Cale L, Harris J, Casey A (2022) Physical activity for health, covid-19 and social media: what, where and why?. Movimento, 28. https://doi.org/10.22456/1982-8918.122533

Pasquini LA, Evangelopoulos N (2016) Sociotechnical stewardship in higher education: a field study of social media policy documents. J Comput High Educ 29:218–239

Pérez-Sanagustín M, Hernández-Leo D, Santos P, Delgado Kloos C, Blat J(2014) Augmenting reality and formality of informal and non-formal settings to enhance blended learning IEEE Trans Learn Technol 7:118–131. https://doi.org/10.1109/TLT.2014.2312719

Pinto M, Leite C (2020) Digital technologies in support of students learning in Higher Education: literature review. Digital Education Review 343–360. https://doi.org/10.1344/der.2020.37.343-360

Pires F, Masanet MJ, Tomasena JM, Scolari CA(2022) Learning with YouTube: beyond formal and informal through new actors, strategies and affordances Convergence 28(3):838–853. https://doi.org/10.1177/1354856521102054

Pritchard A (1969) Statistical bibliography or bibliometrics 25:348

Romero M, Romeu T, Guitert M, Baztán P (2021) Digital transformation in higher education: the UOC case. In ICERI2021 Proceedings (pp. 6695–6703). IATED https://doi.org/10.21125/iceri.2021.1512

Romero-Hall E, Jaramillo Cherrez N (2022) Teaching in times of disruption: faculty digital literacy in higher education during the COVID-19 pandemic. Innovations in Education and Teaching International 1–11. https://doi.org/10.1080/14703297.2022.2030782

Rospigliosi PA(2023) Artificial intelligence in teaching and learning: what questions should we ask of ChatGPT? Interactive Learning Environments 31:1–3. https://doi.org/10.1080/10494820.2023.2180191

Salas-Pilco SZ, Yang Y, Zhang Z(2022) Student engagement in online learning in Latin American higher education during the COVID-19 pandemic: a systematic review. Br J Educ Technol 53(3):593–619. https://doi.org/10.1111/bjet.13190

Selwyn N(2009) The digital native-myth and reality In Aslib proceedings 61(4):364–379. https://doi.org/10.1108/00012530910973776

Selwyn N(2012) Making sense of young people, education and digital technology: the role of sociological theory Oxford Review of Education 38:81–96. https://doi.org/10.1080/03054985.2011.577949

Selwyn N, Facer K(2014) The sociology of education and digital technology: past, present and future Oxford Rev Educ 40:482–496. https://doi.org/10.1080/03054985.2014.933005

Selwyn N, Banaji S, Hadjithoma-Garstka C, Clark W(2011) Providing a platform for parents? Exploring the nature of parental engagement with school Learning Platforms J Comput Assist Learn 27:314–323. https://doi.org/10.1111/j.1365-2729.2011.00428.x

Selwyn N, Aagaard J (2020) Banning mobile phones from classrooms-an opportunity to advance understandings of technology addiction, distraction and cyberbullying. Br J Educ Technol 52. https://doi.org/10.1111/bjet.12943

Selwyn N, O’Neill C, Smith G, Andrejevic M, Gu X (2021) A necessary evil? The rise of online exam proctoring in Australian universities. Media Int Austr 1329878X2110058. https://doi.org/10.1177/1329878x211005862

Selwyn N, Pangrazio L, Nemorin S, Perrotta C (2019) What might the school of 2030 be like? An exercise in social science fiction. Learn, Media Technol 1–17. https://doi.org/10.1080/17439884.2020.1694944

Selwyn, N (2016) What works and why?* Understanding successful technology enabled learning within institutional contexts 2016 Final report Appendices (Part B). Monash University Griffith University

Sjöberg D, Holmgren R (2021) Informal workplace learning in swedish police education-a teacher perspective. Vocations and Learning. https://doi.org/10.1007/s12186-021-09267-3

Strotmann A, Zhao D (2012) Author name disambiguation: what difference does it make in author-based citation analysis? J Am Soc Inf Sci Technol 63:1820–1833

Article   CAS   Google Scholar  

Sutherland R, Facer K, Furlong R, Furlong J(2000) A new environment for education? The computer in the home. Comput Educ 34:195–212. https://doi.org/10.1016/s0360-1315(99)00045-7

Szeto E, Cheng AY-N, Hong J-C(2015) Learning with social media: how do preservice teachers integrate YouTube and Social Media in teaching? Asia-Pac Educ Res 25:35–44. https://doi.org/10.1007/s40299-015-0230-9

Tang E, Lam C(2014) Building an effective online learning community (OLC) in blog-based teaching portfolios Int High Educ 20:79–85. https://doi.org/10.1016/j.iheduc.2012.12.002

Taskin Z, Al U(2019) Natural language processing applications in library and information science Online Inf Rev 43:676–690. https://doi.org/10.1108/oir-07-2018-0217

Tegtmeyer K, Ibsen L, Goldstein B(2001) Computer-assisted learning in critical care: from ENIAC to HAL Crit Care Med 29:N177–N182. https://doi.org/10.1097/00003246-200108001-00006

Article   CAS   PubMed   Google Scholar  

Timotheou S, Miliou O, Dimitriadis Y, Sobrino SV, Giannoutsou N, Cachia R, Moné AM, Ioannou A(2023) Impacts of digital technologies on education and factors influencing schools' digital capacity and transformation: a literature review. Educ Inf Technol 28(6):6695–6726. https://doi.org/10.1007/s10639-022-11431-8

Trujillo Maza EM, Gómez Lozano MT, Cardozo Alarcón AC, Moreno Zuluaga L, Gamba Fadul M (2016) Blended learning supported by digital technology and competency-based medical education: a case study of the social medicine course at the Universidad de los Andes, Colombia. Int J Educ Technol High Educ 13. https://doi.org/10.1186/s41239-016-0027-9

Turin O, Friesem Y(2020) Is that media literacy?: Israeli and US media scholars’ perceptions of the field J Media Lit Educ 12:132–144. https://doi.org/10.1007/s11192-009-0146-3

Van Eck NJ, Waltman L (2019) VOSviewer manual. Universiteit Leiden

Vratulis V, Clarke T, Hoban G, Erickson G(2011) Additive and disruptive pedagogies: the use of slowmation as an example of digital technology implementation Teach Teach Educ 27:1179–1188. https://doi.org/10.1016/j.tate.2011.06.004

Wang CL, Dai J, Xu LJ (2022) Big data and data mining in education: a bibliometrics study from 2010 to 2022. In 2022 7th International Conference on Cloud Computing and Big Data Analytics ( ICCCBDA ) (pp. 507-512). IEEE. https://doi.org/10.1109/icccbda55098.2022.9778874

Wang CL, Dai J, Zhu KK, Yu T, Gu XQ (2023) Understanding the continuance intention of college students toward new E-learning spaces based on an integrated model of the TAM and TTF. Int J Hum-Comput Int 1–14. https://doi.org/10.1080/10447318.2023.2291609

Wong L-H, Boticki I, Sun J, Looi C-K(2011) Improving the scaffolds of a mobile-assisted Chinese character forming game via a design-based research cycle Comput Hum Behav 27:1783–1793. https://doi.org/10.1016/j.chb.2011.03.005

Wu R, Yu Z (2023) Do AI chatbots improve students learning outcomes? Evidence from a meta-analysis. Br J Educ Technol. https://doi.org/10.1111/bjet.13334

Yang D, Zhou J, Shi D, Pan Q, Wang D, Chen X, Liu J (2022) Research status, hotspots, and evolutionary trends of global digital education via knowledge graph analysis. Sustainability 14:15157–15157. https://doi.org/10.3390/su142215157

Yu T, Dai J, Wang CL (2023) Adoption of blended learning: Chinese university students’ perspectives. Humanit Soc Sci Commun 10:390. https://doi.org/10.3390/su142215157

Yu Z (2022) Sustaining student roles, digital literacy, learning achievements, and motivation in online learning environments during the COVID-19 pandemic. Sustainability 14:4388. https://doi.org/10.3390/su14084388

Za S, Spagnoletti P, North-Samardzic A(2014) Organisational learning as an emerging process: the generative role of digital tools in informal learning practices Br J Educ Technol 45:1023–1035. https://doi.org/10.1111/bjet.12211

Zhang X, Chen Y, Hu L, Wang Y (2022) The metaverse in education: definition, framework, features, potential applications, challenges, and future research topics. Front Psychol 13:1016300. https://doi.org/10.3389/fpsyg.2022.1016300

Zhou M, Dzingirai C, Hove K, Chitata T, Mugandani R (2022) Adoption, use and enhancement of virtual learning during COVID-19. Education and Information Technologies. https://doi.org/10.1007/s10639-022-10985-x

Download references

Acknowledgements

This research was supported by the Zhejiang Provincial Social Science Planning Project, “Mechanisms and Pathways for Empowering Classroom Teaching through Learning Spaces under the Strategy of High-Quality Education Development”, the 2022 National Social Science Foundation Education Youth Project “Research on the Strategy of Creating Learning Space Value and Empowering Classroom Teaching under the background of ‘Double Reduction’” (Grant No. CCA220319) and the National College Student Innovation and Entrepreneurship Training Program of China (Grant No. 202310337023).

Author information

Authors and affiliations.

College of Educational Science and Technology, Zhejiang University of Technology, Zhejiang, China

Chengliang Wang, Xiaojiao Chen, Yidan Liu & Yuhui Jing

Graduate School of Business, Universiti Sains Malaysia, Minden, Malaysia

Department of Management, The Chinese University of Hong Kong, Hong Kong, China

College of Humanities and Social Sciences, Beihang University, Beijing, China

You can also search for this author in PubMed   Google Scholar

Contributions

Conceptualization: Y.J., C.W.; methodology, C.W.; software, C.W., Y.L.; writing-original draft preparation, C.W., Y.L.; writing-review and editing, T.Y., Y.L., C.W.; supervision, X.C., T.Y.; project administration, Y.J.; funding acquisition, X.C., Y.L. All authors read and approved the final manuscript. All authors have read and approved the re-submission of the manuscript.

Corresponding author

Correspondence to Yuhui Jing .

Ethics declarations

Ethical approval.

Ethical approval was not required as the study did not involve human participants.

Informed consent

Informed consent was not required as the study did not involve human participants.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Wang, C., Chen, X., Yu, T. et al. Education reform and change driven by digital technology: a bibliometric study from a global perspective. Humanit Soc Sci Commun 11 , 256 (2024). https://doi.org/10.1057/s41599-024-02717-y

Download citation

Received : 11 July 2023

Accepted : 17 January 2024

Published : 12 February 2024

DOI : https://doi.org/10.1057/s41599-024-02717-y

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

This article is cited by

A meta-analysis of learners’ continuance intention toward online education platforms.

• Chengliang Wang

Education and Information Technologies (2024)

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

• Open access
• Published: 28 December 2018

Digital technology and practices for school improvement: innovative digital school model

• Liisa Ilomäki 1 &
• Minna Lakkala 1  

Research and Practice in Technology Enhanced Learning volume  13 , Article number:  25 ( 2018 ) Cite this article

78k Accesses

51 Citations

13 Altmetric

Metrics details

A Correction to this article was published on 18 March 2019

This article has been updated

The aim of this study was to create a model which describes the main elements for improving schools with digital technology and helps to reveal differences between schools and identify their best practices and challenges. The innovative digital school model (IDI school) offers a framework for research but also a research-based model for schools to examine their own practices with digital technologies. The model combines previous research on school improvement, creation of innovations, and digital technology in education as a special case of innovations and learning as knowledge creation to define six main elements describing an innovative, digital school: visions of the school, leadership, practices of the teaching community, pedagogical practices, school-level knowledge practices and digital resources. The model was applied to investigate three basic education schools. The results indicate that the model worked: we found essential differences between the schools and their best practices and challenges for improvement. It worked particularly well for those elements, which are mainly the responsibility for leadership inside a school. The differences of various elements between schools were not based on socioeconomic background but on the school-level practices. As a conclusion, we suggest that to improve schools with digital technology, all elements of the model should be included in the evaluation and development process.

Introduction

In today’s world, education is facing major challenges: it is expected to provide children and teenagers with competencies they will need in the future, to consider informal ways of learning, and to apply digital technologies and modern pedagogical methods to answer these challenges (EU, 2010 ). However, schools have not managed to meet all these challenges: e.g. digital technology has not yet been applied much in education, although it is widely in use elsewhere in the society and in work life (EU 2013 ; Livingstone 2012 ); students do not acquire sufficient competence at school to undertake university studies (such as collaboration, planning, independent learning, digital competence or working with knowledge) (Hautamäki et al. 2012 ; Kiili 2012 ; Lundahl et al. 2010 ); and there are major differences between countries and schools in reaching these skill levels (such as problem-solving skills, OECD 2014 , 2017 ). There have been promising results that some pedagogical practices related to student centredness, real-life activities and group work have increased at schools between 2001 and 2011. Such pedagogical practices are often linked to the use of digital technology (OECD 2014 ).

There is a large body of research about using digital technology in schools, in classrooms and among teachers and students, but often these studies concentrate on only one or two phenomena of education and technology (e.g. classroom cases, or technical competence of teachers and students), thus isolating the object of study from the broader context of a school. Unless a more comprehensive view is adopted in the efforts of developing a school, there is little chance of innovation programmes having any lasting effect (Wikeley et al. 2005 ). Wong and Li ( 2011 ) investigated the connection between information and communication technology (ICT) implementation and pedagogical change. They concluded that organisational interventions and pedagogical interventions interacted with each other in effecting changes in student learning. Korhonen et al. ( 2014 ) introduced an innovative school community model, which addresses the development of four elements: students’ learning and learning environments, teachers’ professionalism, leadership and partnerships, as central to the advancement of educational innovation related to versatile use of digital technology. The model is generic, which leaves considerable room for interpretation in examining how current practices in a school should be evaluated and improved.

To investigate schools, we followed the sociocultural approach to learning (John-Steiner and Mann 1996 ; Packer and Goicoechea 2000 ): a school is an environment of collaborative, social activities of teachers, pupils and other participants; and their activities shape and transform its culture, values, practices and other specific characteristics. This approach also has an impact on our methodological choices: we mainly investigated practices rather than beliefs or thoughts .

The interest in the present study is in exploring the critical elements to be considered and the development processes needed in schools for reforming school education. Our specific focus is on the use of digital technology: how new digital technology has been applied and how it could be used to improve pedagogical and knowledge practices.

School is a complicated object to study: it consists of various administrative levels , from the national policy level to classrooms; various actors , such as school staff and pupils inside a school as well as parents and local school administrators outside a school; contradictory aims , such as aiming to ensure relevant competence levels for pupils in the future, but simultaneously, carrying on the traditions and history of society. For the complexity of a school as a research object, the theoretical background for the present study is multifaceted: research about school improvement, research about innovation, research about pedagogical practices (especially the collaborative knowledge creation traditions) and studies about digital technologies in education.

The connection to societal goals is essential for a school; it forms the external structure and resources for schools—which certainly have a strong impact (e.g. Ranson et al. 2005 )—but the responsibility for improving an individual school from the inside rests with the principal and the teachers. For this reason, the focus in the present study is on the elements and practices inside individual schools, bearing in mind the external factors and stakeholders. The reason for leaving the external administration outside the approach of the study is pragmatic: we want to create a model for schools for their own use, to reflect and improve those practices that they are able to change themselves. An individual school can seldom affect upper-level administrative decisions, but schools always possess some autonomy to make changes in the work of teachers and pupils. As Lemke ( 2001 ) emphasised, educational researchers should be explicit about the level of phenomena and the primary unit of analysis that the investigation is focusing on, but also be aware of the influence of the phenomena at upper and lower levels (e.g. municipal-level administrative decisions or individual teachers’ personal motives). Leclerc et al. ( 2012 ) investigated individual principals and teachers and made school-level conclusions based on these data. This was similar to work by Peck et al. ( 2009 ) when they were investigating innovations in schools. The present study focuses on classroom and school-level practices by interviewing individuals (teachers and principals), observing teaching practices and by conducting surveys for teachers and pupils. We presuppose that there is a strong and essential interaction between the different levels; this is a major starting point of our study.

In the following section, we first describe how the study relates to previous research approaches and then introduce the innovative digital school (IDI school) model: its basic elements and their connection with previous research. The framework has been applied in our study to examine schools. In the empirical section, the application of the model has been examined through case studies from three comprehensive schools.

Review of relevant previous research approaches for developing the model

Research on school improvement and change.

School improvement is aimed at improving student outcomes, wherever the change takes place (Creemers and Reezigt 2005 ). The large body of research about school improvement is one of the cornerstones of understanding the structures and practices of schools, such as leadership practices, teachers’ professional collaboration or pedagogical practices. Studies about school improvement have indicated how schools have benefited from restructuring their common practices, such as teachers’ tasks, activities and learning practices, leadership practices and the ways pedagogical methods are organised, in order to meet the developmental challenges (Crook et al. 2010 ; Harris 2002b ; OECD 2015 ). The elements of consensus about the vision (in vision of the school) and shared leadership (in leadership) are based on the studies presented here.

The school improvement movement and related research are strongly connected to educational systems and the policy-based and societal goals of education. Countries differ in their goals and views about school improvement, and the means for improving education can even be contradictory—leading also to quite different results (Hargreaves 2011 ; OECD 2014 2015 ). In countries such as the UK, the approach has been hierarchical top-down, whereas in the Nordic countries, the emphasis is on democracy, meaning the goal is to give schools and teachers responsibility for the improvement (Sahlberg 2011 ; Wrigley 2003 ). The elements of practices of the teaching society are based on the approach of teachers’ responsibility for the school improvement.

Researchers have defined some necessary characteristics for a school as a learning organisation (Senge et al. 1994 ). These are mutual trust and willingness to engage in open communication by the participants (Creemers and Reezigt 2005 ; Harris 2002b ; Leclerc et al. 2012 ; Senge et al. 1994 ); teachers’ shared values and visions, which focus on student learning (Leclerc et al. 2012 ); and collaborative knowledge-sharing as a tool for continuous growth of both teachers and schools. Knowledge sharing is a fundamental transformation of the teaching profession itself and is a route for creating collaborative cultures (Fullan 2001 ; Leclerc et al. 2012 ; Pedder and MacBeath 2008 ). Furthermore, staff members have opportunities to influence the school’s activities and policies (Harris 2002b ; Newmann et al. 2000 ), teacher collaboration is further supported by practical arrangements such as allocating time for teacher collaboration and teachers assume collective responsibility for attaining goals (Creemers and Reezigt 2005 ; Leclerc et al. 2012 ; Newmann et al. 2000 ). The elements of practices of the teaching community and school-level knowledge practices are based on the studies presented here.

For school improvement, the role of the school principal is essential. The principal manages the processes, motivates, organises and involves the staff in improvement, shares values for creating and supporting common visions (DuFour and Mattos 2013 ; Harris 2002a ) and understands teachers’ learning as a vehicle for the school’s continuous improvement (Earley 2010 ). Leadership affects the atmosphere for collaboration and experimentation (Wong and Li 2011 ). School leadership is best understood as a distributed practice, stretched over the school’s social and situational contexts, which is also beneficial for teachers (Facer 2012 ; OECD 2015 ; Spillane et al. 2004 ). It is an interactive process to build social capacity and trust, and to support networking (Harris 2002a ; Leclerc et al. 2012 ; Resnick and Spillane 2006 ). A challenge for a principal as an educational leader is the requirement for networking with other principals, administrators and other external stakeholders, which provides new perspectives and promotes the creation of effective and sustainable improvement (Hargreaves and Fink 2003 ; Harris 2010 ). The elements of leadership are based on the studies presented here.

In addition to research on school improvement, the research on knowledge work gives essential inspiration on how to view schools as organisations. Brown and Duguid ( 2001 ) emphasised practices and their travelling within an organisation and through sub-cultures. This sharing and collaborative creation of knowledge and practices is realised via boundary objects, such as common ways of working or shared objects to be developed. Brown and Duguid were investigating business firms, but schools are also knowledge work organisations. The elements of development practices (in practices of the teaching community) and common knowledge practices with technology (in school-level knowledge practices) are based on the ideas of Brown and Duguid.

Research on innovation applied in school context

Research concerning innovation provides essential added value to understanding the improvement of pedagogical practices. There are various definitions of innovation, differing between the level of focus and the novelty of the innovation (OECD 2010 , 2014 ). Some definitions regard only fundamentally new change as innovation, some also accept inclusion of issues that are novel in the context of the users. Messmann and Mulder ( 2011 ) defined an innovation as follows: ‘products or processes that are new and applicable for a certain individual, group or organisation and that are useful for the same or a different individual, group or organization’ (p. 66). This definition is close to the approach adopted in the present study. The emergence, acceptance and distribution of innovations that focus on the connection between individuals and organisations are especially important when answering the question about how educational innovations are adopted and what are the conditions for their dissemination.

An educational innovation succeeds or fails with the teachers who shape it (Lieberman and Pointer Mace 2008 ). In every significant change, the locus of innovations in practice could be traced to insights and initiatives of individuals, and collective negotiations and actions through which the changes have been achieved (Peck et al. 2009 ). Messmann and Mulder ( 2011 ) found in their study that the most powerful processes of learning and innovation took place in informal professional and personal relationships and in teachers’ communities. Teachers were motivated to work for change, and their positive individual image was framed by the experience of social support by colleagues and the supervisor as well as a stimulating climate for innovation. This also created a social norm that innovative work was appreciated. Several matters facilitated innovative work behaviour: competence, impact, responsibility for change, motivation for change, supervisor’s support, participative safety, supportive atmosphere and job complexity (see also Kunnari and Ilomäki 2016 ). Furthermore, in studies of teachers’ learning in innovation projects, experiments in practice and teacher learning go hand in hand (Bakkenes et al. 2010 ; Ilomäki et al. 2017 ). According to Bakkenes et al. ( 2010 ), informal learning brought fewer positive results than organised learning, especially reciprocal working with a peer or in a collaborative project team. Pedder and MacBeath ( 2008 ) argued that for schools (in the UK), the challenge appears to be in reasserting the values of learning, risk-taking, critical introspection, experimentation and innovation at all levels of the school organisation, and putting these into practice. Preconditions for innovation in organisations resemble the characteristics of learning communities: supporting teachers’ competence, autonomy and collegiality motivate teachers to change their teaching approaches (Lam et al. 2010 ; OECD 2015 ).

The elements of vision of the school and pedagogical collaboration and sharing of expertise and development practices (in practices of the teaching community) are based on the studies presented here.

Technology adoption as an innovation in school

The expectations about rapid acceptance and implementation of digital technology into educational practices have not been fulfilled (EU 2013 ), although some promising results indicate the connection between new pedagogical practices (= less teacher-centred) and the use of digital technology (Donnelly et al. 2011 ; Overbay et al. 2010 ; OECD 2014 ). In schools, technology is often still used for prevailing teaching methods, such as information sharing, or doing simple exercises, rather than for promoting collaborative or creative activities, solving complex problems or improving students’ digital competence (Livingstone 2012 ; OECD 2010 ).

Two alternative explanations for transforming educational practices associated with ICT have been suggested (Cuban et al. 2001 ; Twining et al. 2013 ): The first is a ‘slow revolution’ and support for existing practices, in which small changes accumulate over time and create a slow-motion transformation towards new ways of working. Only routines are replaced, and no changes are made in learning content or pedagogical practices. This explanation is anchored to the notion of a time lag between the invention of new technology, the adoption of innovations and the slow spread of its virtues through the general population. According to this explanation, the adoption of technology is an inevitable result which will come about anyway. The second explanation, ‘active transformation’ tries to account for the sustaining of teacher-centred practices: teachers and school make plans and decide how technology should be implemented in how best to answer to the specific challenges the school has. The curriculum content and/or processes will be changed, and these are changes that could not have taken place without digital technology.

There is a large body of studies about how digital technology has been implemented in education; e.g. what resources schools, teachers and students have; how much digital technology is used in classrooms; and what practices digital technology is used for (OECD 2010 , 2011 , 2014 , 2015 ). First, it is essential that teachers and students have the opportunity to learn to use digital technology, and second, that they have meaningful and necessary resources to use it. Teachers’ digital competence, related to pedagogical understanding of using technology in education, is the corner stone of supporting students’ digital competence (Hakkarainen et al. 2000 , 2001 ). The elements of pedagogical practices and digital resources are based on the studies presented here.

Research on learning as knowledge creation

Those theoretical approaches emphasising learning as collaborative knowledge creation (Bereiter 2002 ; Paavola and Hakkarainen 2005 ; Hong and Sullivan 2009 ) have strongly influenced our views concerning the pedagogical development in schools through digital technologies. According to these approaches, teaching should primarily promote knowledge innovation and collective advancement of shared knowledge products (Scardamalia and Bereiter 2006 ; Hong and Sullivan 2009 ). Arguments for these approaches are the requirement to promote adaptive expertise, collaboration skills and capabilities to work creatively with knowledge, which are the competencies needed in education, working life and society in general. Recent discussions concerning the learning of ‘21st Century Skills’ have similarities with these ideas: school learning should focus more on supporting the development of the relevant competencies that are needed to cope with the challenges of the unknown future, instead of concentrating on content learning and routine tasks (Ananiadou and Claro 2009 ; Bell 2010 ).

Features of pedagogical practices representing the collaborative knowledge creation approach include learners’ engagement, goal-oriented production of knowledge objects for relevant purpose, collective efforts and resources and versatile use of modern technologies (Robin 2008 ; Bell 2010 ; Scardamalia and Bereiter 2006 ; Tan and McWilliam 2009 ). The role of technological applications in such practices is often to provide flexible tools for communication and networking, co-authoring of shared knowledge products and managing joint working processes (Lakkala et al. 2009 ). The elements of pedagogical practices are based on the studies presented in the two previous chapters.

Scardamalia and Bereiter ( 1999 ) suggested that to help students to succeed in the knowledge society, schools should become knowledge-building organisations, in which students are members, not clients. Their suggestions are in line with the ideas of learning as knowledge creation (in which tradition they have a profound contribution). The element of pupils’ involvement (in school-level knowledge practices) is based on the this approach.

The elements of innovative digital school

Based on previous research approaches reviewed above and our own studies (Ilomäki and Lakkala 2011 ; Lakkala and Ilomäki 2013 ), we created the innovative digital school (IDI school) model for investigating whether schools use digital technology in an innovative way to improve pedagogical and working practices. In developing the model, we have emphasised leaning on relevant previous research approaches to avoid criticisms about creating a model based on occasional empirical findings, which leads to a quasi-theoretical model (Wikeley et al. 2005 ). However, we have also used a data-driven approach with extensive data from everyday practices of schools in order to avoid the gap between the theoretical model and ordinary practices in the field. Such data-driven elements, also acknowledged somewhat by research, are especially elements in school-level practices: physical premises (Cleveland and Fisher 2014 ; Gislason 2010 ) and pupils’ involvement in school level activities (Katsenou et al. 2015 ; Svanbjörnsdóttir et al. 2016 ). Table  1 presents the relationship between the elements of IDI School model with relevant research approaches, the main conclusions of previous studies related to the elements of our model and the main references.

The elements are presented in visual form in Fig.  1 .

The innovative digital school model: elements of a school regarded as relevant for developing schools through digital technology

Aims and research questions of the study

The reason for developing the IDI school model was to offer a framework for research but also to provide a research-based model for schools to reflect on, understand and improve their own practices to achieve sustainable pedagogical improvements with the help of digital technologies. There is a need for research-based, practice-oriented methods that help schools and teachers themselves reflect and investigate their own practices and thus improve them (Angelides et al. 2004 ). The aim of the present study was to examine how the model can be used to evaluate the existing practices of the schools used as examples and to make recommendations for improving the practices. The following research questions were constructed:

How does the innovative digital school model help to identify good practices and points for improvement in using digital technology for school change in the example schools?

How does the model reveal the essential differences in using digital technology for school change between the example schools?

The study is an explanatory multiple case study for explaining how the theoretical model used supports the description of the cases and how the cases differed from each other (Yin 2014 ). The purpose is to increase understanding of the cases (Merriam 1998 ) and to create analytic generalisations for other cases and situations (Yin 2014 ). The study relies on holistic data collection strategies, following the mixed methods approach (Johnson and Onwuegbuzie 2004 ). The use of several approaches and methods leads to better understanding of the objects of investigation and mixing various methods gives a more accurate picture of what is going on, while different methods help to answer slightly different questions (Todd et al. 2004 ). They provide an opportunity to present a greater diversity of views (Teddle and Tashakkori 2003 ) and help us to understand complex phenomena (Newmann et al. 2000 ). In the study, the mixed methods followed the triangulation design model, the variant of multilevel research (Creswell and Plano Clark 2007 ) in which different methods are used to address different levels within the system (school) and the findings are merged into one overall interpretation.

Three basic education schools (grades 1–9) participated in the study. They all are located in the metropolitan area of Helsinki. The city’s education department is the local organiser of education and in principle; all schools have equal access to resources. The local administration organises the technical resources (network connections, computers and other digital tools, the virtual learning environment and other applications). The city has also provided good opportunities for in-service training about digital technology. However, the schools also have some capacity to acquire resources of their own choice, such as by participating in national development projects, or in voluntary teacher training events. All teachers have a university degree and they are qualified teachers.

All the schools are located in suburbs.

School A is located in a residential area of single-family houses. In the area, the unemployment rate was 4.8%, the proportion of inhabitants with a higher education background was 24.5% of the total, the number of inhabitants with a foreign background was 3.6% and the income per residence was €87,645 (Tikkanen and Selander 2014 ). The curriculum of the school emphasises environmental education and sustainable development. It is also the member of a programme which aims to reduce bullying. There were 360 pupils at school A in 2015.

School B is located in an area of small houses and blocks of flats. In the area, the unemployment rate was 7.1%, the number of inhabitants with a higher education background was 11.9%, the number of inhabitants with a foreign background was 9.9% and the income per residence was €57,335 (Tikkanen and Selander 2014 ). The school has no emphasis on any one subject; it aims to be a safe local school. There were 640 pupils at school B in 2015.

School C is located in an area of mainly blocks of flats. In the area, the unemployment rate was 15.1%, the number of inhabitants with a higher education background was 3.2%, the number of inhabitants with a foreign background was 23% and the income per residence was €32,182 (Tikkanen and Selander 2014 ). The school emphasises creativity and handicrafts, and it has two special classes emphasising digital technology from 3rd to 9th grade. The school has several special education classes, and it has organised preparatory teaching for immigrant pupils. There were 375 pupils in the school in 2015.

Participants

Participants of the study were principals (one from each school), teachers with permanent positions and 9th grade pupils. Principals and teachers were not asked for their age; the mean of pupils’ age varied from 15.3 to 15.6 between schools. Table  2 shows the number of participants and their gender.

The response rates to the survey of teachers and students at each school varied as described in Table 2 . Schools and their principals, teachers and pupils participated voluntarily in the study. Permission to participate in the study was sought from parents of the pupils concerning surveys and the videotaping of lessons.

Measures and data collection

From each school, the following data were collected:

Lesson observations

Five subject teachers using digital technology in teaching were recruited from each school for classroom observations and interviews. The lessons in which digital technology was somehow used by the teacher or pupils were chosen for observation. A pre-planned observation sheet of phenomena to be observed was used; the focus was on classroom practices, such as the nature of assignments, pupils’ activities in completing the assignments, the use of digital technology, pupils’ and teachers’ interaction regarding the assignment and technology as well as the focus of the teacher’s guidance. The teachers of the lessons that were observed were interviewed briefly before and after each lesson, concerning their observation about the goals and practices of the lesson. The lessons and the short interviews were videotaped; the videos were used to complement written observation notes. In Table  3 is a list of the lessons observed.

The principal and five teachers at each the school were interviewed using a semi-structured interview. The interviews focussed on the following themes: the use of digital technology in teaching, the school’s vision, the principal’s professional competence and its development, teachers’ collaboration practices and school community and the role of the principal. The principal was also asked about leadership issues. The interviews lasted about 1 h.

Data about the use of digital technologies were collected through questionnaires from pupils and teachers. Both questionnaires were based on questionnaires developed in previous studies (Hakkarainen et al. 2000 ; Hakkarainen et al. 2001 ), and for this study, they were modified to take into account recent technological development (e.g. questions about the use of Internet were added).

The teacher questionnaire was sent to all teachers with permanent positions at the schools. It consisted of questions concerning the following topics:

Digital competence: 17 Likert-type statements (1 = not at all, 5 = very well); e.g. How well do you manage spread sheet applications , e.g. Excel

The use of digital technology: 41 statements concerning the use at school and at home, the use of various Internet services, the use of various digital applications with pupils (answer options were not at all—seldom—monthly—weekly—daily)

The need for support and training in using technology: Four Likert-type statements (1 = completely inadequate, 7 = completely adequate)

The usefulness of digital technology in some pedagogical practices: 20 Likert-type statements (1 = totally useless, 7 = totally useful); e.g. Small - scale project works , e.g. information search for understanding a topic .

The pupil questionnaire was sent to 9th grade pupils. The questionnaire consisted of questions concerning the following topics:

Digital competence: 17 Likert-type statements (1 = not at all, 5 = very well)

The use of digital technology: 33 statements concerning the use at school and at home, the use of various Internet services, the use of various digital applications at school (answer options were not at all—seldom—monthly—weekly—daily)

In which subjects is ICT used at school, also the frequency: Seven statements concerning school subjects (answer options were not at all—seldom—monthly—weekly—daily)

Data analysis

Each type of data was first analysed separately as described below.

Observation notes and related short interviews were used to categorise the pedagogical approach of each lesson. The classification was created by the researchers through abductive use of theory-informed and data-grounded analysis on the data (Timmermans and Tavory 2012 ). The pedagogical infrastructure framework (Lakkala and Ilomäki 2015 ) was applied to define the elements examined in the practices: technical structures (role and organisation of technology use), social structures (role and nature of collaboration), epistemic structures (practices of using and creating knowledge) and cognitive structures (cognitive challenge of tasks, support for pupils’ self-regulation and metaskills). Three categories were created for defining the prevailing pedagogical approach of each lesson:

Structured content learning : Technology was used for teacher presentations or structured practicing (e.g. drill-and-practice tasks), individual tasks, focus on learning factual and declarative knowledge, low cognitive challenge and no explicit attention to metacognitive aspects of working

Learner - centred activating tasks : Technology was used for information seeking or minor authoring tasks (e.g. short essays), mainly individual tasks but some sharing between pupils, small-scale knowledge production, mid-level cognitive challenge, but no explicit attention to metacognitive aspects of working

Collaborative knowledge creation : Versatile use of technical applications for knowledge creation (e.g. reports), working mainly based on pair or group work, open-ended task lasting more than one lesson, high cognitive challenge and modelling of working strategies

The interviews were transcribed verbatim and then analysed following a theory-driven content analysis, using Atlas.ti software (version 7.1.5). The elements of the IDI school model (see Fig. 1 ) were used as categories to define which sections in each interview described which phenomenon of the school practices. The interview questions were designed to address the elements of the model, but in the analysis, we also considered that an answer referring to any of the elements might emerge under any question. In constructing the case descriptions of schools, the coding in Atlas.ti was used to extract all interview excerpts from an individual school concerning a certain school model element, in order to make the judgement and description of the nature and level of practices in that school.

Teacher and pupil surveys

The data were analysed with IBM SPSS 22. The means of items were compared using one-way ANOVA and Tamhane’s T2 post hoc tests.

Integration of the results from individual data sources

The dimensions and levels of each sub-element were constructed descriptively by combining the analysis results of separate data sets. The analysis was of iterative explanation building (Yin 2014 ): The analysis criteria, based on the IDI school model and described above, were first compared with the empirical evidence from the first case, and then revised and compared with the evidence from the other cases.

The dimensions of each phenomenon (elements of the IDI school model; see Fig. 1 ) and the data produced information about each element. Each element was scored in the following way: 1 (low level), 2 (average level) and 3 (high level). The scores were based on the analysis of all data sources, and the researchers together decided the scoring. In addition, the scores of the main elements were constructed as the means of the sub-elements. In Appendix , the analysis framework of the phenomena and the data is presented.

The results are first presented in the order of data and data analysis; the integration of the results is presented after that.

Practices at each school, according to the interviews and classroom observations

The practices are presented following the order of the elements of the IDI School model in Table 1 .

Visions concerning digital technology related mainly to technical skills and resources. The visions were emerging; most teachers shared them, but the visions were not fully clear in teachers’ minds. The school had several common development projects going on and the importance of development activities was emphasised in the interviews.

Shared leadership came true in systematically organised teacher teams, which included all teachers, and the active role of the executive team. The principal’s networking included basic collaboration inside school and with municipal school administrators and parents. The principal acted as an enabler of teachers’ development efforts (e.g., organising resources for training), but also as the promoter of new development initiatives.

Teachers had various established collaboration practices, such as pedagogical workshops, co-teaching between teachers or sharing of teaching plans and materials through virtual forums. The school had multiple development practices, e.g. national and international projects, or periodic joint reflection of teaching. The teachers interviewed actively collaborated with colleagues at the same school, but they did not do much networking outside the school.

Teachers’ perceptions of digital technology in education focussed on aspects related to motivation, increased variability in methods or increased student-centredness and learning effectiveness, but there were few mentions about collaborative or creative activities. The usage of digital tools in teaching included a range of methods, from drill-and-practice tasks to challenging long-term project work. Three of the five lessons observed represented collaborative knowledge creation practices; some teachers appeared to use advanced pedagogical methods with digital technology.

Plans for developing common school-level practices, e.g. about media usage and study practices, had been started. A joint Media Week was organised annually. A virtual learning platform was established as an information channel for teachers, and its usage with pupils was actively promoted. Teachers’ experience of the school premises was that they were quite flexible, but some teachers mentioned the lack of a computer laboratory and the distribution of computers as problems. Pupils were involved in school-level activities in various ways; e.g. the pupils’ media team was responsible for documenting school events, and a training event in which pupils guided teachers to use social media had been organised. School-level networking was based on the activity of some teachers and their classes participated in national and international projects.

Most interviewees thought that too few computers were available for teaching, and that login in the laptops took too much time in lessons. The teachers had common plans about which digital skills to teach to pupils in each subject and grade. Digitally more-competent teachers had organised training sessions for less-competent colleagues about the central applications, and teachers were encouraged to participate in in-service courses organised by the city.

Most of the teachers interviewed shared the opinion that there was no explicit vision in the school about digital technology. Some interviewees mentioned ensuring that pupils had good basic digital skills, whilst others emphasised the improvement in teachers’ digital competence, or flexible digital resources. Attitudes towards development efforts were positive, and some projects with other schools were going on, and there were plans for developing the school’s practices. However, the development interests appeared to be dependent on the motivation of individual teachers.

The teachers were divided into three administrative teams, each of which was allocated tasks based on needs; the teams had some responsibility of their own. The principal had established collaboration with the vice principals, the executive team and the principals of nearby schools, but there were no other explicit networks. The principal was described positively: the creator of a positive atmosphere, a pedagogical leader and a provider of resources for professional development.

Pedagogical collaboration included team discussions, some co-teaching practices, sharing of materials and informal discussions; it was mainly based on subject-specific groups and spontaneous and voluntary participation. The school had one common development programme (about learning to learn skills), but otherwise, development efforts included participation in training events and projects depended on the teachers’ own initiative. Two teachers mentioned an external organisation as a point of contact, but otherwise, networking included conventional partners: the city’s teacher training unit, teachers’ friends or parents. One teacher had no collaborators outside school.

Teachers’ pedagogical perceptions about digital technology included benefits concerning increased motivation, usage as a presentation tool, variation in methods and a useful writing tool. None of the teachers explicitly mentioned more challenging project- or inquiry-based methods or collaborative learning, but two of the lessons that were observed represented such practices.

Communication and sharing of materials among teachers was organised through web-applications, but otherwise no common knowledge practices were mentioned at the school, nor between teachers or pupils. Also, ICT courses for pupils were voluntary. Some teachers mentioned old-fashioned, inflexible premises and computer laboratories as a weakness; the problem was visible also in the lesson observations. One teacher had used older pupils as guides for younger pupils in technology use; otherwise, nobody described any practices for involving pupils in school-level activities. The interviews did not reveal any established school-level networks besides neighbouring schools participating in a common project.

Concerning the utility of digital resources, the teachers were not satisfied with the fixed computer laboratories and the shortage of equipment, especially mobile tools (like tablets). They were satisfied with the technical support but did not mention any examples of pedagogical support.

Digital visions appeared not to be shared visions; the teachers interviewed mentioned basic digital skills, increasing technology use and more versatile practices, or explicitly said that they were unaware what the vision is. The experience of the atmosphere was as supportive of development efforts, and the school participated in various national and international projects.

Leadership was shared through subject-based and task-based teams, and some teachers had taken the responsibility for development projects. The principal had active collaboration with local institutions at various educational levels, and she had taken an active role in renewing common practices.

The teachers had many collaboration practices: working in teams or projects, informal discussions, sharing of ideas and materials and interdisciplinary co-teaching. The interviewees mentioned development practices such as projects and training sessions, but participation in them happened only occasionally and participation was voluntary, depending on the teacher. The teachers had networks with various stakeholders in institutions related to their subject, e.g. the church, music college or police.

Teachers’ perceptions about technology in education included conventional issues, such as individualised teaching, up-to-date information sources or useful tools for pupils’ work, but in general, teachers’ opinions were very positive. The pedagogical practices that were mentioned with technology were versatile but not very innovative, like individual knowledge production or rehearsal of content. None of the lessons that were observed included challenging collaborative knowledge creation activities.

The teachers had made common plans about the teaching of ICT and media communication to different grades of students, and web-applications were used for information sharing between teachers. Other common knowledge practices were not mentioned in the interviews. Some teachers experienced old, inflexible school premises as a challenge for advancing digitalisation, but a new room for project learning was under construction. Pupils’ involvement in school-level responsibilities and activities was not mentioned. The school had collaboration arrangements with external organisations through multiple national and international development projects.

The utility of technical resources was experienced as being at quite a good level, but the heterogeneity of teachers’ digital competence was mentioned as a challenge. Teachers had good opportunities to participate in courses organised by the city, and there had been some internal training events, but the emphasis had been on technical skills, not on pedagogical issues.

Results of questionnaires with teachers and pupils

The results are presented in the order of the elements of the IDI school model shown in Table 1 .

Results of the teacher questionnaire

Perceptions of using digital technology in education.

Teachers were asked about the usefulness of digital technology in various pedagogical assignments. Table  4 shows the means, standard deviations (SDs) of teachers’ perceptions and the p value of statistical differences.

There were statistically significant differences in the following perceptions of the usefulness of digital technology: At school A, teachers’ evaluation scores were statistically significantly lower than the scores of teachers at the other schools in the following pedagogical practices: small-scale project work F (2,54) = 12.841, p  = .000; practicing skills F (2,54) = 10,866, p  = .000; small-scale products (like writings during one lesson) F (2,54) = 12.256, p  = .000; net discussions related to the topic F (2,54) = 6.412, p  = .003; and presenting information and support for illustration F (2,54) = 12.148, p  = .000. Tamhane’s T2 post-hoc comparisons were used to calculate the differences between the schools.

Pedagogical practices with digital technology

Teachers were asked about the use of various digital applications and Internet services in their own teaching; there were no statistically significant differences between schools in how much they reported using various applications and the Internet.

Teachers were also asked about using digital technology in various pedagogical practices. In Table  5 , the means and SDs of all practices are presented. There were a few statistically significant differences in the reported use of digital technology.

The statistically significant differences were found in the following items: small-scale projects F (2,54) = 13.233, practicing skills, F (2,54) = 10.988, p  = .000; small-scale products (like writings) F (2,54) = 9.084, p  = .000; and information presenting and support for illustration F (2,54) = 5.934, p  = .005. Tamhane’s T2 post-hoc comparisons were used for calculating the differences between the schools.

Teachers’ digital competence

The results showed, first, that there were no statistically significant differences between schools in teachers’ self-evaluated digital competence, and that teachers evaluated their competence in basic digital application as being quite high (scale 1–5), such as using email (mean 4.7), searching for information on the Internet (mean 4.7), word processing (mean 4.4), loading files from the Internet (mean 4.2) and using the digital learning environment (mean 3.8). These formed a group of basic digital competence. The second group of applications were using spreadsheets (mean 3.2), digital image processing (mean 3.1), graphics (mean 2.9) and social forums (mean 2.9). The lowest means were in virtual meeting tools (mean 2.3), creating www-pages (mean 2.3), publishing tools (mean 2.2), writing a blog (mean 2.2), publishing www-pages (mean 2.0), producing information to wiki (mean 1.9), voice and music (mean 1.9) and programming (mean 1.4).

Pedagogical and technological training and support

Figure  2 shows the means of teachers’ need for support and training for using digital technology.

Teachers’ need for support and training of digital technology

The evaluation of teachers at school A was that they needed both technical and pedagogical training less than teachers at the two other schools, and there was a statistically significant difference between schools A and B in need for technical training: F (2,54) = 9.993, p  = .000; and in need for pedagogical training: F (2,54) = 12.719, p  = .000, indicated with * in Fig.  2 .

Results of the pupil questionnaire

Pupils were asked which applications they use at school. In Table  6 , the means and SDs of those applications in which there were statistically significant differences between the schools are described.

The statistical significance of differences in means between the pupils of schools was analysed by using one-way ANOVA. The analysis indicated statistically significant differences in the means in the following items: using word processing: F (2,172) = 18.909, p  = .000; using spreadsheets: F (2,172) = 16.686, p  = .000; using email: F (2,172) = 38.490, p  = .000; using social forums: F (2,172) = 9.940, p  = .000; publishing in a web blog: F (2,172) = 22.253, p  = .000; using learning environments: F (2,172) = 17.316, p  = .000; publish pictures, texts or reports: F (2,172) = 5.811, p  = .004; develop my thoughts about the topic in a collaborative discussion: F (2,172) = 14.735, p  = .000; teacher guidance through the net for independent learning: F (2,172) = 9.678, p  = .000; freedom to surf in the Internet when assignments are done: F (2,172) = 15.361, p  = .000; and contact with pupils in other schools via email or the Internet: F (2,172) = 8.367, p  = .000; information search from the Internet: F (2,172) = 22.464, p  = .000; publishing in the Internet: F (2,172) = 7.281, p  = .001. Tamhane’s T2 post-hoc comparisons were used for calculating the differences between the schools.

There was also a difference in the statement about the use of ICT during leisure time for school work, in which pupils at school A had higher scores than pupils at the other schools. The statistically significant differences were between school A ( M  = 3.7, SD = .553) and schools B ( M  = 2.3, SD = .833) and C ( M  = 2.2, SD .956) ( F (2,172) = 55.259, p  = .000).

Pupils’ digital competence

Pupils at all three schools liked to use ICT at school, and there were no statistically significant differences concerning the statements measuring this: the use of ICT is easy ( M  = 4.2, SD = 1.034), the use of ICT makes learning more interesting ( M  = 3.9, SD = 1.111) and pupils would like to use ICT more at school ( M  = 3.8, SD = 1.192). Furthermore, there were no statistically significant differences in the use of technology at home and during leisure time.

Pupils also evaluated their competence in using various digital applications. The statistically significant differences in means and SDs between the pupils from the three schools are described in Table  7 .

The differences were analysed by using one-way ANOVA. No differences were found in applications which tend to be less used in schools, such as digital image processing, publishing tools, voice and music applications or programming. The analysis indicated statistically significant differences in means between pupils of participating schools in the following items: word processing F (2,172) = 13.287, p  = .000; spreadsheets F (2,172) = 15.092, p  = .000; email F (2,172) = 10.002, p  = .000; information search from the Internet F (2,172) = 6.492, p  = .002; writing a web blog, F (2,172) = 9.441, p  = .000; and using virtual learning environments F (2,172) = 9.042, p  = .000. Tamhane’s T2 post-hoc comparisons were used for calculating the differences between the schools.

Overview of the level of practices in the schools

In Table  8 , the results of the separate data sets have been integrated and scored for each school.

The scores show differences between schools: schools A and C are ‘strong’ schools in several major elements. At school A, digital resources are at an especially high level, and in general, school-level working practices are at a high level. At school C, leadership practices and teaching community practices are at a high level. School B has the lowest scores in every major element. In the ‘ Discussion ’ section, we will discuss about the differences more in detail.

In the study, we investigated the practices at three schools based on six elements defined in the innovative digital school model. We aimed to find out, first, if those elements could help in defining good practices and suggestions for improvement for developing the schools with digital technology; and second, if the model revealed essential differences between the schools.

Good practices and points for improvement in the example schools

In order to answer the first research question about how the IDI school model helps to identify good practices and points to be improved in using digital technology for school change, we describe the practices of each school separately.

Among the characteristics of school A were advanced and established practices in shared leadership, practices of the teaching community, advanced pedagogical practices with technology and school-level knowledge practices, including involvement of pupils and systematic promotion of their digital competence through pedagogical activities. However, shared visions about digital technology were only emerging, teachers’ digital competence was only average and the perceptions in the pedagogical usage of technology had considerable variety between teachers, although there were examples of inspiring pedagogical methods. Teachers did not report needing support for using technology which probably indicates both quite a good level of digital competence and well-organised support practices in the school. Pupils’ self-reported digital competence was at a high level especially concerning basic applications. Pupils reported using technology quite often during leisure time for school-related activities, and at school for various basic activities, but also for collaboration and networking. Based on the results, the following suggestions for improvements can be made for school A: (1) the teaching staff should focus on crystallising and sharing the school’s visions in using digital technology as the basis for further development (elements A1 and A2); (2) teachers should share their pedagogical ideas and experiments, e.g. in organised meetings and workshops (elements C1 and C2); and (3) teachers should develop their digital competence, such as by making use of the training resources made available by the city and by organising school-level small-scale training (elements F2 and F4).

School B had some shared leadership practices and the principal was appreciated, but otherwise the school was not very advanced in any of the measures. Attitudes towards development efforts were positive, but established practices were lacking. There were teachers who collaborated with each other, participated in development projects and used digital technology in teaching in advanced ways, but activity was based on teachers’ own initiative and voluntariness. Especially at the school level, knowledge practices were minimal, both concerning the promotion of pupils’ involvement and digital competence, and school-level networking. Teachers at the school reported needing both technical and pedagogical support in using digital technology. For school B, based on the results, the following suggestions for improvements can be made: (1) it is important to create a common vision for developing the use of digital technology (element A1) and promote development orientation among teachers (element A3). (2) The principal and the management team should create and organise systematic common practices to carry out improvements in all developmental areas (elements in C). (3) The digital resources should be evaluated and developed (all elements in F) and especially teachers’ digital competence should be improved (elements F3 and F4).

School C represents a school with high-level leadership practices, and a strong collaboration culture both inside the school and in the active external networking of both the principal, teachers and the whole school. The school had a strong development orientation in general, but it had not yet become true in the school-level knowledge practices, digital resources or advanced practices of using technology in teaching. School C has much potential for improvement, and based on the results, the following suggestions for improvements can be made: (1) the usage of digital technology for school improvement should be more deliberate through agreements of shared visions (elements A1 and A2); (2) the school should create systematic development of pedagogical and knowledge practices (elements D and E); and (3) all pupils’ and teachers’ digital competence should be improved, both with pedagogical practices (element D2) and training and support (elements F2, F3 and F4).

Differences between the schools investigated

To answer the second research question about how the model reveals essential differences in digital technology for school change, we compared the practices of schools by summarising the results of data analyses.

The results of the study indicate that there were some clear differences between the schools, although they also had a lot in common, especially in the principal’s role and teachers’ digital competence; common characteristics might be a result of common policies and practices of the city in these issues. Such elements, which are strongly dependent on school-level decisions, differed between the schools. Included here are teachers’ pedagogical practices and school community’s practices, including sharing of vision-level decisions. According to previous studies (Vieluf et al. 2012 ; OECD 2014 ), shared community-level practices are central to sustainable school improvement, but currently they represent practices which are not yet widespread in schools and require extending the teachers’ professional role beyond only taking responsibility for their own teaching in classrooms.

A clear difference between the three schools was in the presence or absence of practices involving pupils in school-level activities. Only at school A had shared, established practices for pupil engagement at school-level been developed, such as responsible pupil teams (e.g. media and environment teams) or pupils as guides in using digital technology. Various participatory practices presume seeing pupils in an active role in the classroom or at school, not only as objects of teaching during lessons (Facer 2012 ; Kehoe 2015 ; Pereira et al. 2014 ).

Also, the nature of pedagogical practices with digital technology differed between schools. At school A, pupils reported using digital technology more than pupils at the other two schools, both in the classroom and at home for school-related activities. The use focused on general applications and pedagogically ‘advanced’ practices, such as using a virtual learning environment and collaborating via the web. These practices probably helped to improve pupils’ basic digital competence: the regular use of digital tools was an essential condition for competence learning (see also OECD 2011 ; Aesaert et al. 2015 ). Furthermore, classroom practices were most advanced at school A and a comparisons of the teachers’ survey answers between the schools indicated that teachers at school A used and believed less in teacher-centred practices with digital technology than teachers at school C.

The innovative digital school model was not developed primarily for detailed comparisons of differences between schools. A more useful approach is to examine school profiles: the shape of the profile demonstrates the emphasis on the practices inside a school, and the level of the profile elements helps each school to position its strengths and development needs compared with reference schools. Figure  3 presents the results of Table  9 in a visual form illustrating the profiles of the three schools investigated.

A summary of the scores of the three schools in the elements of the IDI school model

The profiles demonstrate the differences between the schools: school A has quite advanced practices in all elements; school C is high in school-level practices involving teachers and the principal, but only average in practices directly affecting pupils; and school B is least-developed in all elements, but highest-developed in leadership and digital resources. We propose that one reason for the differences between schools is the level of vision and how well it is shared among teacher community. Schools A and C had remarkably higher scores in the elements of goals and the vision compared with school B (although even schools A and C could improve on this). These results are in line with previous research according to which an explicated and shared vision is a key element in school improvement and change (see, e.g. Senge et al. 1994 ; Antinluoma et al. 2018). At school B, the vision and goals, pedagogical practices with digital technology and school-level knowledge practices were all at a low level, although the digital resources are almost the same as at school C. For benefitting from digital technology in improving pedagogy, collaborative visions and efforts especially focusing on that are needed (Laurillard 2008 ); technology does not change pedagogical practices per se, which describes the situation at school B. At both schools A and C, the elements related to vision, leadership and teacher community received good or even high scores, but school A was more advanced in pedagogical practices with technology. It seems that to develop high-level pedagogical practices with technology, deliberate effort is needed.

Conclusions

Validity of the innovative digital school model.

The two aims of the IDI school model, to reveal good practices and points for development, as well as to expose differences, were fulfilled, from which we interpret that analytic generalisation (Yin 2014 ) from the model is possible. With qualitative data (classroom observations and interviews), we were able to identify new and innovative practices in the school context, developed in the schools for their individual needs. The quantitative data supported the findings based on qualitative data. Innovative practices were found, especially at the school which was evaluated as being the most advanced in all elements. One of the schools was least-developed in all the measures investigated, and the third school was in between: it had a strong development culture generally, but the focus of the development work had not been on using digital technology as a vehicle for change. In the latter two schools, digital technology was taken into use by individual teachers and often without integrating pedagogy and technology.

The IDI school model as a framework for investigating differences worked particularly well for those elements which are mainly the responsibility for leadership inside a school (visions of the school, practices of teaching community and school-level knowledge practices); there were clear differences in these between the schools, especially according to the qualitative data. The three schools had differences even though they each follow the same curriculum, and the same detailed legislation. The teachers’ educational background is homogeneous, and the schools are located in the same city, which is responsible for providing the resources for all the city’s schools. The role of the city probably explains why there were no statistically significant differences between teachers’ self-estimated digital competence and the use of digital technology in general.

Results of the qualitative and quantitative data were somewhat contradictory in the use of digital technology in classrooms. In the teacher surveys, there were no statistically significant differences between schools, but there were in the pupil surveys. Our explanation is that pupils use technology in some lessons so much that it affects the overall experience, and that pupils in 9th grade use technology more than pupils in lower grades.

Another contradictory issue in the surveys was the result of pedagogical practices. The teachers participating in the observations and interviews were probably more interested in digital technology and their practices were more advanced than the practices reported in the survey by many more teachers. As Kivinen et al. ( 2016 ) suggested, the technology use of the majority of teachers might represent the use of technology per se, which leads to a pragmatic solution in which technology does not support a knowledge creation approach in learning but is used for practical experiments and learner-centred activities.

The schools that were examined are located in areas of different socioeconomic backgrounds. The results do not show differences based on the background, which probably indicates the homogeneity of Finnish schools. All schools receive the same resources from the city, and parents do not make financial contributions for the education. The school from the area of lowest socioeconomic status has participated in various projects during years, and this has promoted the capacity of the teaching staff. Teachers’ development orientation has supported the school to develop advanced practices regardless of challenging socioeconomic background of the pupils.

The results of the study proved that mixed methods are needed when investigating the practices of a whole school. Using only the survey data would not have revealed some of the central differences between the schools and would have given a quite narrow view of the situation at each school. For the qualitative data, it would not have informed about the use of digital technology and the competence in using it. Collecting qualitative data requires more resources than using only surveys. However, we experienced that our data collection model (five teacher interviews and lesson observations, a principal interview and a survey of teachers and highest grade of pupils) was a reasonably inexpensive and valid way to examine the practices of a school.

Practical implications

The IDI school model is an attempt to address the need for practice-oriented methods that help schools and teachers to reflect on their own practices and improve them (Angelides et al. 2004 ), and to narrow the gap between empirical research and practical school work (Wikeley et al. 2005 ), especially related to the change processes of implementing new digital technologies in education.

The IDI school model can be used in schools as a shared conceptual framework for collective reflection, discussion and strategy planning. We have already had some promising experiences about using it in the in-service training of teachers and principals. The model can also be applied to collect best-practice examples from different schools and disseminate them to other schools, or to make school visits and benchmarking of practices more systematic.

At the municipal and national level, educational administrators may have an interest in evaluating the status of using digital technology in schools. As our study witnessed, quantitative data have limitations in describing collaborative pedagogical and working practices. Qualitative methods are important, but there is a need for accessible methods for collecting data widely about the current state of art in schools. The methods, experiences and results of the present study can work as a starting point for developing scalable methods.

As a policy-level implication, we suggest that local and national school administration focus on schools as knowledge work organisations when aiming to improvements, such as to increase the quality of pedagogical and knowledge practices with digital technology in schools. We suggest that all elements of the innovative digital school model be considered, and that the start should be committing the staff to change, by creating shared visions and aims about pedagogical development through digital technology, and by supporting school-level practices including both pupils and teachers.

Future research

In the present study, we used data from three schools to examine the applicability and validity of the IDI school model for evaluating the development of schools through digital technology. All three schools were in the same city and had similar municipal resources for digital technology and in-service teacher training, which allowed differences to be revealed, especially in those practices that schools can influence individually in that context. In future research, it would be important to test the model with a larger collection of schools from different contexts (size, location, socioeconomic background, etc.) and from different countries and cultures, thus also confirming the validation of the model.

Another interesting line of research would be to conduct studies in which the development of the same schools was followed longitudinally. Such studies could include interventional aspects: the investigated schools would get feedback and support from researchers to develop their practices further, and new data would be collected after some period for evaluating the influence of deliberate development efforts.

In the future, schools will face even more challenges and requirements that the school community will have to answer. The best and most effective schools reflect their practices and constantly improve their ways of working. We believe that the innovative digital school model offers a tool for schools and for researchers involved in this work.

Change history

18 march 2019.

In the original publication of this article (Ilomäki & Lakkala, 2018) the appendix Table 9 in PDF is in wrong place due to typeset mistake, which should be at the end of the manuscript. The original PDF version has been corrected.

Aesaert, K., Van Nijlen, D., Vanderlinde, R., Tondeur, J., Devlieger, I., & van Braak, J. (2015). The contribution of pupil, classroom and school level characteristics to primary school pupils’ ICT competences: a performance-based approach. Computers & Education, 87 , 55–69 https://doi.org/10.1016/j.compedu.2015.03.014 .

Google Scholar  

Ananiadou, K., & Claro, M., (2009). 21st Century skills and competences for new millennium learners in OECD countries. OECD Education Working Papers, No. 41, OECD Publishing. http://repositorio.minedu.gob.pe/handle/123456789/2529

Angelides, P., Leigh, J., & Gibbs, P. (2004). Analyzing practice for improving schools: the study of vignettes. School Effectiveness and School Improvement, 15 (3–4), 467–485 https://doi.org/10.1080/09243450512331383282 .

Bakkenes, I., Vermunt, J., & Wubbels, T. (2010). Teacher learning in the context of educational innovation: learning activities and learning outcomes of experienced teachers. Learning and Instruction, 20 , 533–548 https://doi.org/10.1016/j.learninstruc.2009.09.001 .

Bell, S. (2010). Project-based learning for the 21st century: Skills for the future. The Clearing House, 83 (2), 39–43 https://doi.org/10.1080/00098650903505415 .

Bereiter, C. (2002). Education and mind in the knowledge age . Mahwah: Lawrence Erlbaum Associates.

Brown, J. S., & Duguid, P. (2001). Knowledge and organization: a social-practice perspective. Organization Science, 12 (2), 198–213 https://doi.org/10.1287/orsc.12.2.198.10116 .

Chapman, C. (2008). Towards a framework for school-to-school networking in challenging circumstances. Educational Research, 50 (4), 403–420 Retrieved from https://search.proquest.com/docview/61969279?accountid=11365 .

Cleveland, B., & Fisher, K. (2014). The evaluation of physical learning environments: a critical review of the literature. Learning Environments Research, 17 (1), 1–28 https://link.springer.com/article/10.1007/s10984-013-9149-3 .

Creemers, B., & Reezigt, G. (2005). Linking school effectiveness and school improvement: the background and outline of the project. School Effectiveness and School Improvement, 16 (3), 359–371 https://doi.org/10.1080/09243450500234484 .

Creswell, J., & Plano Clark, V. (2007). Designing and conducting mixed methods research . Thousand Oaks, CA: Sage Publications.

Crook, C., Harrison, C., Farrington-Flint, L., Tomás, C., & Underwood, J. (2010). The impact of technology: Value-added classroom practice . Becta http://oro.open.ac.uk/34523/ .

Cuban, L., Kirkpatrick, H., & Peck, C. (2001). High access and low use of technologies in high school classrooms: explaining an apparent paradox. American Educational Research Journal, 38 , 813–834 https://doi.org/10.3102/00028312038004813 .

Donnelly, D., McGarr, O., & O’Reilly, J. (2011). A framework for teachers’ integration of ICT into their classroom practice. Computers & Education, 57 , 1469–1483 Retrieved October 16, 2016 from https://www.learntechlib.org/p/50758/ .

DuFour, R., & Mattos, M. (2013). How do principals really improve schools? Educational Leadership, 70 (7), 34–40 http://www.ascd.org/publications/educational-leadership/apr13/vol70/num07/How-Do-Principals-Really-Improve-Schools%C2%A2.aspx .

Earley, P. (2010). State of the nation’: a discussion of some of the project’s key findings. The Curriculum Journal, 21 , 473–483 https://doi.org/10.1080/09585176.2010.529699 .

EU. (2010). 2010 joint progress report of the council and the commission on the implementation of the ‘education and training 2010 work programme’. Official Journal of the European Union, C, 117 http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:C:2010:117:0001:0007:EN:PDF .

EU. (2013). Survey of schools: ICT in education. Final study report. Benchmarking access, use and attitudes to technology in Europe’s schools . European Commission https://doi.org/10.2759/94499 .

Facer, K. (2012). Taking the 21st century seriously: young people, education and socio-technical futures. Oxford Review of Education, 38 (1), 97–113 https://doi.org/10.1080/03054985.2011.577951 .

Fullan, M. (2001). The new meaning of educational change (3. ed.) . London: Routledge Falmer.

Gislason, N. (2010). Architectural design and the learning environment: A framework for school design research. Learning Environments Research, 13 (2), 127–145 https://link.springer.com/article/10.1007%2Fs10984-010-9071-x .

Hakkarainen, K., Ilomäki, L., Lipponen, L., Muukkonen, H., Rahikainen, M., Tuominen, T., Lakkala, M., & Lehtinen, E. (2000). Students’ skills and practices of using ICT: results of a national assessment in Finland. Computers and Education, 34 , 103–117. https://doi.org/10.1016/S0360-1315(00)00007-5 .

Hakkarainen, K., Muukkonen, H., Lipponen, L., Ilomäki, L., Rahikainen, M., & Lehtinen, E. (2001). Teachers information and communication technology (ICT) skills and practices of using ICT and their pedagogical thinking. Journal of Technology and Teacher Education, 9 (2), 181–197.

Hargreaves, A. (2011). Foreword. In P. Sahlberg (Ed.), Finnish Lessons. Can the world learn from the educational change in Finland? (pp. XV-XX. New York, NY: Teachers College Press.

Hargreaves, A., & Fink, D. (2003). Sustaining leadership. Phi Delta Kappan, 84 , 693–700 https://doi.org/10.1177%2F003172170308400910 .

Harris, A. (2002a). Effective leadership in schools facing challenging contexts. School Leadership and Management, 22 , 15–26 https://doi.org/10.1080/13632430220143024a .

Harris, A. (2002b). School improvement. What’s in it for schools? Great Britain: RoutledgeFalmer.

Harris, A. (2010). Leading system transformation. School Leadership and Management, 30 , 197–207 https://doi.org/10.1080/13632434.2010.494080 .

Hautamäki, J., Säkkinen, T., Tenhunen, M.-L., Ursin, J., Vuorinen, J., Kamppi, P., & Knubb-Manninen, G. (2012). Lukion tuottamat jatkokoulutusvalmiudet korkeakoulutuksen näkökulmasta , The competencies for further studies provided by the upper secondary school from the perspective of higher education . Jyväskylä: Koulutuksen arviointineuvosto.

Hong, H. Y., & Sullivan, F. R. (2009). An idea-centered, principle-based design approach to support learning as knowledge creation. Educational Technology Research and Development, 57 (5), 613–627 https://link.springer.com/article/10.1007/s11423-009-9122-0 .

Ilomäki, L., & Lakkala, M. (2011). Koulu, digitaalinen teknologia ja toimivat käytännöt. [School, digital technology and meaningful practices]. In M. Kankaanranta & S. Vahtivuori-Hänninen (Eds.), Opetusteknologia koulun arjessa II [Educational Technology at School’s Everyday Life] (pp. 47–67). Jyväskylä: Koulutuksen tutkimuslaitos, Jyväskylän yliopisto. http://urn.fi/URN:ISBN:978-951-39-4616-6

Ilomäki, L., Lakkala, M., Toom, A., & Muukkonen, H. (2017). Teacher learning within a multinational project in an upper secondary school. Education Research International, 2017 , 1614262 https://doi.org/10.1155/2017/1614262 .

Johnson, B., & Onwuegbuzie, A. (2004). Mixed methods research: a research paradigm whose time has come. Educational Researcher, 33 , 14–26 https://doi.org/10.3102%2F0013189X033007014 .

John-Steiner, V., & Mann, H. (1996). Sociocultural approaches to learning and development: a Vygotskian framework. Educational Psychologist, 31 , 191–206 https://doi.org/10.1080/00461520.1996.9653266 .

Katsenou, C., Flogaitis, E., & Liarakou, G. (2015). Action research to encourage pupils’ active participation in the sustainable school. Applied Environmental Education & Communication, 14 (1), 14–22. https://doi.org/10.1080/1533015X.2014.994820 .

Kehoe, I. (2015). The cost of performance? Students’ learning about acting as change agents in their schools Discourse. Studies in the Cultural Politics of Education, 36 (1), 106–119 https://doi.org/10.1080/01596306.2013.841356 .

Kiili, C. (2012). Online reading as an individual and social practice , Jyväskylä studies in education, psychology and social research 441 . Jyväskylä: Jyväskylä University Printing House https://jyx.jyu.fi/dspace/bitstream/handle/123456789/38394/978-951-39-4795-8.pdf?sequence=1 .

Kivinen, O., Piiroinen, T., & Saikkonen, L. (2016). Two viewpoints on the challenges of ICT in education: knowledge-building theory vs. a pragmatist conception of learning in social action. Oxford Review of Education, 42 (4), 377–390 https://doi.org/10.1080/03054985.2016.1194263 .

Korhonen, T., Lavonen, J., Kukkonen, M., Sormunen, K., & Juuti, K. (2014). The innovative school as an environment for the design of educational innovations. In H. Niemi, J. Multisilta, L. Lipponen, & M. Vivitsou (Eds.), Finnish innovations and technologies in schools (pp. 99–113). Rotterdam: Sense Publishers.

Kunnari, I., & Ilomäki, L. (2016). Reframing teachers’ work for educational innovation. Innovations in Education and Teaching International, 53(2), 167–178. https://doi.org/10.1080/14703297.2014.978351 .

Lakkala, M., Paavola, S., Kosonen, K., Muukkonen, H., Bauters, M., & Markkanen, H. (2009). Main functionalities of the Knowledge Practices Environment (KPE) affording knowledge creation practices in education. In C. O'Malley, D. Suthers, P. Reimann, & A. Dimitracopoulou (Eds.), Computer Supported Collaborative Learning Practices: CSCL2009 Conference Proceedings. (pp. 297–306). Rhodes, Creek: International Society of the Learning Sciences (ISLS).

Lakkala,M. & Ilomäki, L. (2013). Lukioiden valmiudet siirtyä sähköiseen ylioppilastutkintoon: kahden lukion tapaustutkimus [The readiness of high schools to the electronic matriculation examination: A case study of two high schools]. Vantaa: Vantaan sivistystoimi. https://www.vantaa.fi/instancedata/prime_product_julkaisu/vantaa/embeds/vantaawwwstructure/122594_Lukioiden_valmiudet_siirtya_sahkoiseen_ylioppilastutkintoon_kahden_lukion_tapaustutkimus.pdf .

Lakkala, M. & Ilomäki, L. (2015). A case study of developing ICT-supported pedagogy through a collegial practice transfer process. Computers & Education, 90, 1–12. https://doi.org/10.1016/j.compedu.2015.09.001 .

Lam, S., Cheng, R., & Choy, H. (2010). School support and teacher motivation to implement project-based learning. Learning and Instruction, 20 , 487–497 https://doi.org/10.1016/j.learninstruc.2009.07.003 .

Laurillard, D. (2008). Technology enhanced learning as a tool for pedagogical innovation. Journal of Philosophy in Education, 42 (3–4), 521–533 https://doi.org/10.1111/j.1467-9752.2008.00658.x .

Leclerc, M., Moreau, A. C., Dumouchel, C., & Sallafranque-st-Louis, F. (2012). Factors that promote progression in schools functioning as professional learning community. International Journal of Education Policy & Leadership, 7 (7), 1–14 https://doi.org/10.22230/ijepl.2012v7n7a417 .

Lemke, J. L. (2001). The long and the short of it: comments on multiple timescale studies of human activity. The Journal of the Learning Sciences, 10 , 17–26 https://doi.org/10.1207/S15327809JLS10-1-2_3 .

Lieberman, A., & Pointer Mace, D. (2008). Teacher learning: the key to educational reform. Journal of Teacher Education, 59 (3), 226–234 https://doi.org/10.1177%2F0022487108317020 .

Livingstone, S. (2012). Critical reflections on the benefits of ICT in education. Oxford Review of Education, 38 (1), 9–24 https://doi.org/10.1080/03054985.2011.577938 .

Lundahl, L., Erixon Arreman, I., Lundström, U., & Rönnberg, L. (2010). Setting things right? Swedish upper secondary school reform in a 40-year perspective. European Journal of Education, 45 (1), 46–59 https://doi.org/10.1111/j.1465-3435.2009.01414.x .

Merriam, S. (1998). Qualitative Research and Case Study Applications in Education. San Francisco, CA: Jossey-Bass.

Messmann, G., & Mulder, R. (2011). Innovative work behaviour in vocational colleges: understanding how and why innovations are developed. Vocations and Learning, 4 , 63–84 https://doi.org/10.1007/s12186-010-9049-y .

Newmann, F. M., King, M. B., & Youngs, P. (2000). Professional development that addresses school capacity: lessons from urban primary schools. American Journal of Education, 108 , 259–299 https://doi.org/10.1086/444249 .

OECD. (2010). Inspired by technology, driven by pedagogy. A systemic approach to technology-based school innovations. In Educational research and innovation . Paris: OECD Publishing https://doi.org/10.1787/9789264094437-en .

OECD. (2011). PISA 2009 results: students on line: digital technologies and performance (volume VI) . Paris: OECD Publishing https://doi.org/10.1787/9789264112995-en .

OECD. (2014). PISA 2012 Results: creative problem solving: students’ skills in tackling real-life problems (volume V) . Paris: OECD Publishing https://doi.org/10.1787/9789264208070-en .

OECD. (2015). Building responsive schools for 21st-century learners. In Schools for 21st-Century learners: strong leaders, confident teachers, innovative approaches . Paris: OECD Publishing https://doi.org/10.1787/9789264231191-en .

OECD. (2017). PISA 2015 Results. In Summary in English: collaborative problem solving . Paris: OECD Publishing https://doi.org/10.1787/93ba004b-en .

Overbay, A., Patterson, A., Vasu, E., & Grable, L. (2010). Constructivism and technology: findings from the IMPACTing leadership project. Educational Media International, 47 (2), 103–120 Retrieved October 16, 2018 from https://www.learntechlib.org/p/107127/ .

Paavola, S., & Hakkarainen, K. (2005). The knowledge creation metaphor—an emergent epistemological approach to learning. Science & Education, 14 , 535–557 https://doi.org/10.1007/s11191-004-5157-0 .

Packer, M. J., & Goicoechea, J. (2000). Sociocultural and constructivist theories of learning: ontology, not just epistemology. Educational Psychologist, 35 (4), 227–241 https://doi.org/10.1207/S15326985EP3504_02 .

Peck, C., Gallucci, C., Sloan, T., & Lippincott, A. (2009). Organizational learning and program renewal in teacher education: a socio-cultural theory of learning, innovation and change. Educational Research Review, 4 , 1–25 https://doi.org/10.1016/j.edurev.2008.06.001 .

Pedder, D., & MacBeath, J. (2008). Organisational learning approaches to school leadership and management: teachers’ values and perceptions of practice. School Effectiveness and School Improvement, 19 (2), 207–224 https://doi.org/10.1080/09243450802047899 .

Pereira, F., Mouraz, A., & Figueiredo, C. (2014). Student participation in school life: the “student voice” and mitigated democracy. Croatian Journal of Education, 16 (4), 935–975 https://doi.org/10.15516/cje.v16i4.742 .

Ranson, S., Farrell, C., Peim, N., & Smith, P. (2005). Does governance matter for school improvement? School Effectiveness and School Improvement, 16 (3), 305–325 https://doi.org/10.1080/09243450500114108 .

Resnick, L., & Spillane, J. (2006). From individual learning to organizational designs for learning. In L. Verschaffel, F. Dochy, M. Boekaerts, & S. Vosniadou (Eds.), Instructional psychology: past, present and future trends. Sixteen essays in honor of Erik De Corte , Advances in learning and instruction series . Oxford: Pergamon.

Robin, B. R. (2008). Digital storytelling: a powerful technology tool for the 21st century classroom. Theory Into Practice, 47 (3), 220–228 https://doi.org/10.1080/00405840802153916 .

Rogers, E. (2003). Diffusion of innovations. New York: Free Press.

Sahlberg, P. (2011). Finnish lessons. Can the world learn from the educational change in Finland? New York, NY: Teachers College Press.

Scardamalia, M., & Bereiter, C. (1999). Schools as knowledge building organizations. In D. Keating & C. Hertzman (Eds.), Today’schildren, tomorrow’s society: the developmental health and wealth of nations (pp. 274–289). New York: Guilford.

Scardamalia, M., & Bereiter, C. (2006). Knowledge building: theory, pedagogy, and technology. In K. Sawyer (Ed.), Cambridge handbook of the learning sciences (pp. 97–118). New York, NY: Cambridge University Press.

Scimeca, S., Dumitru, P., Durando, M., Gilleran, A., Joyce, A., & Vuorikari, R. (2009). European schoolnet: enabling school networking. European Journal of Education, 44 (4), 475–492 Retrieved from https://search.proquest.com/docview/61841165?accountid=11365 .

Senge, P., Kleiner, A., Roberts, C., Ross, R. B., & Smith, B. (1994). The fifth discipline fieldbook: strategies and tools for building a learning organization . New York, NY: Doubleday.

Spillane, J. P., Halverson, R., & Diamond, J. (2004). Towards a theory of leadership practice: a distributed perspective. Journal of Curriculum Studies, 36 (1), 3–34 https://doi.org/10.1080/0022027032000106726 .

Svanbjörnsdóttir, B. M., Macdonald, A., & Frímannsson, G. H. (2016). Views of learning and a sense of community among students, paraprofessionals and parents in developing a school culture towards a professional learning community. Professional Development in Education, 42 (4), 589–609. https://doi.org/10.1080/19415257.2015.1047037 .

Tan, J. P., & McWilliam, E. (2009). From literacy to multiliteracies: diverse learners and pedagogical practice. Pedagogies, 4 (3), 213–225 https://doi.org/10.1080/15544800903076119 .

Teddle, C. & Tashakkori, A. (2003). Major issues and controversies in the use of mixed methods in the social and behavioral studies. In A. Tashakkori and C. Teddle (Eds.), Handbook of mixed methods in social and behavioral research (pp. 3–50). Thousand Oaks: Sage Publications.

Tikkanen, T. & Selander, P. (Eds.) (2014). Helsinki by District 2013. City of Helsinki urban facts: Helsinki. Retrieved 16.10.2018 from 14_04_22 helsinki alueittain 2013 tikkanen.pdf.

Timmermans, S., & Tavory, I. (2012). Theory construction in qualitative research: from grounded theory to abductive analysis. Sociological Theory, 30 (3), 167–186 https://doi.org/10.1177%2F0735275112457914 .

Todd, Z., Nerlich, B., & McKeown, S. (2004). Introduction. In Z. Todd, B. Nerlich, S. McKeown, & D. Clarke (Eds.), Mixing methods in psychology. The integration of qualitative and quantitative methods in theory and practice (pp. 3–16). USA, NT: Psychology Press.

Twining, P., Raffaghelli, J., Albion, P., & Knezek, D. (2013). Moving education into the digital age: the contribution of teachers’ professional development. Journal of Computer Assisted Learning, 29 , 426–437 https://doi.org/10.1111/jcal.12031 .

Vieluf, S., Kaplan, D., Klieme, E., & Bayer, S. (2012). Teaching practices and pedagogical innovation: evidence from TALIS . Paris: OECD Publishing https://doi.org/10.1787/9789264123540-en .

Wikeley, F., Stoll, L., & Murillo, J. (2005). Evaluating effective school improvement: case studies of programmes in eight European countries and their contribution to the effective school improvement model. School Effectiveness and School Improvement, 16 (4), 387–405 https://doi.org/10.1080/09243450500234617 .

Wong, E. M. L., & Li, S. C. (2011). Framing ICT implementation in a context of educational change: a structural equation modelling analysis. Australasian Journal of Educational Technology, 27 (2), 361–379 https://ajet.org.au/index.php/AJET/article/download/975/249 .

Wrigley, T. (2003). Is ‘School Effectiveness’ Anti‐Democratic? British Journal of Educational Studies 51(2), 89–112.

Yin, R. K. (2014). Case study research. Design and methods. Fifth Edition . USA: Sage.

Download references

Acknowledgements

We are grateful to Jari Lipsanen for the help and guidance of the statistical analyses, to Ian Dobson who has, with patience and excellent comments, checked the language and to anonymous reviewers, whose comments helped to improve and clarify the article.

This work was supported by Tekes—the Finnish Funding Agency for Innovation [grant number 40233/09] and by City of Helsinki, Media Center.

Availability of data and materials

The data is available in the University of Helsinki from the authors.

Author information

Authors and affiliations.

Faculty of Educational Sciences, University of Helsinki, P.O. Box 9, 00014, Helsinki, Finland

Liisa Ilomäki & Minna Lakkala

You can also search for this author in PubMed   Google Scholar

Contributions

Both authors contributed in the study equally, on the design, data collection and analysis as well as writing the article.

Corresponding author

Correspondence to Liisa Ilomäki .

Ethics declarations

Competing interests.

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Additional information

The original version of this article was revised: In the original publication of this article (Ilomäki & Lakkala, 2018) the appendix Table 9 in PDF is in wrong place due to typeset mistake, which should be at the end of the manuscript. The original PDF version has been corrected.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Reprints and permissions

About this article

Cite this article.

Ilomäki, L., Lakkala, M. Digital technology and practices for school improvement: innovative digital school model. RPTEL 13 , 25 (2018). https://doi.org/10.1186/s41039-018-0094-8

Download citation

Received : 04 May 2018

Accepted : 03 December 2018

Published : 28 December 2018

DOI : https://doi.org/10.1186/s41039-018-0094-8

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Lower secondary
• School improvement
• Digital technology

• Skip to main content
• Accessibility help

Information

We use cookies to collect anonymous data to help us improve your site browsing experience.

Click 'Accept all cookies' to agree to all cookies that collect anonymous data. To only allow the cookies that make the site work, click 'Use essential cookies only.' Visit 'Set cookie preferences' to control specific cookies.

Your cookie preferences have been saved. You can change your cookie settings at any time.

Literature Review on the Impact of Digital Technology on Learning and Teaching

This literature review was commissioned by the Scottish Government to explore how the use of digital technology for learning and teaching can support teachers, parents, children and young people in improving outcomes and achieving our ambitions for education in Scotland

Digital learning and raising attainment

Key findings

There is conclusive evidence that digital equipment, tools and resources can, where effectively used, raise the speed and depth of learning in science and mathematics for primary and secondary age learners. There is indicative evidence that the same can be said for some aspects of literacy, especially writing and comprehension. Digital technologies appear to be appropriate means to improve basic literacy and numeracy skills, especially in primary settings.

The effect sizes are generally similar to other educational interventions that are effective in raising attainment, though the use of digital learning has other benefits. Also, the extent of the effect may be dampened by the level of capability of teachers to use digital learning tools and resources effectively to achieve learning outcomes. More effective use of digital teaching to raise attainment includes the ability of teachers to identify how digital tools and resources can be used to achieve learning outcomes and adapting their approach, as well as having knowledge and understanding of the technology. This applies in all schools.

Where learners use digital learning at home as well as school for formal and non-formal learning activities these have positive effects on their attainment, because they have extended their learning time. This is particularly important for secondary age learners.

The assessment framework, set out in Annex 2 , identifies a number of educational benefits that digital learning and teaching has the potential to help learners aged 5 to 18 to realise, through the opportunity to learn in different ways, access more sources of information, and be tested and get feedback differently. In terms of raising attainment, these benefits include short term outcomes, such as having a greater feeling of control over learning and more confidence to practise a skill, through to medium term outcomes such as faster acquisition of knowledge and skills, and improved impacts in terms of learners achieving higher exam or test results where digital technology has been used.

In this section, the impact of digital technology on children's attainment in a range of areas is discussed, followed by the impact on aspects of numeracy, literacy and science learning.

Raising children's attainment

There is a substantial body of research that has examined the impact of digital tools and resources on children's attainment in a range of areas.

Higgins et al (2012) provide a summary of research findings from studies with experimental and quasi-experimental designs, which have been combined in meta-analyses to assess the impact of digital learning in schools. Their search identified 48 studies which synthesised empirical research of the impact of digital tools and resources on the attainment of school age learners (5-18 year olds).

They found consistent but small positive associations between digital learning and educational outcomes. For example, Harrison et al (2004) identified statistically significant findings, positively associating higher levels of ICT use with school achievement at each Key Stage in England, and in English, maths, science, modern foreign languages and design technology. Somekh et al (2007) identified a link between high levels of ICT use and improved school performance. They found that the rate of improvement in tests in English at the end of primary education was faster in ICT Test Bed education authorities in England than in equivalent comparator areas. However, Higgins et al note that while these associations show, on average, schools with higher than average levels of ICT provision also have learners who perform slightly higher than average, it may be the case that high performing schools are more likely to be better equipped or more prepared to invest in technology or more motivated to bring about improvement.

Higgins et al report that in general analyses of the impact of digital technology on learning, the typical overall effect size is between 0.3 and 0.4 - just slightly below the overall average for researched interventions in education (Sipe & Curlette, 1997; Hattie, 2008) and no greater than other researched changes to teaching to raise attainment, such as peer tutoring or more focused feedback to learners. The range of effect sizes is also very wide (-0.03 to 1.05),which suggests that it is essential to take into account the differences between technologies and how they are used.

Table 4: Summary of meta-analyses published between 2000 and 2012 (in Higgins et al 2012)

In an earlier meta-analysis, Liao et al (2007), considered the effects of digital tools and resources on elementary school learners' achievement in Taiwan. Synthesizing research comparing the effects of digital learning (equipment, tools and resources) with traditional instruction on elementary school learners' achievement, they considered quantitative and qualitative information from 48 studies including over 5,000 learners. Of the 48 studies, 44 (92%) showed positive effects in favour of a computer assisted intervention, while four (8%) were negative and favoured a traditional instruction method. Nearly 60% of the studies examined the effects of computer aided instruction for teaching mathematics or science. Another 11% of the studies concentrated on the teaching of reading and language. They found an overall positive effect size across all the studies of 0.45 (study-weighted grand mean), which is considered to be a moderate effect, with a wide range of effect sizes (from 0.25 to 2.67).

No significant differences were found between subject areas, and the authors suggest that digital learning has the potential to be implemented in many different subject areas. They found that the two subjects that showed the highest effects were reading and languages, which had a high positive effect size of 0.7. Studies using computer simulations also had higher effects. The authors suggest this may be because simulations can provide learners with the opportunity to engage in a learning activity which could not be replicated in a classroom.

More qualitative studies have identified how improvements in attainment are achieved. From a wide study of primary and secondary schools in England that were early adopters in using digital learning and teaching, Jewitt et al (2011) concluded that:

• Using digital resources provided learners with more time for active learning in the classroom;
• Digital tools and resources provided more opportunity for active learning outside the classroom, as well as providing self-directed spaces, such as blogs and forums, and access to games with a learning benefit;
• Digital resources provided learners with opportunities to choose the learning resources;
• The resources provided safer spaces for formative assessment and feedback.

The sections below focus on specific key areas of attainment: literacy, numeracy, and science learning.

There is a large body of research that has examined the impact of digital equipment, tools and resources on children's literacy. The effects are generally positive, though not as large as the effects found where digital learning is used to improve numeracy, and consistent in finding that ICT helps improve reading and writing skills, as well as developing speaking and listening skills.

Effect of context

Archer and Savage (2014) undertook a meta-analysis to reassess the outcomes presented in three previous meta-analyses considering the impact of digital learning on language and literacy learning: Slavin et al (2008 and 2009) and Torgenson and Zhu (2003). Overall they found a relatively small average positive effect size of 0.18, with a few of the studies having a negative effect and three studies showing moderate to large effect sizes. The authors found that programmes with a small number of participants tended to show larger effect sizes than larger programmes but that not all were statistically significant.

Archer and Savage sought to understand whether the context within which the digital tool or resource was used has an impact on outcomes. In particular, they examined whether training and support given to the teachers or other staff delivering the programme had an impact. The authors found that training and support could be identified in around half of the studies and that it did appear to have a positive impact on the effectiveness of the literacy intervention, with the average effect size rising to 0.57. The authors conclude that this indicates the importance of including implementation factors, such as training and support, when considering the relative effectiveness of digital learning and teaching.

Effect on specific literacy skills

In their meta-analysis, Higgins et al (2012) found that digital learning has a greater impact on writing than on reading or spelling. For example, Torgenson and Zhu (2003) reviewed the impact of using digital technology on the literacy competences of 5-16 year-olds in English and found effect sizes on spelling (0.2) and reading (0.28) much lower than the high effect size for writing (0.89).

In their meta-analysis of studies investigating the effects of digital technology on primary schools in Taiwan, Laio et al (2007) considered studies over a range of curriculum areas; 11 of which addressed the effects of using digital learning in one or more literacy competence. They found no significant differences in effect size between the different subject areas, suggesting the potential for digital technology to raise outcomes is equal across different subjects. However, they did note that the two areas that showed the highest effect sizes (over 0.7) were reading and comprehension.

Effect of specific digital tools and resources

Somekh et al (2007) evaluated the Primary School Whiteboard Expansion ( PSWB ) project in England. They found that the length of time learners were taught with interactive whiteboards ( IWB s) was a major factor in learner attainment at the end of primary schooling, and that there were positive impacts on literacy (and numeracy) once teachers had experienced sustained use and the technology had become embedded in pedagogical practice. This equated to improvements at Key Stage 2 writing (age 11), where boys with low prior attainment made 2.5 months of additional progress.

Hess (2014) investigated the impact of using e-readers and e-books in the classroom, among 9-10 year olds in the USA . The e-books were used in daily teacher-led guided reading groups, replacing traditional print books in these sessions. Teachers also regularly used the e-readers in sessions where the class read aloud, and e-readers were available to learners during the school day for silent reading. The study found a significant difference in reading assessment scores for the group using the e-readers. Scores improved for both male and female learners and the gap between males and females decreased.

The use of digital tools and resources also appears to affect levels of literacy. Lysenko and Abrami (2014) investigated the use of two digital tools on reading comprehension for elementary school children (aged 6-8) in Quebec, Canada. The first was a multimedia tool which linked learning activities to interactive digital stories. The tool included games to engage learners in reading and writing activities, and instructions were provided orally to promote listening comprehension. The second tool was a web-based electronic portfolio in which learners could create a personalised portfolio of their reading and share work with peers, teachers and parents to get feedback. The authors found that in classes where both tools were used together during the whole school year learners performed significantly better both in vocabulary and reading comprehension (with medium-level effect sizes) than learners in classes where the tools were not part of English language instruction.

Rosen and Beck-Hill (2012) reported on a study programme that incorporated an interactive core curriculum and a digital teaching platform. At the time of their report it was available for 9-11 year old learners in English language, arts and mathematics classes in Dallas, Texas. The online platform contained teaching and learning tools. Learners were assessed using standardised tests administered before the programme and after a year's participation. The results of increased achievement scores demonstrated that in each of the two school year groups covered, the experimental learners significantly outperformed the control learners in reading and maths scores. In observations in classrooms that used the programme, the researchers observed higher teacher-learner interaction, a greater number and type of teaching methods per class, more frequent and complex examples of differentiation processes and skills, more frequent opportunities for learner collaboration, and significantly higher learner engagement. The authors report that the teaching pedagogy observed in the classrooms differed significantly from that observed in more traditional classrooms. The teachers following the programme commented that the digital resources made planning and implementing 'differentiation' more feasible. This is differentiation of teaching in terms of content, process, and product, to reflect learners' readiness, interests, and learning profile, through varied instructional and management strategies.

Effect of the amount and quality of digital technology use

The uses of digital technology and access to it appear to be critical factors. Lee et al (2009) analysed how in the US 15-16 year-old learners' school behaviour and standardised test scores in literacy are related to computer use. Learners were asked how many hours a day they typically used a computer for school work and for other activities. The results indicated that the learners who used the computer for one hour a day for both school work and other activities had significantly better reading test scores and more positive teacher evaluations for their classroom behaviours than any other groups [5] . This was found while controlling for socio-economic status, which has been shown to be a predictor of test scores in other research. The analysis used data from a national 2002 longitudinal study, and it is likely that learners' usage of computers has increased and changed since that time.

Biagi and Loi (2013), using data from the 2009 Programme for International Student Assessment ( PISA ) and information on how learners used digital technology at school and at home (both for school work and for entertainment), assessed the relationship between the intensity with which learners used digital tools and resources and literacy scores. They examined uses for: gaming activities (playing individual or collective online games), collaboration and communication activities (such as linking with others in on-line chat or discussion forums), information management and technical operations (such as searching for and downloading information) and creating content, knowledge and problem solving activities (such as using computers to do homework or running simulations at school). These were then compared to country specific test scores in reading. The authors found a positive and significant relationship between gaming activity and language attainment in 11 of the 23 countries studied. For the other measures, where relationships existed and were significant, they tended to be negative.

The more recent PISA data study ( OECD , 2015, using 2012 results) also found a positive relationship between the use of computers and better results in literacy where it is evident that digital technology is being used by learners to increase study time and practice [6] . In addition, it found that the effective use of digital tools is related to proficiency in reading.

There is a large body of research which has examined the impact of digital equipment, tools and resources on children's numeracy skills and mathematical competences throughout schooling. Higgins et al (2012) found from their meta-analysis that effect sizes of tested gains in knowledge and understanding tend to be greater in mathematics and science than in literacy. The key benefits found relate to problem solving skills, practising number skills and exploring patterns and relationships (Condie and Monroe, 2007), in addition to increased learner motivation and interest in mathematics.

Effect on specific numeracy skills

Li and Ma's (2010) meta-analysis of the impact of digital learning on school learners' mathematics learning found a generally positive effect. The authors considered 46 primary studies involving a total of over 36,000 learners in primary and secondary schools. About half of the mathematics achievement outcomes were measured by locally-developed or teacher-made instruments, and the other half by standardized tests. Almost all studies were well controlled, employing random assignment of learners to experimental or control conditions.

Overall, the authors found that, on average, there was a high, significantly positive effect of digital technology on mathematics achievement (mean effect size of 0.71), indicating that, in general, learners learning mathematics with the use of digital technology had higher mathematics achievement than those learning without digital technology. The authors found that:

• Although the difference was small, younger school learners (under 13 years old) had higher attainment gains than older secondary school learners;
• Gains were more positive where teaching was more learner-centred than teacher-centred. In this regard, the authors differentiate between traditional models, where the teacher tends to teach to the whole class, and a learner-centred teaching model which is discovery-based (inquiry-oriented) or problem-based (application-oriented) learning;
• Shorter interventions (six months or less) were found to be more effective in promoting mathematics achievement than longer interventions (between six and 12 months). It is suggested that such gains in mathematics achievement are a result of the novelty effects of technology, as suggested in other research, and as learners get familiar with the technology the novelty effects tend to decrease;
• The authors found no significant effects from different types of computer technology on mathematics achievement. Whether it was used as communication media, a tutorial device, or exploratory environment, learners displayed similar results in their mathematics achievement;
• Equally, the authors found no significant relationship between the effect of using digital technology and the characteristics of learners included in the samples for studies, such as gender, ethnicity, or socio-economic characteristics.

The studies by Lee et al (2009) and Biagi and Loi (2013) found similar results for mathematics as they did for reading and literacy in relation to the use of digital equipment. Learners who used a computer at least one hour a day for both school work and other activities had significantly better mathematics test scores and more positive teacher evaluations for their classroom behaviour in mathematics classes than those who did not use the computer. Biagi and Loi (2013) found a significant positive relationship between intensity of gaming activity and maths test scores in 15 countries out of the 23 studied. As with language, the authors found that learners' total use of digital technologies was positively and significantly associated with PISA test scores for maths in 18 of the 23 countries studied.

Studies have found that using digital equipment for formal learning is also associated with increases in learners' motivation for learning mathematics. House and Telese (2011 and 2012) found that:

• For learners aged 13 and 14 in South Korea, for example, those who expressed high levels of enjoyment at learning mathematics, more frequently used computers in their mathematics homework. However, learners who more frequently played computer games and used the internet outside of school tended to report that they did not enjoy learning mathematics;
• Learners in the USA and Japan aged 13 and 14 who showed higher levels of algebra achievement also used computers more at home and at school for school work. Those who used computers most for other activities had lower test scores. In each of the USA and Japan they found that overall computer usage which included use for school work was significantly related to improvements in test scores.

Somekh et al (2007) found that, once the use of IWB s was embedded, in Key Stage 1 mathematics (age 7) in England, high attaining girls made gains of 4.75 months, enabling them to catch up with high attaining boys. In Key Stage 2 mathematics (age 11), average and high attaining boys and girls who had been taught extensively with the IWB made the equivalent of an extra 2.5 to 5 months' progress over the course of two years.

Digital tools and resources can also increase some learners' confidence in mathematics as well as their engagement in new approaches to learning and their mathematical competences. Overcoming learners' anxieties about mathematics and their competence in specific aspects of the subject are common concerns in teaching mathematics which hampers their ability to learn (reported in Huang et al 2014).

Huang et al (2014) researched the outcomes, in Taiwan, from a computer game simulating the purchase of commodities, from which 7 and 8 year-old primary school learners can learn addition and subtraction, and apply mathematical concepts. The model combined games-based learning with a diagnosis system. When the learner made a mistake, the system could detect the type of mistake and present corresponding instructions to help the learner improve their mathematical comprehension and application. The authors compared two learning groups: both used the game-based model but one without the diagnostic, feedback element. They found that the learning achievement post-test showed a significant difference and also that the mathematics anxiety level of the two learner groups was decreased by about 3.5%.

Passey (2011) found that among over 300 schools in England using Espresso digital resources, those that had been using them over a longer period made significantly greater increases in end of primary school numeracy test results than schools which were recent users.

Science learning

Effects on science knowledge and skills

In their meta-analysis, Laio et al (2007) considered 11 studies looking at the impact of digital technology on science learning. These had a moderate average effect size of 0.38 and generally had positive effects. Condie and Monroe (2007) identified that digital learning made science more interesting, authentic and relevant for learners and provided more time for post-experiment analysis and discussion.

In their study of the PISA data, Biagi and Loi (2013) found a significant positive relationship between learners' total use of digital equipment and science test scores in 21 of the 23 countries they studied. They also found evidence of a significant positive relationship between the intensity of using gaming activity and science scores in 13 of the 23 countries they studied. Somekh et al (2007) found that in primary school science all learners, except high attaining girls, made greater progress when given more exposure to IWB s, with low attaining boys making as much as 7.5 months' additional progress.

Effects of specific digital tools and resources

Digital tools and resources generally have a positive effect on learners' science learning. This can be seen from a number of studies assessing outcomes for learners in different stages of education.

Hung et al (2012) explored the effect of using multi-media tools in science learning in an elementary school's science course in Taiwan. Learners were asked to complete a digital storytelling project by taking pictures with digital cameras, developing the story based on the pictures taken, producing a film based on the pictures by adding subtitles and a background, and presenting the story. From the experimental results, the authors found that this approach improved the learners' motivation to learn science, their attitude, problem-solving capability and learning achievements. In addition, interviews found that the learners in the experimental group enjoyed the project-based learning activity and thought it helpful because of the digital storytelling aspect.

Hsu et al (2012) investigated the effects of incorporating self-explanation principles into a digital tool facilitating learners' conceptual learning about light and shadow with 8-9 year old learners in Taiwan. While they found no difference in the overall test scores of the experimental and control groups, they found a statistically significant difference in retention test scores. Those learners who had paid more attention to the self-explanation prompts tended to outperform those in the control group.

Anderson and Barnett's (2013) study, in the US , examined how a digital game used by learners aged 12-13 increased their understanding of electromagnetic concepts, compared to learners who conducted a more traditional inquiry-based investigation of the same concepts. There was a significant difference between the control and experimental groups in gains in knowledge and understanding of physics concepts. Additionally, learners in the experimental group were able to give more nuanced responses about the descriptions of electric fields and the influence of distance on the forces that change experience because of what they learnt during the game.

Güven and Sülün (2012) considered the effects of computer-enhanced teaching in science and technology courses on the structure and properties of matter, such as the periodical table, chemical bonding, and chemical reactions, for 13-14 year olds in Turkey. Their proposition was that computer-enhanced teaching can instil a greater sense of interest in scientific and technological developments, make abstract concepts concrete through simulation and modelling, and help to carry out some dangerous experiments in the classroom setting. They found a significant difference in achievement tests between the mean scores of the group of learners who were taught with the computer-enhanced teaching method and the control group who were taught with traditional teaching methods.

Belland (2009) investigated the extent to which a digital tool improved US middle school children's ability to form scientific arguments. Taking the premise that being able to construct and test an evidence-based argument is critical to learning science, he studied the impact of using a digital problem based learning tool on 12-14 year olds. Learners worked in small groups and were asked to develop and present proposals for spending a grant to investigate an issue relating to the human genome project. Those in the experimental group used an online system which structured the project into stages of scientific enquiry. The system prompted the learners to structure and organise their thinking in particular ways: by prompting the learners individually, sharing group members' ideas, tasking the group to form a consensus view, and prompting the group to assign specific tasks among themselves.

Using pre- and post- test scores to assess the impact on learners' abilities to evaluate arguments, Belland found a high positive effect size of 0.62 for average-achieving learners compared to their peers in the control group. No significant impacts were found for higher or lower-achieving learners. Belland suggests that for high-achieving learners, this may be because they already have good argument making skills and are already able to successfully structure how they approach an issue and gather evidence. The study also used qualitative information to consider how the learners used the digital tool and compared this to how learners in the control group worked. The author found that in the experimental group they made more progress and were more able to divide tasks up between them, which saved time. They also used the tool more and the teacher less to provide support.

Kucukozer et al (2009) examined the impact of digital tools on teaching basic concepts of astronomy to 11-13 year old school children in Turkey. Learners were asked to make predictions about an astronomical phenomenon such as what causes the seasons or the phases of the moon. A digital tool was used to model the predictions and display their results. The learners were then asked to explain the differences and the similarities between their predictions and their observations. In the prediction and explanation phase the learners worked in groups to discuss their ideas and come to a conclusion. In the observation phase they watched the 3D models presented by their teacher. Thereafter, they were asked to discuss and make conclusions about what they had watched. The authors found that instruction supported by observations and the computer modelling was significantly effective in bringing about better conceptual understanding and learning on the subject.

Ingredients of success

Where studies examine the process that brings about positive results from digital learning and teaching compared to traditional approaches, it is evident that these are more likely to be achieved where digital equipment, tools and resources are used for specific learning outcomes and built into a teaching model from the outset. This broadly supports Higgins et al's (2012) conclusions that:

• Digital technology is best used as a supplement to normal teaching rather than as a replacement for it;
• It is not whether technology is used (or not) which makes the difference, but how well the technology is applied to support teaching and learning by teachers;
• More effective schools and teachers are more likely to use digital technologies effectively than other schools.

Differences in effect sizes and the extent that learners achieve positive gains in attainment are ascribed by most authors of the studies above to:

• The quality of teaching and the ability of teachers to use the digital equipment and tools effectively for lessons;
• The preparation and training teachers are given to use equipment and tools;
• The opportunities teachers have to see how digital resources can be used and pedagogies adapted (Rosen and Beck-Hill, 2012; Belland, 2009).

Teachers have to adapt to learner-centred approaches to learning if they are to use digital tools and resources (Li and Ma, 2010).

As well as ensuring digital tools and resources are supporting learning goals, success appears to also be linked to some other factors:

• The availability of equipment and tools within schools (and at home);
• How learners use digital equipment. Higgins et al (2012) found that collaborative use of technology (in pairs or small groups) is usually more effective than individual use, though some learners - especially younger children - may need guidance in how to collaborate effectively and responsibly;
• The extent that teaching continues to innovate using digital tools and resources (Higgins et al, 2012).

Fullan (2013) suggested four criteria that schools should meet if their use of digital technology to support increased attainment is to be successful. These were that systems should be engaging for learners and teachers; easy to adapt and use; ubiquitous - with access to the technology 24/7; and steeped in real life problem solving.

Fullan and Donnelly (2013) developed these themes further, proposing an evaluation tool to enable educators to systematically evaluate new companies, products and school models, using the context of what they have seen as necessary for success. Questions focus on the three key criteria of pedagogy (clarity and quality of intended outcome, quality of pedagogy and the relationship between teacher and learner, and quality of assessment platform and functioning); system change (implementation support, value for money, and whole system change potential) and technology (quality of user experience/model design, ease of adaptation, and comprehensiveness and integration).

Email: Catriona Rooke

There is a problem

Thanks for your feedback

Your feedback helps us to improve this website. Do not give any personal information because we cannot reply to you directly.

• Open access
• Published: 27 December 2022

How does technology challenge teacher education?

• Lina Kaminskienė 1 ,
• Sanna Järvelä 2 &
• Erno Lehtinen 1 , 3  

International Journal of Educational Technology in Higher Education volume  19 , Article number:  64 ( 2022 ) Cite this article

11k Accesses

10 Citations

9 Altmetric

Metrics details

The paper presents an overview of challenges and demands related to teachers’ digital skills and technology integration into educational content and processes. The paper raises a debate how technologies have created new skills gaps in pre-service and in-service teacher training and how that affected traditional forms of teacher education. Accordingly, it is discussed what interventions might be applicable to different contexts to address these challenges. It is argued that technologies should be viewed both as the field where new competences should be developed and at the same time as the method used in developing learning environments for teacher students.

Introduction

In the last few decades, national authorities and multinational organisations have emphasised the importance of increasing the use of information and communication technologies (ICT) in schools and universities (Flecknoe, 2002 ; Roztocki et al., 2019 ; UNESCO ICT Competency Framework for Teachers, 2018 ). This poses a double challenge for teacher education: determining how new technologies can be used to improve the quality of learning experiences that student teachers receive during their university studies and identifying what kinds of new skills future teachers will need for teaching in technologically rich school environments. Several of the arguments in favour of greater use of ICT in schools are based on the belief that due to the general digitisation of the workforce, it is vital that students acquire good digital skills at an early age. However, it has also been argued that the use of ICT will be engaging for students and can thus result in better learning outcomes (Cheung & Slavin, 2013 ; Gloria, 2015 ). A number of large meta-analyses have shown that intervention studies utilising technology have positive effects on students’ motivation and learning (Fadda et al., 2022 ; Wouters et al., 2013 ).

However, when large-scale national and international evaluation studies have examined the relationship between the use of ICT and student achievement, the results have been mixed. Researchers have reported that re-analyses of large international evaluation studies, including the Trends in International Mathematics and Science Study (TIMMS) and Programme for International Student Assessment (PISA), indicate that there is either no relationship or a negative relationship between the frequency of ICT use in teaching and students’ achievements (Eickelmann et al., 2016 ; Papanastasiou et al., 2003 ).

The mixed results regarding the impact of ICT suggest that there are qualitative differences between the ways in which technology is implemented. In intervention studies, ICT applications have typically been used with careful planning, with extensive professional development for teachers conducting the experiment and with continuous support from researchers, whereas large-scale evaluation studies have focused on regular classrooms without such support. When technology is implemented in these latter situations, the pedagogical quality of the technology application is determined by the teachers’ knowledge and skills. However, there is a great deal of variation in the knowledge and competences of teachers when it comes to using technology in their classrooms (Valtonen et al., 2016 ). This highlights the importance of the in-service training of teachers while at the same time calling attention to pre-service teachers’ opportunities to acquire the competencies necessary to implement technology in their classrooms.

What does the development of technology mean for teacher education and how does it challenge traditional forms and content of the discipline? During the past few decades, a large number of policy documents and scientific studies have addressed this issue (Bakir, 2015 ). Different perspectives can be taken into account when discussing the influence of technology on teacher education. Several studies have examined the skills that will be necessary to apply technology to pedagogical practice in the future. This topic has been explored from several perspectives, such as teacher students’ technology literacy or their knowledge of technological pedagogical content (Mishra & Koehler, 2006 ). To ensure that future teachers possess adequate technical skills, standards and recommendations have been developed regarding the content of teacher education programmes. Rather than simply focusing on basic technological skills, the main emphasis has been on the knowledge and skills associated with the pedagogical use of technology (Erstad et al., 2021 ). Moreover, technology can provide many opportunities to develop novel methods to improve the quality of teacher education, such as the development of new methods for conducting research in the field of teacher education. In this special issue, these issues are addressed from a variety of theoretical and methodological perspectives.

Technology integration and content in teacher education

Given the many ways in which technology can be used in education, pre-service teachers’ pedagogical and technical competences in using ICT in teaching have different dimensions. For instance, Tondeur et al. ( 2017 ) developed a test to measure pre-service teachers’ ICT abilities and applied it to a large sample of Belgian teacher candidates. According to the findings, there are two dimensions to ICT competences: (1) competencies for supporting students’ use of ICT in class and (2) competencies for using ICT to create instructional materials. Some studies have also reported barriers that hinder the organisation of adequate teacher education to develop these skills, such as faculty beliefs and skills (Bakir, 2015 ; Polly et al., 2010 ).

The experiences pre-service teachers acquire during their teacher studies have also been shown to influence how willing and skilled they are when it comes to integrating technology in their classrooms (Agyei & Voogt, 2011 ). Additionally, studies have shown that the opportunity for teacher students to observe advanced technology applications in real-world settings is important for their future professional development (Gromseth et al., 2010 ). Hence, it is not sufficient to take formal courses in ICT or educational technology without applying these skills in the classroom.

Finnish pre-service teacher education stresses not only ICT skills but also teachers’ competences, such as strategic learning skills and collaboration competences. Häkkinen et al. ( 2017 ) identified five profiles among Finnish first-year pre-service teachers (N = 872) using perceptions of the teachers’ strategic learning skills and collaboration dispositions and investigated what background variables explained membership to those profiles. The most robust factor explaining membership in the profiles was life satisfaction. For example, pre-service teachers in a profile group with high strategic learning skills and high collaboration dispositions showed the highest anticipated life satisfaction after 5 years. Their results demonstrate the need to develop both ICT skills and learning competences in pre-service teacher education.

Use of technological innovations in organising teacher education

An analysis of the nature of experience and how practise can optimally enhance expertise has demonstrated the importance of deliberate practise (Ericsson et al., 1993 ), defined as an intensive practise that is purposefully focused on developing specific aspects of performance. To achieve this, it is necessary to have the opportunity to practise the most demanding aspects of performance with a large number of repetitions. Feedback from a tutor or coach also plays an important role in deliberate practise (Ericsson, 1993 ). Although deliberate practise has already been applied in some teacher education studies (e.g. Bronkhorst, 2011 ), its main aspect, repeated practise of challenging tasks and informed feedback, has been difficult to apply in traditional teacher education settings. Recent studies have shown that technology can contribute to the development of training methods that better reflect the main principles of deliberate practise.

There is a long tradition of using video technology in teacher education; such technology was first applied in a systematic manner in the 1970s (Nagro & Cornelius, 2013 ). Since then, a number of video-assisted instructional designs have been developed to provide teacher students with opportunities to learn from expert teachers, reflect on their own teaching behaviour and practise professional skills that would not otherwise be possible without this technology. Digital videos that are easy to use and various web platforms that facilitate the sharing and annotation of videos have opened up new opportunities for the development of novel learning environments in the field of teacher education (e.g. Sommerhoff et al., 2022 ). In the past few years, models for using videos recorded by mobile eye-tracking technology in teacher education have also been developed (e.g. Pouta et al., 2021 ).

Simulations are widely used in medical education, and a meta-analysis found that deliberate practise with simulations is superior to traditional clinical training in medical education (McGaghie et al., 2011 ). The use of simulations in teacher education is also gradually increasing. In their review of the use of simulations in teacher education, Theelen et al. ( 2019 ) synthesised the findings of 15 studies that applied computer-based simulations in teacher education. Several studies have demonstrated (Ferdig & Pytash, 2020 ; Samuelsson et al., 2022 ) that classroom simulations increase students’ self-efficacy and confidence in their teaching abilities. Classroom simulations were also found to have a positive impact on the development of classroom management skills.

New technologies can also be used in research on teaching and learning. Digitalisation has provided more ways to collect data and understand the teaching–learning process with multiple data channels and modalities. Multiple layers of data can be collected from contextual interactions, such as high-quality video data, psychophysiological measures and computer logs. With learning analytics, for example, these data can be used to create teacher dashboards, thus fulfilling students’ need for teacher scaffolds (Knoop-van Campen & Molenaar, 2020 ). Recently, eye-tracking technology has also been used to analyse teachers’ and student teachers’ abilities to notice relevant events in classrooms (Gegenfurtner et al., 2020 ; Pouta et al., 2021 ). Eye-movement technology, which has been used to model expert performance in other professional fields (Gegenfurtner et al., 2017 ), could also lead to promising training methods for teacher education.

Articles in this special issue

Three of the articles (Basilotta‑Gómez‑Pablos et al., 2022 ; Peciuliauskiene et al.’s, 2022 ; Kulaksız & Toran, 2022 ) included in this special issue deal with digital competences and interventions aimed at enhancing them.

Basilotta‑Gómez‑Pablos et al. (in this issue) synthesised 56 studies on higher education teachers’ digital competences. The authors used special software called SciMAT to analyse the content of the articles and to present thematic networks. Their review of the literature revealed that the topic is timely and that the number of relevant studies is increasing rapidly. The reviewed studies generally relied on teachers’ self-reports and self-evaluations of their abilities. Overall, the results indicated that the participants were aware of their insufficient knowledge and skills in the area of digital technology. According to the synthesis, many of the articles describe teachers’ experiences of various projects and activities aimed at improving their digital competences; however, many of these articles describe informal learning using internet tools and social networks. The authors conclude that their review clearly shows the gap in the evaluation of teachers’ competence in teaching and learning practice. Their recommendation is that more interventions and training programmes be created to support the development of teachers’ digital competence.

The recent challenges in education caused by the pandemic situation raised teachers’ awareness on the gap of their digital skills.

Despite the developed national or EU digital competences frameworks the trend remains that the development of digital skills is not systemic and lacks coherence in in-service and pre-service teacher education.

Further studies may bring more insights regarding more effective interventions to teaching practices with a wider application of digital technologies.

Peciuliauskiene et al.’s ( 2022 ) paper presents the results of their survey of two Lithuanian universities that offer teacher education programmes. Their questionnaire focused on information literacy (search and evaluation) and ICT self-efficacy. According to their results, both information literacy variables predicted teacher students’ ICT self-efficacy. Additionally, there was an indirect relationship between information evaluation and ICT self-efficacy. The findings of the study are discussed in terms of their theoretical and practical implications. The research indicates that information search ability does not depend on a person’s digital nativity, contrary to what is sometimes assumed when referring to the younger generation of pre-service teachers. As an ICT literacy component, information evaluation has become particularly pertinent during the COVID-19 situation and recent challenges related to distinguishing credible information from the vast amount of fake news and propaganda. It is also noted that optimal time and resources should be planned for the development of information search and evaluation abilities; however, more time should be allocated for the development of information search literacy, as it directly predicts pre-service teachers’ ICT self-efficacy. Based on the findings of this study, we identify the following trends and implications for further studies:

ICT self-efficacy of teachers contribute to the enhancement of teaching and learning process however, ICT self-efficacy should not be limited to specific ICT skills but rather on rethinking the organisation of the teaching process and rethinking the principles of teaching. In other words, the development of digital skills alone without integrating them with specific pedagogical content knowledge and teaching strategies would be less beneficial.

Further studies could be focused on how digital skills development could be better aligned with the development of teacher pedagogical strategies and specific subject areas.

The starting point of Kulaksız and Toran’s study ( 2022 ) was the observation that, despite pre-service teachers’ participation in courses on ICT integration, these teachers are still not confident about their competences to apply their knowledge in practice. In their study, Kulaksız and Toran used the so-called praxeological approach, which aims to produce beneficial knowledge and skills and to organise a democratic and participatory environment. The results indicate that the participants were prepared to transform their skills into practical pedagogical situations due to the personal development they experienced during and after the completion of the co-created course. As pre-service teachers could co-create the technology course, this allowed them to develop not only digital competences but also self-regulated learning skills, collaborative project development skills and peer mentoring skills, which contributed to building their sustainable motivation—an important component of teachers’ self-efficacy. This article highlights that.

Teachers’ motivation is increased through participatory design in their professional development practices which allows to achieve a more holistic development of digital skills in combination with cognitive and non-cognitive competences.

Further studies on how co-teaching contributes to digital skills development and innovative teaching strategies would allow to find more attractive models for teachers professional development.

The aforementioned three articles about digital competencies provide different perspectives on the issue, but they all emphasise the complexity of those competencies while also suggesting new approaches to deal with these challenges. In two of the articles, technology was not the focus of the studies but a method used in developing learning environments for teacher students.

In their study, Martin et al. (in this issue) investigated whether a video-based multimedia application about classroom teaching could be used to enhance teacher students’ professional vision. A teacher’s professional vision is their ability to observe and interpret important events in the classroom and determine the most appropriate teaching activities related to these events. Teacher education faces a variety of challenges because teacher students are unable to readily translate the knowledge they learn through formal teacher education into situation-specific skills that can be applied in actual classroom settings. The aim of the study was to help students make this translation with the aid of a video-based simulation developed using the findings of multimedia research. The simulation presented the classroom videos as short segments and provided prompts aimed at facilitating the students’ self-explanations. In the intervention study, they applied two versions of the video-based simulation: one with features based on multimedia research and one without these features. During the training, the segmented simulation with the self-explanation prompt resulted in increased noticing of relevant events in the teaching–learning process. In the comparison of pre-test and post-test results, all groups participating in the video training developed considerably in their professional vision, but the video simulation with the two multimedia elements did not differ significantly from the video training without these elements. The authors concluded that further research on the optimal implementation of the simulation is needed. Major issues raised by this article were:

It confirms previous studies which have shown that the importance of the use of classroom videos in teacher education.

When new methods (e.g. the multimedia elements added to the videos) are applied in interventions, it is important to pay attention to qualitative changes in learning processes and not only on immediate learning gains.

The results also indicate that methods which have been effective in one context do not necessarily work in a new environment.

Nickl et al.’s study (in this issue) examined how video-based simulations can be used to enhance pre-service teachers’ assessment skills. The aim was to analyse individual learning processes in a simulated environment by taking into account learners’ cognitive and motivational-affective characteristics. Their study applied a person-oriented approach to analyse how these learner characteristics relate to students’ situated learning experiences and performance. In the latent profile analysis, three profiles were identified: one with high knowledge and average motivation-affect, one with high motivation-affect and average knowledge and one with below average knowledge and motivation-affect. Based on the results, it was confirmed that the motivated profile resulted in positive motivational experiences in the situation, while the knowledgeable profile resulted in relevant cognitive demands when working on the tasks. Situational experiences were also found to be related to learning outcomes when working with the simulation. In comparison to the other profiles, the cognitive profile demonstrated the most effective navigation and deep learning processes. The authors concluded that the identification of learner profiles is a promising approach that can uncover individual learner needs when working in technology-based learning environments. There lessons to learn from this study include:

Learners prior learning and their personal characteristics can strongly mediate the outcomes of intervention programs

Person oriented statistical analyses are promising approaches to focus on sub-groups with unique profiles.

The challenge is to decide which individual characteristics are relevant in explaining varying effects of interventions.

Practices which stimulate change

This collection of papers disclosing different aspects of the application of technology in todays’ teacher education clearly highlights that the development of digital competences should become an integral part of pre-service and in-service teacher education. In line with the holistic view on teacher competencies (Metsäpelto et al., 2022 ), this special issue suggests that digital competences appear as a strong component within cognitive and non-cognitive competences that contribute to high-quality teaching.

The results of the studies presented in this special issue strongly reflect recent studies (Falloon, 2020 ; Lin et al., 2022 ) demonstrating that the development of mere digital skills is not sufficient; we should instead structure teacher education to promote the development of digital teaching competences, including ICT attitudes, ICT skills, data literacy and deep pedagogical understanding of the opportunities and limitations of the use of technology in education. Digitally competent teachers are more capable of integrating technologies into their regular teaching practices while also creating more appropriate conditions for personalised learning (Schmid & Petko, 2019 ). This is important because large international evaluation studies (OECD, 2014 , 2019 , 2020 ) have shown that the inadequate use of technology can be harmful for student learning.

It should also be noted that the COVID-19 pandemic created additional challenges for teachers and contributed to changes in their teaching practices and digital habits (Blume, 2020 ). Reflecting school situations caused by COVID-19, numerous studies from the last 2 years have revealed a much wider scope of application of digital technologies in education and a need to create active interactions with learners (Greenhow et al., 2021 ). They have also identified a widening gap between learners who are more digitally advanced and less digitally competent teachers (Blume, 2020 ).

Likewise, simulations and other technological applications can be used to provide richer learning opportunities in teacher education. These new tools can help develop more effective models to connect theoretical content and practical skills. A big challenge for teacher education is to create opportunities for students to deliberately practise skills that are needed in classroom teaching while at the same time deepening student teachers’ theoretical understanding of teaching–learning processes.

Availability of data and materials

The paper is based on openly available data.

Agyei, D. D., & Voogt, J. (2011). Exploring the potential of the Will Skill Tool model in Ghana: Predicting prospective and practicing teachers’ use of technology. Computers & Education, 56 , 91–100. https://doi.org/10.1016/j.compedu.2010.08.017

Article   Google Scholar  

Bakir, N. (2015). An exploration of contemporary realities of technology and teacher education: Lessons learned. Journal of Digital Learning in Teacher Education, 31 (3), 117–130. https://doi.org/10.1080/21532974.2015.1040930

Basilotta-Gómez-Pablos, V., Matarranz, M., Casado-Aranda, L. A., et al. (2022) Teachers’ digital competencies in higher education: a systematic literature review. International Journal of Educational Technology in Higher Education, 19 , 8 (2022). https://doi.org/10.1186/s41239-021-00312-8

Blume, C. (2020). German teachers’ digital habitus and their pandemic pedagogy. Post Digit Science Education, 2 , 879–905. https://doi.org/10.1007/s42438-020-00174-9

Bronkhorst, L. H., Meijer, P. C., Koster, B., & Vermung, J. D. (2011). Fostering meaning-oriented learning and deliberate practice in teacher education. Teaching and Teacher Education, 27 , 1120–1130. https://doi.org/10.1016/j.tate.2011.05.008

Cheung, A. C., & Slavin, R. E. (2013). The effectiveness of education technology applications for enhancing mathematics achievement in K-12 classrooms: A meta-analysis. Educational Research Review, 9 , 88–113.

Eickelmann, B., Gerick, J., & Koop, C. (2016). ICT use in mathematics lessons and the mathematics achievement of secondary school students by international comparison: Which role do school level factors play? Education and Information Technologies, 22 , 1527–1551. https://doi.org/10.1007/s10639-016-9498-5

Ericsson, K. A., Krampe, R. T., & Tesch-Römer, C. (1993). The role of deliberate practice in the acquisition of expert performance. Psychological Review, 100 , 363–406. https://doi.org/10.1037/0033-295X.87.3.215

Erstad, O., Kjällander, S., & Järvelä, S. (2021). Facing the challenges of ‘digital competence’—A Nordic agenda on curriculum development for the 21st century. Nordic Journal of Digital Literacy, 16 (2), 77–87. https://doi.org/10.18261/issn.1891-943x-2021-02-02

Fadda, D., Pellegrini, M., Vivanet, G., & Callegher, C. Z. (2022). Effects of digital games on student motivation in mathematics: A meta-analysis in K-12. Journal of Computer Assisted Learning, 38 , 304–325.

Falloon, G. (2020). From digital literacy to digital competence: The teacher digital competency (TDC) framework. Educational Technology Research and Development, 68 , 2449–2472. https://doi.org/10.1007/s11423-020-09767-4

Ferdig RE, Pytash KE. What teacher educators should have learned from 2020, AACE – Association for the Advancement of Computing in Education , 229–242; 2020.

Flecknoe, M. (2002). How can ICT help us to improve education? Innovations in Education and Teaching International, 39 (4), 271–279. https://doi.org/10.1080/13558000210161061

Gegenfurtner, A., Lehtinen, E., Jarodzka, H., & Säljö, R. (2017). Effects of eye movement modeling examples on adaptive expertise in medical image diagnosis. Computers & Education, 13 , 212–225. https://doi.org/10.1016/j.compedu.2017.06.001

Gegenfurtner, A., Lewalter, D., Lehtinen, E., Schmidt, M., & Gruber, H. (2020). Teacher expertise and professional vision: Examining knowledge-based reasoning of pre-service teachers, in-service teachers, and school principals. Frontiers in Education . https://doi.org/10.3389/feduc.2020.00059

Gloria, M. (2015). A Meta-Analysis of the Relationship between E-Learning and Students’ Academic Achievement in Higher Education. Journal of Education and Practice, 6 (9), 6–9.

Google Scholar  

Greenhow, Ch., Cathy, L. C., & Willet, B. S. (2021). The educational response to Covid-19 across two countries: A critical examination of initial digital pedagogy adoption. Technology, Pedagogy and Education, 30 (1), 7–25. https://doi.org/10.1080/1475939X.2020.1866654

Gronseth, S., Brush, T., Ottenbreit-Leftwich, A., Strycker, J., Abaci, S., Easterling, W., Roman, T., Shin, S., & van Leusen, P. (2010). Equipping the next generation of teachers. Journal of Digital Learning in Teacher Education, 27 (1), 30–36. https://doi.org/10.1080/21532974.2010.10784654

Häkkinen, P., Järvelä, S., Mäkitalo-Siegl, K., Ahonen, A., Näykki, P., & Valtonen, T. (2017). Preparing teacher students for 21st century learning practices (PREP 21): A framework for enhancing collaborative problem solving and strategic learning skills. Teachers and Teaching: Theory and Practice, 23 (1), 25–41. https://doi.org/10.1080/13540602.2016.1203772

Huang, Y., Miller, K. F., Cortina, K., & Richter, D. (2020). Teachers’ professional vision in action. Comparing expert and novice teacher’s real-life eye movements in the classroom. Zeitschrift Für Pädagogische Psychologie, 1 , 18.

Knoop-vanCampen, C., & Molenaar, I. (2020). How teachers integrate dashboards into their feedback practices. Frontline Learning Research, 8 (4), 37–51. https://doi.org/10.14786/flr.v8i4.641

Kulaksız, T., & Toran, M. (2022) Development of pre-service early childhood teachers’ technology integrations skills through a praxeological approach. International Journal of Educational Technology in Higher Education, 19 , 36. https://doi.org/10.1186/s41239-022-00344-8

Lin, R., Yang, J., Jiang, F., et al. (2022). Does teacher’s data literacy and digital teaching competence influence empowering students in the classroom? Evidence from China. Education and Information Technologies . https://doi.org/10.1007/s10639-022-11274-3

McGaghie, W. C., Issenberg, S. B., Cohen, E. R., Barsuk, J. H., & Wayne, D. B. (2011). Does simulation-based medical education with deliberate practice yield better results than traditional clinical education? A meta-analytic comparative review of the evidence. Academic Medicine, 86 (6), 706–711. https://doi.org/10.1097/ACM.0b013e318217e119

Metsäpelto, R. L., Poikkeus, A. M., Heikkilä, M., et al. (2022). A multidimensional adapted process model of teaching. Educational Assessment, Evaluation and Accountability, 34 , 143–172. https://doi.org/10.1007/s11092-021-09373-9

Mishra, P., & Koehler, M. J. (2006). Technological pedagogical content knowledge: A framework for teacher knowledge. Teachers College Record, 108 (6), 1017–1054.

Nagro, S. A., & Cornelius, K. E. (2013). Evaluating the evidence base of video analysis: A special education teacher development tool. Teacher Education and Special Education, 36 (4), 312–329. https://doi.org/10.1177/0888406413501090

OECD. (2014). TALIS 2013 results: An international perspective on teaching and learning. TALIS, OECD Publishing, Paris, . https://doi.org/10.1787/9789264196261-en

OECD. (2019). TALIS 2018 results (volume I): Teachers and school leaders as lifelong learners. TALIS, OECD Publishing, Paris, . https://doi.org/10.1787/1d0bc92a-en

OECD. (2020). TALIS 2018 results (volume II): Teachers and school leaders as valued professionals. TALIS, OECD Publishing, Paris, . https://doi.org/10.1787/19cf08df-en

Papanastasiou, E. C., Zembylas, M., & Vrasidas, C. (2003). Can computer use hurt science achievement? The USA results from PISA. Journal of Science Education and Technology, 12 (3), 325–332.

Peciuliauskiene, P., Tamoliune, G. & Trepule, E. (2022). Exploring the roles of information search and information evaluation literacy and pre-service teachers’ ICT self-efficacy in teaching. International Journal of Educational Technology in Higher Education, 19 , 33. https://doi.org/10.1186/s41239-022-00339-5

Polly, D., Mims, C., Shepherd, C. E., & Inan, F. (2010). Evidence of impact: Transforming teacher education with preparing tomorrow’s teachers to teach with technology (PT3) grants. Teaching and Teacher Education, 26 , 863–870.

Pouta, M., Palonen, T., & Lehtinen, E. (2021). Student teachers’ and experienced teachers’ professional vision of students’ rational number concept understanding. Educational Psychology Review., 33 (1), 109–128. https://doi.org/10.1007/s10648-020-09536-y

Roztocki, N., Soja, P., & Weistroffer, H. R. (2019). The role of information and communication technologies in socioeconomic development: Towards a multi-dimensional framework. Information Technology for Development, 25 (2), 171–183. https://doi.org/10.1080/02681102.2019.1596654

Samuelsson, M., Samuelsson, J., & Thorsten, A. (2022). Simulation training-a boost for pre-service teachers’ efficacy beliefs. Computers and Education Open, 3 , 100074. https://doi.org/10.1016/j.caeo.2022.100074

Schmid, R., & Petko, D. (2019). Does the use of educational technology in personalized learning environments correlate with self-reported digital skills and beliefs of secondary-school students? Computers & Education, 136 , 75–86. https://doi.org/10.1016/j.compedu.2019.03.006

Sommerhoff, D., Codreanu, E., Nickl, M., Ufer, S., & Seidel, T. (2022). Pre-service teachers’ learning of diagnostic skills in a video-based simulation: Effects of conceptual vs interconnecting prompts on judgment accuracy and the diagnostic process. Learning and Instruction . https://doi.org/10.1016/j.learninstruc.2022.101689

Theelen, H., Van den Beem, A., & den Brok, P. (2019). Classroom simulations in teacher education to support preservice teachers’ interpersonal competence: A systematic literature review. Computers & Education, 129 , 14–26. https://doi.org/10.1016/j.compedu.2018.10.015

Tondeur, J., Aesaert, K., Pynoo, B., van Braak, F., & J.N., & Erstad, O. (2017). Developing a validated instrument to measure preserviceteachers’ ICT competencies: Meeting the demands of the21st century. British Journal of Educational Technology, 48 (2), 462–472. https://doi.org/10.1111/bjet.12380

UNESCO ICT Competency Framework for Teachers. (2018). UNESCO, Paris: France. https://unesdoc.unesco.org/ark:/48223/pf0000265721

Valtonen, T., Sointu, E., Kukkonen, J., Häkkinen, P., Järvelä, S., Ahonen, A., Näykki, P., Pöysä-Tarhonen, J., & Mäkitalo-Siegl, K. (2016). Insights into Finnish first-year pre-service teachers’ twenty-first century skills. Education and Information Technologies, 22 , 2055–2069. https://doi.org/10.1007/s10639-016-9529-2

Wouters, P., van Nimwegen, C., van Oostendorp, H., & van der Spek, E. D. (2013). A meta-analysis of the cognitive and motivational effects of serious games. Journal of Educational Psychology, 105 (2), 249–265. https://doi.org/10.1037/a0031311

Download references

Author information

Authors and affiliations.

Education Academy, Vytautas Magnus University, Kaunas, Lithuania

Lina Kaminskienė & Erno Lehtinen

Faculty of Education and Psychology, University of Oulu, Oulu, Finland

Sanna Järvelä

Department of Teacher Education, University of Turku, Turku, Finland

Erno Lehtinen

You can also search for this author in PubMed   Google Scholar

Contributions

This is a collaboratively developed and discussed paper. The contribution of each author is equal. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Lina Kaminskienė .

Ethics declarations

Competing interests.

The authors state that they have no conflicting interests regarding the ideas of this study.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Kaminskienė, L., Järvelä, S. & Lehtinen, E. How does technology challenge teacher education?. Int J Educ Technol High Educ 19 , 64 (2022). https://doi.org/10.1186/s41239-022-00375-1

Download citation

Published : 27 December 2022

DOI : https://doi.org/10.1186/s41239-022-00375-1

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Teachers’ digital competences
• Teachers’ initial and continuous training
• Technology integration
• Digital innovations in education

• Northeastern University
• College of Professional Studies
• School of Education
• Education Theses and Dissertations
• Education Doctoral Theses

The impact of digital education on learning and teaching

Your file is being processed

Return to Education Doctoral Theses

A systematic literature review of empirical research on ChatGPT in education

• Open access
• Published: 26 May 2024
• Volume 3 , article number  60 , ( 2024 )

Cite this article

You have full access to this open access article

• Yazid Albadarin   ORCID: orcid.org/0009-0005-8068-8902 1 ,
• Mohammed Saqr 1 ,
• Nicolas Pope 1 &
• Markku Tukiainen 1  

365 Accesses

Explore all metrics

Over the last four decades, studies have investigated the incorporation of Artificial Intelligence (AI) into education. A recent prominent AI-powered technology that has impacted the education sector is ChatGPT. This article provides a systematic review of 14 empirical studies incorporating ChatGPT into various educational settings, published in 2022 and before the 10th of April 2023—the date of conducting the search process. It carefully followed the essential steps outlined in the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA 2020) guidelines, as well as Okoli’s (Okoli in Commun Assoc Inf Syst, 2015) steps for conducting a rigorous and transparent systematic review. In this review, we aimed to explore how students and teachers have utilized ChatGPT in various educational settings, as well as the primary findings of those studies. By employing Creswell’s (Creswell in Educational research: planning, conducting, and evaluating quantitative and qualitative research [Ebook], Pearson Education, London, 2015) coding techniques for data extraction and interpretation, we sought to gain insight into their initial attempts at ChatGPT incorporation into education. This approach also enabled us to extract insights and considerations that can facilitate its effective and responsible use in future educational contexts. The results of this review show that learners have utilized ChatGPT as a virtual intelligent assistant, where it offered instant feedback, on-demand answers, and explanations of complex topics. Additionally, learners have used it to enhance their writing and language skills by generating ideas, composing essays, summarizing, translating, paraphrasing texts, or checking grammar. Moreover, learners turned to it as an aiding tool to facilitate their directed and personalized learning by assisting in understanding concepts and homework, providing structured learning plans, and clarifying assignments and tasks. However, the results of specific studies (n = 3, 21.4%) show that overuse of ChatGPT may negatively impact innovative capacities and collaborative learning competencies among learners. Educators, on the other hand, have utilized ChatGPT to create lesson plans, generate quizzes, and provide additional resources, which helped them enhance their productivity and efficiency and promote different teaching methodologies. Despite these benefits, the majority of the reviewed studies recommend the importance of conducting structured training, support, and clear guidelines for both learners and educators to mitigate the drawbacks. This includes developing critical evaluation skills to assess the accuracy and relevance of information provided by ChatGPT, as well as strategies for integrating human interaction and collaboration into learning activities that involve AI tools. Furthermore, they also recommend ongoing research and proactive dialogue with policymakers, stakeholders, and educational practitioners to refine and enhance the use of AI in learning environments. This review could serve as an insightful resource for practitioners who seek to integrate ChatGPT into education and stimulate further research in the field.

Similar content being viewed by others

Empowering learners with ChatGPT: insights from a systematic literature exploration

Incorporating AI in foreign language education: An investigation into ChatGPT’s effect on foreign language learners

Large language models in education: A focus on the complementary relationship between human teachers and ChatGPT

Avoid common mistakes on your manuscript.

1 Introduction

Educational technology, a rapidly evolving field, plays a crucial role in reshaping the landscape of teaching and learning [ 82 ]. One of the most transformative technological innovations of our era that has influenced the field of education is Artificial Intelligence (AI) [ 50 ]. Over the last four decades, AI in education (AIEd) has gained remarkable attention for its potential to make significant advancements in learning, instructional methods, and administrative tasks within educational settings [ 11 ]. In particular, a large language model (LLM), a type of AI algorithm that applies artificial neural networks (ANNs) and uses massively large data sets to understand, summarize, generate, and predict new content that is almost difficult to differentiate from human creations [ 79 ], has opened up novel possibilities for enhancing various aspects of education, from content creation to personalized instruction [ 35 ]. Chatbots that leverage the capabilities of LLMs to understand and generate human-like responses have also presented the capacity to enhance student learning and educational outcomes by engaging students, offering timely support, and fostering interactive learning experiences [ 46 ].

The ongoing and remarkable technological advancements in chatbots have made their use more convenient, increasingly natural and effortless, and have expanded their potential for deployment across various domains [ 70 ]. One prominent example of chatbot applications is the Chat Generative Pre-Trained Transformer, known as ChatGPT, which was introduced by OpenAI, a leading AI research lab, on November 30th, 2022. ChatGPT employs a variety of deep learning techniques to generate human-like text, with a particular focus on recurrent neural networks (RNNs). Long short-term memory (LSTM) allows it to grasp the context of the text being processed and retain information from previous inputs. Also, the transformer architecture, a neural network architecture based on the self-attention mechanism, allows it to analyze specific parts of the input, thereby enabling it to produce more natural-sounding and coherent output. Additionally, the unsupervised generative pre-training and the fine-tuning methods allow ChatGPT to generate more relevant and accurate text for specific tasks [ 31 , 62 ]. Furthermore, reinforcement learning from human feedback (RLHF), a machine learning approach that combines reinforcement learning techniques with human-provided feedback, has helped improve ChatGPT’s model by accelerating the learning process and making it significantly more efficient.

This cutting-edge natural language processing (NLP) tool is widely recognized as one of today's most advanced LLMs-based chatbots [ 70 ], allowing users to ask questions and receive detailed, coherent, systematic, personalized, convincing, and informative human-like responses [ 55 ], even within complex and ambiguous contexts [ 63 , 77 ]. ChatGPT is considered the fastest-growing technology in history: in just three months following its public launch, it amassed an estimated 120 million monthly active users [ 16 ] with an estimated 13 million daily queries [ 49 ], surpassing all other applications [ 64 ]. This remarkable growth can be attributed to the unique features and user-friendly interface that ChatGPT offers. Its intuitive design allows users to interact seamlessly with the technology, making it accessible to a diverse range of individuals, regardless of their technical expertise [ 78 ]. Additionally, its exceptional performance results from a combination of advanced algorithms, continuous enhancements, and extensive training on a diverse dataset that includes various text sources such as books, articles, websites, and online forums [ 63 ], have contributed to a more engaging and satisfying user experience [ 62 ]. These factors collectively explain its remarkable global growth and set it apart from predecessors like Bard, Bing Chat, ERNIE, and others.

In this context, several studies have explored the technological advancements of chatbots. One noteworthy recent research effort, conducted by Schöbel et al. [ 70 ], stands out for its comprehensive analysis of more than 5,000 studies on communication agents. This study offered a comprehensive overview of the historical progression and future prospects of communication agents, including ChatGPT. Moreover, other studies have focused on making comparisons, particularly between ChatGPT and alternative chatbots like Bard, Bing Chat, ERNIE, LaMDA, BlenderBot, and various others. For example, O’Leary [ 53 ] compared two chatbots, LaMDA and BlenderBot, with ChatGPT and revealed that ChatGPT outperformed both. This superiority arises from ChatGPT’s capacity to handle a wider range of questions and generate slightly varied perspectives within specific contexts. Similarly, ChatGPT exhibited an impressive ability to formulate interpretable responses that were easily understood when compared with Google's feature snippet [ 34 ]. Additionally, ChatGPT was compared to other LLMs-based chatbots, including Bard and BERT, as well as ERNIE. The findings indicated that ChatGPT exhibited strong performance in the given tasks, often outperforming the other models [ 59 ].

Furthermore, in the education context, a comprehensive study systematically compared a range of the most promising chatbots, including Bard, Bing Chat, ChatGPT, and Ernie across a multidisciplinary test that required higher-order thinking. The study revealed that ChatGPT achieved the highest score, surpassing Bing Chat and Bard [ 64 ]. Similarly, a comparative analysis was conducted to compare ChatGPT with Bard in answering a set of 30 mathematical questions and logic problems, grouped into two question sets. Set (A) is unavailable online, while Set (B) is available online. The results revealed ChatGPT's superiority in Set (A) over Bard. Nevertheless, Bard's advantage emerged in Set (B) due to its capacity to access the internet directly and retrieve answers, a capability that ChatGPT does not possess [ 57 ]. However, through these varied assessments, ChatGPT consistently highlights its exceptional prowess compared to various alternatives in the ever-evolving chatbot technology.

The widespread adoption of chatbots, especially ChatGPT, by millions of students and educators, has sparked extensive discussions regarding its incorporation into the education sector [ 64 ]. Accordingly, many scholars have contributed to the discourse, expressing both optimism and pessimism regarding the incorporation of ChatGPT into education. For example, ChatGPT has been highlighted for its capabilities in enriching the learning and teaching experience through its ability to support different learning approaches, including adaptive learning, personalized learning, and self-directed learning [ 58 , 60 , 91 ]), deliver summative and formative feedback to students and provide real-time responses to questions, increase the accessibility of information [ 22 , 40 , 43 ], foster students’ performance, engagement and motivation [ 14 , 44 , 58 ], and enhance teaching practices [ 17 , 18 , 64 , 74 ].

On the other hand, concerns have been also raised regarding its potential negative effects on learning and teaching. These include the dissemination of false information and references [ 12 , 23 , 61 , 85 ], biased reinforcement [ 47 , 50 ], compromised academic integrity [ 18 , 40 , 66 , 74 ], and the potential decline in students' skills [ 43 , 61 , 64 , 74 ]. As a result, ChatGPT has been banned in multiple countries, including Russia, China, Venezuela, Belarus, and Iran, as well as in various educational institutions in India, Italy, Western Australia, France, and the United States [ 52 , 90 ].

Clearly, the advent of chatbots, especially ChatGPT, has provoked significant controversy due to their potential impact on learning and teaching. This indicates the necessity for further exploration to gain a deeper understanding of this technology and carefully evaluate its potential benefits, limitations, challenges, and threats to education [ 79 ]. Therefore, conducting a systematic literature review will provide valuable insights into the potential prospects and obstacles linked to its incorporation into education. This systematic literature review will primarily focus on ChatGPT, driven by the aforementioned key factors outlined above.

However, the existing literature lacks a systematic literature review of empirical studies. Thus, this systematic literature review aims to address this gap by synthesizing the existing empirical studies conducted on chatbots, particularly ChatGPT, in the field of education, highlighting how ChatGPT has been utilized in educational settings, and identifying any existing gaps. This review may be particularly useful for researchers in the field and educators who are contemplating the integration of ChatGPT or any chatbot into education. The following research questions will guide this study:

What are students' and teachers' initial attempts at utilizing ChatGPT in education?

What are the main findings derived from empirical studies that have incorporated ChatGPT into learning and teaching?

2 Methodology

To conduct this study, the authors followed the essential steps of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA 2020) and Okoli’s [ 54 ] steps for conducting a systematic review. These included identifying the study’s purpose, drafting a protocol, applying a practical screening process, searching the literature, extracting relevant data, evaluating the quality of the included studies, synthesizing the studies, and ultimately writing the review. The subsequent section provides an extensive explanation of how these steps were carried out in this study.

2.1 Identify the purpose

Given the widespread adoption of ChatGPT by students and teachers for various educational purposes, often without a thorough understanding of responsible and effective use or a clear recognition of its potential impact on learning and teaching, the authors recognized the need for further exploration of ChatGPT's impact on education in this early stage. Therefore, they have chosen to conduct a systematic literature review of existing empirical studies that incorporate ChatGPT into educational settings. Despite the limited number of empirical studies due to the novelty of the topic, their goal is to gain a deeper understanding of this technology and proactively evaluate its potential benefits, limitations, challenges, and threats to education. This effort could help to understand initial reactions and attempts at incorporating ChatGPT into education and bring out insights and considerations that can inform the future development of education.

2.2 Draft the protocol

The next step is formulating the protocol. This protocol serves to outline the study process in a rigorous and transparent manner, mitigating researcher bias in study selection and data extraction [ 88 ]. The protocol will include the following steps: generating the research question, predefining a literature search strategy, identifying search locations, establishing selection criteria, assessing the studies, developing a data extraction strategy, and creating a timeline.

2.3 Apply practical screen

The screening step aims to accurately filter the articles resulting from the searching step and select the empirical studies that have incorporated ChatGPT into educational contexts, which will guide us in answering the research questions and achieving the objectives of this study. To ensure the rigorous execution of this step, our inclusion and exclusion criteria were determined based on the authors' experience and informed by previous successful systematic reviews [ 21 ]. Table 1 summarizes the inclusion and exclusion criteria for study selection.

2.4 Literature search

We conducted a thorough literature search to identify articles that explored, examined, and addressed the use of ChatGPT in Educational contexts. We utilized two research databases: Dimensions.ai, which provides access to a large number of research publications, and lens.org, which offers access to over 300 million articles, patents, and other research outputs from diverse sources. Additionally, we included three databases, Scopus, Web of Knowledge, and ERIC, which contain relevant research on the topic that addresses our research questions. To browse and identify relevant articles, we used the following search formula: ("ChatGPT" AND "Education"), which included the Boolean operator "AND" to get more specific results. The subject area in the Scopus and ERIC databases were narrowed to "ChatGPT" and "Education" keywords, and in the WoS database was limited to the "Education" category. The search was conducted between the 3rd and 10th of April 2023, which resulted in 276 articles from all selected databases (111 articles from Dimensions.ai, 65 from Scopus, 28 from Web of Science, 14 from ERIC, and 58 from Lens.org). These articles were imported into the Rayyan web-based system for analysis. The duplicates were identified automatically by the system. Subsequently, the first author manually reviewed the duplicated articles ensured that they had the same content, and then removed them, leaving us with 135 unique articles. Afterward, the titles, abstracts, and keywords of the first 40 manuscripts were scanned and reviewed by the first author and were discussed with the second and third authors to resolve any disagreements. Subsequently, the first author proceeded with the filtering process for all articles and carefully applied the inclusion and exclusion criteria as presented in Table  1 . Articles that met any one of the exclusion criteria were eliminated, resulting in 26 articles. Afterward, the authors met to carefully scan and discuss them. The authors agreed to eliminate any empirical studies solely focused on checking ChatGPT capabilities, as these studies do not guide us in addressing the research questions and achieving the study's objectives. This resulted in 14 articles eligible for analysis.

2.5 Quality appraisal

The examination and evaluation of the quality of the extracted articles is a vital step [ 9 ]. Therefore, the extracted articles were carefully evaluated for quality using Fink’s [ 24 ] standards, which emphasize the necessity for detailed descriptions of methodology, results, conclusions, strengths, and limitations. The process began with a thorough assessment of each study's design, data collection, and analysis methods to ensure their appropriateness and comprehensive execution. The clarity, consistency, and logical progression from data to results and conclusions were also critically examined. Potential biases and recognized limitations within the studies were also scrutinized. Ultimately, two articles were excluded for failing to meet Fink’s criteria, particularly in providing sufficient detail on methodology, results, conclusions, strengths, or limitations. The review process is illustrated in Fig.  1 .

The study selection process

2.6 Data extraction

The next step is data extraction, the process of capturing the key information and categories from the included studies. To improve efficiency, reduce variation among authors, and minimize errors in data analysis, the coding categories were constructed using Creswell's [ 15 ] coding techniques for data extraction and interpretation. The coding process involves three sequential steps. The initial stage encompasses open coding , where the researcher examines the data, generates codes to describe and categorize it, and gains a deeper understanding without preconceived ideas. Following open coding is axial coding , where the interrelationships between codes from open coding are analyzed to establish more comprehensive categories or themes. The process concludes with selective coding , refining and integrating categories or themes to identify core concepts emerging from the data. The first coder performed the coding process, then engaged in discussions with the second and third authors to finalize the coding categories for the first five articles. The first coder then proceeded to code all studies and engaged again in discussions with the other authors to ensure the finalization of the coding process. After a comprehensive analysis and capturing of the key information from the included studies, the data extraction and interpretation process yielded several themes. These themes have been categorized and are presented in Table  2 . It is important to note that open coding results were removed from Table  2 for aesthetic reasons, as it included many generic aspects, such as words, short phrases, or sentences mentioned in the studies.

2.7 Synthesize studies

In this stage, we will gather, discuss, and analyze the key findings that emerged from the selected studies. The synthesis stage is considered a transition from an author-centric to a concept-centric focus, enabling us to map all the provided information to achieve the most effective evaluation of the data [ 87 ]. Initially, the authors extracted data that included general information about the selected studies, including the author(s)' names, study titles, years of publication, educational levels, research methodologies, sample sizes, participants, main aims or objectives, raw data sources, and analysis methods. Following that, all key information and significant results from the selected studies were compiled using Creswell’s [ 15 ] coding techniques for data extraction and interpretation to identify core concepts and themes emerging from the data, focusing on those that directly contributed to our research questions and objectives, such as the initial utilization of ChatGPT in learning and teaching, learners' and educators' familiarity with ChatGPT, and the main findings of each study. Finally, the data related to each selected study were extracted into an Excel spreadsheet for data processing. The Excel spreadsheet was reviewed by the authors, including a series of discussions to ensure the finalization of this process and prepare it for further analysis. Afterward, the final result being analyzed and presented in various types of charts and graphs. Table 4 presents the extracted data from the selected studies, with each study labeled with a capital 'S' followed by a number.

This section consists of two main parts. The first part provides a descriptive analysis of the data compiled from the reviewed studies. The second part presents the answers to the research questions and the main findings of these studies.

3.1 Part 1: descriptive analysis

This section will provide a descriptive analysis of the reviewed studies, including educational levels and fields, participants distribution, country contribution, research methodologies, study sample size, study population, publication year, list of journals, familiarity with ChatGPT, source of data, and the main aims and objectives of the studies. Table 4 presents a comprehensive overview of the extracted data from the selected studies.

3.1.1 The number of the reviewed studies and publication years

The total number of the reviewed studies was 14. All studies were empirical studies and published in different journals focusing on Education and Technology. One study was published in 2022 [S1], while the remaining were published in 2023 [S2]-[S14]. Table 3 illustrates the year of publication, the names of the journals, and the number of reviewed studies published in each journal for the studies reviewed.

3.1.2 Educational levels and fields

The majority of the reviewed studies, 11 studies, were conducted in higher education institutions [S1]-[S10] and [S13]. Two studies did not specify the educational level of the population [S12] and [S14], while one study focused on elementary education [S11]. However, the reviewed studies covered various fields of education. Three studies focused on Arts and Humanities Education [S8], [S11], and [S14], specifically English Education. Two studies focused on Engineering Education, with one in Computer Engineering [S2] and the other in Construction Education [S3]. Two studies focused on Mathematics Education [S5] and [S12]. One study focused on Social Science Education [S13]. One study focused on Early Education [S4]. One study focused on Journalism Education [S9]. Finally, three studies did not specify the field of education [S1], [S6], and [S7]. Figure  2 represents the educational levels in the reviewed studies, while Fig.  3 represents the context of the reviewed studies.

Educational levels in the reviewed studies

Context of the reviewed studies

3.1.3 Participants distribution and countries contribution

The reviewed studies have been conducted across different geographic regions, providing a diverse representation of the studies. The majority of the studies, 10 in total, [S1]-[S3], [S5]-[S9], [S11], and [S14], primarily focused on participants from single countries such as Pakistan, the United Arab Emirates, China, Indonesia, Poland, Saudi Arabia, South Korea, Spain, Tajikistan, and the United States. In contrast, four studies, [S4], [S10], [S12], and [S13], involved participants from multiple countries, including China and the United States [S4], China, the United Kingdom, and the United States [S10], the United Arab Emirates, Oman, Saudi Arabia, and Jordan [S12], Turkey, Sweden, Canada, and Australia [ 13 ]. Figures  4 and 5 illustrate the distribution of participants, whether from single or multiple countries, and the contribution of each country in the reviewed studies, respectively.

The reviewed studies conducted in single or multiple countries

The Contribution of each country in the studies

3.1.4 Study population and sample size

Four study populations were included: university students, university teachers, university teachers and students, and elementary school teachers. Six studies involved university students [S2], [S3], [S5] and [S6]-[S8]. Three studies focused on university teachers [S1], [S4], and [S6], while one study specifically targeted elementary school teachers [S11]. Additionally, four studies included both university teachers and students [S10] and [ 12 , 13 , 14 ], and among them, study [S13] specifically included postgraduate students. In terms of the sample size of the reviewed studies, nine studies included a small sample size of less than 50 participants [S1], [S3], [S6], [S8], and [S10]-[S13]. Three studies had 50–100 participants [S2], [S9], and [S14]. Only one study had more than 100 participants [S7]. It is worth mentioning that study [S4] adopted a mixed methods approach, including 10 participants for qualitative analysis and 110 participants for quantitative analysis.

3.1.5 Participants’ familiarity with using ChatGPT

The reviewed studies recruited a diverse range of participants with varying levels of familiarity with ChatGPT. Five studies [S2], [S4], [S6], [S8], and [S12] involved participants already familiar with ChatGPT, while eight studies [S1], [S3], [S5], [S7], [S9], [S10], [S13] and [S14] included individuals with differing levels of familiarity. Notably, one study [S11] had participants who were entirely unfamiliar with ChatGPT. It is important to note that four studies [S3], [S5], [S9], and [S11] provided training or guidance to their participants before conducting their studies, while ten studies [S1], [S2], [S4], [S6]-[S8], [S10], and [S12]-[S14] did not provide training due to the participants' existing familiarity with ChatGPT.

3.1.6 Research methodology approaches and source(S) of data

The reviewed studies adopted various research methodology approaches. Seven studies adopted qualitative research methodology [S1], [S4], [S6], [S8], [S10], [S11], and [S12], while three studies adopted quantitative research methodology [S3], [S7], and [S14], and four studies employed mixed-methods, which involved a combination of both the strengths of qualitative and quantitative methods [S2], [S5], [S9], and [S13].

In terms of the source(s) of data, the reviewed studies obtained their data from various sources, such as interviews, questionnaires, and pre-and post-tests. Six studies relied on interviews as their primary source of data collection [S1], [S4], [S6], [S10], [S11], and [S12], four studies relied on questionnaires [S2], [S7], [S13], and [S14], two studies combined the use of pre-and post-tests and questionnaires for data collection [S3] and [S9], while two studies combined the use of questionnaires and interviews to obtain the data [S5] and [S8]. It is important to note that six of the reviewed studies were quasi-experimental [S3], [S5], [S8], [S9], [S12], and [S14], while the remaining ones were experimental studies [S1], [S2], [S4], [S6], [S7], [S10], [S11], and [S13]. Figures  6 and 7 illustrate the research methodologies and the source (s) of data used in the reviewed studies, respectively.

Research methodologies in the reviewed studies

Source of data in the reviewed studies

3.1.7 The aim and objectives of the studies

The reviewed studies encompassed a diverse set of aims, with several of them incorporating multiple primary objectives. Six studies [S3], [S6], [S7], [S8], [S11], and [S12] examined the integration of ChatGPT in educational contexts, and four studies [S4], [S5], [S13], and [S14] investigated the various implications of its use in education, while three studies [S2], [S9], and [S10] aimed to explore both its integration and implications in education. Additionally, seven studies explicitly explored attitudes and perceptions of students [S2] and [S3], educators [S1] and [S6], or both [S10], [S12], and [S13] regarding the utilization of ChatGPT in educational settings.

3.2 Part 2: research questions and main findings of the reviewed studies

This part will present the answers to the research questions and the main findings of the reviewed studies, classified into two main categories (learning and teaching) according to AI Education classification by [ 36 ]. Figure  8 summarizes the main findings of the reviewed studies in a visually informative diagram. Table 4 provides a detailed list of the key information extracted from the selected studies that led to generating these themes.

The main findings in the reviewed studies

4 Students' initial attempts at utilizing ChatGPT in learning and main findings from students' perspective

4.1 virtual intelligent assistant.

Nine studies demonstrated that ChatGPT has been utilized by students as an intelligent assistant to enhance and support their learning. Students employed it for various purposes, such as answering on-demand questions [S2]-[S5], [S8], [S10], and [S12], providing valuable information and learning resources [S2]-[S5], [S6], and [S8], as well as receiving immediate feedback [S2], [S4], [S9], [S10], and [S12]. In this regard, students generally were confident in the accuracy of ChatGPT's responses, considering them relevant, reliable, and detailed [S3], [S4], [S5], and [S8]. However, some students indicated the need for improvement, as they found that answers are not always accurate [S2], and that misleading information may have been provided or that it may not always align with their expectations [S6] and [S10]. It was also observed by the students that the accuracy of ChatGPT is dependent on several factors, including the quality and specificity of the user's input, the complexity of the question or topic, and the scope and relevance of its training data [S12]. Many students felt that ChatGPT's answers were not always accurate and most of them believed that it requires good background knowledge to work with.

4.2 Writing and language proficiency assistant

Six of the reviewed studies highlighted that ChatGPT has been utilized by students as a valuable assistant tool to improve their academic writing skills and language proficiency. Among these studies, three mainly focused on English education, demonstrating that students showed sufficient mastery in using ChatGPT for generating ideas, summarizing, paraphrasing texts, and completing writing essays [S8], [S11], and [S14]. Furthermore, ChatGPT helped them in writing by making students active investigators rather than passive knowledge recipients and facilitated the development of their writing skills [S11] and [S14]. Similarly, ChatGPT allowed students to generate unique ideas and perspectives, leading to deeper analysis and reflection on their journalism writing [S9]. In terms of language proficiency, ChatGPT allowed participants to translate content into their home languages, making it more accessible and relevant to their context [S4]. It also enabled them to request changes in linguistic tones or flavors [S8]. Moreover, participants used it to check grammar or as a dictionary [S11].

4.3 Valuable resource for learning approaches

Five studies demonstrated that students used ChatGPT as a valuable complementary resource for self-directed learning. It provided learning resources and guidance on diverse educational topics and created a supportive home learning environment [S2] and [S4]. Moreover, it offered step-by-step guidance to grasp concepts at their own pace and enhance their understanding [S5], streamlined task and project completion carried out independently [S7], provided comprehensive and easy-to-understand explanations on various subjects [S10], and assisted in studying geometry operations, thereby empowering them to explore geometry operations at their own pace [S12]. Three studies showed that students used ChatGPT as a valuable learning resource for personalized learning. It delivered age-appropriate conversations and tailored teaching based on a child's interests [S4], acted as a personalized learning assistant, adapted to their needs and pace, which assisted them in understanding mathematical concepts [S12], and enabled personalized learning experiences in social sciences by adapting to students' needs and learning styles [S13]. On the other hand, it is important to note that, according to one study [S5], students suggested that using ChatGPT may negatively affect collaborative learning competencies between students.

4.4 Enhancing students' competencies

Six of the reviewed studies have shown that ChatGPT is a valuable tool for improving a wide range of skills among students. Two studies have provided evidence that ChatGPT led to improvements in students' critical thinking, reasoning skills, and hazard recognition competencies through engaging them in interactive conversations or activities and providing responses related to their disciplines in journalism [S5] and construction education [S9]. Furthermore, two studies focused on mathematical education have shown the positive impact of ChatGPT on students' problem-solving abilities in unraveling problem-solving questions [S12] and enhancing the students' understanding of the problem-solving process [S5]. Lastly, one study indicated that ChatGPT effectively contributed to the enhancement of conversational social skills [S4].

4.5 Supporting students' academic success

Seven of the reviewed studies highlighted that students found ChatGPT to be beneficial for learning as it enhanced learning efficiency and improved the learning experience. It has been observed to improve students' efficiency in computer engineering studies by providing well-structured responses and good explanations [S2]. Additionally, students found it extremely useful for hazard reporting [S3], and it also enhanced their efficiency in solving mathematics problems and capabilities [S5] and [S12]. Furthermore, by finding information, generating ideas, translating texts, and providing alternative questions, ChatGPT aided students in deepening their understanding of various subjects [S6]. It contributed to an increase in students' overall productivity [S7] and improved efficiency in composing written tasks [S8]. Regarding learning experiences, ChatGPT was instrumental in assisting students in identifying hazards that they might have otherwise overlooked [S3]. It also improved students' learning experiences in solving mathematics problems and developing abilities [S5] and [S12]. Moreover, it increased students' successful completion of important tasks in their studies [S7], particularly those involving average difficulty writing tasks [S8]. Additionally, ChatGPT increased the chances of educational success by providing students with baseline knowledge on various topics [S10].

5 Teachers' initial attempts at utilizing ChatGPT in teaching and main findings from teachers' perspective

5.1 valuable resource for teaching.

The reviewed studies showed that teachers have employed ChatGPT to recommend, modify, and generate diverse, creative, organized, and engaging educational contents, teaching materials, and testing resources more rapidly [S4], [S6], [S10] and [S11]. Additionally, teachers experienced increased productivity as ChatGPT facilitated quick and accurate responses to questions, fact-checking, and information searches [S1]. It also proved valuable in constructing new knowledge [S6] and providing timely answers to students' questions in classrooms [S11]. Moreover, ChatGPT enhanced teachers' efficiency by generating new ideas for activities and preplanning activities for their students [S4] and [S6], including interactive language game partners [S11].

5.2 Improving productivity and efficiency

The reviewed studies showed that participants' productivity and work efficiency have been significantly enhanced by using ChatGPT as it enabled them to allocate more time to other tasks and reduce their overall workloads [S6], [S10], [S11], [S13], and [S14]. However, three studies [S1], [S4], and [S11], indicated a negative perception and attitude among teachers toward using ChatGPT. This negativity stemmed from a lack of necessary skills to use it effectively [S1], a limited familiarity with it [S4], and occasional inaccuracies in the content provided by it [S10].

5.3 Catalyzing new teaching methodologies

Five of the reviewed studies highlighted that educators found the necessity of redefining their teaching profession with the assistance of ChatGPT [S11], developing new effective learning strategies [S4], and adapting teaching strategies and methodologies to ensure the development of essential skills for future engineers [S5]. They also emphasized the importance of adopting new educational philosophies and approaches that can evolve with the introduction of ChatGPT into the classroom [S12]. Furthermore, updating curricula to focus on improving human-specific features, such as emotional intelligence, creativity, and philosophical perspectives [S13], was found to be essential.

5.4 Effective utilization of CHATGPT in teaching

According to the reviewed studies, effective utilization of ChatGPT in education requires providing teachers with well-structured training, support, and adequate background on how to use ChatGPT responsibly [S1], [S3], [S11], and [S12]. Establishing clear rules and regulations regarding its usage is essential to ensure it positively impacts the teaching and learning processes, including students' skills [S1], [S4], [S5], [S8], [S9], and [S11]-[S14]. Moreover, conducting further research and engaging in discussions with policymakers and stakeholders is indeed crucial for the successful integration of ChatGPT in education and to maximize the benefits for both educators and students [S1], [S6]-[S10], and [S12]-[S14].

6 Discussion

The purpose of this review is to conduct a systematic review of empirical studies that have explored the utilization of ChatGPT, one of today’s most advanced LLM-based chatbots, in education. The findings of the reviewed studies showed several ways of ChatGPT utilization in different learning and teaching practices as well as it provided insights and considerations that can facilitate its effective and responsible use in future educational contexts. The results of the reviewed studies came from diverse fields of education, which helped us avoid a biased review that is limited to a specific field. Similarly, the reviewed studies have been conducted across different geographic regions. This kind of variety in geographic representation enriched the findings of this review.

In response to RQ1 , "What are students' and teachers' initial attempts at utilizing ChatGPT in education?", the findings from this review provide comprehensive insights. Chatbots, including ChatGPT, play a crucial role in supporting student learning, enhancing their learning experiences, and facilitating diverse learning approaches [ 42 , 43 ]. This review found that this tool, ChatGPT, has been instrumental in enhancing students' learning experiences by serving as a virtual intelligent assistant, providing immediate feedback, on-demand answers, and engaging in educational conversations. Additionally, students have benefited from ChatGPT’s ability to generate ideas, compose essays, and perform tasks like summarizing, translating, paraphrasing texts, or checking grammar, thereby enhancing their writing and language competencies. Furthermore, students have turned to ChatGPT for assistance in understanding concepts and homework, providing structured learning plans, and clarifying assignments and tasks, which fosters a supportive home learning environment, allowing them to take responsibility for their own learning and cultivate the skills and approaches essential for supportive home learning environment [ 26 , 27 , 28 ]. This finding aligns with the study of Saqr et al. [ 68 , 69 ] who highlighted that, when students actively engage in their own learning process, it yields additional advantages, such as heightened motivation, enhanced achievement, and the cultivation of enthusiasm, turning them into advocates for their own learning.

Moreover, students have utilized ChatGPT for tailored teaching and step-by-step guidance on diverse educational topics, streamlining task and project completion, and generating and recommending educational content. This personalization enhances the learning environment, leading to increased academic success. This finding aligns with other recent studies [ 26 , 27 , 28 , 60 , 66 ] which revealed that ChatGPT has the potential to offer personalized learning experiences and support an effective learning process by providing students with customized feedback and explanations tailored to their needs and abilities. Ultimately, fostering students' performance, engagement, and motivation, leading to increase students' academic success [ 14 , 44 , 58 ]. This ultimate outcome is in line with the findings of Saqr et al. [ 68 , 69 ], which emphasized that learning strategies are important catalysts of students' learning, as students who utilize effective learning strategies are more likely to have better academic achievement.

Teachers, too, have capitalized on ChatGPT's capabilities to enhance productivity and efficiency, using it for creating lesson plans, generating quizzes, providing additional resources, generating and preplanning new ideas for activities, and aiding in answering students’ questions. This adoption of technology introduces new opportunities to support teaching and learning practices, enhancing teacher productivity. This finding aligns with those of Day [ 17 ], De Castro [ 18 ], and Su and Yang [ 74 ] as well as with those of Valtonen et al. [ 82 ], who revealed that emerging technological advancements have opened up novel opportunities and means to support teaching and learning practices, and enhance teachers’ productivity.

In response to RQ2 , "What are the main findings derived from empirical studies that have incorporated ChatGPT into learning and teaching?", the findings from this review provide profound insights and raise significant concerns. Starting with the insights, chatbots, including ChatGPT, have demonstrated the potential to reshape and revolutionize education, creating new, novel opportunities for enhancing the learning process and outcomes [ 83 ], facilitating different learning approaches, and offering a range of pedagogical benefits [ 19 , 43 , 72 ]. In this context, this review found that ChatGPT could open avenues for educators to adopt or develop new effective learning and teaching strategies that can evolve with the introduction of ChatGPT into the classroom. Nonetheless, there is an evident lack of research understanding regarding the potential impact of generative machine learning models within diverse educational settings [ 83 ]. This necessitates teachers to attain a high level of proficiency in incorporating chatbots, such as ChatGPT, into their classrooms to create inventive, well-structured, and captivating learning strategies. In the same vein, the review also found that teachers without the requisite skills to utilize ChatGPT realized that it did not contribute positively to their work and could potentially have adverse effects [ 37 ]. This concern could lead to inequity of access to the benefits of chatbots, including ChatGPT, as individuals who lack the necessary expertise may not be able to harness their full potential, resulting in disparities in educational outcomes and opportunities. Therefore, immediate action is needed to address these potential issues. A potential solution is offering training, support, and competency development for teachers to ensure that all of them can leverage chatbots, including ChatGPT, effectively and equitably in their educational practices [ 5 , 28 , 80 ], which could enhance accessibility and inclusivity, and potentially result in innovative outcomes [ 82 , 83 ].

Additionally, chatbots, including ChatGPT, have the potential to significantly impact students' thinking abilities, including retention, reasoning, analysis skills [ 19 , 45 ], and foster innovation and creativity capabilities [ 83 ]. This review found that ChatGPT could contribute to improving a wide range of skills among students. However, it found that frequent use of ChatGPT may result in a decrease in innovative capacities, collaborative skills and cognitive capacities, and students' motivation to attend classes, as well as could lead to reduced higher-order thinking skills among students [ 22 , 29 ]. Therefore, immediate action is needed to carefully examine the long-term impact of chatbots such as ChatGPT, on learning outcomes as well as to explore its incorporation into educational settings as a supportive tool without compromising students' cognitive development and critical thinking abilities. In the same vein, the review also found that it is challenging to draw a consistent conclusion regarding the potential of ChatGPT to aid self-directed learning approach. This finding aligns with the recent study of Baskara [ 8 ]. Therefore, further research is needed to explore the potential of ChatGPT for self-directed learning. One potential solution involves utilizing learning analytics as a novel approach to examine various aspects of students' learning and support them in their individual endeavors [ 32 ]. This approach can bridge this gap by facilitating an in-depth analysis of how learners engage with ChatGPT, identifying trends in self-directed learning behavior, and assessing its influence on their outcomes.

Turning to the significant concerns, on the other hand, a fundamental challenge with LLM-based chatbots, including ChatGPT, is the accuracy and quality of the provided information and responses, as they provide false information as truth—a phenomenon often referred to as "hallucination" [ 3 , 49 ]. In this context, this review found that the provided information was not entirely satisfactory. Consequently, the utilization of chatbots presents potential concerns, such as generating and providing inaccurate or misleading information, especially for students who utilize it to support their learning. This finding aligns with other findings [ 6 , 30 , 35 , 40 ] which revealed that incorporating chatbots such as ChatGPT, into education presents challenges related to its accuracy and reliability due to its training on a large corpus of data, which may contain inaccuracies and the way users formulate or ask ChatGPT. Therefore, immediate action is needed to address these potential issues. One possible solution is to equip students with the necessary skills and competencies, which include a background understanding of how to use it effectively and the ability to assess and evaluate the information it generates, as the accuracy and the quality of the provided information depend on the input, its complexity, the topic, and the relevance of its training data [ 28 , 49 , 86 ]. However, it's also essential to examine how learners can be educated about how these models operate, the data used in their training, and how to recognize their limitations, challenges, and issues [ 79 ].

Furthermore, chatbots present a substantial challenge concerning maintaining academic integrity [ 20 , 56 ] and copyright violations [ 83 ], which are significant concerns in education. The review found that the potential misuse of ChatGPT might foster cheating, facilitate plagiarism, and threaten academic integrity. This issue is also affirmed by the research conducted by Basic et al. [ 7 ], who presented evidence that students who utilized ChatGPT in their writing assignments had more plagiarism cases than those who did not. These findings align with the conclusions drawn by Cotton et al. [ 13 ], Hisan and Amri [ 33 ] and Sullivan et al. [ 75 ], who revealed that the integration of chatbots such as ChatGPT into education poses a significant challenge to the preservation of academic integrity. Moreover, chatbots, including ChatGPT, have increased the difficulty in identifying plagiarism [ 47 , 67 , 76 ]. The findings from previous studies [ 1 , 84 ] indicate that AI-generated text often went undetected by plagiarism software, such as Turnitin. However, Turnitin and other similar plagiarism detection tools, such as ZeroGPT, GPTZero, and Copyleaks, have since evolved, incorporating enhanced techniques to detect AI-generated text, despite the possibility of false positives, as noted in different studies that have found these tools still not yet fully ready to accurately and reliably identify AI-generated text [ 10 , 51 ], and new novel detection methods may need to be created and implemented for AI-generated text detection [ 4 ]. This potential issue could lead to another concern, which is the difficulty of accurately evaluating student performance when they utilize chatbots such as ChatGPT assistance in their assignments. Consequently, the most LLM-driven chatbots present a substantial challenge to traditional assessments [ 64 ]. The findings from previous studies indicate the importance of rethinking, improving, and redesigning innovative assessment methods in the era of chatbots [ 14 , 20 , 64 , 75 ]. These methods should prioritize the process of evaluating students' ability to apply knowledge to complex cases and demonstrate comprehension, rather than solely focusing on the final product for assessment. Therefore, immediate action is needed to address these potential issues. One possible solution would be the development of clear guidelines, regulatory policies, and pedagogical guidance. These measures would help regulate the proper and ethical utilization of chatbots, such as ChatGPT, and must be established before their introduction to students [ 35 , 38 , 39 , 41 , 89 ].

In summary, our review has delved into the utilization of ChatGPT, a prominent example of chatbots, in education, addressing the question of how ChatGPT has been utilized in education. However, there remain significant gaps, which necessitate further research to shed light on this area.

7 Conclusions

This systematic review has shed light on the varied initial attempts at incorporating ChatGPT into education by both learners and educators, while also offering insights and considerations that can facilitate its effective and responsible use in future educational contexts. From the analysis of 14 selected studies, the review revealed the dual-edged impact of ChatGPT in educational settings. On the positive side, ChatGPT significantly aided the learning process in various ways. Learners have used it as a virtual intelligent assistant, benefiting from its ability to provide immediate feedback, on-demand answers, and easy access to educational resources. Additionally, it was clear that learners have used it to enhance their writing and language skills, engaging in practices such as generating ideas, composing essays, and performing tasks like summarizing, translating, paraphrasing texts, or checking grammar. Importantly, other learners have utilized it in supporting and facilitating their directed and personalized learning on a broad range of educational topics, assisting in understanding concepts and homework, providing structured learning plans, and clarifying assignments and tasks. Educators, on the other hand, found ChatGPT beneficial for enhancing productivity and efficiency. They used it for creating lesson plans, generating quizzes, providing additional resources, and answers learners' questions, which saved time and allowed for more dynamic and engaging teaching strategies and methodologies.

However, the review also pointed out negative impacts. The results revealed that overuse of ChatGPT could decrease innovative capacities and collaborative learning among learners. Specifically, relying too much on ChatGPT for quick answers can inhibit learners' critical thinking and problem-solving skills. Learners might not engage deeply with the material or consider multiple solutions to a problem. This tendency was particularly evident in group projects, where learners preferred consulting ChatGPT individually for solutions over brainstorming and collaborating with peers, which negatively affected their teamwork abilities. On a broader level, integrating ChatGPT into education has also raised several concerns, including the potential for providing inaccurate or misleading information, issues of inequity in access, challenges related to academic integrity, and the possibility of misusing the technology.

Accordingly, this review emphasizes the urgency of developing clear rules, policies, and regulations to ensure ChatGPT's effective and responsible use in educational settings, alongside other chatbots, by both learners and educators. This requires providing well-structured training to educate them on responsible usage and understanding its limitations, along with offering sufficient background information. Moreover, it highlights the importance of rethinking, improving, and redesigning innovative teaching and assessment methods in the era of ChatGPT. Furthermore, conducting further research and engaging in discussions with policymakers and stakeholders are essential steps to maximize the benefits for both educators and learners and ensure academic integrity.

It is important to acknowledge that this review has certain limitations. Firstly, the limited inclusion of reviewed studies can be attributed to several reasons, including the novelty of the technology, as new technologies often face initial skepticism and cautious adoption; the lack of clear guidelines or best practices for leveraging this technology for educational purposes; and institutional or governmental policies affecting the utilization of this technology in educational contexts. These factors, in turn, have affected the number of studies available for review. Secondly, the utilization of the original version of ChatGPT, based on GPT-3 or GPT-3.5, implies that new studies utilizing the updated version, GPT-4 may lead to different findings. Therefore, conducting follow-up systematic reviews is essential once more empirical studies on ChatGPT are published. Additionally, long-term studies are necessary to thoroughly examine and assess the impact of ChatGPT on various educational practices.

Despite these limitations, this systematic review has highlighted the transformative potential of ChatGPT in education, revealing its diverse utilization by learners and educators alike and summarized the benefits of incorporating it into education, as well as the forefront critical concerns and challenges that must be addressed to facilitate its effective and responsible use in future educational contexts. This review could serve as an insightful resource for practitioners who seek to integrate ChatGPT into education and stimulate further research in the field.

Data availability

The data supporting our findings are available upon request.

Abbreviations

• Artificial intelligence

AI in education

Large language model

Artificial neural networks

Chat Generative Pre-Trained Transformer

Recurrent neural networks

Long short-term memory

Reinforcement learning from human feedback

Natural language processing

Preferred Reporting Items for Systematic Reviews and Meta-Analyses

AlAfnan MA, Dishari S, Jovic M, Lomidze K. ChatGPT as an educational tool: opportunities, challenges, and recommendations for communication, business writing, and composition courses. J Artif Intell Technol. 2023. https://doi.org/10.37965/jait.2023.0184 .

Article   Google Scholar  

Ali JKM, Shamsan MAA, Hezam TA, Mohammed AAQ. Impact of ChatGPT on learning motivation. J Engl Stud Arabia Felix. 2023;2(1):41–9. https://doi.org/10.56540/jesaf.v2i1.51 .

Alkaissi H, McFarlane SI. Artificial hallucinations in ChatGPT: implications in scientific writing. Cureus. 2023. https://doi.org/10.7759/cureus.35179 .

Anderson N, Belavý DL, Perle SM, Hendricks S, Hespanhol L, Verhagen E, Memon AR. AI did not write this manuscript, or did it? Can we trick the AI text detector into generated texts? The potential future of ChatGPT and AI in sports & exercise medicine manuscript generation. BMJ Open Sport Exerc Med. 2023;9(1): e001568. https://doi.org/10.1136/bmjsem-2023-001568 .

Ausat AMA, Massang B, Efendi M, Nofirman N, Riady Y. Can chat GPT replace the role of the teacher in the classroom: a fundamental analysis. J Educ. 2023;5(4):16100–6.

Google Scholar  

Baidoo-Anu D, Ansah L. Education in the Era of generative artificial intelligence (AI): understanding the potential benefits of ChatGPT in promoting teaching and learning. Soc Sci Res Netw. 2023. https://doi.org/10.2139/ssrn.4337484 .

Basic Z, Banovac A, Kruzic I, Jerkovic I. Better by you, better than me, chatgpt3 as writing assistance in students essays. 2023. arXiv preprint arXiv:2302.04536 .‏

Baskara FR. The promises and pitfalls of using chat GPT for self-determined learning in higher education: an argumentative review. Prosiding Seminar Nasional Fakultas Tarbiyah dan Ilmu Keguruan IAIM Sinjai. 2023;2:95–101. https://doi.org/10.47435/sentikjar.v2i0.1825 .

Behera RK, Bala PK, Dhir A. The emerging role of cognitive computing in healthcare: a systematic literature review. Int J Med Inform. 2019;129:154–66. https://doi.org/10.1016/j.ijmedinf.2019.04.024 .

Chaka C. Detecting AI content in responses generated by ChatGPT, YouChat, and Chatsonic: the case of five AI content detection tools. J Appl Learn Teach. 2023. https://doi.org/10.37074/jalt.2023.6.2.12 .

Chiu TKF, Xia Q, Zhou X, Chai CS, Cheng M. Systematic literature review on opportunities, challenges, and future research recommendations of artificial intelligence in education. Comput Educ Artif Intell. 2023;4:100118. https://doi.org/10.1016/j.caeai.2022.100118 .

Choi EPH, Lee JJ, Ho M, Kwok JYY, Lok KYW. Chatting or cheating? The impacts of ChatGPT and other artificial intelligence language models on nurse education. Nurse Educ Today. 2023;125:105796. https://doi.org/10.1016/j.nedt.2023.105796 .

Cotton D, Cotton PA, Shipway JR. Chatting and cheating: ensuring academic integrity in the era of ChatGPT. Innov Educ Teach Int. 2023. https://doi.org/10.1080/14703297.2023.2190148 .

Crawford J, Cowling M, Allen K. Leadership is needed for ethical ChatGPT: Character, assessment, and learning using artificial intelligence (AI). J Univ Teach Learn Pract. 2023. https://doi.org/10.53761/1.20.3.02 .

Creswell JW. Educational research: planning, conducting, and evaluating quantitative and qualitative research [Ebook]. 4th ed. London: Pearson Education; 2015.

Curry D. ChatGPT Revenue and Usage Statistics (2023)—Business of Apps. 2023. https://www.businessofapps.com/data/chatgpt-statistics/

Day T. A preliminary investigation of fake peer-reviewed citations and references generated by ChatGPT. Prof Geogr. 2023. https://doi.org/10.1080/00330124.2023.2190373 .

De Castro CA. A Discussion about the Impact of ChatGPT in education: benefits and concerns. J Bus Theor Pract. 2023;11(2):p28. https://doi.org/10.22158/jbtp.v11n2p28 .

Deng X, Yu Z. A meta-analysis and systematic review of the effect of Chatbot technology use in sustainable education. Sustainability. 2023;15(4):2940. https://doi.org/10.3390/su15042940 .

Eke DO. ChatGPT and the rise of generative AI: threat to academic integrity? J Responsib Technol. 2023;13:100060. https://doi.org/10.1016/j.jrt.2023.100060 .

Elmoazen R, Saqr M, Tedre M, Hirsto L. A systematic literature review of empirical research on epistemic network analysis in education. IEEE Access. 2022;10:17330–48. https://doi.org/10.1109/access.2022.3149812 .

Farrokhnia M, Banihashem SK, Noroozi O, Wals AEJ. A SWOT analysis of ChatGPT: implications for educational practice and research. Innov Educ Teach Int. 2023. https://doi.org/10.1080/14703297.2023.2195846 .

Fergus S, Botha M, Ostovar M. Evaluating academic answers generated using ChatGPT. J Chem Educ. 2023;100(4):1672–5. https://doi.org/10.1021/acs.jchemed.3c00087 .

Fink A. Conducting research literature reviews: from the Internet to Paper. Incorporated: SAGE Publications; 2010.

Firaina R, Sulisworo D. Exploring the usage of ChatGPT in higher education: frequency and impact on productivity. Buletin Edukasi Indonesia (BEI). 2023;2(01):39–46. https://doi.org/10.56741/bei.v2i01.310 .

Firat, M. (2023). How chat GPT can transform autodidactic experiences and open education.  Department of Distance Education, Open Education Faculty, Anadolu Unive .‏ https://orcid.org/0000-0001-8707-5918

Firat M. What ChatGPT means for universities: perceptions of scholars and students. J Appl Learn Teach. 2023. https://doi.org/10.37074/jalt.2023.6.1.22 .

Fuchs K. Exploring the opportunities and challenges of NLP models in higher education: is Chat GPT a blessing or a curse? Front Educ. 2023. https://doi.org/10.3389/feduc.2023.1166682 .

García-Peñalvo FJ. La percepción de la inteligencia artificial en contextos educativos tras el lanzamiento de ChatGPT: disrupción o pánico. Educ Knowl Soc. 2023;24: e31279. https://doi.org/10.14201/eks.31279 .

Gilson A, Safranek CW, Huang T, Socrates V, Chi L, Taylor A, Chartash D. How does ChatGPT perform on the United States medical Licensing examination? The implications of large language models for medical education and knowledge assessment. JMIR Med Educ. 2023;9: e45312. https://doi.org/10.2196/45312 .

Hashana AJ, Brundha P, Ayoobkhan MUA, Fazila S. Deep Learning in ChatGPT—A Survey. In   2023 7th international conference on trends in electronics and informatics (ICOEI) . 2023. (pp. 1001–1005). IEEE. https://doi.org/10.1109/icoei56765.2023.10125852

Hirsto L, Saqr M, López-Pernas S, Valtonen T. (2022). A systematic narrative review of learning analytics research in K-12 and schools.  Proceedings . https://ceur-ws.org/Vol-3383/FLAIEC22_paper_9536.pdf

Hisan UK, Amri MM. ChatGPT and medical education: a double-edged sword. J Pedag Educ Sci. 2023;2(01):71–89. https://doi.org/10.13140/RG.2.2.31280.23043/1 .

Hopkins AM, Logan JM, Kichenadasse G, Sorich MJ. Artificial intelligence chatbots will revolutionize how cancer patients access information: ChatGPT represents a paradigm-shift. JNCI Cancer Spectr. 2023. https://doi.org/10.1093/jncics/pkad010 .

Househ M, AlSaad R, Alhuwail D, Ahmed A, Healy MG, Latifi S, Sheikh J. Large Language models in medical education: opportunities, challenges, and future directions. JMIR Med Educ. 2023;9: e48291. https://doi.org/10.2196/48291 .

Ilkka T. The impact of artificial intelligence on learning, teaching, and education. Minist de Educ. 2018. https://doi.org/10.2760/12297 .

Iqbal N, Ahmed H, Azhar KA. Exploring teachers’ attitudes towards using CHATGPT. Globa J Manag Adm Sci. 2022;3(4):97–111. https://doi.org/10.46568/gjmas.v3i4.163 .

Irfan M, Murray L, Ali S. Integration of Artificial intelligence in academia: a case study of critical teaching and learning in Higher education. Globa Soc Sci Rev. 2023;8(1):352–64. https://doi.org/10.31703/gssr.2023(viii-i).32 .

Jeon JH, Lee S. Large language models in education: a focus on the complementary relationship between human teachers and ChatGPT. Educ Inf Technol. 2023. https://doi.org/10.1007/s10639-023-11834-1 .

Khan RA, Jawaid M, Khan AR, Sajjad M. ChatGPT—Reshaping medical education and clinical management. Pak J Med Sci. 2023. https://doi.org/10.12669/pjms.39.2.7653 .

King MR. A conversation on artificial intelligence, Chatbots, and plagiarism in higher education. Cell Mol Bioeng. 2023;16(1):1–2. https://doi.org/10.1007/s12195-022-00754-8 .

Kooli C. Chatbots in education and research: a critical examination of ethical implications and solutions. Sustainability. 2023;15(7):5614. https://doi.org/10.3390/su15075614 .

Kuhail MA, Alturki N, Alramlawi S, Alhejori K. Interacting with educational chatbots: a systematic review. Educ Inf Technol. 2022;28(1):973–1018. https://doi.org/10.1007/s10639-022-11177-3 .

Lee H. The rise of ChatGPT: exploring its potential in medical education. Anat Sci Educ. 2023. https://doi.org/10.1002/ase.2270 .

Li L, Subbareddy R, Raghavendra CG. AI intelligence Chatbot to improve students learning in the higher education platform. J Interconnect Netw. 2022. https://doi.org/10.1142/s0219265921430325 .

Limna P. A Review of Artificial Intelligence (AI) in Education during the Digital Era. 2022. https://ssrn.com/abstract=4160798

Lo CK. What is the impact of ChatGPT on education? A rapid review of the literature. Educ Sci. 2023;13(4):410. https://doi.org/10.3390/educsci13040410 .

Luo W, He H, Liu J, Berson IR, Berson MJ, Zhou Y, Li H. Aladdin’s genie or pandora’s box For early childhood education? Experts chat on the roles, challenges, and developments of ChatGPT. Early Educ Dev. 2023. https://doi.org/10.1080/10409289.2023.2214181 .

Meyer JG, Urbanowicz RJ, Martin P, O’Connor K, Li R, Peng P, Moore JH. ChatGPT and large language models in academia: opportunities and challenges. Biodata Min. 2023. https://doi.org/10.1186/s13040-023-00339-9 .

Mhlanga D. Open AI in education, the responsible and ethical use of ChatGPT towards lifelong learning. Soc Sci Res Netw. 2023. https://doi.org/10.2139/ssrn.4354422 .

Neumann, M., Rauschenberger, M., & Schön, E. M. (2023). “We Need To Talk About ChatGPT”: The Future of AI and Higher Education.‏ https://doi.org/10.1109/seeng59157.2023.00010

Nolan B. Here are the schools and colleges that have banned the use of ChatGPT over plagiarism and misinformation fears. Business Insider . 2023. https://www.businessinsider.com

O’Leary DE. An analysis of three chatbots: BlenderBot, ChatGPT and LaMDA. Int J Intell Syst Account, Financ Manag. 2023;30(1):41–54. https://doi.org/10.1002/isaf.1531 .

Okoli C. A guide to conducting a standalone systematic literature review. Commun Assoc Inf Syst. 2015. https://doi.org/10.17705/1cais.03743 .

OpenAI. (2023). https://openai.com/blog/chatgpt

Perkins M. Academic integrity considerations of AI large language models in the post-pandemic era: ChatGPT and beyond. J Univ Teach Learn Pract. 2023. https://doi.org/10.53761/1.20.02.07 .

Plevris V, Papazafeiropoulos G, Rios AJ. Chatbots put to the test in math and logic problems: A preliminary comparison and assessment of ChatGPT-3.5, ChatGPT-4, and Google Bard. arXiv (Cornell University) . 2023. https://doi.org/10.48550/arxiv.2305.18618

Rahman MM, Watanobe Y (2023) ChatGPT for education and research: opportunities, threats, and strategies. Appl Sci 13(9):5783. https://doi.org/10.3390/app13095783

Ram B, Verma P. Artificial intelligence AI-based Chatbot study of ChatGPT, google AI bard and baidu AI. World J Adv Eng Technol Sci. 2023;8(1):258–61. https://doi.org/10.30574/wjaets.2023.8.1.0045 .

Rasul T, Nair S, Kalendra D, Robin M, de Oliveira Santini F, Ladeira WJ, Heathcote L. The role of ChatGPT in higher education: benefits, challenges, and future research directions. J Appl Learn Teach. 2023. https://doi.org/10.37074/jalt.2023.6.1.29 .

Ratnam M, Sharm B, Tomer A. ChatGPT: educational artificial intelligence. Int J Adv Trends Comput Sci Eng. 2023;12(2):84–91. https://doi.org/10.30534/ijatcse/2023/091222023 .

Ray PP. ChatGPT: a comprehensive review on background, applications, key challenges, bias, ethics, limitations and future scope. Internet Things Cyber-Phys Syst. 2023;3:121–54. https://doi.org/10.1016/j.iotcps.2023.04.003 .

Roumeliotis KI, Tselikas ND. ChatGPT and Open-AI models: a preliminary review. Future Internet. 2023;15(6):192. https://doi.org/10.3390/fi15060192 .

Rudolph J, Tan S, Tan S. War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education. J Appl Learn Teach. 2023. https://doi.org/10.37074/jalt.2023.6.1.23 .

Ruiz LMS, Moll-López S, Nuñez-Pérez A, Moraño J, Vega-Fleitas E. ChatGPT challenges blended learning methodologies in engineering education: a case study in mathematics. Appl Sci. 2023;13(10):6039. https://doi.org/10.3390/app13106039 .

Sallam M, Salim NA, Barakat M, Al-Tammemi AB. ChatGPT applications in medical, dental, pharmacy, and public health education: a descriptive study highlighting the advantages and limitations. Narra J. 2023;3(1): e103. https://doi.org/10.52225/narra.v3i1.103 .

Salvagno M, Taccone FS, Gerli AG. Can artificial intelligence help for scientific writing? Crit Care. 2023. https://doi.org/10.1186/s13054-023-04380-2 .

Saqr M, López-Pernas S, Helske S, Hrastinski S. The longitudinal association between engagement and achievement varies by time, students’ profiles, and achievement state: a full program study. Comput Educ. 2023;199:104787. https://doi.org/10.1016/j.compedu.2023.104787 .

Saqr M, Matcha W, Uzir N, Jovanović J, Gašević D, López-Pernas S. Transferring effective learning strategies across learning contexts matters: a study in problem-based learning. Australas J Educ Technol. 2023;39(3):9.

Schöbel S, Schmitt A, Benner D, Saqr M, Janson A, Leimeister JM. Charting the evolution and future of conversational agents: a research agenda along five waves and new frontiers. Inf Syst Front. 2023. https://doi.org/10.1007/s10796-023-10375-9 .

Shoufan A. Exploring students’ perceptions of CHATGPT: thematic analysis and follow-up survey. IEEE Access. 2023. https://doi.org/10.1109/access.2023.3268224 .

Sonderegger S, Seufert S. Chatbot-mediated learning: conceptual framework for the design of Chatbot use cases in education. Gallen: Institute for Educational Management and Technologies, University of St; 2022. https://doi.org/10.5220/0010999200003182 .

Book   Google Scholar  

Strzelecki A. To use or not to use ChatGPT in higher education? A study of students’ acceptance and use of technology. Interact Learn Environ. 2023. https://doi.org/10.1080/10494820.2023.2209881 .

Su J, Yang W. Unlocking the power of ChatGPT: a framework for applying generative AI in education. ECNU Rev Educ. 2023. https://doi.org/10.1177/20965311231168423 .

Sullivan M, Kelly A, McLaughlan P. ChatGPT in higher education: Considerations for academic integrity and student learning. J ApplLearn Teach. 2023;6(1):1–10. https://doi.org/10.37074/jalt.2023.6.1.17 .

Szabo A. ChatGPT is a breakthrough in science and education but fails a test in sports and exercise psychology. Balt J Sport Health Sci. 2023;1(128):25–40. https://doi.org/10.33607/bjshs.v127i4.1233 .

Taecharungroj V. “What can ChatGPT do?” analyzing early reactions to the innovative AI chatbot on Twitter. Big Data Cognit Comput. 2023;7(1):35. https://doi.org/10.3390/bdcc7010035 .

Tam S, Said RB. User preferences for ChatGPT-powered conversational interfaces versus traditional methods. Biomed Eng Soc. 2023. https://doi.org/10.58496/mjcsc/2023/004 .

Tedre M, Kahila J, Vartiainen H. (2023). Exploration on how co-designing with AI facilitates critical evaluation of ethics of AI in craft education. In: Langran E, Christensen P, Sanson J (Eds).  Proceedings of Society for Information Technology and Teacher Education International Conference . 2023. pp. 2289–2296.

Tlili A, Shehata B, Adarkwah MA, Bozkurt A, Hickey DT, Huang R, Agyemang B. What if the devil is my guardian angel: ChatGPT as a case study of using chatbots in education. Smart Learn Environ. 2023. https://doi.org/10.1186/s40561-023-00237-x .

Uddin SMJ, Albert A, Ovid A, Alsharef A. Leveraging CHATGPT to aid construction hazard recognition and support safety education and training. Sustainability. 2023;15(9):7121. https://doi.org/10.3390/su15097121 .

Valtonen T, López-Pernas S, Saqr M, Vartiainen H, Sointu E, Tedre M. The nature and building blocks of educational technology research. Comput Hum Behav. 2022;128:107123. https://doi.org/10.1016/j.chb.2021.107123 .

Vartiainen H, Tedre M. Using artificial intelligence in craft education: crafting with text-to-image generative models. Digit Creat. 2023;34(1):1–21. https://doi.org/10.1080/14626268.2023.2174557 .

Ventayen RJM. OpenAI ChatGPT generated results: similarity index of artificial intelligence-based contents. Soc Sci Res Netw. 2023. https://doi.org/10.2139/ssrn.4332664 .

Wagner MW, Ertl-Wagner BB. Accuracy of information and references using ChatGPT-3 for retrieval of clinical radiological information. Can Assoc Radiol J. 2023. https://doi.org/10.1177/08465371231171125 .

Wardat Y, Tashtoush MA, AlAli R, Jarrah AM. ChatGPT: a revolutionary tool for teaching and learning mathematics. Eurasia J Math, Sci Technol Educ. 2023;19(7):em2286. https://doi.org/10.29333/ejmste/13272 .

Webster J, Watson RT. Analyzing the past to prepare for the future: writing a literature review. Manag Inf Syst Quart. 2002;26(2):3.

Xiao Y, Watson ME. Guidance on conducting a systematic literature review. J Plan Educ Res. 2017;39(1):93–112. https://doi.org/10.1177/0739456x17723971 .

Yan D. Impact of ChatGPT on learners in a L2 writing practicum: an exploratory investigation. Educ Inf Technol. 2023. https://doi.org/10.1007/s10639-023-11742-4 .

Yu H. Reflection on whether Chat GPT should be banned by academia from the perspective of education and teaching. Front Psychol. 2023;14:1181712. https://doi.org/10.3389/fpsyg.2023.1181712 .

Zhu C, Sun M, Luo J, Li T, Wang M. How to harness the potential of ChatGPT in education? Knowl Manag ELearn. 2023;15(2):133–52. https://doi.org/10.34105/j.kmel.2023.15.008 .

Download references

The paper is co-funded by the Academy of Finland (Suomen Akatemia) Research Council for Natural Sciences and Engineering for the project Towards precision education: Idiographic learning analytics (TOPEILA), Decision Number 350560.

Author information

Authors and affiliations.

School of Computing, University of Eastern Finland, 80100, Joensuu, Finland

Yazid Albadarin, Mohammed Saqr, Nicolas Pope & Markku Tukiainen

You can also search for this author in PubMed   Google Scholar

Contributions

YA contributed to the literature search, data analysis, discussion, and conclusion. Additionally, YA contributed to the manuscript’s writing, editing, and finalization. MS contributed to the study’s design, conceptualization, acquisition of funding, project administration, allocation of resources, supervision, validation, literature search, and analysis of results. Furthermore, MS contributed to the manuscript's writing, revising, and approving it in its finalized state. NP contributed to the results, and discussions, and provided supervision. NP also contributed to the writing process, revisions, and the final approval of the manuscript in its finalized state. MT contributed to the study's conceptualization, resource management, supervision, writing, revising the manuscript, and approving it.

Corresponding author

Correspondence to Yazid Albadarin .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

See Table  4

The process of synthesizing the data presented in Table  4 involved identifying the relevant studies through a search process of databases (ERIC, Scopus, Web of Knowledge, Dimensions.ai, and lens.org) using specific keywords "ChatGPT" and "education". Following this, inclusion/exclusion criteria were applied, and data extraction was performed using Creswell's [ 15 ] coding techniques to capture key information and identify common themes across the included studies.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Albadarin, Y., Saqr, M., Pope, N. et al. A systematic literature review of empirical research on ChatGPT in education. Discov Educ 3 , 60 (2024). https://doi.org/10.1007/s44217-024-00138-2

Download citation

Received : 22 October 2023

Accepted : 10 May 2024

Published : 26 May 2024

DOI : https://doi.org/10.1007/s44217-024-00138-2

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Large language models
• Educational technology
• Systematic review

Advertisement

• Find a journal
• Publish with us
• Track your research

Digital Transformation & Innovation in Auditing: Insights from a Review of Academic Research

In 2021, the International Auditing and Assurance Standards Board (IAASB) asked members of the International Association for Accounting Education and Research (IAAER) to conduct a literature review examining digital transformation in the external audit setting. The review was intended to inform the IAASB’s standard-setting initiatives related to using technology in audit engagements.

In January 2022, a paper titled “ The Effects of Person-Specific, Task, and Environmental Factors on Digital Transformation and Innovation in Auditing: A Review of the Literature ,” prepared by Dereck Barr-Pulliam, Helen Brown-Liburd and Ivy Munoko, was published detailing the findings from this study. This article sets out some of the insights the IAASB gained from reviewing this research and discussing it with the paper’s authors.

Using technology in an audit continues to evolve and, by examining relevant literature published over the last 20 years, insights can be learned about evolving trends and the trajectory of digital transformation in audit.

The paper’s authors were among the first to conduct an extensive review of the growing academic literature on digital transformation in the external audit arena. The study identified an increasing interest in publishing digital transformation-related research, as demonstrated by the increase in volume of research over recent years, but indicated that research on external auditors’ use of emerging technologies is still at an early stage. This latter point could reflect the fact that many emerging technologies are yet to achieve widespread adoption due to their complexity of implementation and use.

The research identified person, task and environmental factors which affected digital transformation in audit engagements and distinguished between the types of analytics used by auditors—descriptive, diagnostic, predictive and prescriptive—and the research findings relevant to each. The findings suggest very few studies have examined the more complex predictive and prescriptive analytics.

The diagram below identifies factors that positively influence adoption of the digital audit; these themes are explored in more detail throughout this article. 

Person related factors impact enthusiasm and willingness to adopt technology in the audit

The research identified some key person-specific factors influencing the adoption of technology. When discussed with the original paper’s authors, this was highlighted as the most significant reason for a lag in technology adoption.

1)     Training and skills

The study noted that accounting curricula lags accounting practice due to the complexities in augmenting university course content and that missing components include an understanding of the information lifecycle and the technologies of the information system. The study also notes a need for greater emphasis on helping students become more agile and use more critical thinking when interacting with disruptive technology, and some studies identified possible approaches to address including use of case studies. Tool-specific training was noted as a critical driver for using technology with the timeliness of this training (i.e., before busy season) being a way to encourage greater use.

2)     Auditor Characteristics and Behavior

The review of research identified certain auditor behaviors that foster greater commitment to technology adoption with growth mindset, agility and critical thinking recognized as behaviors that positively influence the reliance on and support for technology in the audit. Research into whether mindset (fixed or growth) moderates the effect of inspection risk on auditors’ reliance on data analytics tools found that when inspection risk is high, those with a fixed mindset rely less on data analytic tools than those with growth mindset. This finding in conjunction with others related to concerns about regulators’ response to and acceptance of emerging technologies provides valuable insight into key barriers to technology adoption.

Some research noted the impact of algorithm aversion, which suggests individuals will be more likely to discount computer-generated advice or evidence more heavily than human advice or evidence. One example of this is a study that found that auditors who received contradictory evidence from an artificial intelligence system (for example, where used to evaluate a complex estimate) proposed smaller adjustments to management’s initial estimate, mainly when the underlying inputs and assumptions are objective.

3)     Stakeholder/external attitudes

The review identified several publications that presented research performed to understand perceptions of and behavioral responses to using analytics in the audit. Findings suggest that stakeholder views (including peer reviewers and regulators) influence auditors’ willingness to adopt technology.

Whilst a primary benefit of data analytics is increased audit quality, some research indicated that peer reviewers, external reviewers and key stakeholders viewed quality as largely unaffected by using data analytic techniques as an alternative to traditional audit procedures.

It is clear from the research that confidence in using automated tools and techniques by auditors and various stakeholders in audit outcomes is key to enabling increased adoption of technology on engagements.

Task related factors such as structure and complexity impact technology adoption

The research identified variations in audit task complexity and noted the importance of understanding how using emerging technology in the audit interacts with task complexity to impact judgement quality.

1)     Task complexity

Descriptive analytics were noted as most widely used of all the advanced analytics types, particularly data visualization—which is used to better understand an entity’s financial performance and for population testing, as well as for business insights. Research indicates that when data visualization is appropriately integrated into audit tasks it can improve decision making. However, as the data becomes more voluminous and the analytic more complex, there are challenges for the auditor in understanding and interpreting this data and making appropriate judgements regarding treatment of anomalies.

Studies of auditors’ use of diagnostic analytics indicates task complexity moderates the effectiveness of technology used in the audit, particularly when it gives rise to a high number of anomalies—potentially significantly more than would require investigation in a traditional sample test.

Unstructured tasks such as the use of advanced data analytic techniques, like clustering to identify patterns in data that could signal higher risk areas, may increase complexity because the auditor must process a higher number of information cues (i.e., larger data sets), combine the information in an unspecified way (e.g., identify patterns) or adapt to changes in required actions or information cues (i.e., identify higher risk areas).

As the technology being deployed becomes more complex, there is a risk that auditors experience information processing and cognitive limitations (e.g., information overload) when analyzing and interpreting output from data analytic tools. A decision aid, framework, or an accepted systematic approach can help with practical challenges faced when potentially large numbers of outliers result from full population testing. Research identified that higher levels of false positives associated with data analytics can also negatively influence the extent to which auditors exhibit professional skepticism. However, it was noted that this can be mitigated by consistently rewarding auditors for exhibiting appropriate skepticism.

2)     Examples of technology driving audit quality improvements and audit efficiency

Despite challenges around task complexity, several publications reviewed as part of the study identified examples of automated tools and techniques that could positively impact audit quality, as well as potentially improving the audit experience.

• Exogenous Data – some research looked at the use of exogenous data combined with company data to gain deeper insights. Findings indicated benefits of using this data but stressed the importance of carefully evaluating how the exogenous data linked to financial accounts.
• Benchmarking – research noted the use of appropriate benchmarking and incorporation of relevant information can improve auditors’ performance of analytical procedures.
• Machine learning – research identified benefits in using machine learning to develop independent estimates to compare to management’s estimates with studies showing that these are generally more accurate and benefit from the model being retrained each year using the actual figures.
• Contract analysis – research identified various AI-enabled techniques used in the audit, such as natural language processing to analyze contracts for unusual terms or clauses enabling a more efficient and effective approach to examining full populations of contracts and related audit tasks.
• Automation – the use of robotic process automation (RPA) technologies to automate routine, repetitive tasks to improve audit efficiency with some research proposing frameworks to use for development of RPA in an audit practice including determining which activities to automate.
• Drones – one study identified the use of drones to support inventory counts and noted that this had driven significant reductions in inspection time as well as reductions in errors.
• Process mining – research indicated that use of this technology is emerging and found that it improved the evaluation of the effectiveness of internal controls over financial reporting.

Environmental factors influence technology adoption in audit

The study highlighted some environmental factors that influenced the adoption of technology in the audit. These factors include client preferences, competitor activity, regulatory response to technology in the audit as well as regional and global shifts towards digitization. The adoption rate, enthusiasm and expectations of these environmental parties directly impact the audit firm’s use of technologies.

The following environmental factors were noted.

1)     A regional and global shift towards digitization, automation and business intelligence – Regional factors such as government influence, competition of audit firms, regulation, advancement of technology and availability of necessary talent play a significant role in the adoption of technology.

2)     Influence of the audit client on adoption of emerging technologies – Factors such as the client’s expectation of auditor use of emerging technology and client support for data access influences how the auditor can deploy emerging technology and the regularity of use. Client expectations regarding additional insights gleaned from using emerging technology coupled with tensions around anticipated audit fee reduction because of using technology impact adoption. Additionally, an expectation gap may exist regarding the level of assurance attained from testing full populations of transactions or related to the evaluation of non-financial information through technology.

3)     Business drive to achieve/maintain competitive advantage – Emerging technologies provide opportunities to increase audit efficiency and effectiveness, for example, through use of Robotic Process Automation (RPA) to automate routine, repetitive audit tasks. A disparity was noted in emerging technologies and the phase of digital transformation across accounting firms with larger firms having innovation leaders or organizations that help identify, develop, and otherwise facilitate the digital transformation journey whilst smaller firms are more likely to use off-the-shelf tools, placing them at a disadvantage in competing for clients and human capital.

4)     Regulator response to adoption of emerging technologies – Uncertainty about regulators’ response and acceptance of emerging technologies can hinder its adoption. Insights provided through using data analytics may be perceived by regulators as a breach of independence impacting audit quality, with a lack of clarity on regulator response to using technology causing “confusion and frustration.” Findings noting a need for regulators to be more proactive in identifying appropriate use of emerging technology in the audit rather than being reactive through identified findings from inspecting completed engagements.

Environmental factors that support the adoption of technology create the right conditions for successful use. Where these factors work against the adoption of technology in the audit, they give the auditor a greater hill to climb to achieve successful technology adoption.

Of the factors noted, the influence of the audit client on an auditor’s adoption of emerging technology seems to be most significant. This factor is particularly important when it comes to supporting the acquisition of data needed to run the technology and in setting an expectation with the auditor of technology use, whilst the auditor needs to appropriately manage expectations around fees and the level of assurance to be provided (reasonable not absolute) where technology is deployed.

Conclusion and takeaways

The research has provided some valuable insights into digital transformation within audit engagements and delineates person-specific, task, and environmental factors that influence adoption of technology. The research recommends that audit firms and practitioners avoid the temptation to run before they can walk. That is, instead, they take a methodical approach to technology adoption by involving all necessary parties and ensuring there are sufficient resources (human capital and technology) to enable the adoption of specific types of data analytic tools.

The research also advises consideration by standard setters and regulators about whether specific guidance on emerging technologies in the audit may help to allay concerns about adoption of these technologies. For example, in regard to artificial intelligence, to potentially mitigate auditor perception of technological innovation as an addition to traditional audit procedures rather than an enhancement.

Finally, the research concludes that a confluence of positive factors is required to achieve more widespread adoption of the digital transformation. The factors require actions by all stakeholders within the audit and assurance ecosystem. Continued collaboration between academia, audit firms, standard setters and regulators can yield significant insight into adoption of emerging technologies in audit.

Danielle Davies

EY Partner and Former IAASB Staff Fellow

Danielle Davies was an IAASB Staff Fellow from November 2021 to May 2023, on secondment from EY where she is a Partner in the UK Assurance Practice. Danielle is a subject matter expert in audit automated tools and techniques and has wide experience in using technology to aid audit and driving change in the UK audit practice. She is also a member of the UK FRC’s Technology Working Group.

While with the IAASB, Danielle’s focus was on supporting the IAASB’s disruptive technology initiative as well as providing advice and input on other technology related matters.

Follow Danielle on LinkedIn