quantitative research in digital education

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• My Account Login
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Review Article
• Open access
• Published: 12 February 2024

Education reform and change driven by digital technology: a bibliometric study from a global perspective

• Chengliang Wang 1 ,
• Xiaojiao Chen 1 ,
• Teng Yu   ORCID: orcid.org/0000-0001-5198-7261 2 , 3 ,
• Yidan Liu 1 , 4 &
• Yuhui Jing 1  

Humanities and Social Sciences Communications volume  11 , Article number:  256 ( 2024 ) Cite this article

5690 Accesses

1 Citations

1 Altmetric

Metrics details

• Development studies
• Science, technology and society

Amidst the global digital transformation of educational institutions, digital technology has emerged as a significant area of interest among scholars. Such technologies have played an instrumental role in enhancing learner performance and improving the effectiveness of teaching and learning. These digital technologies also ensure the sustainability and stability of education during the epidemic. Despite this, a dearth of systematic reviews exists regarding the current state of digital technology application in education. To address this gap, this study utilized the Web of Science Core Collection as a data source (specifically selecting the high-quality SSCI and SCIE) and implemented a topic search by setting keywords, yielding 1849 initial publications. Furthermore, following the PRISMA guidelines, we refined the selection to 588 high-quality articles. Using software tools such as CiteSpace, VOSviewer, and Charticulator, we reviewed these 588 publications to identify core authors (such as Selwyn, Henderson, Edwards), highly productive countries/regions (England, Australia, USA), key institutions (Monash University, Australian Catholic University), and crucial journals in the field ( Education and Information Technologies , Computers & Education , British Journal of Educational Technology ). Evolutionary analysis reveals four developmental periods in the research field of digital technology education application: the embryonic period, the preliminary development period, the key exploration, and the acceleration period of change. The study highlights the dual influence of technological factors and historical context on the research topic. Technology is a key factor in enabling education to transform and upgrade, and the context of the times is an important driving force in promoting the adoption of new technologies in the education system and the transformation and upgrading of education. Additionally, the study identifies three frontier hotspots in the field: physical education, digital transformation, and professional development under the promotion of digital technology. This study presents a clear framework for digital technology application in education, which can serve as a valuable reference for researchers and educational practitioners concerned with digital technology education application in theory and practice.

Similar content being viewed by others

A bibliometric analysis of knowledge mapping in Chinese education digitalization research from 2012 to 2022

Digital transformation and digital literacy in the context of complexity within higher education institutions: a systematic literature review

Education big data and learning analytics: a bibliometric analysis

Introduction.

Digital technology has become an essential component of modern education, facilitating the extension of temporal and spatial boundaries and enriching the pedagogical contexts (Selwyn and Facer, 2014 ). The advent of mobile communication technology has enabled learning through social media platforms (Szeto et al. 2015 ; Pires et al. 2022 ), while the advancement of augmented reality technology has disrupted traditional conceptions of learning environments and spaces (Perez-Sanagustin et al., 2014 ; Kyza and Georgiou, 2018 ). A wide range of digital technologies has enabled learning to become a norm in various settings, including the workplace (Sjöberg and Holmgren, 2021 ), home (Nazare et al. 2022 ), and online communities (Tang and Lam, 2014 ). Education is no longer limited to fixed locations and schedules, but has permeated all aspects of life, allowing learning to continue at any time and any place (Camilleri and Camilleri, 2016 ; Selwyn and Facer, 2014 ).

The advent of digital technology has led to the creation of several informal learning environments (Greenhow and Lewin, 2015 ) that exhibit divergent form, function, features, and patterns in comparison to conventional learning environments (Nygren et al. 2019 ). Consequently, the associated teaching and learning processes, as well as the strategies for the creation, dissemination, and acquisition of learning resources, have undergone a complete overhaul. The ensuing transformations have posed a myriad of novel issues, such as the optimal structuring of teaching methods by instructors and the adoption of appropriate learning strategies by students in the new digital technology environment. Consequently, an examination of the principles that underpin effective teaching and learning in this environment is a topic of significant interest to numerous scholars engaged in digital technology education research.

Over the course of the last two decades, digital technology has made significant strides in the field of education, notably in extending education time and space and creating novel educational contexts with sustainability. Despite research attempts to consolidate the application of digital technology in education, previous studies have only focused on specific aspects of digital technology, such as Pinto and Leite’s ( 2020 ) investigation into digital technology in higher education and Mustapha et al.’s ( 2021 ) examination of the role and value of digital technology in education during the pandemic. While these studies have provided valuable insights into the practical applications of digital technology in particular educational domains, they have not comprehensively explored the macro-mechanisms and internal logic of digital technology implementation in education. Additionally, these studies were conducted over a relatively brief period, making it challenging to gain a comprehensive understanding of the macro-dynamics and evolutionary process of digital technology in education. Some studies have provided an overview of digital education from an educational perspective but lack a precise understanding of technological advancement and change (Yang et al. 2022 ). Therefore, this study seeks to employ a systematic scientific approach to collate relevant research from 2000 to 2022, comprehend the internal logic and development trends of digital technology in education, and grasp the outstanding contribution of digital technology in promoting the sustainability of education in time and space. In summary, this study aims to address the following questions:

RQ1: Since the turn of the century, what is the productivity distribution of the field of digital technology education application research in terms of authorship, country/region, institutional and journal level?

RQ2: What is the development trend of research on the application of digital technology in education in the past two decades?

RQ3: What are the current frontiers of research on the application of digital technology in education?

Literature review

Although the term “digital technology” has become ubiquitous, a unified definition has yet to be agreed upon by scholars. Because the meaning of the word digital technology is closely related to the specific context. Within the educational research domain, Selwyn’s ( 2016 ) definition is widely favored by scholars (Pinto and Leite, 2020 ). Selwyn ( 2016 ) provides a comprehensive view of various concrete digital technologies and their applications in education through ten specific cases, such as immediate feedback in classes, orchestrating teaching, and community learning. Through these specific application scenarios, Selwyn ( 2016 ) argues that digital technology encompasses technologies associated with digital devices, including but not limited to tablets, smartphones, computers, and social media platforms (such as Facebook and YouTube). Furthermore, Further, the behavior of accessing the internet at any location through portable devices can be taken as an extension of the behavior of applying digital technology.

The evolving nature of digital technology has significant implications in the field of education. In the 1890s, the focus of digital technology in education was on comprehending the nuances of digital space, digital culture, and educational methodologies, with its connotations aligned more towards the idea of e-learning. The advent and subsequent widespread usage of mobile devices since the dawn of the new millennium have been instrumental in the rapid expansion of the concept of digital technology. Notably, mobile learning devices such as smartphones and tablets, along with social media platforms, have become integral components of digital technology (Conole and Alevizou, 2010 ; Batista et al. 2016 ). In recent times, the burgeoning application of AI technology in the education sector has played a vital role in enriching the digital technology lexicon (Banerjee et al. 2021 ). ChatGPT, for instance, is identified as a novel educational technology that has immense potential to revolutionize future education (Rospigliosi, 2023 ; Arif, Munaf and Ul-Haque, 2023 ).

Pinto and Leite ( 2020 ) conducted a comprehensive macroscopic survey of the use of digital technologies in the education sector and identified three distinct categories, namely technologies for assessment and feedback, mobile technologies, and Information Communication Technologies (ICT). This classification criterion is both macroscopic and highly condensed. In light of the established concept definitions of digital technology in the educational research literature, this study has adopted the characterizations of digital technology proposed by Selwyn ( 2016 ) and Pinto and Leite ( 2020 ) as crucial criteria for analysis and research inclusion. Specifically, this criterion encompasses several distinct types of digital technologies, including Information and Communication Technologies (ICT), Mobile tools, eXtended Reality (XR) Technologies, Assessment and Feedback systems, Learning Management Systems (LMS), Publish and Share tools, Collaborative systems, Social media, Interpersonal Communication tools, and Content Aggregation tools.

Methodology and materials

Research method: bibliometric.

The research on econometric properties has been present in various aspects of human production and life, yet systematic scientific theoretical guidance has been lacking, resulting in disorganization. In 1969, British scholar Pritchard ( 1969 ) proposed “bibliometrics,” which subsequently emerged as an independent discipline in scientific quantification research. Initially, Pritchard defined bibliometrics as “the application of mathematical and statistical methods to books and other media of communication,” however, the definition was not entirely rigorous. To remedy this, Hawkins ( 2001 ) expanded Pritchard’s definition to “the quantitative analysis of the bibliographic features of a body of literature.” De Bellis further clarified the objectives of bibliometrics, stating that it aims to analyze and identify patterns in literature, such as the most productive authors, institutions, countries, and journals in scientific disciplines, trends in literary production over time, and collaboration networks (De Bellis, 2009 ). According to Garfield ( 2006 ), bibliometric research enables the examination of the history and structure of a field, the flow of information within the field, the impact of journals, and the citation status of publications over a longer time scale. All of these definitions illustrate the unique role of bibliometrics as a research method for evaluating specific research fields.

This study uses CiteSpace, VOSviewer, and Charticulator to analyze data and create visualizations. Each of these three tools has its own strengths and can complement each other. CiteSpace and VOSviewer use set theory and probability theory to provide various visualization views in fields such as keywords, co-occurrence, and co-authors. They are easy to use and produce visually appealing graphics (Chen, 2006 ; van Eck and Waltman, 2009 ) and are currently the two most widely used bibliometric tools in the field of visualization (Pan et al. 2018 ). In this study, VOSviewer provided the data necessary for the Performance Analysis; Charticulator was then used to redraw using the tabular data exported from VOSviewer (for creating the chord diagram of country collaboration); this was to complement the mapping process, while CiteSpace was primarily utilized to generate keyword maps and conduct burst word analysis.

Data retrieval

This study selected documents from the Science Citation Index Expanded (SCIE) and Social Science Citation Index (SSCI) in the Web of Science Core Collection as the data source, for the following reasons:

(1) The Web of Science Core Collection, as a high-quality digital literature resource database, has been widely accepted by many researchers and is currently considered the most suitable database for bibliometric analysis (Jing et al. 2023a ). Compared to other databases, Web of Science provides more comprehensive data information (Chen et al. 2022a ), and also provides data formats suitable for analysis using VOSviewer and CiteSpace (Gaviria-Marin et al. 2019 ).

(2) The application of digital technology in the field of education is an interdisciplinary research topic, involving technical knowledge literature belonging to the natural sciences and education-related literature belonging to the social sciences. Therefore, it is necessary to select Science Citation Index Expanded (SCIE) and Social Science Citation Index (SSCI) as the sources of research data, ensuring the comprehensiveness of data while ensuring the reliability and persuasiveness of bibliometric research (Hwang and Tsai, 2011 ; Wang et al. 2022 ).

After establishing the source of research data, it is necessary to determine a retrieval strategy (Jing et al. 2023b ). The choice of a retrieval strategy should consider a balance between the breadth and precision of the search formula. That is to say, it should encompass all the literature pertaining to the research topic while excluding irrelevant documents as much as possible. In light of this, this study has set a retrieval strategy informed by multiple related papers (Mustapha et al. 2021 ; Luo et al. 2021 ). The research by Mustapha et al. ( 2021 ) guided us in selecting keywords (“digital” AND “technolog*”) to target digital technology, while Luo et al. ( 2021 ) informed the selection of terms (such as “instruct*,” “teach*,” and “education”) to establish links with the field of education. Then, based on the current application of digital technology in the educational domain and the scope of selection criteria, we constructed the final retrieval strategy. Following the general patterns of past research (Jing et al. 2023a , 2023b ), we conducted a specific screening using the topic search (Topics, TS) function in Web of Science. For the specific criteria used in the screening for this study, please refer to Table 1 .

Literature screening

Literature acquired through keyword searches may contain ostensibly related yet actually unrelated works. Therefore, to ensure the close relevance of literature included in the analysis to the research topic, it is often necessary to perform a manual screening process to identify the final literature to be analyzed, subsequent to completing the initial literature search.

The manual screening process consists of two steps. Initially, irrelevant literature is weeded out based on the title and abstract, with two members of the research team involved in this phase. This stage lasted about one week, resulting in 1106 articles being retained. Subsequently, a comprehensive review of the full text is conducted to accurately identify the literature required for the study. To carry out the second phase of manual screening effectively and scientifically, and to minimize the potential for researcher bias, the research team established the inclusion criteria presented in Table 2 . Three members were engaged in this phase, which took approximately 2 weeks, culminating in the retention of 588 articles after meticulous screening. The entire screening process is depicted in Fig. 1 , adhering to the PRISMA guidelines (Page et al. 2021 ).

The process of obtaining and filtering the necessary literature data for research.

Data standardization

Nguyen and Hallinger ( 2020 ) pointed out that raw data extracted from scientific databases often contains multiple expressions of the same term, and not addressing these synonymous expressions could affect research results in bibliometric analysis. For instance, in the original data, the author list may include “Tsai, C. C.” and “Tsai, C.-C.”, while the keyword list may include “professional-development” and “professional development,” which often require merging. Therefore, before analyzing the selected literature, a data disambiguation process is necessary to standardize the data (Strotmann and Zhao, 2012 ; Van Eck and Waltman, 2019 ). This study adopted the data standardization process proposed by Taskin and Al ( 2019 ), mainly including the following standardization operations:

Firstly, the author and source fields in the data are corrected and standardized to differentiate authors with similar names.

Secondly, the study checks whether the journals to which the literature belongs have been renamed in the past over 20 years, so as to avoid the influence of periodical name change on the analysis results.

Finally, the keyword field is standardized by unifying parts of speech and singular/plural forms of keywords, which can help eliminate redundant entries in the knowledge graph.

Performance analysis (RQ1)

This section offers a thorough and detailed analysis of the state of research in the field of digital technology education. By utilizing descriptive statistics and visual maps, it provides a comprehensive overview of the development trends, authors, countries, institutions, and journal distribution within the field. The insights presented in this section are of great significance in advancing our understanding of the current state of research in this field and identifying areas for further investigation. The use of visual aids to display inter-country cooperation and the evolution of the field adds to the clarity and coherence of the analysis.

Time trend of the publications

To understand a research field, it is first necessary to understand the most basic quantitative information, among which the change in the number of publications per year best reflects the development trend of a research field. Figure 2 shows the distribution of publication dates.

Time trend of the publications on application of digital technology in education.

From the Fig. 2 , it can be seen that the development of this field over the past over 20 years can be roughly divided into three stages. The first stage was from 2000 to 2007, during which the number of publications was relatively low. Due to various factors such as technological maturity, the academic community did not pay widespread attention to the role of digital technology in expanding the scope of teaching and learning. The second stage was from 2008 to 2019, during which the overall number of publications showed an upward trend, and the development of the field entered an accelerated period, attracting more and more scholars’ attention. The third stage was from 2020 to 2022, during which the number of publications stabilized at around 100. During this period, the impact of the pandemic led to a large number of scholars focusing on the role of digital technology in education during the pandemic, and research on the application of digital technology in education became a core topic in social science research.

Analysis of authors

An analysis of the author’s publication volume provides information about the representative scholars and core research strengths of a research area. Table 3 presents information on the core authors in adaptive learning research, including name, publication number, and average number of citations per article (based on the analysis and statistics from VOSviewer).

Variations in research foci among scholars abound. Within the field of digital technology education application research over the past two decades, Neil Selwyn stands as the most productive author, having published 15 papers garnering a total of 1027 citations, resulting in an average of 68.47 citations per paper. As a Professor at the Faculty of Education at Monash University, Selwyn concentrates on exploring the application of digital technology in higher education contexts (Selwyn et al. 2021 ), as well as related products in higher education such as Coursera, edX, and Udacity MOOC platforms (Bulfin et al. 2014 ). Selwyn’s contributions to the educational sociology perspective include extensive research on the impact of digital technology on education, highlighting the spatiotemporal extension of educational processes and practices through technological means as the greatest value of educational technology (Selwyn, 2012 ; Selwyn and Facer, 2014 ). In addition, he provides a blueprint for the development of future schools in 2030 based on the present impact of digital technology on education (Selwyn et al. 2019 ). The second most productive author in this field, Henderson, also offers significant contributions to the understanding of the important value of digital technology in education, specifically in the higher education setting, with a focus on the impact of the pandemic (Henderson et al. 2015 ; Cohen et al. 2022 ). In contrast, Edwards’ research interests focus on early childhood education, particularly the application of digital technology in this context (Edwards, 2013 ; Bird and Edwards, 2015 ). Additionally, on the technical level, Edwards also mainly prefers digital game technology, because it is a digital technology that children are relatively easy to accept (Edwards, 2015 ).

Analysis of countries/regions and organization

The present study aimed to ascertain the leading countries in digital technology education application research by analyzing 75 countries related to 558 works of literature. Table 4 depicts the top ten countries that have contributed significantly to this field in terms of publication count (based on the analysis and statistics from VOSviewer). Our analysis of Table 4 data shows that England emerged as the most influential country/region, with 92 published papers and 2401 citations. Australia and the United States secured the second and third ranks, respectively, with 90 papers (2187 citations) and 70 papers (1331 citations) published. Geographically, most of the countries featured in the top ten publication volumes are situated in Australia, North America, and Europe, with China being the only exception. Notably, all these countries, except China, belong to the group of developed nations, suggesting that economic strength is a prerequisite for fostering research in the digital technology education application field.

This study presents a visual representation of the publication output and cooperation relationships among different countries in the field of digital technology education application research. Specifically, a chord diagram is employed to display the top 30 countries in terms of publication output, as depicted in Fig. 3 . The chord diagram is composed of nodes and chords, where the nodes are positioned as scattered points along the circumference, and the length of each node corresponds to the publication output, with longer lengths indicating higher publication output. The chords, on the other hand, represent the cooperation relationships between any two countries, and are weighted based on the degree of closeness of the cooperation, with wider chords indicating closer cooperation. Through the analysis of the cooperation relationships, the findings suggest that the main publishing countries in this field are engaged in cooperative relationships with each other, indicating a relatively high level of international academic exchange and research internationalization.

In the diagram, nodes are scattered along the circumference of a circle, with the length of each node representing the volume of publications. The weighted arcs connecting any two points on the circle are known as chords, representing the collaborative relationship between the two, with the width of the arc indicating the closeness of the collaboration.

Further analyzing Fig. 3 , we can extract more valuable information, enabling a deeper understanding of the connections between countries in the research field of digital technology in educational applications. It is evident that certain countries, such as the United States, China, and England, display thicker connections, indicating robust collaborative relationships in terms of productivity. These thicker lines signify substantial mutual contributions and shared objectives in certain sectors or fields, highlighting the interconnectedness and global integration in these areas. By delving deeper, we can also explore potential future collaboration opportunities through the chord diagram, identifying possible partners to propel research and development in this field. In essence, the chord diagram successfully encapsulates and conveys the multi-dimensionality of global productivity and cooperation, allowing for a comprehensive understanding of the intricate inter-country relationships and networks in a global context, providing valuable guidance and insights for future research and collaborations.

An in-depth examination of the publishing institutions is provided in Table 5 , showcasing the foremost 10 institutions ranked by their publication volume. Notably, Monash University and Australian Catholic University, situated in Australia, have recorded the most prolific publications within the digital technology education application realm, with 22 and 10 publications respectively. Moreover, the University of Oslo from Norway is featured among the top 10 publishing institutions, with an impressive average citation count of 64 per publication. It is worth highlighting that six institutions based in the United Kingdom were also ranked within the top 10 publishing institutions, signifying their leading position in this area of research.

Analysis of journals

Journals are the main carriers for publishing high-quality papers. Some scholars point out that the two key factors to measure the influence of journals in the specified field are the number of articles published and the number of citations. The more papers published in a magazine and the more citations, the greater its influence (Dzikowski, 2018 ). Therefore, this study utilized VOSviewer to statistically analyze the top 10 journals with the most publications in the field of digital technology in education and calculated the average citations per article (see Table 6 ).

Based on Table 6 , it is apparent that the highest number of articles in the domain of digital technology in education research were published in Education and Information Technologies (47 articles), Computers & Education (34 articles), and British Journal of Educational Technology (32 articles), indicating a higher article output compared to other journals. This underscores the fact that these three journals concentrate more on the application of digital technology in education. Furthermore, several other journals, such as Technology Pedagogy and Education and Sustainability, have published more than 15 articles in this domain. Sustainability represents the open access movement, which has notably facilitated research progress in this field, indicating that the development of open access journals in recent years has had a significant impact. Although there is still considerable disagreement among scholars on the optimal approach to achieve open access, the notion that research outcomes should be accessible to all is widely recognized (Huang et al. 2020 ). On further analysis of the research fields to which these journals belong, except for Sustainability, it is evident that they all pertain to educational technology, thus providing a qualitative definition of the research area of digital technology education from the perspective of journals.

Temporal keyword analysis: thematic evolution (RQ2)

The evolution of research themes is a dynamic process, and previous studies have attempted to present the developmental trajectory of fields by drawing keyword networks in phases (Kumar et al. 2021 ; Chen et al. 2022b ). To understand the shifts in research topics across different periods, this study follows past research and, based on the significant changes in the research field and corresponding technological advancements during the outlined periods, divides the timeline into four stages (the first stage from January 2000 to December 2005, the second stage from January 2006 to December 2011, the third stage from January 2012 to December 2017; and the fourth stage from January 2018 to December 2022). The division into these four stages was determined through a combination of bibliometric analysis and literature review, which presented a clear trajectory of the field’s development. The research analyzes the keyword networks for each time period (as there are only three articles in the first stage, it was not possible to generate an appropriate keyword co-occurrence map, hence only the keyword co-occurrence maps from the second to the fourth stages are provided), to understand the evolutionary track of the digital technology education application research field over time.

2000.1–2005.12: germination period

From January 2000 to December 2005, digital technology education application research was in its infancy. Only three studies focused on digital technology, all of which were related to computers. Due to the popularity of computers, the home became a new learning environment, highlighting the important role of digital technology in expanding the scope of learning spaces (Sutherland et al. 2000 ). In specific disciplines and contexts, digital technology was first favored in medical clinical practice, becoming an important tool for supporting the learning of clinical knowledge and practice (Tegtmeyer et al. 2001 ; Durfee et al. 2003 ).

2006.1–2011.12: initial development period

Between January 2006 and December 2011, it was the initial development period of digital technology education research. Significant growth was observed in research related to digital technology, and discussions and theoretical analyses about “digital natives” emerged. During this phase, scholars focused on the debate about “how to use digital technology reasonably” and “whether current educational models and school curriculum design need to be adjusted on a large scale” (Bennett and Maton, 2010 ; Selwyn, 2009 ; Margaryan et al. 2011 ). These theoretical and speculative arguments provided a unique perspective on the impact of cognitive digital technology on education and teaching. As can be seen from the vocabulary such as “rethinking”, “disruptive pedagogy”, and “attitude” in Fig. 4 , many scholars joined the calm reflection and analysis under the trend of digital technology (Laurillard, 2008 ; Vratulis et al. 2011 ). During this phase, technology was still undergoing dramatic changes. The development of mobile technology had already caught the attention of many scholars (Wong et al. 2011 ), but digital technology represented by computers was still very active (Selwyn et al. 2011 ). The change in technological form would inevitably lead to educational transformation. Collins and Halverson ( 2010 ) summarized the prospects and challenges of using digital technology for learning and educational practices, believing that digital technology would bring a disruptive revolution to the education field and bring about a new educational system. In addition, the term “teacher education” in Fig. 4 reflects the impact of digital technology development on teachers. The rapid development of technology has widened the generation gap between teachers and students. To ensure smooth communication between teachers and students, teachers must keep up with the trend of technological development and establish a lifelong learning concept (Donnison, 2009 ).

In the diagram, each node represents a keyword, with the size of the node indicating the frequency of occurrence of the keyword. The connections represent the co-occurrence relationships between keywords, with a higher frequency of co-occurrence resulting in tighter connections.

2012.1–2017.12: critical exploration period

During the period spanning January 2012 to December 2017, the application of digital technology in education research underwent a significant exploration phase. As can be seen from Fig. 5 , different from the previous stage, the specific elements of specific digital technology have started to increase significantly, including the enrichment of technological contexts, the greater variety of research methods, and the diversification of learning modes. Moreover, the temporal and spatial dimensions of the learning environment were further de-emphasized, as noted in previous literature (Za et al. 2014 ). Given the rapidly accelerating pace of technological development, the education system in the digital era is in urgent need of collaborative evolution and reconstruction, as argued by Davis, Eickelmann, and Zaka ( 2013 ).

In the domain of digital technology, social media has garnered substantial scholarly attention as a promising avenue for learning, as noted by Pasquini and Evangelopoulos ( 2016 ). The implementation of social media in education presents several benefits, including the liberation of education from the restrictions of physical distance and time, as well as the erasure of conventional educational boundaries. The user-generated content (UGC) model in social media has emerged as a crucial source for knowledge creation and distribution, with the widespread adoption of mobile devices. Moreover, social networks have become an integral component of ubiquitous learning environments (Hwang et al. 2013 ). The utilization of social media allows individuals to function as both knowledge producers and recipients, which leads to a blurring of the conventional roles of learners and teachers. On mobile platforms, the roles of learners and teachers are not fixed, but instead interchangeable.

In terms of research methodology, the prevalence of empirical studies with survey designs in the field of educational technology during this period is evident from the vocabulary used, such as “achievement,” “acceptance,” “attitude,” and “ict.” in Fig. 5 . These studies aim to understand learners’ willingness to adopt and attitudes towards new technologies, and some seek to investigate the impact of digital technologies on learning outcomes through quasi-experimental designs (Domínguez et al. 2013 ). Among these empirical studies, mobile learning emerged as a hot topic, and this is not surprising. First, the advantages of mobile learning environments over traditional ones have been empirically demonstrated (Hwang et al. 2013 ). Second, learners born around the turn of the century have been heavily influenced by digital technologies and have developed their own learning styles that are more open to mobile devices as a means of learning. Consequently, analyzing mobile learning as a relatively novel mode of learning has become an important issue for scholars in the field of educational technology.

The intervention of technology has led to the emergence of several novel learning modes, with the blended learning model being the most representative one in the current phase. Blended learning, a novel concept introduced in the information age, emphasizes the integration of the benefits of traditional learning methods and online learning. This learning mode not only highlights the prominent role of teachers in guiding, inspiring, and monitoring the learning process but also underlines the importance of learners’ initiative, enthusiasm, and creativity in the learning process. Despite being an early conceptualization, blended learning’s meaning has been expanded by the widespread use of mobile technology and social media in education. The implementation of new technologies, particularly mobile devices, has resulted in the transformation of curriculum design and increased flexibility and autonomy in students’ learning processes (Trujillo Maza et al. 2016 ), rekindling scholarly attention to this learning mode. However, some scholars have raised concerns about the potential drawbacks of the blended learning model, such as its significant impact on the traditional teaching system, the lack of systematic coping strategies and relevant policies in several schools and regions (Moskal et al. 2013 ).

2018.1–2022.12: accelerated transformation period

The period spanning from January 2018 to December 2022 witnessed a rapid transformation in the application of digital technology in education research. The field of digital technology education research reached a peak period of publication, largely influenced by factors such as the COVID-19 pandemic (Yu et al. 2023 ). Research during this period was built upon the achievements, attitudes, and social media of the previous phase, and included more elements that reflect the characteristics of this research field, such as digital literacy, digital competence, and professional development, as depicted in Fig. 6 . Alongside this, scholars’ expectations for the value of digital technology have expanded, and the pursuit of improving learning efficiency and performance is no longer the sole focus. Some research now aims to cultivate learners’ motivation and enhance their self-efficacy by applying digital technology in a reasonable manner, as demonstrated by recent studies (Beardsley et al. 2021 ; Creely et al. 2021 ).

The COVID-19 pandemic has emerged as a crucial backdrop for the digital technology’s role in sustaining global education, as highlighted by recent scholarly research (Zhou et al. 2022 ; Pan and Zhang, 2020 ; Mo et al. 2022 ). The online learning environment, which is supported by digital technology, has become the primary battleground for global education (Yu, 2022 ). This social context has led to various studies being conducted, with some scholars positing that the pandemic has impacted the traditional teaching order while also expanding learning possibilities in terms of patterns and forms (Alabdulaziz, 2021 ). Furthermore, the pandemic has acted as a catalyst for teacher teaching and technological innovation, and this viewpoint has been empirically substantiated (Moorhouse and Wong, 2021 ). Additionally, some scholars believe that the pandemic’s push is a crucial driving force for the digital transformation of the education system, serving as an essential mechanism for overcoming the system’s inertia (Romero et al. 2021 ).

The rapid outbreak of the pandemic posed a challenge to the large-scale implementation of digital technologies, which was influenced by a complex interplay of subjective and objective factors. Objective constraints included the lack of infrastructure in some regions to support digital technologies, while subjective obstacles included psychological resistance among certain students and teachers (Moorhouse, 2021 ). These factors greatly impacted the progress of online learning during the pandemic. Additionally, Timotheou et al. ( 2023 ) conducted a comprehensive systematic review of existing research on digital technology use during the pandemic, highlighting the critical role played by various factors such as learners’ and teachers’ digital skills, teachers’ personal attributes and professional development, school leadership and management, and administration in facilitating the digitalization and transformation of schools.

The current stage of research is characterized by the pivotal term “digital literacy,” denoting a growing interest in learners’ attitudes and adoption of emerging technologies. Initially, the term “literacy” was restricted to fundamental abilities and knowledge associated with books and print materials (McMillan, 1996 ). However, with the swift advancement of computers and digital technology, there have been various attempts to broaden the scope of literacy beyond its traditional meaning, including game literacy (Buckingham and Burn, 2007 ), information literacy (Eisenberg, 2008 ), and media literacy (Turin and Friesem, 2020 ). Similarly, digital literacy has emerged as a crucial concept, and Gilster and Glister ( 1997 ) were the first to introduce this concept, referring to the proficiency in utilizing technology and processing digital information in academic, professional, and daily life settings. In practical educational settings, learners who possess higher digital literacy often exhibit an aptitude for quickly mastering digital devices and applying them intelligently to education and teaching (Yu, 2022 ).

The utilization of digital technology in education has undergone significant changes over the past two decades, and has been a crucial driver of educational reform with each new technological revolution. The impact of these changes on the underlying logic of digital technology education applications has been noticeable. From computer technology to more recent developments such as virtual reality (VR), augmented reality (AR), and artificial intelligence (AI), the acceleration in digital technology development has been ongoing. Educational reforms spurred by digital technology development continue to be dynamic, as each new digital innovation presents new possibilities and models for teaching practice. This is especially relevant in the post-pandemic era, where the importance of technological progress in supporting teaching cannot be overstated (Mughal et al. 2022 ). Existing digital technologies have already greatly expanded the dimensions of education in both time and space, while future digital technologies aim to expand learners’ perceptions. Researchers have highlighted the potential of integrated technology and immersive technology in the development of the educational metaverse, which is highly anticipated to create a new dimension for the teaching and learning environment, foster a new value system for the discipline of educational technology, and more effectively and efficiently achieve the grand educational blueprint of the United Nations’ Sustainable Development Goals (Zhang et al. 2022 ; Li and Yu, 2023 ).

Hotspot evolution analysis (RQ3)

The examination of keyword evolution reveals a consistent trend in the advancement of digital technology education application research. The emergence and transformation of keywords serve as indicators of the varying research interests in this field. Thus, the utilization of the burst detection function available in CiteSpace allowed for the identification of the top 10 burst words that exhibited a high level of burst strength. This outcome is illustrated in Table 7 .

According to the results presented in Table 7 , the explosive terminology within the realm of digital technology education research has exhibited a concentration mainly between the years 2018 and 2022. Prior to this time frame, the emerging keywords were limited to “information technology” and “computer”. Notably, among them, computer, as an emergent keyword, has always had a high explosive intensity from 2008 to 2018, which reflects the important position of computer in digital technology and is the main carrier of many digital technologies such as Learning Management Systems (LMS) and Assessment and Feedback systems (Barlovits et al. 2022 ).

Since 2018, an increasing number of research studies have focused on evaluating the capabilities of learners to accept, apply, and comprehend digital technologies. As indicated by the use of terms such as “digital literacy” and “digital skill,” the assessment of learners’ digital literacy has become a critical task. Scholarly efforts have been directed towards the development of literacy assessment tools and the implementation of empirical assessments. Furthermore, enhancing the digital literacy of both learners and educators has garnered significant attention. (Nagle, 2018 ; Yu, 2022 ). Simultaneously, given the widespread use of various digital technologies in different formal and informal learning settings, promoting learners’ digital skills has become a crucial objective for contemporary schools (Nygren et al. 2019 ; Forde and OBrien, 2022 ).

Since 2020, the field of applied research on digital technology education has witnessed the emergence of three new hotspots, all of which have been affected to some extent by the pandemic. Firstly, digital technology has been widely applied in physical education, which is one of the subjects that has been severely affected by the pandemic (Parris et al. 2022 ; Jiang and Ning, 2022 ). Secondly, digital transformation has become an important measure for most schools, especially higher education institutions, to cope with the impact of the pandemic globally (García-Morales et al. 2021 ). Although the concept of digital transformation was proposed earlier, the COVID-19 pandemic has greatly accelerated this transformation process. Educational institutions must carefully redesign their educational products to face this new situation, providing timely digital learning methods, environments, tools, and support systems that have far-reaching impacts on modern society (Krishnamurthy, 2020 ; Salas-Pilco et al. 2022 ). Moreover, the professional development of teachers has become a key mission of educational institutions in the post-pandemic era. Teachers need to have a certain level of digital literacy and be familiar with the tools and online teaching resources used in online teaching, which has become a research hotspot today. Organizing digital skills training for teachers to cope with the application of emerging technologies in education is an important issue for teacher professional development and lifelong learning (Garzón-Artacho et al. 2021 ). As the main organizers and practitioners of emergency remote teaching (ERT) during the pandemic, teachers must put cognitive effort into their professional development to ensure effective implementation of ERT (Romero-Hall and Jaramillo Cherrez, 2022 ).

The burst word “digital transformation” reveals that we are in the midst of an ongoing digital technology revolution. With the emergence of innovative digital technologies such as ChatGPT and Microsoft 365 Copilot, technology trends will continue to evolve, albeit unpredictably. While the impact of these advancements on school education remains uncertain, it is anticipated that the widespread integration of technology will significantly affect the current education system. Rejecting emerging technologies without careful consideration is unwise. Like any revolution, the technological revolution in the education field has both positive and negative aspects. Detractors argue that digital technology disrupts learning and memory (Baron, 2021 ) or causes learners to become addicted and distracted from learning (Selwyn and Aagaard, 2020 ). On the other hand, the prudent use of digital technology in education offers a glimpse of a golden age of open learning. Educational leaders and practitioners have the opportunity to leverage cutting-edge digital technologies to address current educational challenges and develop a rational path for the sustainable and healthy growth of education.

Discussion on performance analysis (RQ1)

The field of digital technology education application research has experienced substantial growth since the turn of the century, a phenomenon that is quantifiably apparent through an analysis of authorship, country/region contributions, and institutional engagement. This expansion reflects the increased integration of digital technologies in educational settings and the heightened scholarly interest in understanding and optimizing their use.

Discussion on authorship productivity in digital technology education research

The authorship distribution within digital technology education research is indicative of the field’s intellectual structure and depth. A primary figure in this domain is Neil Selwyn, whose substantial citation rate underscores the profound impact of his work. His focus on the implications of digital technology in higher education and educational sociology has proven to be seminal. Selwyn’s research trajectory, especially the exploration of spatiotemporal extensions of education through technology, provides valuable insights into the multifaceted role of digital tools in learning processes (Selwyn et al. 2019 ).

Other notable contributors, like Henderson and Edwards, present diversified research interests, such as the impact of digital technologies during the pandemic and their application in early childhood education, respectively. Their varied focuses highlight the breadth of digital technology education research, encompassing pedagogical innovation, technological adaptation, and policy development.

Discussion on country/region-level productivity and collaboration

At the country/region level, the United Kingdom, specifically England, emerges as a leading contributor with 92 published papers and a significant citation count. This is closely followed by Australia and the United States, indicating a strong English-speaking research axis. Such geographical concentration of scholarly output often correlates with investment in research and development, technological infrastructure, and the prevalence of higher education institutions engaging in cutting-edge research.

China’s notable inclusion as the only non-Western country among the top contributors to the field suggests a growing research capacity and interest in digital technology in education. However, the lower average citation per paper for China could reflect emerging engagement or different research focuses that may not yet have achieved the same international recognition as Western counterparts.

The chord diagram analysis furthers this understanding, revealing dense interconnections between countries like the United States, China, and England, which indicates robust collaborations. Such collaborations are fundamental in addressing global educational challenges and shaping international research agendas.

Discussion on institutional-level contributions to digital technology education

Institutional productivity in digital technology education research reveals a constellation of universities driving the field forward. Monash University and the Australian Catholic University have the highest publication output, signaling Australia’s significant role in advancing digital education research. The University of Oslo’s remarkable average citation count per publication indicates influential research contributions, potentially reflecting high-quality studies that resonate with the broader academic community.

The strong showing of UK institutions, including the University of London, The Open University, and the University of Cambridge, reinforces the UK’s prominence in this research field. Such institutions are often at the forefront of pedagogical innovation, benefiting from established research cultures and funding mechanisms that support sustained inquiry into digital education.

Discussion on journal publication analysis

An examination of journal outputs offers a lens into the communicative channels of the field’s knowledge base. Journals such as Education and Information Technologies , Computers & Education , and the British Journal of Educational Technology not only serve as the primary disseminators of research findings but also as indicators of research quality and relevance. The impact factor (IF) serves as a proxy for the quality and influence of these journals within the academic community.

The high citation counts for articles published in Computers & Education suggest that research disseminated through this medium has a wide-reaching impact and is of particular interest to the field. This is further evidenced by its significant IF of 11.182, indicating that the journal is a pivotal platform for seminal work in the application of digital technology in education.

The authorship, regional, and institutional productivity in the field of digital technology education application research collectively narrate the evolution of this domain since the turn of the century. The prominence of certain authors and countries underscores the importance of socioeconomic factors and existing academic infrastructure in fostering research productivity. Meanwhile, the centrality of specific journals as outlets for high-impact research emphasizes the role of academic publishing in shaping the research landscape.

As the field continues to grow, future research may benefit from leveraging the collaborative networks that have been elucidated through this analysis, perhaps focusing on underrepresented regions to broaden the scope and diversity of research. Furthermore, the stabilization of publication numbers in recent years invites a deeper exploration into potential plateaus in research trends or saturation in certain sub-fields, signaling an opportunity for novel inquiries and methodological innovations.

Discussion on the evolutionary trends (RQ2)

The evolution of the research field concerning the application of digital technology in education over the past two decades is a story of convergence, diversification, and transformation, shaped by rapid technological advancements and shifting educational paradigms.

At the turn of the century, the inception of digital technology in education was largely exploratory, with a focus on how emerging computer technologies could be harnessed to enhance traditional learning environments. Research from this early period was primarily descriptive, reflecting on the potential and challenges of incorporating digital tools into the educational setting. This phase was critical in establishing the fundamental discourse that would guide subsequent research, as it set the stage for understanding the scope and impact of digital technology in learning spaces (Wang et al. 2023 ).

As the first decade progressed, the narrative expanded to encompass the pedagogical implications of digital technologies. This was a period of conceptual debates, where terms like “digital natives” and “disruptive pedagogy” entered the academic lexicon, underscoring the growing acknowledgment of digital technology as a transformative force within education (Bennett and Maton, 2010 ). During this time, the research began to reflect a more nuanced understanding of the integration of technology, considering not only its potential to change where and how learning occurred but also its implications for educational equity and access.

In the second decade, with the maturation of internet connectivity and mobile technology, the focus of research shifted from theoretical speculations to empirical investigations. The proliferation of digital devices and the ubiquity of social media influenced how learners interacted with information and each other, prompting a surge in studies that sought to measure the impact of these tools on learning outcomes. The digital divide and issues related to digital literacy became central concerns, as scholars explored the varying capacities of students and educators to engage with technology effectively.

Throughout this period, there was an increasing emphasis on the individualization of learning experiences, facilitated by adaptive technologies that could cater to the unique needs and pacing of learners (Jing et al. 2023a ). This individualization was coupled with a growing recognition of the importance of collaborative learning, both online and offline, and the role of digital tools in supporting these processes. Blended learning models, which combined face-to-face instruction with online resources, emerged as a significant trend, advocating for a balance between traditional pedagogies and innovative digital strategies.

The later years, particularly marked by the COVID-19 pandemic, accelerated the necessity for digital technology in education, transforming it from a supplementary tool to an essential platform for delivering education globally (Mo et al. 2022 ; Mustapha et al. 2021 ). This era brought about an unprecedented focus on online learning environments, distance education, and virtual classrooms. Research became more granular, examining not just the pedagogical effectiveness of digital tools, but also their role in maintaining continuity of education during crises, their impact on teacher and student well-being, and their implications for the future of educational policy and infrastructure.

Across these two decades, the research field has seen a shift from examining digital technology as an external addition to the educational process, to viewing it as an integral component of curriculum design, instructional strategies, and even assessment methods. The emergent themes have broadened from a narrow focus on specific tools or platforms to include wider considerations such as data privacy, ethical use of technology, and the environmental impact of digital tools.

Moreover, the field has moved from considering the application of digital technology in education as a primarily cognitive endeavor to recognizing its role in facilitating socio-emotional learning, digital citizenship, and global competencies. Researchers have increasingly turned their attention to the ways in which technology can support collaborative skills, cultural understanding, and ethical reasoning within diverse student populations.

In summary, the past over twenty years in the research field of digital technology applications in education have been characterized by a progression from foundational inquiries to complex analyses of digital integration. This evolution has mirrored the trajectory of technology itself, from a facilitative tool to a pervasive ecosystem defining contemporary educational experiences. As we look to the future, the field is poised to delve into the implications of emerging technologies like AI, AR, and VR, and their potential to redefine the educational landscape even further. This ongoing metamorphosis suggests that the application of digital technology in education will continue to be a rich area of inquiry, demanding continual adaptation and forward-thinking from educators and researchers alike.

Discussion on the study of research hotspots (RQ3)

The analysis of keyword evolution in digital technology education application research elucidates the current frontiers in the field, reflecting a trajectory that is in tandem with the rapidly advancing digital age. This landscape is sculpted by emergent technological innovations and shaped by the demands of an increasingly digital society.

Interdisciplinary integration and pedagogical transformation

One of the frontiers identified from recent keyword bursts includes the integration of digital technology into diverse educational contexts, particularly noted with the keyword “physical education.” The digitalization of disciplines traditionally characterized by physical presence illustrates the pervasive reach of technology and signifies a push towards interdisciplinary integration where technology is not only a facilitator but also a transformative agent. This integration challenges educators to reconceptualize curriculum delivery to accommodate digital tools that can enhance or simulate the physical aspects of learning.

Digital literacy and skills acquisition

Another pivotal frontier is the focus on “digital literacy” and “digital skill”, which has intensified in recent years. This suggests a shift from mere access to technology towards a comprehensive understanding and utilization of digital tools. In this realm, the emphasis is not only on the ability to use technology but also on critical thinking, problem-solving, and the ethical use of digital resources (Yu, 2022 ). The acquisition of digital literacy is no longer an additive skill but a fundamental aspect of modern education, essential for navigating and contributing to the digital world.

Educational digital transformation

The keyword “digital transformation” marks a significant research frontier, emphasizing the systemic changes that education institutions must undergo to align with the digital era (Romero et al. 2021 ). This transformation includes the redesigning of learning environments, pedagogical strategies, and assessment methods to harness digital technology’s full potential. Research in this area explores the complexity of institutional change, addressing the infrastructural, cultural, and policy adjustments needed for a seamless digital transition.

Engagement and participation

Further exploration into “engagement” and “participation” underscores the importance of student-centered learning environments that are mediated by technology. The current frontiers examine how digital platforms can foster collaboration, inclusivity, and active learning, potentially leading to more meaningful and personalized educational experiences. Here, the use of technology seeks to support the emotional and cognitive aspects of learning, moving beyond the transactional view of education to one that is relational and interactive.

Professional development and teacher readiness

As the field evolves, “professional development” emerges as a crucial area, particularly in light of the pandemic which necessitated emergency remote teaching. The need for teacher readiness in a digital age is a pressing frontier, with research focusing on the competencies required for educators to effectively integrate technology into their teaching practices. This includes familiarity with digital tools, pedagogical innovation, and an ongoing commitment to personal and professional growth in the digital domain.

Pandemic as a catalyst

The recent pandemic has acted as a catalyst for accelerated research and application in this field, particularly in the domains of “digital transformation,” “professional development,” and “physical education.” This period has been a litmus test for the resilience and adaptability of educational systems to continue their operations in an emergency. Research has thus been directed at understanding how digital technologies can support not only continuity but also enhance the quality and reach of education in such contexts.

Ethical and societal considerations

The frontier of digital technology in education is also expanding to consider broader ethical and societal implications. This includes issues of digital equity, data privacy, and the sociocultural impact of technology on learning communities. The research explores how educational technology can be leveraged to address inequities and create more equitable learning opportunities for all students, regardless of their socioeconomic background.

Innovation and emerging technologies

Looking forward, the frontiers are set to be influenced by ongoing and future technological innovations, such as artificial intelligence (AI) (Wu and Yu, 2023 ; Chen et al. 2022a ). The exploration into how these technologies can be integrated into educational practices to create immersive and adaptive learning experiences represents a bold new chapter for the field.

In conclusion, the current frontiers of research on the application of digital technology in education are multifaceted and dynamic. They reflect an overarching movement towards deeper integration of technology in educational systems and pedagogical practices, where the goals are not only to facilitate learning but to redefine it. As these frontiers continue to expand and evolve, they will shape the educational landscape, requiring a concerted effort from researchers, educators, policymakers, and technologists to navigate the challenges and harness the opportunities presented by the digital revolution in education.

Conclusions and future research

Conclusions.

The utilization of digital technology in education is a research area that cuts across multiple technical and educational domains and continues to experience dynamic growth due to the continuous progress of technology. In this study, a systematic review of this field was conducted through bibliometric techniques to examine its development trajectory. The primary focus of the review was to investigate the leading contributors, productive national institutions, significant publications, and evolving development patterns. The study’s quantitative analysis resulted in several key conclusions that shed light on this research field’s current state and future prospects.

(1) The research field of digital technology education applications has entered a stage of rapid development, particularly in recent years due to the impact of the pandemic, resulting in a peak of publications. Within this field, several key authors (Selwyn, Henderson, Edwards, etc.) and countries/regions (England, Australia, USA, etc.) have emerged, who have made significant contributions. International exchanges in this field have become frequent, with a high degree of internationalization in academic research. Higher education institutions in the UK and Australia are the core productive forces in this field at the institutional level.

(2) Education and Information Technologies , Computers & Education , and the British Journal of Educational Technology are notable journals that publish research related to digital technology education applications. These journals are affiliated with the research field of educational technology and provide effective communication platforms for sharing digital technology education applications.

(3) Over the past two decades, research on digital technology education applications has progressed from its early stages of budding, initial development, and critical exploration to accelerated transformation, and it is currently approaching maturity. Technological progress and changes in the times have been key driving forces for educational transformation and innovation, and both have played important roles in promoting the continuous development of education.

(4) Influenced by the pandemic, three emerging frontiers have emerged in current research on digital technology education applications, which are physical education, digital transformation, and professional development under the promotion of digital technology. These frontier research hotspots reflect the core issues that the education system faces when encountering new technologies. The evolution of research hotspots shows that technology breakthroughs in education’s original boundaries of time and space create new challenges. The continuous self-renewal of education is achieved by solving one hotspot problem after another.

The present study offers significant practical implications for scholars and practitioners in the field of digital technology education applications. Firstly, it presents a well-defined framework of the existing research in this area, serving as a comprehensive guide for new entrants to the field and shedding light on the developmental trajectory of this research domain. Secondly, the study identifies several contemporary research hotspots, thus offering a valuable decision-making resource for scholars aiming to explore potential research directions. Thirdly, the study undertakes an exhaustive analysis of published literature to identify core journals in the field of digital technology education applications, with Sustainability being identified as a promising open access journal that publishes extensively on this topic. This finding can potentially facilitate scholars in selecting appropriate journals for their research outputs.

Limitation and future research

Influenced by some objective factors, this study also has some limitations. First of all, the bibliometrics analysis software has high standards for data. In order to ensure the quality and integrity of the collected data, the research only selects the periodical papers in SCIE and SSCI indexes, which are the core collection of Web of Science database, and excludes other databases, conference papers, editorials and other publications, which may ignore some scientific research and original opinions in the field of digital technology education and application research. In addition, although this study used professional software to carry out bibliometric analysis and obtained more objective quantitative data, the analysis and interpretation of data will inevitably have a certain subjective color, and the influence of subjectivity on data analysis cannot be completely avoided. As such, future research endeavors will broaden the scope of literature screening and proactively engage scholars in the field to gain objective and state-of-the-art insights, while minimizing the adverse impact of personal subjectivity on research analysis.

Data availability

The datasets analyzed during the current study are available in the Dataverse repository: https://doi.org/10.7910/DVN/F9QMHY

Alabdulaziz MS (2021) COVID-19 and the use of digital technology in mathematics education. Educ Inf Technol 26(6):7609–7633. https://doi.org/10.1007/s10639-021-10602-3

Arif TB, Munaf U, Ul-Haque I (2023) The future of medical education and research: is ChatGPT a blessing or blight in disguise? Med Educ Online 28. https://doi.org/10.1080/10872981.2023.2181052

Banerjee M, Chiew D, Patel KT, Johns I, Chappell D, Linton N, Cole GD, Francis DP, Szram J, Ross J, Zaman S (2021) The impact of artificial intelligence on clinical education: perceptions of postgraduate trainee doctors in London (UK) and recommendations for trainers. BMC Med Educ 21. https://doi.org/10.1186/s12909-021-02870-x

Barlovits S, Caldeira A, Fesakis G, Jablonski S, Koutsomanoli Filippaki D, Lázaro C, Ludwig M, Mammana MF, Moura A, Oehler DXK, Recio T, Taranto E, Volika S(2022) Adaptive, synchronous, and mobile online education: developing the ASYMPTOTE learning environment. Mathematics 10:1628. https://doi.org/10.3390/math10101628

Article   Google Scholar  

Baron NS(2021) Know what? How digital technologies undermine learning and remembering J Pragmat 175:27–37. https://doi.org/10.1016/j.pragma.2021.01.011

Batista J, Morais NS, Ramos F (2016) Researching the use of communication technologies in higher education institutions in Portugal. https://doi.org/10.4018/978-1-5225-0571-6.ch057

Beardsley M, Albó L, Aragón P, Hernández-Leo D (2021) Emergency education effects on teacher abilities and motivation to use digital technologies. Br J Educ Technol 52. https://doi.org/10.1111/bjet.13101

Bennett S, Maton K(2010) Beyond the “digital natives” debate: towards a more nuanced understanding of students’ technology experiences J Comput Assist Learn 26:321–331. https://doi.org/10.1111/j.1365-2729.2010.00360.x

Buckingham D, Burn A (2007) Game literacy in theory and practice 16:323–349

Google Scholar  

Bulfin S, Pangrazio L, Selwyn N (2014) Making “MOOCs”: the construction of a new digital higher education within news media discourse. In: The International Review of Research in Open and Distributed Learning 15. https://doi.org/10.19173/irrodl.v15i5.1856

Camilleri MA, Camilleri AC(2016) Digital learning resources and ubiquitous technologies in education Technol Knowl Learn 22:65–82. https://doi.org/10.1007/s10758-016-9287-7

Chen C(2006) CiteSpace II: detecting and visualizing emerging trends and transient patterns in scientific literature J Am Soc Inf Sci Technol 57:359–377. https://doi.org/10.1002/asi.20317

Chen J, Dai J, Zhu K, Xu L(2022) Effects of extended reality on language learning: a meta-analysis Front Psychol 13:1016519. https://doi.org/10.3389/fpsyg.2022.1016519

Article   PubMed   PubMed Central   Google Scholar  

Chen J, Wang CL, Tang Y (2022b) Knowledge mapping of volunteer motivation: a bibliometric analysis and cross-cultural comparative study. Front Psychol 13. https://doi.org/10.3389/fpsyg.2022.883150

Cohen A, Soffer T, Henderson M(2022) Students’ use of technology and their perceptions of its usefulness in higher education: International comparison J Comput Assist Learn 38(5):1321–1331. https://doi.org/10.1111/jcal.12678

Collins A, Halverson R(2010) The second educational revolution: rethinking education in the age of technology J Comput Assist Learn 26:18–27. https://doi.org/10.1111/j.1365-2729.2009.00339.x

Conole G, Alevizou P (2010) A literature review of the use of Web 2.0 tools in higher education. Walton Hall, Milton Keynes, UK: the Open University, retrieved 17 February

Creely E, Henriksen D, Crawford R, Henderson M(2021) Exploring creative risk-taking and productive failure in classroom practice. A case study of the perceived self-efficacy and agency of teachers at one school Think Ski Creat 42:100951. https://doi.org/10.1016/j.tsc.2021.100951

Davis N, Eickelmann B, Zaka P(2013) Restructuring of educational systems in the digital age from a co-evolutionary perspective J Comput Assist Learn 29:438–450. https://doi.org/10.1111/jcal.12032

De Belli N (2009) Bibliometrics and citation analysis: from the science citation index to cybermetrics, Scarecrow Press. https://doi.org/10.1111/jcal.12032

Domínguez A, Saenz-de-Navarrete J, de-Marcos L, Fernández-Sanz L, Pagés C, Martínez-Herráiz JJ(2013) Gamifying learning experiences: practical implications and outcomes Comput Educ 63:380–392. https://doi.org/10.1016/j.compedu.2012.12.020

Donnison S (2009) Discourses in conflict: the relationship between Gen Y pre-service teachers, digital technologies and lifelong learning. Australasian J Educ Technol 25. https://doi.org/10.14742/ajet.1138

Durfee SM, Jain S, Shaffer K (2003) Incorporating electronic media into medical student education. Acad Radiol 10:205–210. https://doi.org/10.1016/s1076-6332(03)80046-6

Dzikowski P(2018) A bibliometric analysis of born global firms J Bus Res 85:281–294. https://doi.org/10.1016/j.jbusres.2017.12.054

van Eck NJ, Waltman L(2009) Software survey: VOSviewer, a computer program for bibliometric mapping Scientometrics 84:523–538 https://doi.org/10.1007/s11192-009-0146-3

Edwards S(2013) Digital play in the early years: a contextual response to the problem of integrating technologies and play-based pedagogies in the early childhood curriculum Eur Early Child Educ Res J 21:199–212. https://doi.org/10.1080/1350293x.2013.789190

Edwards S(2015) New concepts of play and the problem of technology, digital media and popular-culture integration with play-based learning in early childhood education Technol Pedagogy Educ 25:513–532 https://doi.org/10.1080/1475939x.2015.1108929

Article   MathSciNet   Google Scholar  

Eisenberg MB(2008) Information literacy: essential skills for the information age DESIDOC J Libr Inf Technol 28:39–47. https://doi.org/10.14429/djlit.28.2.166

Forde C, OBrien A (2022) A literature review of barriers and opportunities presented by digitally enhanced practical skill teaching and learning in health science education. Med Educ Online 27. https://doi.org/10.1080/10872981.2022.2068210

García-Morales VJ, Garrido-Moreno A, Martín-Rojas R (2021) The transformation of higher education after the COVID disruption: emerging challenges in an online learning scenario. Front Psychol 12. https://doi.org/10.3389/fpsyg.2021.616059

Garfield E(2006) The history and meaning of the journal impact factor JAMA 295:90. https://doi.org/10.1001/jama.295.1.90

Article   PubMed   Google Scholar  

Garzón-Artacho E, Sola-Martínez T, Romero-Rodríguez JM, Gómez-García G(2021) Teachers’ perceptions of digital competence at the lifelong learning stage Heliyon 7:e07513. https://doi.org/10.1016/j.heliyon.2021.e07513

Gaviria-Marin M, Merigó JM, Baier-Fuentes H(2019) Knowledge management: a global examination based on bibliometric analysis Technol Forecast Soc Change 140:194–220. https://doi.org/10.1016/j.techfore.2018.07.006

Gilster P, Glister P (1997) Digital literacy. Wiley Computer Pub, New York

Greenhow C, Lewin C(2015) Social media and education: reconceptualizing the boundaries of formal and informal learning Learn Media Technol 41:6–30. https://doi.org/10.1080/17439884.2015.1064954

Hawkins DT(2001) Bibliometrics of electronic journals in information science Infor Res 7(1):7–1. http://informationr.net/ir/7-1/paper120.html

Henderson M, Selwyn N, Finger G, Aston R(2015) Students’ everyday engagement with digital technology in university: exploring patterns of use and “usefulness J High Educ Policy Manag 37:308–319 https://doi.org/10.1080/1360080x.2015.1034424

Huang CK, Neylon C, Hosking R, Montgomery L, Wilson KS, Ozaygen A, Brookes-Kenworthy C (2020) Evaluating the impact of open access policies on research institutions. eLife 9. https://doi.org/10.7554/elife.57067

Hwang GJ, Tsai CC(2011) Research trends in mobile and ubiquitous learning: a review of publications in selected journals from 2001 to 2010 Br J Educ Technol 42:E65–E70. https://doi.org/10.1111/j.1467-8535.2011.01183.x

Hwang GJ, Wu PH, Zhuang YY, Huang YM(2013) Effects of the inquiry-based mobile learning model on the cognitive load and learning achievement of students Interact Learn Environ 21:338–354. https://doi.org/10.1080/10494820.2011.575789

Jiang S, Ning CF (2022) Interactive communication in the process of physical education: are social media contributing to the improvement of physical training performance. Universal Access Inf Soc, 1–10. https://doi.org/10.1007/s10209-022-00911-w

Jing Y, Zhao L, Zhu KK, Wang H, Wang CL, Xia Q(2023) Research landscape of adaptive learning in education: a bibliometric study on research publications from 2000 to 2022 Sustainability 15:3115–3115. https://doi.org/10.3390/su15043115

Jing Y, Wang CL, Chen Y, Wang H, Yu T, Shadiev R (2023b) Bibliometric mapping techniques in educational technology research: a systematic literature review. Educ Inf Technol 1–29. https://doi.org/10.1007/s10639-023-12178-6

Krishnamurthy S (2020) The future of business education: a commentary in the shadow of the Covid-19 pandemic. J Bus Res. https://doi.org/10.1016/j.jbusres.2020.05.034

Kumar S, Lim WM, Pandey N, Christopher Westland J (2021) 20 years of electronic commerce research. Electron Commer Res 21:1–40

Kyza EA, Georgiou Y(2018) Scaffolding augmented reality inquiry learning: the design and investigation of the TraceReaders location-based, augmented reality platform Interact Learn Environ 27:211–225. https://doi.org/10.1080/10494820.2018.1458039

Laurillard D(2008) Technology enhanced learning as a tool for pedagogical innovation J Philos Educ 42:521–533. https://doi.org/10.1111/j.1467-9752.2008.00658.x

Li M, Yu Z (2023) A systematic review on the metaverse-based blended English learning. Front Psychol 13. https://doi.org/10.3389/fpsyg.2022.1087508

Luo H, Li G, Feng Q, Yang Y, Zuo M (2021) Virtual reality in K-12 and higher education: a systematic review of the literature from 2000 to 2019. J Comput Assist Learn. https://doi.org/10.1111/jcal.12538

Margaryan A, Littlejohn A, Vojt G(2011) Are digital natives a myth or reality? University students’ use of digital technologies Comput Educ 56:429–440. https://doi.org/10.1016/j.compedu.2010.09.004

McMillan S(1996) Literacy and computer literacy: definitions and comparisons Comput Educ 27:161–170. https://doi.org/10.1016/s0360-1315(96)00026-7

Mo CY, Wang CL, Dai J, Jin P (2022) Video playback speed influence on learning effect from the perspective of personalized adaptive learning: a study based on cognitive load theory. Front Psychology 13. https://doi.org/10.3389/fpsyg.2022.839982

Moorhouse BL (2021) Beginning teaching during COVID-19: newly qualified Hong Kong teachers’ preparedness for online teaching. Educ Stud 1–17. https://doi.org/10.1080/03055698.2021.1964939

Moorhouse BL, Wong KM (2021) The COVID-19 Pandemic as a catalyst for teacher pedagogical and technological innovation and development: teachers’ perspectives. Asia Pac J Educ 1–16. https://doi.org/10.1080/02188791.2021.1988511

Moskal P, Dziuban C, Hartman J (2013) Blended learning: a dangerous idea? Internet High Educ 18:15–23

Mughal MY, Andleeb N, Khurram AFA, Ali MY, Aslam MS, Saleem MN (2022) Perceptions of teaching-learning force about Metaverse for education: a qualitative study. J. Positive School Psychol 6:1738–1745

Mustapha I, Thuy Van N, Shahverdi M, Qureshi MI, Khan N (2021) Effectiveness of digital technology in education during COVID-19 pandemic. a bibliometric analysis. Int J Interact Mob Technol 15:136

Nagle J (2018) Twitter, cyber-violence, and the need for a critical social media literacy in teacher education: a review of the literature. Teach Teach Education 76:86–94

Nazare J, Woolf A, Sysoev I, Ballinger S, Saveski M, Walker M, Roy D (2022) Technology-assisted coaching can increase engagement with learning technology at home and caregivers’ awareness of it. Comput Educ 188:104565

Nguyen UP, Hallinger P (2020) Assessing the distinctive contributions of simulation & gaming to the literature, 1970-2019: a bibliometric review. Simul Gaming 104687812094156. https://doi.org/10.1177/1046878120941569

Nygren H, Nissinen K, Hämäläinen R, Wever B(2019) Lifelong learning: formal, non-formal and informal learning in the context of the use of problem-solving skills in technology-rich environments Br J Educ Technol 50:1759–1770. https://doi.org/10.1111/bjet.12807

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, Moher D (2021) The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. Int J Surg 88:105906

Pan SL, Zhang S(2020) From fighting COVID-19 pandemic to tackling sustainable development goals: an opportunity for responsible information systems research Int J Inf Manage 55:102196. https://doi.org/10.1016/j.ijinfomgt.2020.102196

Pan X, Yan E, Cui M, Hua W(2018) Examining the usage, citation, and diffusion patterns of bibliometric mapping software: a comparative study of three tools J Informetr 12:481–493. https://doi.org/10.1016/j.joi.2018.03.005

Parris Z, Cale L, Harris J, Casey A (2022) Physical activity for health, covid-19 and social media: what, where and why?. Movimento, 28. https://doi.org/10.22456/1982-8918.122533

Pasquini LA, Evangelopoulos N (2016) Sociotechnical stewardship in higher education: a field study of social media policy documents. J Comput High Educ 29:218–239

Pérez-Sanagustín M, Hernández-Leo D, Santos P, Delgado Kloos C, Blat J(2014) Augmenting reality and formality of informal and non-formal settings to enhance blended learning IEEE Trans Learn Technol 7:118–131. https://doi.org/10.1109/TLT.2014.2312719

Pinto M, Leite C (2020) Digital technologies in support of students learning in Higher Education: literature review. Digital Education Review 343–360. https://doi.org/10.1344/der.2020.37.343-360

Pires F, Masanet MJ, Tomasena JM, Scolari CA(2022) Learning with YouTube: beyond formal and informal through new actors, strategies and affordances Convergence 28(3):838–853. https://doi.org/10.1177/1354856521102054

Pritchard A (1969) Statistical bibliography or bibliometrics 25:348

Romero M, Romeu T, Guitert M, Baztán P (2021) Digital transformation in higher education: the UOC case. In ICERI2021 Proceedings (pp. 6695–6703). IATED https://doi.org/10.21125/iceri.2021.1512

Romero-Hall E, Jaramillo Cherrez N (2022) Teaching in times of disruption: faculty digital literacy in higher education during the COVID-19 pandemic. Innovations in Education and Teaching International 1–11. https://doi.org/10.1080/14703297.2022.2030782

Rospigliosi PA(2023) Artificial intelligence in teaching and learning: what questions should we ask of ChatGPT? Interactive Learning Environments 31:1–3. https://doi.org/10.1080/10494820.2023.2180191

Salas-Pilco SZ, Yang Y, Zhang Z(2022) Student engagement in online learning in Latin American higher education during the COVID-19 pandemic: a systematic review. Br J Educ Technol 53(3):593–619. https://doi.org/10.1111/bjet.13190

Selwyn N(2009) The digital native-myth and reality In Aslib proceedings 61(4):364–379. https://doi.org/10.1108/00012530910973776

Selwyn N(2012) Making sense of young people, education and digital technology: the role of sociological theory Oxford Review of Education 38:81–96. https://doi.org/10.1080/03054985.2011.577949

Selwyn N, Facer K(2014) The sociology of education and digital technology: past, present and future Oxford Rev Educ 40:482–496. https://doi.org/10.1080/03054985.2014.933005

Selwyn N, Banaji S, Hadjithoma-Garstka C, Clark W(2011) Providing a platform for parents? Exploring the nature of parental engagement with school Learning Platforms J Comput Assist Learn 27:314–323. https://doi.org/10.1111/j.1365-2729.2011.00428.x

Selwyn N, Aagaard J (2020) Banning mobile phones from classrooms-an opportunity to advance understandings of technology addiction, distraction and cyberbullying. Br J Educ Technol 52. https://doi.org/10.1111/bjet.12943

Selwyn N, O’Neill C, Smith G, Andrejevic M, Gu X (2021) A necessary evil? The rise of online exam proctoring in Australian universities. Media Int Austr 1329878X2110058. https://doi.org/10.1177/1329878x211005862

Selwyn N, Pangrazio L, Nemorin S, Perrotta C (2019) What might the school of 2030 be like? An exercise in social science fiction. Learn, Media Technol 1–17. https://doi.org/10.1080/17439884.2020.1694944

Selwyn, N (2016) What works and why?* Understanding successful technology enabled learning within institutional contexts 2016 Final report Appendices (Part B). Monash University Griffith University

Sjöberg D, Holmgren R (2021) Informal workplace learning in swedish police education-a teacher perspective. Vocations and Learning. https://doi.org/10.1007/s12186-021-09267-3

Strotmann A, Zhao D (2012) Author name disambiguation: what difference does it make in author-based citation analysis? J Am Soc Inf Sci Technol 63:1820–1833

Article   CAS   Google Scholar  

Sutherland R, Facer K, Furlong R, Furlong J(2000) A new environment for education? The computer in the home. Comput Educ 34:195–212. https://doi.org/10.1016/s0360-1315(99)00045-7

Szeto E, Cheng AY-N, Hong J-C(2015) Learning with social media: how do preservice teachers integrate YouTube and Social Media in teaching? Asia-Pac Educ Res 25:35–44. https://doi.org/10.1007/s40299-015-0230-9

Tang E, Lam C(2014) Building an effective online learning community (OLC) in blog-based teaching portfolios Int High Educ 20:79–85. https://doi.org/10.1016/j.iheduc.2012.12.002

Taskin Z, Al U(2019) Natural language processing applications in library and information science Online Inf Rev 43:676–690. https://doi.org/10.1108/oir-07-2018-0217

Tegtmeyer K, Ibsen L, Goldstein B(2001) Computer-assisted learning in critical care: from ENIAC to HAL Crit Care Med 29:N177–N182. https://doi.org/10.1097/00003246-200108001-00006

Article   CAS   PubMed   Google Scholar  

Timotheou S, Miliou O, Dimitriadis Y, Sobrino SV, Giannoutsou N, Cachia R, Moné AM, Ioannou A(2023) Impacts of digital technologies on education and factors influencing schools' digital capacity and transformation: a literature review. Educ Inf Technol 28(6):6695–6726. https://doi.org/10.1007/s10639-022-11431-8

Trujillo Maza EM, Gómez Lozano MT, Cardozo Alarcón AC, Moreno Zuluaga L, Gamba Fadul M (2016) Blended learning supported by digital technology and competency-based medical education: a case study of the social medicine course at the Universidad de los Andes, Colombia. Int J Educ Technol High Educ 13. https://doi.org/10.1186/s41239-016-0027-9

Turin O, Friesem Y(2020) Is that media literacy?: Israeli and US media scholars’ perceptions of the field J Media Lit Educ 12:132–144. https://doi.org/10.1007/s11192-009-0146-3

Van Eck NJ, Waltman L (2019) VOSviewer manual. Universiteit Leiden

Vratulis V, Clarke T, Hoban G, Erickson G(2011) Additive and disruptive pedagogies: the use of slowmation as an example of digital technology implementation Teach Teach Educ 27:1179–1188. https://doi.org/10.1016/j.tate.2011.06.004

Wang CL, Dai J, Xu LJ (2022) Big data and data mining in education: a bibliometrics study from 2010 to 2022. In 2022 7th International Conference on Cloud Computing and Big Data Analytics ( ICCCBDA ) (pp. 507-512). IEEE. https://doi.org/10.1109/icccbda55098.2022.9778874

Wang CL, Dai J, Zhu KK, Yu T, Gu XQ (2023) Understanding the continuance intention of college students toward new E-learning spaces based on an integrated model of the TAM and TTF. Int J Hum-Comput Int 1–14. https://doi.org/10.1080/10447318.2023.2291609

Wong L-H, Boticki I, Sun J, Looi C-K(2011) Improving the scaffolds of a mobile-assisted Chinese character forming game via a design-based research cycle Comput Hum Behav 27:1783–1793. https://doi.org/10.1016/j.chb.2011.03.005

Wu R, Yu Z (2023) Do AI chatbots improve students learning outcomes? Evidence from a meta-analysis. Br J Educ Technol. https://doi.org/10.1111/bjet.13334

Yang D, Zhou J, Shi D, Pan Q, Wang D, Chen X, Liu J (2022) Research status, hotspots, and evolutionary trends of global digital education via knowledge graph analysis. Sustainability 14:15157–15157. https://doi.org/10.3390/su142215157

Yu T, Dai J, Wang CL (2023) Adoption of blended learning: Chinese university students’ perspectives. Humanit Soc Sci Commun 10:390. https://doi.org/10.3390/su142215157

Yu Z (2022) Sustaining student roles, digital literacy, learning achievements, and motivation in online learning environments during the COVID-19 pandemic. Sustainability 14:4388. https://doi.org/10.3390/su14084388

Za S, Spagnoletti P, North-Samardzic A(2014) Organisational learning as an emerging process: the generative role of digital tools in informal learning practices Br J Educ Technol 45:1023–1035. https://doi.org/10.1111/bjet.12211

Zhang X, Chen Y, Hu L, Wang Y (2022) The metaverse in education: definition, framework, features, potential applications, challenges, and future research topics. Front Psychol 13:1016300. https://doi.org/10.3389/fpsyg.2022.1016300

Zhou M, Dzingirai C, Hove K, Chitata T, Mugandani R (2022) Adoption, use and enhancement of virtual learning during COVID-19. Education and Information Technologies. https://doi.org/10.1007/s10639-022-10985-x

Download references

Acknowledgements

This research was supported by the Zhejiang Provincial Social Science Planning Project, “Mechanisms and Pathways for Empowering Classroom Teaching through Learning Spaces under the Strategy of High-Quality Education Development”, the 2022 National Social Science Foundation Education Youth Project “Research on the Strategy of Creating Learning Space Value and Empowering Classroom Teaching under the background of ‘Double Reduction’” (Grant No. CCA220319) and the National College Student Innovation and Entrepreneurship Training Program of China (Grant No. 202310337023).

Author information

Authors and affiliations.

College of Educational Science and Technology, Zhejiang University of Technology, Zhejiang, China

Chengliang Wang, Xiaojiao Chen, Yidan Liu & Yuhui Jing

Graduate School of Business, Universiti Sains Malaysia, Minden, Malaysia

Department of Management, The Chinese University of Hong Kong, Hong Kong, China

College of Humanities and Social Sciences, Beihang University, Beijing, China

You can also search for this author in PubMed   Google Scholar

Contributions

Conceptualization: Y.J., C.W.; methodology, C.W.; software, C.W., Y.L.; writing-original draft preparation, C.W., Y.L.; writing-review and editing, T.Y., Y.L., C.W.; supervision, X.C., T.Y.; project administration, Y.J.; funding acquisition, X.C., Y.L. All authors read and approved the final manuscript. All authors have read and approved the re-submission of the manuscript.

Corresponding author

Correspondence to Yuhui Jing .

Ethics declarations

Ethical approval.

Ethical approval was not required as the study did not involve human participants.

Informed consent

Informed consent was not required as the study did not involve human participants.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Wang, C., Chen, X., Yu, T. et al. Education reform and change driven by digital technology: a bibliometric study from a global perspective. Humanit Soc Sci Commun 11 , 256 (2024). https://doi.org/10.1057/s41599-024-02717-y

Download citation

Received : 11 July 2023

Accepted : 17 January 2024

Published : 12 February 2024

DOI : https://doi.org/10.1057/s41599-024-02717-y

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

This article is cited by

A meta-analysis of learners’ continuance intention toward online education platforms.

• Chengliang Wang

Education and Information Technologies (2024)

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here .

Loading metrics

Open Access

Peer-reviewed

Research Article

On the quality of quantitative instruments to measure digital competence in higher education: A systematic mapping study

Contributed equally to this work with: Rafael Saltos-Rivas, Pavel Novoa-Hernández, Rocío Serrano Rodríguez

Roles Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Software, Visualization, Writing – original draft, Writing – review & editing

Affiliation Facultad de Filosofía Letras y Ciencias de la Educación de la Universidad Técnica de Manabí, Portoviejo, Ecuador

Roles Formal analysis, Investigation, Project administration, Software, Supervision, Writing – original draft, Writing – review & editing

* E-mail: [email protected]

Affiliation Escuela de Ciencias Empresariales, Universidad Católica del Norte, Coquimbo, Chile

Roles Conceptualization, Funding acquisition, Project administration, Supervision, Writing – review & editing

Affiliation Facultad de Ciencias de la Educación, Universidad de Córdoba, Cordoba, Spain

• Rafael Saltos-Rivas, 
• Pavel Novoa-Hernández, 
• Rocío Serrano Rodríguez

• Published: September 10, 2021
• https://doi.org/10.1371/journal.pone.0257344
• Reader Comments

In this study, we report on a Systematic Mapping Study (SMS) on how the quality of the quantitative instruments used to measure digital competencies in higher education is assured. 73 primary studies were selected from the published literature in the last 10 years in order to 1) characterize the literature, 2) evaluate the reporting practice of quality assessments, and 3) analyze which variables explain such reporting practices. The results indicate that most of the studies focused on medium to large samples of European university students, who attended social science programs. Ad hoc, self-reported questionnaires measuring various digital competence areas were the most commonly used method for data collection. The studies were mostly published in low tier journals. 36% of the studies did not report any quality assessment, while less than 50% covered both groups of reliability and validity assessments at the same time. In general, the studies had a moderate to high depth of evidence on the assessments performed. We found that studies in which several areas of digital competence were measured were more likely to report quality assessments. In addition, we estimate that the probability of finding studies with acceptable or good reporting practices increases over time.

Citation: Saltos-Rivas R, Novoa-Hernández P, Serrano Rodríguez R (2021) On the quality of quantitative instruments to measure digital competence in higher education: A systematic mapping study. PLoS ONE 16(9): e0257344. https://doi.org/10.1371/journal.pone.0257344

Editor: José Gutiérrez-Pérez, University of Granada: Universidad de Granada, SPAIN

Received: March 23, 2021; Accepted: August 29, 2021; Published: September 10, 2021

Copyright: © 2021 Saltos-Rivas et al. This is an open access article distributed under the terms of the Creative Commons Attribution License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: All data are publicly available on the OSF platform ( https://osf.io/me36k ).

Funding: The author(s) received no specific funding for this work.

Competing interests: The authors have declared that no competing interests exist.

1 Introduction

In a world governed by Information and Communication Technologies (ICTs) [ 1 ], most of the essential processes of modern society are automatized in one way or another [ 2 ]. For this reason, it is essential that professionals have sufficient ICT skills and competencies, adapting to the demands of the modern working world [ 3 , 4 ]. Higher education institutions play an essential role in this context [ 5 ], that is, by integrating different strategies to provide these digital competencies to their educational community [ 6 ] and building coherent evaluation processes and instruments. The latter is particularly important not only for diagnosing the educational community, but also to verify the extent to which an intervention program has been effective.

The act of evaluating digital competencies is, from a theoretical perspective, a measurement task. As a consequence, it largely depends on the quality of the employed instrument. Regardless of the structure of the instrument, its quality is given by the fulfillment degree of two psychometric properties: reliability and validity [ 7 ]. As recent reviews showed [ 8 , 9 ], the topic of digital competence in higher education comprises of a large and fertile body of studies. This literature has been characterized on several occasions from different perspectives: ranging from concept use [ 10 , 11 ] to organizational infrastructures, strategic leadership, and teaching practices [ 12 ].

However, the evaluation process itself and, more specifically, the quality of the employed instrument are two important topics that have not been extensively addressed in the past. So far, there is still an uncertainty about how and to what extent studies ensure that the instruments used are adequate to measure digital competencies in higher education. In our opinion, characterizing current literature on these issues is relevant for both researchers and practitioners. In the first case, researchers are provided with an overview on 1) the main studies’ features and 2) the trends in reporting quality assessments. As a consequence, current literature is assessed and some important research opportunities to explore in the near future are identified. Additionally, such characterization would help us to reflect on what we have done well and what we have not done when conducting or reporting on quality assessments. In the case of practitioners, since literature is assessed according to how the quality of the employed instruments has been assured, they are provided with a top-quality list of studies that serve as a good starting point for reusing previous experiences.

In order to shed light on these issues, in this paper, we conducted a systematic mapping study [ 13 ]. More specifically, we aim at 1) characterizing the literature demographically and methodologically, 2) describing how and to what extent studies assured the quality of employed instruments, and 3) identifying what studies’ features explain certain reporting practices. We specifically focused on literature using quantitative instruments in the form of questionnaires. The rationale behind this move, as noted by [ 11 ], is that these data collection methods are among the most commonly employed by researchers in this field. Therefore, it is expected that the results obtained can characterize the vast majority of published studies on this topic.

2 Digital competencies in higher education

Currently, no consensus exists about what entails digital competencies in the context of higher education [ 9 – 11 ]. This is mainly because, as noted by [ 11 ], the definition of digital competence is context-dependent and, therefore, it is possible to find various positions both in the scientific context [ 14 ] and in the of the definition of government policies [ 3 ]. In addition to this, we have to take into account that students and academic staffs demand for specific competencies, which are not necessarily the same [ 8 ].

Among the thirty definitions reviewed in [ 11 ], perhaps the most complete is the one provided by [ 3 ] in the context of defining policies. The author defined digital competencies as follows: “the set of knowledge, skills, attitudes, strategies and awareness which are required when ICT and digital media are used to perform tasks, resolve problems, communicate, manage information, collaborate, create and share content, and build knowledge in an effective, efficient and adequate way, in a critical, creative, autonomous, flexible, ethical and a sensible form for work, entertainment, participation, learning, socialization, consumption and empowerment.” (p. 3).

In the same line, but in a scientific context [ 10 ], defined it as the composition of the following: “(1) technical competence, (2) the ability to use digital technologies in a meaningful way for working, studying and in everyday life, (3) the ability to evaluate digital technologies critically, and (4) motivation to participate and commit in the digital culture.” (p. 655).

Another definition of digital competence can be realized by approaching this concept through the frameworks defined in both the scientific context [ 8 ] and in that of government policies [ 15 ]. A notable example in the latter case is the Digital Competence Framework for Citizens (DigComp) created by the European Commission. This framework has gone through 3 fundamental versions. In the first one, DigComp 1.0 [ 16 ], 21 digital competencies (dimension 2) described in terms of knowledge, skills and attitudes were considered (dimension 4). Moreover, these competencies were organized in 5 areas (dimension 1): Information, Communication, Content-creation, Safety and Problem-solving. To assess how competent a citizen is, DigComp 1.0 proposes three proficiency levels (dimension 3): A (foundation level), B (intermediate level) and C (advanced level). The framework also includes examples of use on the application of digital competencies for different purposes (dimension 5).

The main contribution of the second version, DigComp 2.0 [ 17 ], was the redefinition of some concepts and terms (dimensions 1 and 2). However, it maintained the five areas of digital competencies of the first version. Finally, the third version, which was named DigComp 2.1 [ 18 ], established 8 levels of proficiency (dimension 3) instead of the 3 defined by DigComp 1.0. These levels were defined with consecutive numbers from 1 to 8 with the following distribution: Foundation (1 and 2), Intermediate (3 and 4), Advanced (5 and 6) and Highly specialized (7 and 8). It is important to note that DigComp 2.1 did not include an update to dimension 4 related to knowledge, skills and attitudes. Instead, the authors focused more on dimension 5, that is, by showing examples of use of the framework in the employment and learning contexts.

From the above-mentioned definitions, it is easy to conclude that measuring the degree or level of digital competence involves at least three areas: knowledge, skills and attitudes. In other words, the development of an instrument for measuring digital competencies should include assessment items related to these three competence areas.

Existing literature on the subject includes several works related to our research topic and type (secondary study). Table 1 summarizes the current related studies. These studies were selected from the systematic search explained in Sec. 4, but considering only secondary studies published in the period 2016-2020.

• PPT PowerPoint slide
• PNG larger image
• TIFF original image

https://doi.org/10.1371/journal.pone.0257344.t001

As seen in the table, only five studies addressed the topic of digital competencies evaluation. They are [ 8 , 11 , 19 – 21 ]. However [ 19 ], did not focus on higher education and the analyzed studies are only from Latin America. Similarly [ 8 ], did not focus on higher education and is not a systematic review. In the case of [ 11 ], the authors included some relevant factors for the evaluation process, such as the method of data collection (instrument) and the study area of the participants. However, it is not clear which works actually evaluated digital competencies. Regarding these two factors, the authors concluded that most of the studies use mixed methods or surveys for measuring, and are based on populations from different knowledge areas. An important limitation of this study is that it dates back to 2018, so more recent contributions are not present in the review. The study conducted by [ 20 ] found that programs and actions developed by the HEIs leave out the development of competencies in content creation and safety. To identify this gap, the authors used the DigComp 2.1 framework [ 18 ] as a reference. Finally, in the meta-analysis developed by [ 21 ], the authors found that, in the field of higher education in Latin America, the proportion of students and teachers with digital skills is moderate (64%), with no notable differences between both types of populations.

Regardless of the progress achieved by the above mentioned studies, some important aspects of the process of evaluating digital competencies in higher education remain unexplored. This is the case for the quality of the instruments employed for conducting such an evaluation.

3 Quality of quantitative instruments

When developing a quantitative instrument, it is important to assess its quality [ 7 , 28 , 29 ]. This is a process that mainly depends on assessing its reliability and validity [ 29 ]. From a psychometric perspective, the first property states whether the instrument provides the same (or similar) results under similar conditions or inputs, while the second one states whether it measures what is supposed to be measured [ 28 ]. Several methods to conduct these assessments have been proposed in the past [ 7 ] and have been classified into different categories. For instance, the detailed review provided by [ 30 ] identified four types of both reliability and validity assessments as shown in Table 2 .

https://doi.org/10.1371/journal.pone.0257344.t002

Ideally, a study developing or administering an instrument should present enough details about these eight assessments types. However, this is not always possible because the presence of research limitations (e.g., time, lack of another instrument to compare with, access to the participants). In any case, it is important to conduct an assessment of at least one of these types in order to guarantee a suitable degree of consistency and accuracy for the instrument [ 28 ]. Even if the instrument have been proposed and validated in a previous study, it is a good practice to check its reliability and validity [ 28 ].

3.1 Related work on quality evaluation of quantitative instruments

Critically evaluating the quality of quantitative instruments is not a new research topic and has been developed for quite some time in various areas of knowledge, such as education, psychology and health. In what follows, we will review some of the most important reported experiences, emphasizing the conclusions related to the quality of the instruments considered.

In [ 31 ] the authors focused on evaluating the quality of the methods used in high-quality trials of continuing medical education. Of the 136 studies selected, only 34.6% reported reliability or validity assessments. In the same context of medical education, in [ 32 ] the authors reviewed the instruments and questionnaires used for peer review published up to May 2010. In line with the results of the previous study [ 31 ], it was found that most of the questionnaires did not provide sufficient psychometric data.

Smokeless tobacco dependence measures were the focus of the review conducted in [ 33 ]. From the 4 selected studies, the authors conclude that the instruments analyzed have limitations in terms of reliability and validity. The same difficulties were detected in the critical synthesis developed in [ 34 ] on the so-called Implicit Relational Assessment Procedure, a computer-based psychological measure. From 31 studies published before March 2013, the authors conclude that although there is growing evidence of validity in the studies, they lack sufficient reliability to ensure replicability.

In a more extensive work where 53 studies published in the period 1995-2012 were included [ 35 ], reviewed studies that administered the Parenting Style and Dimensions Questionnaire instrument. In this case, the authors highlight that only a few studies involved complex reliability and validity assessments.

The apathy scales validated in generic and specific neurodegenerative disease populations was the focus of the review conducted in [ 36 ]. Of the 16 studies analyzed, the authors found a great heterogeneity of results. More specifically, the methodological quality of the studies ranged from poor to excellent.

In an educational context [ 37 ], analyzed the validity and reliability of the structured objective clinical evaluation (OSCE) with nursing students. By reviewing 19 papers published up to April 2016, the authors concluded that validity and reliability was adequate in most studies. However, considering that one of the selection criteria in the search conducted by the authors was precisely to include psychometric assessments, this result was somewhat expected. A more objective conclusion is obtained if the 14 studies excluded by the authors that did not meet this criterion are taken into account. In this sense, the 19 studies represent approximately 58% of the relevant studies.

A great heterogeneity of results was also observed by [ 38 ] in their evaluation on the replicability, comparability and validity of quality assessment tools for urban green spaces. This work was based on 15 primary studies published up to July 2019.

In [ 39 ] the authors summarized the instruments used to measure constructs of marital quality by analyzing 91 primary studies. As the authors indicate, most of the instruments reported include sufficient exploratory evidence of construct validity, but without explicitly defining the construct under study.

In the context of nursing education, the review developed by [ 40 ] aimed to determine how valid and reliable simulated patient scenarios are. Relying on 17 studies found in the Cumulative Index to Nursing and Allied Health Literature, the authors conclude that academics are inconsistent in developing both reliable and valid simulated scenarios.

Similarly, in [ 41 ] a comprehensive review of the literature was conducted on instruments measuring the competencies of special educators. In total, 20 instruments reported by 29 studies were characterized. The authors found that only 11 instruments (e.g. 55%) have evidence of reliability and validity assessment.

What these experiences tell us is that there are serious problems related to the quality of quantitative instruments. This is an issue that affects several areas of knowledge, including education sciences. However, the current state of the instruments used to measure digital competencies in higher education remains to be known. Thus, the results of our work will allow us to verify, among other things, to what extent this field would be affected by this issue.

4 Methodology

This paper follows the methodology described by [ 13 ] for conducting systematic mapping studies. In turn, this is a type of secondary study [ 42 ]. It is also a correlation study since we aim at analyzing the association between the variables under study.

According to the selected guide, mapping is achieved through three main steps: planning, conducting, and reporting. The following sections describe how the first two steps were developed, while the third one is fulfilled by writing this paper.

The selection and data extraction processes were carried out in parallel by two authors. In order to evaluate the concordance of the results of these processes, we relied on Cohen’s kappa ( κ ). However, it is important to clarify that although we could have used other more sophisticated indicators [ 43 ], we consider Cohen’s kappa to be sufficient for our purposes. Our decision is in line with previous research such as [ 44 ], where Cohen’s kappa was employed for similar aims.

4.1 Research questions

The main goal of this research is to provide an overview of how literature related to the evaluation of digital competencies in higher education reports on the quality of the employed instruments (e.g. questionnaires). We consider the literature published during the period from January, 2010 to July, 2020, which is a time frame commonly used in literature reviews in the field [ 9 , 12 , 45 ].

More specifically, we were interested in answering the following research questions:

• RQ1) . What are the main demographic and methodological features of the studies evaluating digital competencies in higher education?
• RQ2.1) . What types of assessments and what specific methods are most often reported?
• RQ2.2) . How comprehensive and how deep are these quality reports?
• RQ2.3) . What studies achieve the best balance between coverage and depth?
• RQ3.1) . How do quality reporting practices evolve over time?

To design the search formula for finding the relevant studies, we proceeded as follows. First, we used the PICO tool [ 46 ] to identify the relevant terms according to the population under study, intervention method, comparison group, and outcomes. Second, careful readings of similar reviews such as that of [ 11 , 21 ] were useful in order to complement the results of applying the PICO tool. As a result, the following search formula was defined:

(“digital competence” OR “digital literacy” OR “digital literacies”) AND (undergraduate OR postgraduate OR freshmen OR sophomore OR junior OR senior OR preservice OR teacher OR junior OR university OR “higher education” OR college OR tertiary OR “academic staff” OR professor OR lecturer) AND (evaluate OR assess OR appraise OR validate OR evaluation OR assessment OR appraisal OR validation OR evaluation OR assessing OR appraising OR validating)

This formula was used to search three relevant databases: Scopus, Web of Science, and ERIC (Education Resources Information Center). These databases cover a great part of the scientific literature about Education Sciences and are widely used in the review studies related to this field [ 11 , 12 ].

The results obtained from applying the above formula are shown in Table 3 . It is worth noting that the search was conducted in July 2020.

https://doi.org/10.1371/journal.pone.0257344.t003

4.3 Selection of the studies

During the selection, we followed several steps which are summarized in Fig 1 . We considered the following as inclusion and exclusion criteria:

Inclusion criteria :

• Studies measuring digital competencies quantitatively in the context of higher education
• Studies published in the period of 2010 to July, 2020
• Studies published as journal articles

Exclusion criteria :

• Studies published in conference proceedings, book chapters
• Non-peer reviewed studies
• Studies published in books, technical reports, editorials
• Duplicates of other studies

https://doi.org/10.1371/journal.pone.0257344.g001

The selection process was conducted by two authors independently, from the Screening step up to the Eligibility step ( Fig 1 ). In the first case, the observed agreement and accounting for chance agreement were p 0 = 0.928 and κ = 0.856 (p-value< 0.001), respectively. As for the Eligibility step, these values were p 0 = 0.973 and κ = 0.938 (p-value< 0.001), respectively. In the case of the 36 studies rejected after reading their full texts, the particular reasons were as follows: 23 due to the exclusive use of qualitative data collection techniques, and 13 for not specifically assessing digital competencies.

4.4 Data extraction

Data extraction was conducted based on the template depicted in Table 4 . As it can be observed, 10 variables related to both demographic and methodological features were considered. Regarding reliability and validity assessments, 4 different types were included for both groups (8 in total) [ 30 ]. For each type, we recorded the extent to which the study reported on the assessments, that is, Not mentioned , Only mentioned , Referenced (a previous study), and Details are provided . These dimensions were mapped into numerical values (0, 1, 2 and 3, respectively) in order to compute specific indicators for characterizing the studies’ reporting practices. Such indicators are described in the next section.

https://doi.org/10.1371/journal.pone.0257344.t004

As in the selection process, data extraction was carried out by two authors in order to mitigate personal biases, especially when evaluating studies. As a result, the observed agreement and accounting for chance agreement were p 0 = 0.965 and κ = 0.904 (p-value< 0.001), respectively.

4.5 Analysis and classification

Analysis and classification of the studies was carried out after data extraction. The obtained results were tabulated and visually summarized as shown in Sec. 5. A complete list of the reviewed studies and their corresponding classification according to the used data extraction template can be located at the following link https://osf.io/me36k .

In order to characterize the reporting practice of the studies, we defined three indicators, which are computed from the data extracted. They measure three different features of the studies. The first one, which we have called External Coverage represents how exhaustive the study is in conducting both reliability and validity assessments of any kind. Here, three cases are possible: 1) the study has not reported on any group of assessments, 2) the study reported on a single group (reliability or validity), and 3) the study reported on both groups. These three cases were labeled and numerically coded as None = 0.0, Moderate = 0.5 and High = 1.0, respectively.

Similarly, the second indicator, that we named Internal Coverage , is devoted to measure how comprehensive the study is inside the group of assessments it reported on. We have defined this indicator as the proportion between the number of assessment types conducted by the study and the number of possible assessments within the reported group or groups. Given that each group (reliability or validity) has 4 types of assessments, the total of possible assessments will be 4 if the study only reports on a single group, while it will be 8 if it reports on both groups at the same time. In turn, this indicator will range from 0 (conducting no assessment types at all) to 1 (conducting all assessment types withing the group or groups), with other values in between corresponding to several degrees of coverage.

The third indicator, Reporting Depth , measures how deep the study reports on the conducted assessments, that is, regardless of its external or internal coverage. It is computed as the normalized average from the study’s reporting levels achieved in the types of assessment that were conducted. These reporting levels are the ones defined in Table 4 , which also have numerical codes. For example, a study conducting only one type of assessment with a reporting level of Referenced = 2 will have a Reporting Depth of 2/(1 ⋅ 3) = 0.667. Note that we divided by 1 ⋅ 3 because only one (1) type of assessment was conducted, while the maximum value a reporting level that may be achieved is 3 (corresponding to Details are provided ). Therefore, this indicator ranges from 0 (no depth) to 1 (maximum depth). Of course, the latter case corresponds to the studies providing details in all of the conducted assessments.

From these indicators it is possible not only to rank the studies, but also to characterize their reporting practices. Note that this is necessary in order to answer research questions RQ2.3 , RQ3 , and RQ3.1 . Here, three different approaches from multi-criteria decision analysis can be adopted [ 47 ]: 1) Full aggregation approach, 2) Outranking approach, or 3) Goal, Aspiration or Reference-level approach. Since, in our context, it is possible to certainly know both the ideal and the anti-ideal studies according to the three indicators we defined above, the third approach was adopted. Specifically, we applied the Technique of Order Preference Similarity to the Ideal Solution (TOPSIS) method [ 48 ], which assumes that the best alternative (study) is the one with the shortest distance to the ideal alternative and the furthest distance from the anti-ideal alternative [ 47 ]. Note that the ideal study is the one with External Coverage = Internal Coverage = Reporting Depth = 1. Contrarily, the anti-ideal study is the one with External Coverage = Internal Coverage = Reporting Depth = 0. Following the steps from the TOPSIS method [ 47 ], we computed the Relative Closeness coefficient for each study using euclidean distance and the same weights for the three indicators. This coefficient ranges from 0 to 1, where a value approaching 1 means that the study is close to the ideal study, while a value approaching 0 means the opposite. Conceptually, this ratio provides a good insight into how satisfactory the quality assessment reporting process was in the study.

https://doi.org/10.1371/journal.pone.0257344.g002

In addition to the above, we considered it fair to evaluate the studies from the point of view of their dissemination in the literature. In this sense, we have considered an indicator that we have called the Dissemination Index , which quantifies the degree to which the study has been cited by other works. Although this indicator does not accurately capture the real use of the instrument proposed by the study, it does give us an estimated idea of its impact on the scientific community. Formally, we have calculated this index as the rate of the number of citations of the study divided by the number of years it has been published. In this way, we seek to mitigate the possible advantage in the number of citations that studies published in earlier years may have over those published more recently. This index is a non-negative continuous magnitude with a minimum value of 0. A value close to 0 indicates that the study had low dissemination. The number of citations for each study was obtained from Google Scholar (scholar.google.com), while the calculation of the number of years was based on the year 2021. We selected Google Scholar for two reasons. The first is that this database has broad coverage (not only of the scientific literature, but also of the so-called gray literature) [ 49 ], and on the other hand, it serves as an independent reference to mitigate the bias of considering citations exclusively by the databases used in our systematic mapping study.

Finally, it is worth mentioning that, in order to find out what demographic and methodological features explain studies’ reporting practices ( RQ3.1 ), we proceeded with an association analysis. Specifically, we conducted a Pearson’s Chi-squared test for testing whether each of the demographic and methodological variables are significantly associated with the Reporting Practices . We supplemented this analysis by calculating Cramér’s V , which is an effect size measurement for the Chi-square test [ 50 ]. Although other indicators could be equally effective in characterizing the strength of association (e.g., V 2 ), we decided on this measure because it is intuitive and common in the context of association analysis of nominal categorical variables [ 50 ]. Further, for those significant associations (p-values below 0.05), we proceed with a post hoc analysis based on the standardized residuals of the Pearson’s Chi-squared test [ 51 ]. To address RQ3.1 , we relied on an ordinal logistic regression model for describing Reporting Practice as a function of the variable Year . We have selected this regression model because of the nature of the variables involved.

4.6 Validity assessment

As suggested by [ 13 ], the validity of a systematic mapping study should be conducted by analyzing the main threats occurring in achieving the following validity types:

• Descriptive validity . The main thread here is that subjective studies have less descriptive validity than quantitative ones. However, our study is based in the count of data using a well-structured template (see Table 4 ). Consequently, we assumed that this threat is under control.
• Theoretical validity . Study selection and data extraction are important sources of threats affecting this validity type. However, we employed snowballing sampling (backward and forward) in order to mitigate the possibility of not including relevant studies. Additionally, authors reviewed each other’s steps in order to control the bias when selecting and extracting data. Regarding the quality of the sample of studies, it is clear that it is high since it comes from databases with great coverage of high quality venues.
• Generalizability . To assess this type of validity, we have to consider the internal and external generalizability of the results and methodology. In the first case, it is clear that results from this research can be generalized in the context of digital competence evaluations in higher education. So, internal generalizability is guaranteed. External generalizability is not guaranteed since digital competencies are not only measured in the context of higher education. Additionally, we have focused on quantitative instruments only, so our results describe an specific group of studies. With respect to the methodology, since systematic mapping studies and association analysis are general research methods, internal and external generalizability is guaranteed.
• Interpretive validity . In this case, threats may exist here because authors have worked together in previous research. So, it is possible that similar judgments when selecting and analyzing the primary studies arose.
• Reproducibility . We consider that reproducibility is guaranteed. This is because enough details are provided in this paper, that is, by following a systematic guide like the proposed by [ 13 ].

In the following section, we present the results obtained from our analysis and classification of the studies. We organized them according to the research questions defined in Sec. 4.1.

5.1 Demographic and methodological features (RQ1)

Fig 3 summarizes the distribution of studies according to demographic features. More specifically, Fig 3a shows that evaluating digital competence in higher education is a research topic with an increasing number of studies over time. The special case of July 2020, which corresponds to the first half of 2020, contains 13 studies. A number that is clearly higher than half that of the previous year. Fig 3b and 3c indicate that most of the studies originate from Europe (47%), and are based on undergraduate students (63%) coming from Social Sciences programs (53%). Regarding the reputation of the journal, different results arise from the SJR and JCR indicators. While for the SJR, most of the studies have been published in venues with a quartile assigned ( Fig 3e ), the opposite occurs for the JCR indicator ( Fig 3f ). In the latter’s case, only 22% of the studies were published in top-tier journals (Q1 or Q2).

https://doi.org/10.1371/journal.pone.0257344.g003

Taking into account the considered methodological features, Fig 3 reveals some interesting patterns. For instance, in regard to the sample size, Fig 3g shows that Large size samples are barely more frequent (38%) studied than the others. From Fig 3h it is clear that researchers have been focused in measuring several domains of digital competencies (78%) at the same time. More specifically, the employed instruments are mainly ad hoc (64%), that is, proposed by the authors of the study as shown in Fig 3i . According to Fig 3j , such measurements have been mostly self-assessments (79%), that is, based on participants perceptions about their own level of digital competence.

5.2 Reporting practices of quality assessments (RQ2)

In this section, we answered several questions about how the studies reported on the quality of the employed instruments. To this end, Fig 4 summarizes the main results we obtained.

https://doi.org/10.1371/journal.pone.0257344.g004

5.2.1 What types of assessments and what specific methods are most often reported? (RQ2.1).

From Fig 4a , it is possible to observe that Internal Consistency is the most reported type of reliability assessment by far. About 50% of the studies provided enough details or referenced another study. Consistent with this result, Fig 4b shows that 86% of the studies conducting reliability assessments relied on Cronbach’s alpha, which is indeed a typical method for measuring internal consistency [ 30 ].

A different situation occurs with the validity assessments ( Fig 4c ). In this case, the studies are more evenly distributed. However, it is interesting to see that although Content validity is the most frequently reported assessment, about the half of the studies reporting on this provided no more detail than just mentioning that they did it. Fig 4d is more specific on how the assessment was conducted. Expert judgment , a typical method for content validity, is present in 36% of the studies conducting validity assessments. Interestingly, Exploratory Factor Analysis (EFA) and Confirmatory Factor Analysis (CFA) are present in 25% of the studies. They are used for conducting construct validity assessments.

5.2.2 How comprehensive and how deep are these quality reports? (RQ2.2).

To characterize the reporting practices of the studies, we relied on the indicators described in Sec. 4.5. From Fig 4e , we can see that, in the case of External Coverage , 49% of the studies present a high coverage (reporting reliability and validity at the same time). However, 36% did not report on any group at all, while 15% did it moderately (that is, on a single group). Regarding the other two indicators, box plots from Fig 4f show that while more than 50% of studies achieve extreme values of Reporting Depth (0 or 1), in Internal Coverage , the maximum value is 0.75. In both cases, we see that at least the 25% of studies (1st Quartile) are equal to the minimum value of the 2nd Quartile, which is indeed 0. This is consistent with 36% of the studies with no External Coverage , as shown in Fig 4e . In fact, having no External Coverage at all implies no Internal Coverage and no Reporting Depth . So, the 36% of studies that are in this latter case correspond to those without any quality reporting practice at all.

5.2.3 What studies achieve the best balance between coverage and depth in their reports? (RQ2.3).

Table 5 shows the studies arranged in descending order according to their Relative Closeness (the indicator we defined in Sec. 4.5). Also, note that in the first column, we have included the ordinal variable Reporting Practice , which allows for categorizing the studies through three groups: None , Acceptable , and Good . Studies belonging to the first group, which are also the ones with Relative Closeness = 0, were omitted from Table 5 due to space limitations. However, they can be accessed in the resource provided in Sec. 4.5.

https://doi.org/10.1371/journal.pone.0257344.t005

To complement this analysis, the rightmost column of Table 5 contains the dissemination index of the studies. It is easy to see that this characteristic allows us to classify the studies differently than Relative Closeness. In order to facilitate the identification of studies with high dissemination, we have divided the set of studies into tertiles. In the first one (T1) are the studies with dissemination index in the interval [0, 2.5], in the second (T2) those with values in the interval (2.5, 7.33], and in the third one (T3) those with the values in (7.33, 64.7]. Of course, those belonging to the T3 are the most widely disseminated studies of all. From these intervals, it is noteworthy that the distribution that the studies follow is not normal (e.g., it is highly skewed to the right), indicating that most of the studies have low dissemination rates compared to the highest value study (64.7). The 14 studies belonging to the latter category are marked with * in Table 5 . Note that these studies have a dissemination index greater than or equal to 8.0 and would be, according to our analysis, those that achieve an adequate balance between quality and dissemination. To obtain an overview of how the studies are distributed according to these two characteristics, Table 6 summarizes the number for each variable level of Reporting Practice and Level of Dissemination . The results show a generally homogeneous distribution among the combinations of levels, although it is remarkable that the lowest number of studies (5) corresponds to those with Good reporting practices and Low levels of dissemination. A Chi-squared test confirms that there are no differences between these groups of studies ( χ 2 = 2.897, df = 4, p-value = 0.575).

https://doi.org/10.1371/journal.pone.0257344.t006

5.3 What studies’ characteristics are more likely associated with certain reporting practices? (RQ3)

Table 7 shows the results obtained for an association analysis between Reporting Practice and studies’ demographic and methodological variables. Note that we also included the indicator of Cramér’s V for quantifying, in the range [0, 1], the strength of the association between variables. The larger this indicator, the stronger is the association between the variables. As Table 7 shows, only Measured dimension resulted in a significant association with Reporting Practice (p-value < 0.05). Cramér’s V indicates a low strength in this association (= 0.303).

https://doi.org/10.1371/journal.pone.0257344.t007

In order to find out which dimensions account for this significant association, we proceed with a post hoc analysis of the standardized residuals [ 51 ]. Table 8 summarized the results for this analysis. We observe that only one pair of dimensions was significantly associated ( Several and None ). This negative association means that studies focused on measuring several areas of digital competence are more likely to report on quality assessments.

https://doi.org/10.1371/journal.pone.0257344.t008

5.3.1 How do quality reporting practices evolve over time? (RQ3.1).

Finally, we addressed the question of how studies’ reporting practices have evolved over the years. In this case, we estimated an ordered logistic regression model [ 99 ]. The corresponding model resulted in a significant coefficient for Year (= 0.279) and a standard error of 1.373 e − 04. Brant’s test for checking the proportional odds assumption [ 100 ] gave a χ 2 = 0.040 and a probability (p-value) equal to 0.850. Therefore, the relationship between each pair of outcome groups is the same under the coefficient estimate.

In order to better understand the effects of this coefficient, Fig 5 shows the predicted probabilities of each dimension of Reporting Practice as a function of the years. This plot shows a clear pattern in the evolution of the studies’ reporting practices. Note that the dominant dimension was None (no reporting at all) up to 2017. From this year up to the present (July 2020), Acceptable and Good reporting practices become more likely to appear in literature. We may also observe a decreasing trend over time in the predicted probabilities related to practice of no reporting quality assessments ( None ). On the contrary, predicted probabilities related to the practices of providing acceptable and good reports increase along the years. In the specific case of Acceptable reporting practices, the probabilities become steady from 2017 to the present.

The shaded ribbons correspond to the 95% confidence interval of the predicted values.

https://doi.org/10.1371/journal.pone.0257344.g005

6 Discussion and conclusion

From the results obtained, we observe that the number of studies evaluating digital competence in higher education has grown from 2010 to the present. This increasing behavior has been also reported by similar studies such as [ 11 , 27 ]. Another important pattern arising from our study is the predominance of studies coming from Europe and which are based on undergraduate students from Social Sciences programs. This is somehow expected if we consider that evaluating digital competence in higher education is a research topic related to this discipline in this geographical area. So, these results suggest that researchers have been focused on studying what is perhaps the closest population to them: undergraduate students from Social Sciences. Other higher education actors from different disciplines, continents or with different roles (e.g., academic staff) have been less or not studied at all. This is consistent with the results reported in other similar studies [ 101 ], and this gap constitutes a clear opportunity for further research studying those underrepresented populations.

As a positive aspect, we observed most of the studies have been based on medium or large samples (involving more than 100 participants). Additionally, our results revealed that ad hoc questionnaires for measuring several dimensions of digital competencies were in the preference of the researchers. More specifically, the employed instruments mostly relied on the self-assessment of the participants. Although measuring several dimensions is a positive aspect of the studies, the high presence of self-assessment, ad hoc instruments is, from our viewpoint, a weakness in this field. In the case of ad hoc instruments, it would be a symptom of the lack of consensus on what digital competencies are in the context of higher education [ 11 ], or perhaps a consequence of divergent research purposes. In any case, this undermines the reuse of existing instruments, which indisputably affects the opportunity to improve and validate them on a large scale. As a related consequence, the reported results (even for populations with similar characteristics) are increasingly heterogeneous as time passes, making it difficult to draw general and precise conclusions from them.

With respect to the quality of the venue where the studies were published, different results from the SJR and JCR indicators were observed. While most studies appear in journals recognized by the SJR indicator, this is not the case for the JCR indicator. These differences are explained in part by the large coverage of SJR, as compared to that of JCR [ 102 ]. Since JCR is a more exclusive indicator, the results suggest that, under this indicator, most of the studies are published in low-tier journals.

In regard to the reported quality assessments, the obtained results are somewhat consistent with the use of questionnaires as data collection methods. It is for this reason that assessments of internal consistency (through Cronbach’s alpha), face validity (through pilot studies), content validity (through Expert judgment ) and construct validity (through factor analysis) abound. Taking into account that, in this list, only internal consistency is a type of reliability assessment, it is clear that most studies report more types of validity than reliability assessments.

From the proposed indicators for appraising studies’ reporting practice, we observed that less than half of the studies conducted assessments from reliability and validity at the same time ( External Coverage ). In addition, the proportion of types of assessments conducted within each group ( Internal Coverage ), was lower than 50%, which indicates poor coverage. Regarding the depth in providing evidence supporting the quality assessments ( Reporting Depth ), the results showed that more than the half of the studies provided good levels. Overall, these results indicate that serious issues exist when conducting and reporting quality assessments of the employed instruments. It seems, however, that this is not a problem typical to digital competence evaluation in higher education, but a more general one in Educational Sciences. Studies such as that of [ 32 ] in the context of medical education and, more recently [ 41 ], in special education identified similar problems. Interestingly, we found no association between the degree of dissemination of the studies and their quality. This result indicates that studies with low quality in their reporting practices have statistically the same dissemination, in terms of citations, as those with acceptable or good quality.

The statistical analysis conducted, aimed at identifying what studies’ features could help to understand certain reporting practices, revealed just a few insights. We found only one significant, negative association between measuring several digital competence areas at the same time, and no reporting of any quality assessment at all. This suggests that those studies devoted to measuring several areas of digital competence are more likely to report on quality assessments of the employed instruments. Although in part explained by the lack of enough data, the absence of significant associations in the case of the other variables is a warning sign. For instance, we might expect that those studies published in top-tier venues are more likely to exhibit good reporting practices. In the same line, we would like to see more concern in conducting, and hence, reporting quality assessments by those studies using ad hoc, self-assessment instruments. However, no evidence was found supporting these beliefs. The good news is that, according to our estimates, the tendency to report more and with better quality evaluations of the instruments used to measure digital competencies in higher education is growing over time. Although slight, this growth is significant.

In summary, it is clear that, based on the collected evidence, we can not trust in all of the available instruments published till date. The list of studies we provided, which rank them according to their degree of coverage and depth in reporting quality assessments, is expected to help researchers and practitioners in identifying relevant instruments to advance in the field.

7 Implications

The results obtained by our study have important implications from both practical and research perspectives. First, our demographic and methodological characterization of the studies implies that practitioners and policymakers have a rich body of prior experience that can be taken into account when measuring digital competencies in higher education settings. Similarly, researchers have clear opportunities for future research. The evaluation of less studied populations (e.g., university professors from regions such as South America, North America or Africa) or validating existing instruments through comparative studies, are two examples of these opportunities.

Second, the fact that more than half of the studies do not conduct reliability and validity assessments at the same time, and those that do, cover less than half of the criteria within each category, implies that the selection of an existing instrument to measure digital competencies in higher education should be done with care. From a research perspective, this result implies that more emphasis should be placed on ensuring that the instruments used meet adequate levels of reliability and validity. Research opportunities exist in this context, especially related to the development of validation studies that are based on proposed instruments with little evidence of quality assessments.

Third, the fact that the quality and dissemination of the studies do not correlate is a warning sign regarding the selection of the most appropriate instruments in higher education. This implies that the practitioner, as well as the researcher, should not be guided purely by the quantity of citations of the study, but also the quality of the instrument in terms of the psychometric properties that we have considered in this paper. In the same line, our results imply that the selection of instruments based on demographic or methodological similarities is not reliable. Therefore, emphasis should be placed on the specific evidences reported by the studies.

Fourth, the evolution of studies towards the inclusion of a greater number of quality assessments has as its main implication that, in the near future, it should be easier to find better validated instruments to measure digital competencies in higher education. This augurs a favorable scenario for practitioners and researchers. However, much remains to be done. The great heterogeneity of approaches to evidence the quality of a quantitative instrument is a clear indication of the absence of a “standard” in this field of research. Future research could focus on proposing, for each measurement scenario, which methods should be applied to ensure the reliability and validity of the instrument under consideration.

In a more general context, the fact that academia is placing increasing interest in measuring digital competencies is a good sign that awareness is growing of the importance of understanding and monitoring the achievement of the UN Sustainable Development Goals (SDGs) [ 103 ]. As precisely mentioned in [ 104 ], ICTs are considered key catalysts for the achievement of the 17 SGDs. In this context, our results contribute to raising awareness of the importance of correctly measuring digital competencies in higher education, a key step to know how far we have progressed and what still needs to be done.

8 Limitations

Regardless of the relevance of the results obtained, this research has important limitations. First, we based our results on a limited sample extracted only from journal articles that were published in the last 10 years. Therefore, we do not know the extent to which these results also apply to studies published in different venues and on different dates. Similarly, our study is limited by the variables used for the characterization of the studies. Therefore, there is a possibility that other variables (not included here) not only better describe the studies, but also more appropriately explain the presence of certain reporting practices.

Another important limitation is that we have evaluated the studies according to the practices and levels (depth) with which they report quality assessments. Therefore, in no way do our results indicate which instruments are more appropriate for measuring digital competence in higher education. We are aware that this is a much more complex task which depends on several factors including context. Therefore, in the future, it will be necessary to develop more research to answer these and other related scientific questions. In this sense, an important question is, “How do qualitative data collection techniques guarantee the quality of measurement?” Our future research will focus on providing answers to these issues.

• 1. Konsbruck Robert L. Impacts of information technology on society in the new century; 2002. [Cited 12 December 2020]. Available from: https://www.zurich.ibm.com/pdf/news/Konsbruck.pdf .
• View Article
• Google Scholar
• 3. Ferrari A. Digital Competence in Practice: An Analysis of Frameworks. Institute for Prospective Technological Studies (Joint Research Centre); 2012. Available from: https://op.europa.eu/s/pcz6 .
• 6. Ala-Mutka K, Punie Y, Redecker C. Digital competence for lifelong learning. European Commission, Joint Research Centre, Institute for Prospective Technological Studies; 2008.
• 7. Mueller RO, Knapp TR. Reliability and validity. In: Hancock GR, Stapleton LM, Mueller RO, editors. The Reviewer’s Guide to Quantitative Methods in the Social Sciences. 2nd ed. Routledge; 2019. p. 397–401.
• 8. Prendes MP, Gutiérrez I, Martínez F. Competencia digital: una necesidad del profesorado universitario en el siglo XXI. Revista de Educación a Distancia (RED). 2018;(56).
• 15. Johnson L, Levine A, Smith R, Stone S. The 2010 Horizon Report. Austin, Texas: The New Media Consortium; 2010. Available from: https://files.eric.ed.gov/fulltext/ED510220.pdf .
• 16. Ferrari A. DIGCOMP: A Framework for Developing and Understanding Digital Competence in Europe. Eur 26035 ed. Luxembourg: Publications Office of the European Union; 2013.
• 17. Vuorikari R, Punie Y, Carretero S, Van Den Brande L. DigComp 2.0: The Digital Competence Framework for Citizens. Update Phase 1: The Conceptual Reference Model. Eur 27948 en ed. June. Luxembourg: Publication Office of the European Union; 2016.
• 18. Carretero S, Vuorikari R, Punie Y. DigComp 2.1: The Digital Competence Framework for Citizens with eight proficiency levels and examples of use. Eur 28558 en ed. Luxembourg: Publications Office of the European Union; 2017.
• 22. Gibson PF, Smith S. Digital literacies: Preparing pupils and students for their information journey in the twenty-first century. Information and Learning Science,. 2018.
• 28. Bandalos DL. Measurement Theory and Applications for the Social Sciences. Methodology in the Social Sciences. Guilford Publications; 2018.
• PubMed/NCBI
• 46. Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, et al. Cochrane Handbook for Systematic Reviews of Interventions. Wiley Cochrane Series. Wiley; 2019.
• 47. Ishizaka A, Nemery P. Multi-Criteria Decision Analysis. John Wiley & Sons Ltd; 2013.
• 50. Kvålseth TO. Association Measures for Nominal Categorical Variables. In: Lovric M, editor. International Encyclopedia of Statistical Science. Berlin, Heidelberg: Springer Berlin Heidelberg; 2011. p. 61–64. Available from: https://doi.org/10.1007/978-3-642-04898-2_122 .
• 87. Harati A, Rahmatizadeh S, Valizadeh-Haghi S. Allied medical sciences students’ experiences with technology: Are they digitally literate? Library Philosophy and Practice. 2018;2018.
• 99. Harrell FE. Ordinal Logistic Regression. In: Regression Modeling Strategies: With Applications to Linear Models, Logistic and Ordinal Regression, and Survival Analysis. Cham: Springer International Publishing; 2015. p. 311–325.
• 103. SDGs. The sustainable development goals report 2019. United Nations publication issued by the Department of Economic and Social Affairs; 2019. Available from: https://undocs.org/E/2019/68 .
• 104. Nations U. ICTs as a catalyst for sustainable development; 2016. Available from: https://sustainabledevelopment.un.org/index.php?page=view&type=20000&nr=579&menu=2993 .

Quantitative research in education : Journals

• Computers and education "Computers & Education aims to increase knowledge and understanding of ways in which digital technology can enhance education, through the publication of high-quality research, which extends theory and practice."
• Journal of educational and behavioral statistics "Cosponsored by the American Statistical Association, the Journal of Educational and Behavioral Statistics (JEBS) publishes articles that are original and useful to those applying statistical approaches to problems and issues in educational or behavioral research. Typical papers present new methods of analysis."
• Research in higher education "Research in Higher Education publishes studies that examine issues pertaining to postsecondary education. The journal is open to studies using a wide range of methods, but has particular interest in studies that apply advanced quantitative research methods to issues in postsecondary education or address postsecondary education policy issues."
• << Previous: Recent print books
• Next: Databases >>
• Background information
• Recent e-books
• Recent print books
• Connect to Stanford e-resources
• Last Updated: Jan 23, 2024 12:46 PM
• URL: https://guides.library.stanford.edu/quantitative_research_in_ed

• Technical Support
• Find My Rep

You are here

Quantitative Research in Education A Primer

• Wayne K. Hoy - Ohio State University, USA
• Curt M. Adams - University of Oklahoma, USA
• Description

“ The book provides a reference point for beginning educational researchers to grasp the most pertinent elements of designing and conducting research… ”

— Megan Tschannen-Moran, The College of William & Mary

Quantitative Research in Education: A Primer, Second Edition is a brief and practical text designed to allay anxiety about quantitative research. Award-winning authors Wayne K. Hoy and Curt M. Adams first introduce readers to the nature of research and science, and then present the meaning of concepts and research problems as they dispel notions that quantitative research is too difficult, too theoretical, and not practical. Rich with concrete examples and illustrations, the Primer emphasizes conceptual understanding and the practical utility of quantitative methods while teaching strategies and techniques for developing original research hypotheses.

The Second Edition includes suggestions for empirical investigation and features a new section on self-determination theory, examples from the latest research, a concluding chapter illustrating the practical applications of quantitative research, and much more. This accessible Primer is perfect for students and researchers who want a quick understanding of the process of scientific inquiry and who want to learn how to effectively create and test ideas.

See what’s new to this edition by selecting the Features tab on this page. Should you need additional information or have questions regarding the HEOA information provided for this title, including what is new to this edition, please email [email protected] . Please include your name, contact information, and the name of the title for which you would like more information. For information on the HEOA, please go to http://ed.gov/policy/highered/leg/hea08/index.html .

For assistance with your order: Please email us at [email protected] or connect with your SAGE representative.

SAGE 2455 Teller Road Thousand Oaks, CA 91320 www.sagepub.com

“This text will definitely be useful in providing students with a solid orientation to research design particularly in quantitative research”

“Precision, precision, precision! I think this is a must have companion text for graduate students who have to complete a thesis or dissertation. The author does an outstanding job of cataloging and describing difficult research methods terms in a clear and concise way.”

“Greatest strength is the comprehensiveness of the treatment”

“A reference point for beginning educational researchers to grasp the most pertinent elements of designing and conducting research”

Provides all the essential information for quantitative research in a concise book.

A book on research in education but quite well can be accommodated into other social science areas. A great easy to follow publication especially if someone is new to statistical analysis.

There are two strong chapters in this publication that are clearer and more relevant that the sources presently being used by my students. Chapter 3 is particularly well written and clear and builds a progression in terms of understanding statistics. Chapter 4 is also effective however I would probably place this before Chapter 3. In terms of detail there is probably too much in Chapter 4 on Hypothesis whereas Chapter 3 could be developed perhaps by the inclusion of more examples.

Very helpful book that provides a basis for students undertaking education based research.

For those that are interested in doing research that is quantitative in nature, this book is useful, although we tend to advise a more qualitative approach. Therefore I can see myself dipping in and out of this book as it provides some good explanations and there is follow through. I would have welcomed more working examples as this would have concretised everything a lot more.

This is a good supplement to the research methods module, especially for those students who are entering into the field of education. The quantitative methods discussed are also transferrable to other subjects.

NEW TO THIS EDITION:    

• A new chapter devoted to the practical applications of education research uses the concepts of collective trust, organizational climate, and improvement science to illustrate the utility of a quantitative approach. It also offers guidelines for analyzing and improving the practice of research in education.
• New hypotheses found in a variety of research studies are available for readers to analyze and diagram.
• A new section on self-determination theory has been added to demonstrate the relation between theory and practice.
• A new section on self-regulatory climate gives readers an opportunity to explore an exciting new area that they are likely to encounter in practice.  
• A conceptual description of Hierarchical Linear Modeling (HLM) has been added to help readers understand statistical data organized at more than one level.    

KEY FEATURES:  

• Education-specific concrete examples bring concepts to life and engage readers with relevant, meaningful illustrations.
• Check Your Understanding exercises and questions assess the reader’s ability to understand, value, and apply the content of the chapter.  
• Strat egies and techniques for generating hypotheses help readers understand the process of creating their own hypotheses.
• Key Terms are highlighted in the text when they first appear and then summarized in a list at the end of the chapter to help reinforce key concepts.
• A Glossary concisely and clearly defines all the key terms in the text so readers have immediate access to ideas and concepts needing review.
• Charts throughout the text allow readers to select appropriate statistical techniques for given scenarios.
• The Diagramming Table (in Chapter 4) enables readers to diagram and dissect hypotheses by ensuring the key elements of a hypothesis are considered, analyzed, and understood.
• An Elements of a Proposal section (Appendix A) gives readers directions for developing a quantitative research plan and motivates readers to get started—the most difficult step for many.
• The A Few Writing Tips section (Appendix B) lists a number of salient writing suggestions to help readers avoid common mistakes found in formal writing.

Sample Materials & Chapters

For instructors, select a purchasing option.

• Tools and Resources
• Customer Services
• Original Language Spotlight
• Alternative and Non-formal Education 
• Cognition, Emotion, and Learning
• Curriculum and Pedagogy
• Education and Society
• Education, Change, and Development
• Education, Cultures, and Ethnicities
• Education, Gender, and Sexualities
• Education, Health, and Social Services
• Educational Administration and Leadership
• Educational History
• Educational Politics and Policy
• Educational Purposes and Ideals
• Educational Systems
• Educational Theories and Philosophies
• Globalization, Economics, and Education
• Languages and Literacies
• Professional Learning and Development
• Research and Assessment Methods
• Technology and Education
• Share This Facebook LinkedIn Twitter

Article contents

Digital culture and qualitative methodologies in education.

• Eliane Schlemmer Eliane Schlemmer Universidade do Vale do Rio dos Sinos - UNISOS
• https://doi.org/10.1093/acrefore/9780190264093.013.508
• Published online: 30 September 2019

From a digital culture perspective, this article has as main objective to assess two contemporary qualitative research methods in the field of education with distinct theoretical orientations: the cartographic method as a way of tracing trajectories in research-intervention with a theoretical basis in the biology of knowledge, enactive cognition and inventive cognition; and the cartographic method as a means of identifying and mapping the controversies linked to the different associations between human and non-human actors with a theoretical basis in actor-network theory (ANT). With their own specificities, both methods have been fruitful in the development of qualitative research in the field of education, in the context of digital culture, and more recently, in the hybrid culture of atopic habitation, mainly because they also relate to equally consistent theories and aspects of human cognition, making it possible to detect traces and clues in the fluid associations between actors enhanced by different digital technologies (DT), including data mining and learning analytics. From the Brazilian perspective on the topic, this article approaches the experience of the cartographic method of research intervention as well as the cartography of controversies as tools for developing qualitative research in education. These different forms of the cartographic method have inspired the construction of didactic-pedagogical experiences based on theoretical approaches linked to cognition, producing inventive methodologies and interventionist pedagogical practices. These methodologies and practices, which will be discussed at length in this article, have been developed and validated by the Research Group in Digital Education at Unisinos University at different levels and in varied educational settings.

• qualitative research
• cartography
• digital culture
• multimodality
• pervasiveness

Introduction

A version of this article in its original language.

A social group’s socialization is made visible by a distinctive way of acting, which develops rituals representing emotions, common values, and norms for coexistence, all of which contribute to constructing a culture. This makes it possible to refer, for example, to a pre-digital, digital, or gamer culture, and, more recently, a hybrid culture in atopic habitation.

Research itself is also embedded in a culture, which is evident in the understanding of what science is and how to do science in different areas of knowledge. This way of doing science, of researching, has faced new challenges and implications arising mainly from exponential digital technological growth.

The production of knowledge, as well as its almost instantaneous dissemination, produces broad access to ever greater amounts of information in a very short time, providing direct interaction with the researcher or research group responsible for a given discovery and/or innovation, as well as the constitution of research networks. This movement, so characteristic of contemporary scientific inquiry, instigates dialogue among fields, allowing distinct areas of knowledge to overlap and establish new research contexts. Thus we have seen the emergence of biomedicine, biotechnology, food engineering, and informatics in digital education/education, among many others, posing new methodological research challenges.

Lopes and Schlemmer ( 2017 ), point out that research in education in the context of digital culture has sought theoretical and methodological references that can support the complexity of knowledge production in this context, provoking dialogue and the problematization of the very theoretical and epistemological field that sustains it, in addition to the simple problematization related to the use of specific digital technologies (DT) and their effects on education. In this sense, theorists such as Pierre Lévy, Manuel Castells, Bruno Latour, Michel Maffesoli, Massimo Di Felice, Lúcia Santaella, and André Lemos, among others, problematize the sociocultural dimension of digital technological development, allowing us to broaden the focus of research into the contexts of emerging digital culture and, according to Schlemmer ( 2018 ), hybrid culture in atopic habitation, as well as to understand how this phenomenon relates to the field of education. It is therefore a matter of researching the broader meanings that these cultures produce in the field of teaching and learning, that is, the broader field of education.

With this context in mind, the following is a presentation of the research we have conducted as part of the Research Group in Digital Education, which highlights some of the concerns, strategies, and theoretical and methodological approaches that we have adopted in our research in the field of education in digital culture, and more recently in the hybrid culture in atopic habitation, which has enabled us to engage in broader and deeper dialogue. The objective is to consider the cartographic method of intervention research so as to highlight the research process, showing how we gradually adopted digital culture research methods before arriving at the cartographic method of intervention research and the cartography of controversies, which are discussed at length in this article.

From Digital Culture to Hybrid Culture in Atopic Habitation: Challenges for Qualitative Research in Education

In the field of research on education and digital culture in Brazil, Schlemmer, Lopes, and Molina ( 2012 ) outline some epistemological and methodological challenges on experiments in providing guidance by master’s and doctoral students in education. These challenges involve the construction of research objects and problems in the field of education and digital culture in DT contexts, including virtual learning environments, digital social media, and 3D digital virtual worlds. The authors propose rethinking the term virtual as a demarcation of an epistemological and methodological frontier in the ethnographic research of education and digital culture. They argue that the term digital would be more appropriate in distinguishing this border. They also suggest that the term netnography would be the most appropriate in a theoretical-methodological ethnographic research design in education and digital culture.

With this perspective, Lopes, Schlemmer, and Molina ( 2014b ) present some approaches to developing research and new procedures involving the use of applications (apps) and mobile devices (netbooks, tablets, and smartphones) in ethnographic research based on the project “Escola aumentada: Cartografia digital e mobilidade para a aprendizagem e a cidadania.” For the authors, the development of mobile technologies, geolocation (GPS and locative media), and distributed online databases (cloud computing) created new possibilities for the production of records and interactions in ethnographic research. While the diversity of digital records (text, photo, audio, and video) coupled with indexing and tagging mechanisms (tags, hashtags, geotags, etc.) opens up new possibilities for research, it also requires participants to understand the relevant syntax and new technological procedures for the production, recording, and sharing of information. Some programs and applications for desktop and mobile devices such as NVivo and Evernote, among others, can facilitate this process, contributing to the planning and organization of such production. The authors also refer to participatory research methodologies capable of involving researchers and subjects in the production and analysis of data. The current technological context, they note, can consolidate a new scenario in the field of research in which scholars and subjects act as co-producers of knowledge. In this sense, cartography mediated by digital devices emerges as a rich methodological possibility, involving social actors in the context of local problems. This engagement is a necessary condition for the production of meaning through the shared and mediated use of DT in the process of knowledge construction. This scenario can reconfigure both the research itself and the contract between researchers and subjects, since action and participation become instances of authorship from which all sides speak and produce.

Expanding their experimentation along with the discussion over the cartographic method, Lopes, Schlemmer, and Molina ( 2014a ) carried out an early attempt to map actions linked to a digital inclusion program—Província de São Pedro (PSP)—to distribute netbooks and tablets to teachers and students, initially prioritizing schools in cities participating in the Territories of Peace Program (Programa Territórios de Paz, PTP) to address the following research problem: What is the reasoning for linking an educational program of digital inclusion to a public safety program? The objective of the research was to understand the ties between such programs. Thus, the authors present digital cartography as a methodological proposition for online research and indicate the digital map of culture in Rio Grande do Sul (RS) and digital cartography as technological possibilities to promote greater connectivity between such programs.

Schlemmer and Lopes ( 2016 ) and Schlemmer ( 2016a ) also analyze the potential of the method to inspire new practices in line with the need to understand the phenomenon of learning in all its complexity—social, political, cognitive, affective, and technological—precisely because of its interventionist nature. In this context, the authors present a theoretical, methodological, and technological experiment developed in higher education and inspired by the cartographic method as a way of monitoring and evaluating learning in gamified processes and games from an interventionist perspective developed in a hybrid, multimodal, pervasive, and ubiquitous context.

Lopes and Schlemmer ( 2017 ) problematize the ethical, epistemological, and methodological aspects related to the field of education in digital culture, reflecting on how ethics can dialogue with the choices scholars make when conducting research. The authors present the paths adopted in two surveys conducted between 2010 and 2015 with a state public school in the metropolitan region of Porto Alegre participating in government programs for digital inclusion. Founded on the intervention research cartographic method, they present some of the results of discussions with teachers and students, based on the experience of producing and publishing information on the Internet. They problematize the ethical dilemma of research intervention based on the idea of technological appropriation, as a process that is established from the changes of practices that take place in contexts of analogical school culture and digital culture. They discuss and propose, based on the results of the research, overcoming the ethical dilemma of children and young students participating in surveys involving the publication of online content and the fears regarding media exposure—namely, the production and access to inappropriate content—and inattention in the classroom.

This research demonstrates that the cartographic method has been relevant, especially when the research tries to understand phenomena related to learning in the digital culture or, more recently, in the hybrid culture in an atopic habitation. Associated with this perspective, in the digital technological context are systems of data mining and learning analytics that make it possible to more effectively follow the tracks left by the subjects in the different spaces in which they interact.

Hybrid, in this context, refers to the mix between different elements resulting in a new element composed of the previous ones. For Latour ( 1994 ), the hybrid consists of multiple matrices, mixtures of nature and culture, which is therefore contrary to the separation between culture/nature, human/non human, among other things.

By atopic habitation, Di Felice ( 2009 ) refers to a relationship, a form of communication, characterized by the network interactions between different human and non human collectives, digital and territorial technologies. Atopic habitation “is thus the transient and fluid hybridization of bodies, technologies and landscapes, and as the advent of a new typology of ecosystem, neither organic, nor inorganic, nor static, nor delimitable, but informative and immaterial” (p. 291).

Thus, according to Schlemmer ( 2013 , 2014 , 2015 , 2016a , 2016b , 2016c , 2017 ), the hybrid is understood to be a mixture of space (geographic and digital), presence (physical and digital), technologies (analogue and digital), and culture (pre-digital and digital). It is in this context that the term multimodal is used, which includes the different imbricated educational modalities: the presential-physical modality and online modality and, while online, being able to combine elements of electronic learning, mobile learning, pervasive learning, ubiquitous learning, immersive learning, gamification learning, and game-based learning.

When scholars refer to hybridism in atopic habitation, in multimodality, pervasiveness, and ubiquity, they mean actions and interactions between human actors (HA) and non-human actors (NHA), in geographic and digital spaces, in interactions of different cultures (digital and pre-digital), constituting inseparable networks linking interconnected natures, techniques, and cultures. This suggests that a new understanding of culture and society may be emerging, one that embraces coexistence, co-engendering, mutual respect, solidarity, and the recognition of the other as a legitimate interlocutor. Thus, it is worth seeking to understand what these changes might mean in the area of education and, by extension, for research in education.

It is in this context that two contemporary methodological approaches are presented and discussed within the scope of qualitative research in education, linked to specific theoretical orientations.

The Cartographic Method of Research Intervention in Relation to Cognition Sciences

The cartographic method of intervention research is based on the cartographic method proposed by Deleuze and Guattari ( 1995 ). This method has been developed in Brazil by Kastrup ( 2007 , 2008 ), Passos, Kastrup, and Escóssia ( 2009 ), and Passos, Kastrup, and Tedesco ( 2014 ) as a means of interventionist research.

According to Passos et al. ( 2009 ), this approach originated from concerns about research methodology, which requires more open and, at the same time, inventive procedures. Thus, the theme of cartography emerged as a methodological issue in the face of impasses in cognition research, developed by the research group Cognição e Subjetividades. 1 The method began taking shape when the members of the research group questioned the assumption that knowledge means representing or recognizing reality by configuring the importance of the binomial cognition/creation and calling for a more detailed investigation process into the temporal dimension of knowledge production processes. Thus, the authors defined the concept of cognition as creative, autopoietic (Humberto Maturana and Francisco Varela, 2001 ), or enactive (Francisco Varela, 1988 ).

Maturana and Rezepka ( 2000 ) suggest that the way people attribute meaning and learn has a distinctly human quality, since people are autonomous and autopoietic, in congruence with the environment in which they are inserted. This congruence can cause disturbances in the structure of human beings, promoting learning processes insofar as the structure self-produces to compensate for the disturbance. Thus, for Maturana ( 1993a , 1993b ), learning is the act of transforming within a particular environment of recurrent interactions, and happens as the behavior of an organism varies during its ontogenesis in a manner congruent with the variations of the environment (Maturana, 1998 ). Therefore, when I refer to a medium or a hybrid, it means that the congruence of the subject with this environment causes disruptions in the structure of this subject, which allows him or her to attribute meanings that originate from the action and interaction in that space, thus promoting learning processes as the structure reproduces itself to compensate for the disturbance, and in so doing, compounds its ontogeny.

The process of cognition consists of creating a range of behaviors through conduct within a field of interactions. Knowledge, from this perspective, is not simply representation, but implies a permanent interpretation of action. For Varela ( 1990 ), interpretation and knowledge are emergent (in the sense of emerging) results of action or acting in the world. Thus, greater capacity for cognition consists, to a large extent, of asking the pertinent questions that arise at every moment of our life. These are not predefined but rather enactive: they emerge from action in the world, the relevant aspect being what our common sense deems appropriate within a given context. Thus, knower and what is known, subject and object, determine each other and arise simultaneously. The enactive orientation proposes an intermediate way of transcending both extremes: subject and object define each other and are correlative.

The central point of cognition for Maturana and Varela ( 1997 ) is its ability to elicit meaning: knowledge is not predetermined or established a priori, but implicit in regular processes of cognitive activities themselves. In this way, cognition is not the representation of a world that exists independently, but rather the “production” of a world through the process of living, the “continuous coincidence of our being, our doing and our knowing” (p. 20).

According to Passos ( 2015 ), “to know is to enter a structural coupling with the environment, to interact” (p. 85). However, the understanding of interaction changes in this perspective because it no longer assumes the preexistence of the two terms (organism and environment, subject and object) that interact. To interact in this perspective means to construct oneself and the environment, being, therefore, the “act of knowing reality, an act of affirmation of self, self-surrendering, of autopoiesis. By redefining the cognitive act, its representational sense disappears” (p. 86). In this way, transgression lies in imbuing knowledge with pragmatic value that makes it a performative act, in which to know is to do and vice versa. This delegitimizes the understanding of knowledge through transparency or indifference of the cognitive act: “The whole act of knowing is a form of engagement in the world, of commitment to the world that constitutes itself in this act” (p. 86).

An important element that marks the difference between the enactive approach and any form of biological constructivism or neo-Kantianism is the emphasis on codetermination. In this context, the understanding that conduct is potentially unpredictable marks a departure from the theoretical approach of Maturana and Varela, from other behaviorist and Piagetian approaches.

In this perspective, according to Passos et al. ( 2009 ), subject and object—the poles of cognoscence—are outcomes of cognitive activity rather than conditions. By broadening the concept of cognition and understanding it as linked to creation, the production of knowledge pragmatically and reciprocally shapes the self and the cognitive domain such that cognitive practice engenders subjectivity, overcoming an understanding of dependence of a cognitive subject and a given world, understood as invariant fundamentals. Understanding of cognition as an act of creation brings with it “the problem of the ethical commitment of the cognitive act to the created reality. Production of knowledge, production of subjectivity” (p. 13). The methodological problem is set as follows: “How to study the plane of reality production? What method allows us to follow these processes?” (p. 13). Instead of rules to be applied in the method, the authors offer clues to guide the researcher, since it is not always possible to predetermine every methodological procedure: “The clues that guide the cartographer are like references that contribute to maintaining an attitude of openness to what is happening and of calibrating the course of the research—the meta-hodos of research” (p. 13).

Initially, eight points were proposed to guide the practice of the cartographic method. These were not laid out in hierarchical order but as a rhizome (based on Deleuze & Guattari, 1995 ), referring to each other and forming a set of connections and references to order, develop, and collectivize the cartographer’s experience.

According to Passos et al. ( 2009 ), cartography is an intervention research method (point 1) that aims to track the process, through clues, guiding the course of the research, without establishing a linear path to an end. In this way, it considers the effects of the research process on the research object, the researcher, and the results, and does not simply represent an object: “Cartography seeks to ensure the accuracy of the method without giving up the unpredictability of the process of knowledge production, which is a positive requirement of the ad hoc investigation process” (Kastrup, 2007 , p. 19). What sets it apart from other approaches is the focus on the process and not the end result. With the aim of tracking the process (point 3), clues may arise, which might help to describe, discuss, and, above all, to collect the experience of the cartographer.

In this sense, the cartographer needs to keep in mind that the action of researching his object in motion constitutes a practice in which his path establishes links with the participants inserted into the context of what is being investigated. This composition of agency between heterogeneous actors is expressed by Barros and Kastrup ( 2009 ), drawing from Caiafa ( 2007 ). For these authors, agency implies a relationship of cooperation, a kind of sympathy, which, in addition to a simple feeling of esteem, refers to a composition of bodies implying mutual affection that enable the ethnographer to effectively “enter into relationship with the heterogeneous ones that surround it, to act with them, to write with them” (Barros & Kastrup, 2009 , p. 57).

According to Passos et al. ( 2009 ), cartography as a methodological orientation needs to be articulated using three ideas that make up a plan of action or a research plan: transversality, implication, and the dissolution of the observer’s point of view (point 6). In traditional third- and first-person methodologies, there is always the imposition of a point of view capable of representing or signifying the object at hand. There must be an observer, which implies the “subject–object separation or duality, as well as the imposition of an interpretative reference frame separate from experience” (Passos & Eirado, 2009 , p. 121). These authors discuss the work of Varela, Thompson, and Rosch ( 2003 ), who point out that third-person methodology does not work when studying cognition or the mind, because there is a circularity between knowledge and the known world that is fundamental but often overlooked. This is more evident in studies of cognition because it is not possible to separate the structure that is known from the experience of knowing. This understanding of cognitive experience as its own creation, that is, of both the known object and the subject it knows, which occurs in circular motion, is called “enactive action or approach, modulating the notion of autopoiesis formulated by Maturana and Varela in the 1970s” (Passos & Eirado, 2009 , p. 121).

For Passos and Eirado ( 2009 ), the biology of knowledge, autopoiesis, accepts the challenge of thinking without a foundation, since third-person methodology needs to be complemented by first-person methodology. Woven together, these make it possible to penetrate the circularity that arises in the experience of acquiring knowledge. The cartographer has to avoid merely seeking solutions and testing hypotheses, for “he does not take the self as an object, but the self-emergence processes as the destabilization of the points of view that collapse the experience in the (‘internal’) self” (p. 123). He must inhabit the experience without being bound by any point of view, his main task being to dissolve the observer’s point of view without neglecting observation. Enaction assumes that all experience emerges from experimentation, since it does not refer to what is already a given but rather the emergence of change. Data does not exist a priori, waiting to be gathered, instead, it is constituted in the experience itself. In this way, it is up to the cartographer to accompany this emergence of himself and the world in the experience, and for this it is imperative to be immersed and never immune to the process.

Kastrup and Barros ( 2009 ) argue that the method is not a research model developed through clues, strategies, and procedures. The procedures are embodied in apparatuses (dispositif) that perform important and distinct functions in the operation of cartography. 2 Grounded in Deleuze’s work, they understand apparatuses as “machines that make it possible to see and speak,” composed of lines of visibility, enunciation, force, and subjectification. These apparatuses are aligned with the process of creation, and the work of the researcher-cartographer is to unravel these lines and monitor their effects. The purpose of the apparatus involves three movement functions (point 4): reference (more or less regular apparatus, in which repetition and variation are articulated); explicitation (research territory to be explored, explicitation of the lines that participate in the ongoing production process, inseparable dimensions, research and intervention); and transformation production (“transformation of the relations between the elements/lines/affective, cognitive, institutional, micro and macropolitical vectors, activating movements and sustaining processes of production” [Kastrup & Barros, 2009 , p. 80]). Cartography can produce and transform the reality to be analyzed. In this way, mapping implies intervention.

The cartographer, the person using the cartographic method, does so through “cartographic attention.” Cartographic attention (point 2), according to Kastrup ( 2007 , p. 15), is based on Freud’s concept of “free-floating attention” and Bergson’s concept of “attentive recognition.” It is concentrated and open with four varieties (movements): tracing, touch, landing, and attentive recognition. The cartographer’s work begins with tracing , which involves scanning/sweeping the field, an overview with open and unfocused attention. It is a broader look at something that touches it, beyond the search for information. Touch triggers the selection process, the first meaning, the first analysis performed on the selection process. It is characterized by a quick sense of focus on attention, when something touches, it draws attention, causing it to become alert, but that does not yet define what the cartographer will focus on. The movement that refers to a defined point of attention and focus is the landing , which is to stop, zoom in, choose/define, and indicate that the selected element needs to be inspected more closely for analysis. That is, “the landing gesture indicates that the perception, whether it is visual, auditory or otherwise, makes a stop and the field closes in a kind of zoom. A new territory is formed, the field of observation is reconfigured” (Kastrup, 2009 , p. 43). The fourth and final movement is of Bergson’s attentive recognition , characterized by an investigative attitude about the landing, to which the cartographer’s attention is drawn. It represents analysis itself.

The cartographer’s objective is to map a territory that he/she did not previously inhabit (point 7), to understand the planes of power (point 5)—a moving plane of the reality of things at work in it—and to produce knowledge over the course of research, which involves attention and, with it, the very creation of the field of observation (Escóssia & Tedesco, 2009 ).

Because it is a form of intervention research, the analysis occurs in the process, in the movement of cartography, which makes it possible to carry out the intervention while the process is taking place. Thus, Escóssia and Tedesco ( 2009 ) point to the double direction of the nature of cartography: as a knowledge process that is not restricted to describing or classifying the formal contours of the objects of the world, but in tracing the movement itself that animates them, and as a practice of intervention, where access to the plane of power implies inhabiting it, so that the acts of the cartographer, also a collective of forces, participate and intervene in the changes and in the transformations that occur.

This inhabiting of an existential territory is significantly different from the “application of a theory or the execution of a prescriptive methodological planning, since it implies receiving and being welcomed in the difference that is expressed between the terms of the relation: subject and object, researcher and researched, I and the world” (Alvarez & Passos, 2009 , p. 148). In cartography, one does not “separate theory and practice, spaces of reflection and action. To know, to act and to inhabit a territory are no longer experiences distant from each other” (p. 149).

As far as the cartographic method of research intervention is concerned, where the data are produced Passos and Barros ( 2009 ) emphasize the question of narrativity, that is, it is always narratives that we deal with, being that sometimes the research participants also are cartographers. What each one says, what the situation says implies taking a position in a certain narrativity politics (point 8). 3 This narrative position (ethos of research) is embedded in other policies that are at stake, such as research policies, subjectivity, or cognitive policies. So all production of knowledge comes from an implicitly political position. According to the authors, narrativity politics refers to the position we take in facing the world and ourselves. In this way, “the knowledge we express about ourselves and the world is not only a theoretical problem, but a policy-related problem” (Passos & Barros, 2009 , p. 151).

According to Passos and Barros ( 2009 ), narrativity politics involves two methods and two ways of speaking—extensivism and intensivism—and also two narrative procedures: redundancy (“organizing what in this case is abundance, generating a circulation of meaning that reinforces the clarity of the case, its unity and identity” [p. 158]) and disassembly (“extracting from the larger case the agitation of microcases as microstruggles brought into the scene” [p.161]). In this dismantling process, three characteristics stand out: (1) the procedure to narrate the “case” is due to an increase in the coefficient of deterritorialization; (2) “everything is political”; and (3) everything acquires collective value.

Continuing their elaboration of the eight points of the cartographic method, Passos et al. ( 2014 ) cite thinkers besides Deleuze, Guattari, Maturana, and Varela, introducing Latour, among others, into the discussion regarding the research experience. According to Passos et al. ( 2014 ), the “importance of the research experience points to its inscription on the plane of powers, which constitutes the production plan of both knowledge and known reality” (p. 8). Researchers are immersed in the experience, which distances the cartographic method from other approaches guided by processes such as the “collection,” processing, and analysis of the data, taken as information. In this way, the cartographic method is based on inventive cognition and creative cognition, thus differentiating itself from the idea of the representation of a preexisting world. Therefore, “the cartographic method is not defined by the procedures it adopts, but it is an activity guided by a directive of a nature that is not strictly epistemological, but ethical-aesthetic-political” (p. 9).

Kastrup, Tedesco, and Passos ( 2015 ) point out that the cartographic method is compatible and can be used alongside different techniques, strategies, and research approaches, among them interviews, data analysis, and qualitative or quantitative strategies. In this way, the method is fluid, distinct from methodological models guided by the assumptions of representation. However, research that investigates the experience of research itself must make clear the “firm position of the cartographer with regard to the guideline of research: access/production of the plane of powers that responds to the creation/transformation of experience” (p. 9).

Inventive cognition emerges from the biology of knowledge (Maturana and Varela), of enunciative cognition (Varela), and includes elements of Bergson, Nietzsche, Foucault, Deleuze, and Guattari. According to Kastrup ( 2015 ), thinking stems from stimuli that make you think and does not happen spontaneously, from nothing. The stimuli are, therefore, forces of the present, of a world in movement and accelerated transformation constituting “the unique ground of emergence of thought and novelty” (p. 96). This understanding differs from that held by those who understand cognition from a perspective that Maturana and Varela ( 2001 ) call environmentalist (a realistic assumption of a given world), which “does not allow us to think about the invention of the world itself and above all the world in transformation” (p. 96). Thus, Kastrup ( 2015 ) seeks to understand the “shifts of cognition in the contemporary,” from the encounter of two intercessors for the psychology of cognition: (a) Maturana and Varela, who promote the idea of the biology of knowledge (autopoiesis), by refusing the model of representation and promoting the understanding of cognition as an invention of oneself and of the world; and (b) Deleuze and Guattari, who focus on the transformations currently taking place in cognition. To this understanding, the author adds that, in order to be able to understand the new ways of knowing and living emerging today, it is necessary to affirm the present as a movement of virtualization of currently constituted cognitive forms. The conditions of cognition have in themselves tension between constituted forms and forces of instability: “Forces of the present, which problematize the old forms, placing cognition on the route of experimentation” (Kastrup, 2015 , p. 97).

In this context, Kastrup ( 2015 ) refers to DTs, stating that they cannot be understood as mere objects or as solutions to old problems but as a basis “for creating new problems, new relationships with information, in time, with space, with oneself and with others” (p. 97). 4 Thus, the relation between the constituted forms and the present is not of rupture or of discontinuity, but of coexistence, the conditions of cognition being polytemporal and not invariant or historical. The problem of cognitive functioning is in how the present can provoke “cracks in historical strata, in old mental habits, in established structural couplings and produce novelty,” in addition to understanding it as historically produced. “It is the living gift that coexists with the history of structural couplings. Through this notion, Varela introduces in the studies of cognition the possibility of thinking it into becoming, becoming that makes the history bifurcate” (pp. 98–99).

In the biology of knowledge, with the concept of autopoiesis, Varela resignifies the understanding of learning by, in approaching the problem, placing the actor as the prototypical apprentice. Learning is not, as previous theories proposed, adapting to a given environment, or obtaining knowledge, but experimentation, invention of self and the world. The invention of a work of art is correlated with the production of the artist him or herself. As a novelty comes a theory of action, since for Maturana and Varela the living system is a constantly moving cognitive system in a process of permanent self-production, that is, autopoietic, which can be understood, according to Kastrup ( 2015 ), by the formula BE = DO = KNOW. 5 In this sense, the functioning of the living being is confused with the process of self-creation; according to Varela ( 1990 , p. 99), “doing is ontological.” Understanding cognition as action or practice leads to its permanent modification and not to invariant structures. In addition to the logic of action, cognition refers to flows in conduct (Maturana & Mpodozis, 1992 , p. 18).

Paradoxically, according to Kastrup ( 2015 ), what ensures the flow of the conduct is precisely the crack, the break, the notion of breakdown, described in Varela ( 1990 ) and Varela, Thompson, and Rosch ( 2003 ), as perturbation, a “problematization” of the structures of the living, ranging from engagements with the world, without it being possible to determine a principle that guides this drift toward the pursuit of a superior equilibrium. Breakdowns are the source of the autonomous and creative side of living cognition and arise as a theoretical-scientific formulation for an understanding of cognition that is not restricted to solving problems but is, first of all, the invention of problems. Thus, Varela ( 1990 ) explains the rooting of cognition in the “concrete,” dealing with earlier conceptions that approach cognition from the point of view of logic, general mechanisms, or representation, grouped under the denomination of “abstract” approaches to cognition (p. 102). The breakdown is a cognitive activity that happens in the immediate present and in that concrete actually lives. This “concrete” for Varela ( 1990 ) is not a step for something different, but how we arrived and where we are.

In the perspective of reconciling cognition with concrete Varela presents the notion of enaction (actuation), previously explained. For Kastrup ( 2015 ), this notion refers first to an embodied cognition distinct from the understanding of cognition as a mental process, for it is “tributary to action, resulting from experiences that are not mentally inscribed, but in the body” (p. 103). It is an action guided by local sensory processes and not by the perception of objects or forms. These sensorimotor attachments are not separate from the lived cognition (biological, psychological, and cultural couplings). Thus, the embodiment of knowledge implies social couplings, including linguistic ones, so that the body, in addition to a biological entity, is able to register and mark itself historically and culturally.

In order to exemplify the concept of enaction, linked to the embodiment of knowledge, Varela et al. ( 2003 ) refer to learning a musical instrument, where the musician is taken as a prototype of the learner. In this process of learning, initially the body functions are commanded by the mind, because the process begins with a representation, with symbolic instructions. However, to learn to play an instrument is not to follow rules, and learning truly only happens “when the symbolic relation is transformed into direct coupling of the body with the instrument, eliminating the intermediary of representation” (p. 103). That is, therein is the enaction, actuation, incarnation, or embodiment of knowledge. Thus, for the author, cognition begins to function outside the register of representation, in direct coupling with the matter that the world provides. To learn is not to adapt to the musical instrument, but to act with it. Thus, the notion of acting refers to a collective dimension that appears in the body, at the same time as it indicates the participation of the body in the configuration of the world that is shared by the collective.

This understanding of coupling as agency allows Kastrup ( 2015 ) to move forward in a second sense of the notion of acting— cognition as invention of a world—constructed in the interface between Varela’s cognitive studies and Deleuze and Guattari’s subjectivity production, from whom it takes the concept of agency as “direct communication, without mediation of representation” (p. 104). Communication without subordination, hierarchy, or determinism does not operate by causality, but by reciprocal implication between movements, processes, or heterogeneous flows, by double capture. According to the authors still referring to the learning of a musical instrument, agency refers to the production of a complex apprentice–instrument unit, which produces a process of reciprocal differentiation. The mechanical relationship occurs between previous elements (having the same elements and the same relationships, we will have the same product behavior repeated in the same way) whereas machinic agency, on the other hand, connects flows or processes and creates forms.

Kastrup ( 2015 ) uses again the example of learning a musical instrument to demonstrate that if we understand flute learning, for example, as a machinic agency, “learning is eliminating distances, because one learns between mouth and flute, learns in the middle, on the surface of its coupling, outside the field of representation” (p. 104). That is to say, in this adaptation with the medium, “the blowing motion is able to interact with the arrangement of the instrument and at the same time generate the sound and the apprentice” (p. 105). Thus, coupling should be thought of as a machinic agency or a product of learning, a creative activity always focused on becoming and not a mechanical representation or repetition. This understanding puts an end to the supposed determinism of the object or the environment. The best learner is the one who permanently creates a relationship with the instrument, incessantly reinventing himself as a musician.

The best student, for Kastrup ( 2015 ), is not the one who approaches the world through crystallized habits, but who can always remain in the process of learning, which can also be understood as permanent unlearning. That is, learn is to experience incessantly in order to evade the control of representation, preventing crystallized habits from forming, that is, be alert to continuous variations and rapid resonances, implying, at the same time, a certain lack of attention to the practical schemes of recognition.

Bergson ( 1934 ) theorizes this relation between certain attention and correlative inattention. For this author, there is a pragmatic, utilitarian life that assures learning while solving problems, but there is also an additional attention, which is attention to duration, which ensures learning as the invention of problems.

In this context, it is fundamental to consider that the contemporary world has provoked the emergence of new forms of subjectivity, mainly by the ceaseless and almost omnipresent presence of all kinds of DT, which has accelerated processes of transformation and innovation in the ways of living and engaging, which are more and more open and in flux. In this whirlwind of uncertainty, subjectivity is called upon to reconfigure itself and must learn to deal with breakdowns, with the disturbances that present themselves. On the other hand, this same reality, coupled with the online approach to the most diverse cultures, according to Kastrup ( 2015 ), reveals the precariousness of any supposed foundation that can be provided by the world (p. 108), that is, if we are affected on all sides by disturbances of all nature, solutions are not assured. Therefore, if we want to create new ways of knowing and living, we must invent a world, for learning to live in a world without fundamentals is to invent it by living, remembering that invention of self cannot be achieved without the invention of a related world.

If interpretation and knowledge are emergent results (in the sense of emerging) of action in the world or acting, when spaces are hybrid, multimodal, pervasive, and ubiquitous, and dwelling is atopical, what are the relevant issues that emerge concerning the action and performance of the subjects in these spaces? How do meanings emerge? What world do we produce and invent?

The Cartographic Method as a Means of Identifying and Mapping Controversies and Actor–Network Theory

Recently, actor–network theory (ANT), developed by Latour, Law, and Callon, also recognized subject–object codetermination by emphasizing the participation of non-humans—objects and quasi-objects—in social relations, thus presenting itself as an alternative to the binaries of modernity by eschewing a compartmentalized view of reality. 6

In this perspective, ANT (Latour, 2012 ) provides a new understanding of what is social, presenting the idea that humans establish a social network not only to interact with other people, but with non human elements as well. According to the author, the social is not simply made up of people, but also machines, animals, texts, money, architecture, laboratories, institutions, among other elements. By the principle of connectivity, everything is linked in a network, with multiple inputs, which is always in continuous movement and open to new elements. For the author, social refers to the network of HA and NHA, where the actor is any person, thing, (quasi-)object, or institution that produces agency, that is, something with the ability to produce effects on the network (although indirectly), of being actant. The understanding of agency, therefore, is related to the human and non human actors (actants), similarly, who participate in the actions and provoke transformations in the network, in movement. Thus, in ANT, or sociology of associations, the non human is no longer considered only as an artifact, whose meaning is attributed by the human, but as having agency, because it participates in actions in everyday situations and causes transformations. Non human actors also shape events in the creation of meanings, acting in the reflective and symbolic sphere.

In this context, network is understood from the perspective of a rhizome (based on Deleuze & Guattari, 1995 ), that is, seen as something alive, changeable—as flows, circulations, alliances, and movements of a series of animate and inanimate elements—and not as fixed to a set of actors. It refers to transformations, translations, displacements, therefore, quite distinct from the traditional understanding of a network as a form or structure. The network is the associative movement that forms the social, being rather an instrument of analysis or its object. The actor–network binomial perspective proposes that the actor never acts alone. In acting, it is influenced (constituted) by the networks in which it has connections and, at the same time, it can represent these networks, as well as influence them. In this way, it is never quite clear who is acting. The actor is, at the same time, the builder and receiver of the networks.

For Latour ( 2012 ), the social has no predefined locus, but is understood as provisional, performative, as processes of aggregations, associations, and reassociations between HA and NHA. In order to understand the social, which, therefore, is not what explains but rather what needs to be explained, the author recommends that the actors be followed in their associations and reassociations (cartography). I understand, therefore, that nowadays these associations and reassociations are increasingly constituted in nomadic movements, in an atopic habitation that takes place in hybrid, multimodal, pervasive, and ubiquitous spaces. 7

With regard to science, Latour ( 2016 ) says that every idea only proceeds from multiple deviations and compositions. It is the attribution of science to understand this process and not only the result. For this, it is necessary to retrace the entire chain of deviations and compositions, and what matters in this process is what emerges and forms in the course of the process of composition and deviations of courses of action. Linked to this question, Latour ( 2016 ) in the second letter of Cogitamus, raises the problem of method: How is it possible to analyze deviations and compositions if, in general, they are invisible?

In this context, the concept of proof becomes the protagonist, because, according to the author, it is at the moment of proof that the blunting of deviations and compositions is revealed. This evidence, although there are other forms of evidence, materializes in the panel: everything works well, until it stops working. This is more pedagogical form of expression of proof. The computer, initially understood as a technical object, is now presented as a sociotechnical project: “From simple, my computer has become multiple; of unified, has become disunited; it immediately became mediate; of fast, it became slow” (Latour, 2016 , p. 47). The network, or part of it, with the different elements that together kept it working, now fails, becoming visible. It is at this moment that it is necessary to analyze the links, the relationships, the networks that integrate it.

This perspective, presented by Latour ( 2016 ), although linked to the method he calls “cartography of controversies,” could also be linked to the cartographic method of intervention research, proposed by Kastrup ( 2007 , 2008 ), Passos et al. ( 2009 ), and Passos, Kastrup, and Tedesco ( 2014 ).

While Latour ( 2016 ) refers to pane and understood as a proof, bringing sociotechnical network analysis (HA and ANH) to the context; we could think from the point of view of the subject’s cognition (HA) as a cognitive imbalance (Piaget) or as a breakdown (Varela) understood as a proof, later assumed by Kastrup, Tedesco, and Passos ( 2015 ) from the perspective of the inventive cognition.

With regard to the interactions that take place in this sociotechnical network, the deviations and compositions are visible by the traces that the different actants produce in the movement of associations, which can be accompanied by “another” sociotechnical network formed by HA (teachers) and NHA (mining and data-based systems and learning analytics), providing elements that allow us to trace the cosmogram and understand the process under construction.

Turning to the question of the pane for Latour ( 2016 )—in the context of a sociotechnical network; cognitive imbalance for Piaget and breakdown for Varela—in the context of cognition; a process of investigation begins, so that the initial indetermination begins to be deciphered, the source of the disturbance is found, and a problem is revealed progressively. 8 Hypotheses and solutions are tested and verified, until the problem is solved.

In the case of the computer (NHA), in the context of sociotechnical network (formed by HA and NHA), the pane is repaired and put back to use. In the case of cognition, what in the comprehension of Varela ( 1990 ) and Kastrup, Tedesco, and Passos ( 2015 ) refers to an enactuated, self-engendered process of agency, caused by a breakdown, takes knowledge to a superior equilibrium, not in the perspective of problem-solving but, above all, in the invention of problems. It is not the result, but the process by which the result is achieved. In a way, this is related to the concept of debugging, a result of a metacognitive process (Piaget, 1976 , 1978a , 1978b , 1995 ) that is necessary for computational thinking, which was very present in research related to language programming in the 1980s and 1990s.

From the proof concept, Latour ( 2016 ), in his third letter in Cogitamus, goes on to discuss scientific controversies, stating that the statements pass between two poles: radical doubt and unquestionable certainty: “At the beginning of the exercise, the statement floats; in the end, one must find it solidly anchored in a precise landscape . . .” (p. 81). According to the author, this is where the importance of controversies lies.

The term controversy, according to Latour ( 2016 ), “designates all possible positions, ranging from absolute doubt . . . to indisputable certainty” (p. 79). The word “controversy” describes a shared uncertainty about aspects of science and/or technology that are not yet stabilized. It occurs when there is a disagreement between the actors, that is, “when actors discover that they cannot ignore each other and controversies end when actors manage to work out the solid commitment to live together” (Venturini, 2010 , p. 260). The cartography of controversies consists in mapping the actions of human and non human actors involved in contemporary sociotechnical questions, without, however, assuming an a priori frame or an order to be followed. In this context, according to Latour ( 2012 ), the Actor-Network Theory (ANT) achieves a better understanding of the order after the actors explain all the controversies in which they were involved, that is, “We [social scientists] will not try to discipline, to frame you [the actors] in our categories; we will allow them to stick to their own worlds and only then will we ask for their explanation of how they were established” (p. 44). It is not up to the analyst to define and order the social, but rather to the actors present in the context. If the goal is to restore order, it is best to go through the associations, tracing the connections between the controversies themselves. The pursuit of order, rigor, and pattern is by no means abandoned, just repositioned one step further in the form of abstraction, so that actors can unfold their own and various cosmos, no matter how irrational they may seem.

Latour ( 2012 ) notes that it is possible to trace stronger relationships and discover more revealing patterns when we find a way to record the links between unstable and mutable frames of reference rather than trying to stabilize one. What makes one expand, relate, compare, and organize is what one has to describe. The important thing is not to stop the flow of controversy, because if the actors do not act, they will leave no clues: “No clue, no information, no description—and therefore no conversation” (p. 217). It is in the flow of controversies that one has to find the “firm ground: on shifting sands. Contrary to what is commonly said, relativism is a way of floating in the data, not plunging into it” (p. 46).

The main idea is to give visibility to the different understandings about situations, movements, representativities, influences, and interests. For this, it is necessary to explore, visualize polemics, the movement of action and motion, that is, where mediation flows (Lemos, 2013 ). Controversies are those spaces of dialogue, conflict, negotiation, and action, which the actors reveal by leaving traces.

For Latour ( 2016 ), mapping a controversy is learning to locate all these movements. This implies following the statements from doubts permeated by intermediate states (rumor, opinion, idea, proposition), pros and cons, until they become the

final result, where clear and well-defined inscriptions are evident. However, with the condition of taking it in its motion, and not frozen in an object. Remembering that nothing is definitive, both techniques and sciences do not exist by the simple force of inertia. To exist is to always be in that front line. What we now understand as a certain statement is only the final stage of a controversy and in no way its beginning. (Latour, 2016 , pp. 80–81)

Latour ( 2016 ) states that the two extremes must be considered: fact and opinion, which correspond to two moments in the controversy. Accompanying the controversies is then to describe the ways in which the actors construct and modify the evidence.

Venturini ( 2012 , p. 800) proposes a script to subsidize the creation of cartographies of controversies, which is summarized by Lemos ( 2013 , p. 118): (1) to define the best possible controversy; (2) observe, describe, and maintain that the object is controversial; (3) identify whether the controversy is: cold/hot, present/past, secret/public, difficult to access/accessible, limited/unlimited; (4) apply the lenses to the collection of information (gather statements, opinions, read the specialized literature); (5) identify human and non human actants and sketch the network that connects them; (6) identify cosmogram, ideologies, and worldviews. The cartographer must then identify the representativity, influence, and interest of the actors in the networks.

If there is evidence, Latour ( 2016 ) proposes that two conditions are imposed on the analysis: (1) that it comes from the actors, and (2) that it results from joint activity, therefore, not more than a cogito, but a Cogitamus.

In this context, the author refers to the new passage from the infinite world (modern and post-scientific revolution) to the complicated multiverse or pluriverse, term coined by William James. Thus, if there is no radical discontinuity, the meaning of the word revolution changes. Latour ( 2016 ), based on Sloterdijk, stresses that it is not a matter of revolution or emancipation, but of explicitness, since “history never breaks with the past, but permanently makes more and more explicit with which we have to learn to live, elements that will be compatible or incompatible with existing ones” (p. 115). Thus, the author inaugurates a new “epistemological policy,” in which the researcher’s function is to describe “the agency of all beings that a particular culture links with practical forms of life” (p. 166). This refers to a movement of associations between the different parties participating in a controversy, which can be represented by what the author calls a cosmogram. In order to retract the different parties, Latour ( 2016 ) proposes the description of the associations of convenience, coexistence, opposition, and exclusion between HA and NHA, whose conditions of existence become explicit in the course of the trials submitted by the disputes: “to become sensitive to these lists of associations and logical duels without resorting to the distinction between the rational and the irrational, the modern and the archaic, the systematic and the unsystematic” (Latour, 2016 , pp. 116–117).

Mapping the cosmograms means working with the movement, with the agency distribution drawing of mobility, unlike the paradigm, which works with frames (stabilized theoretical framework). When designing a cosmogram, the analyst does not need to resort to structures, systems, or frames, which limit or even render unviable the monitoring of the connections in formation in the sociotechnical networks. He needs to go through the network, follow the actants in their associations, identifying the controversies, their different visions or world versions (multiverses or pluriverses), showing how the whole world (a complicated and complex pluriverse), a cosmos, reveals itself, emerges, and reconfigures itself in the confrontation of ontologies.

It is within the scope of interactions that occur in this sociotechnical network that the different actants (human and non human) produce traces in the movement of associations. These traces, in the scope of education research, can be accompanied by another sociotechnical network formed by HA (teachers-researchers) and NHA (diverse systems, including those based on mining and data and learning analytics) which, when mapped, provide elements that make it possible to draw the cosmogram (diagram of mediations, motion, mobility), a multiverse, and understand the process under construction—the movement of these associations happening. The aim is to map the controversies: “the study of innovations and controversies is one of the first privileged places where objects can be held longer as visible, disseminated and recognized mediators before they become invisible, non-social intermediaries” (Latour, 2012 , p. 120).

It is important to consider that for Latour ( 1994 ), the logic of thinking of the sciences is vitiated by a need for purification by division, which results in exclusion. According to Melo ( 2011 ), this logic, in a way, freezes the possibility of transformation, because it does not consider the continuous mixture, which produces hybrids incessantly and indefinitely. Instead of imposing one part(s) on another, replacing one or the other(s), as the modern project intended, the parts become the contact with heterogeneous versions, through the adjustment of hybrid practices and interests, through which these parties receive from each other the chance of mutual transformation (Despret, 2002 ). TAR presents the notion of hybrids and generalized symmetry.

It is possible to bring elements of this vision presented by Latour to reflect on the question of culture, since it is not a question of dividing, of classifying, into analogical (pre-digital) culture, digital culture, or cyberculture, or, but to consider the mixture, that is to say, the hybrids that are produced in that mixture, which is resignified and transformed in that coexistence.

Conclusion: Education Qualitative Research in the Hybrid and Multimodal Culture Context in an Atopic Dwelling

Both methods, the cartographic method of intervention research and the cartography of controversies, with their specificities, have proved robust for the development of qualitative research in the field of education, in the context of digital culture and hybrid culture in an atopic dwelling, mainly because they are also related to equally consistent and contemporary theories in terms of human cognition aspects, enabling us to track the traces and clues in the mobility of associations between actants, which are enhanced by different Digital Technologies (DT), including data mining and learning analytics.

These methods have been used in the research developed by the Grupo de Pesquisa Educação Digital (GPe-dU Unisinos/CNPq), in which I am involved at the Universidade do Vale do Rio dos Sinos (UNISINOS), since 2010 . In particular, the cartographic method of intervention research, as well as being appropriate as a research method, has been investigated for its potency with regard to the development of new methodologies and pedagogical practices, due to its interventionist characteristic in accompanying the process (along the course), aligned with the need to understand the phenomenon of learning in its complexity—social, political, cognitive, affective, and technological (Schlemmer, Lopes, & Molina, 2012 ), in contexts of hybridism, multimodality, pervasiveness, and ubiquity. Its power to accompany learning processes has also been investigated as well as the possibility of it being appropriated by teachers and students in their own learning pathways.

Among the research projects that have used the cartographic method of intervention research are: “Escola aumentada: Cartografias digitais para as aprendizagens e a cidadania,” “Gamificação em Espaços de Convivência Híbridos e Multimodais: Uma experiência no ensino superior,” and “Gamificação em Espaços de Convivência Híbridos e Multimodais: A educação na cultura digital.” Under different approaches and in varying contexts, research explores the perspectives of hybridism, multimodality, pervasiveness, and ubiquity, as well as cartographic attention. It was in the context of these projects that the cartographic method of intervention research began to become the object of study, inspiring studies carried out on the formation and qualification of teachers and students (Lopes & Schlemmer, 2017 ; Lopes & Valentini, 2012 ; Schlemmer, 2014 , 2015 ; Schlemmer & Lopes, 2016 ) in the scope of elementary education, graduation in pedagogy and digital games, post-graduation stricto sensu , and continuing teacher training. Our focus has been to explore some elements related to the hybrid, multimodal, pervasive, and ubiquitous culture and the new regimes of action, participation, and socialization of experience.

In the context of elementary education schools, we have outlined some experiences involving cartography with the support of digital media, geolocation, and digital marking (quick response [QR] codes), to provoke experiences of local mapping of people, objects, and places in the public space. In the undergraduate, and graduate continuing teacher education context, as well as in elementary school, we also designed inventive methodologies and pedagogical interventionist, aggregative, and gammatical practices with a cartographic bent, mainly the cartographer attention through the four movements (tracing, touching, landing and attentive recognition) , linked to elements of gamification, with the support of digital media, geolocation, digital coding (QR codes), and augmented reality, in order to provoke learning experiences in a hybrid, multimodal, pervasive, and ubiquitous context. According to Schlemmer and Lopes ( 2016 ), unlike classic cartography (mapping), the idea was to provoke aesthetic and/or informational experiences for the production of meanings about our environment in the case of schools, and on the concepts present in undergraduate and undergraduate academic activities in the case of higher education. The purpose of these experiments is to activate sensibility and cognition as functions of intelligence, as well as registration and sharing as functions of sociability.

Schlemmer ( 2018 ) states that inspiration in the cartographic method of intervention research to develop inventive methodologies and interventionist, aggregative, and gamified pedagogical practices comes from the following elements:

the intervention research perspective, and, during the development of games and gamified processes, at different times, both the teacher and the subjects themselves act as interventionist mediators

the idea of working with learning as an invention of problems (“supplementary” attention, duration, attentive recognition of a context), besides learning how to solve problems (attention to pragmatic utilitarian life)

the proposal to follow the course

the clue metaphor

movements of the cartographer’s attention (tracing, touching, landing, and attentive recognition).

In the context of tracks, according to Schlemmer ( 2018 ), the proposal is to work with the concept of geographic tracks (local/specific points in the community/city), live tracks (people from local community who hold certain knowledge necessary for the development of the missions), online tracks (specialists that do not belong to the local community, but that can appear in video), as characters in Mixed Reality (MR) or in Augmented Reality (AR) (iotized objects).

In the scope of a cartographer’s attention movements, tracing is characterized by the exploration/scanning of the field—in the case of this research, geographic and online spaces in search of clues (information) to understand the processes; the touch triggers the selection process which consists, in this case from Schlemmer ( 2016a , 2017 ), in the selection of geographic clues, online clues, and live clues, to direct the research; and landing refers to stopping, zooming in on lanes, choice/definition, and attentive recognition in the perception of the global context.

The metaphor of the clue, as well as the changing focus of the cartographer’s attention, can also serve as inspiration to understand the composition of tracks, as well as the progression itself in the context of a game or gamified process—gaining achievements—at the same time as achievements can be understood as skills for the own gameplay and sociability. 9

It is important to point out that as a result of this process the inventive methodology |Gamified Learning Processes (GLP) is created (Schlemmer, 2018 ).

The Grupo de Pesquisa em Educação Digital (GPe-dU) has also investigated and developed theoretical, methodological, and technological experiences inspired by the cartographic method of research, as methodology for the monitoring and evaluation of learning in games and gamified processes from an interventionist perspective and developed in a hybrid, multimodal, pervasive, and ubiquitous context. According to Schlemmer and Lopes ( 2016 ) and Schlemmer ( 2016a ) such contexts are more easily subject to the pulverization of the spaces of participation and registration and, therefore, can hinder the exercise of teaching and of the discourse regarding both pedagogical mediation and the evaluation. Thus, the proposal developed by the authors, inspired by the cartographic method of intervention research as a methodology of monitoring and evaluation, allows accompanying the subjects in their different learning pathways, involving analogue technologies and DT, physical and online face-to-face interactions to develop their own missions and projects that, from the perspective of bring your own device (BYOD), can extend beyond the time set for formal education. The fact that the subject possesses a mobile device and is connected creates conditions of possibility for him or her to remain engaged in the process, regardless of time and space. Thus, the processes of monitoring and evaluation can, at different times, be “situated” and still intertwined. By means of clues, designed and planned to provide the hybridization of analogue and digital spaces, it is possible to establish a multimodal context, which is desirable when talking about immersion, agency, and engagement.

More specifically, with regard to the movements proposed by Kastrup ( 2007 , 2008 ) and others in the cartographic method, the development of gamification and game experiments allowed us to evaluate the power of the method and the inadequacies we perceive in our own experience of building the game or gamification. The question that seemed to us most challenging was to guarantee the unpredictability and rhizomatic opening of the cartographic method and attention. If we work from a perspective of “use of” rather than inventiveness, the design of phases of a gamified game or process, for example, which was designed by someone to be “applied” in education or simply played by players, cannot always guarantee the rhizomatic opening that underlies the two proposals of methods previously carried out, because it has a limited context and whose control is not complete developed in the field reconfiguration of the students learning. The possible reconfigurations take place in the circular or linear dimension of success itself when completing the missions. In this sense, an a priori objective always seems limited from the point of view of cartography, but not limiting from the point of view of learning. What is learned opens possibilities, but in the dynamics of the game these are not necessarily unpredictable, since it is a condition that games advance in phases.

The clue metaphor to the tracks conception, according to Schlemmer and Lopes ( 2016 ), was inspiring for game designers or gamification, but, to become inspirational to gaming, we identified the need for players to leave “traces” which may become clues for other players. An interesting strategy would be to insert notebooks as an object/item that loads and can be left somewhere—as in some online and offline role-playing games—into the dynamics of the game or gamification. Another strategy would be to insert HA and/or bots equipped with AI to record and report events, producing clues for the route of the tracks to grow and insert, at continuously, new challenges based on the reconfiguration of the field of knowledge produced by the players themselves. This perspective of valuing, in the context of game or gamification, ways of recording and sharing personal narratives (dynamic clue production) would be an interesting possibility for both teachers and students to map their learning processes—after all, mapping, from the perspective here presented, is a means to track processes.

In this case, it seems important to consider that it is necessary to invest in game dynamics that strengthen and value the narratives of the players (as in the case of Role Playing Game [RPG] or, better still, to work at the level of inventiveness, in which the entire process of the conception and development of the game or gamification is co-constructed by the learning subjects themselves).

It is important to mention that, in the case of the researches we develop, because they are situated within the scope of enactive and inventive cognition, the creation of the game or process is a result of a co-creation process between and with the learning subjects themselves, where the teacher acts as an interventionist, conducting pedagogical mediation. Thus, everything that composes the game or gamified process—from the pre-concept, concept, and development; what is implied in the definition of mechanics and dynamics; to being able to include tracks and progression—are defined by the subjects themselves who, therefore, extrapolate the perspective of knowledge as representation and learning as problem-solving, working on the level of knowledge as interpretation and learning through the invention of problems, from the notion of the rhizome, which is at the base of the development of both the cartographic method of intervention research, as well as the cartography of the controversies. This approach is qualitatively different from the traditional approaches found in games or gamified processes, where the subjects are only users, players, whose learning is by solving problems and their evolution by linear stages or phases.

Most recently, the projects “The City as a Learning Space: Games and Gamification in the Constitution of Hybrid, Multimodal, Pervasive and Ubiquitous Spaces for the Development of Citizenship,” and “The City as a Learning Space: Education for Citizenship in Hybrid, Multimodal, Pervasive and Ubiquitous Contexts,” also developed on the basis of the cartographic method of intervention research, we have more appropriately called the cartography of controversies, in order to better understand, in addition to what occurs at the micro level (enative and inventive cognition), the associations that occur between HA and NHA, which may be due to convenience, coexistence, opposition, and exclusion. These associations also evidence planes of forces and, therefore, political expression of cognition, in the sense that “knowing involves a position in relation to the world and itself, an attitude, an ethos” (Kastrup, Tedesco, & Passos, 2015 , p. 12). These associations can be evidenced in a cosmogram, which makes it possible to draw the distribution of the agency, the diagram of the mediations, in short, the design of the movement, its formation. In this way, the cosmogram works with the empirical and in motion, unlike the paradigm that works with the frame, with the theoretical framework stabilized with the model as structure.

Schlemmer and Lopes ( 2016 ) and Schlemmer ( 2018 ) emphasize that the proposal does not consist in a transposition of the method or methods, but rather an experimentation with the logic behind them, as well as some of its elements, which are linked to others, in this case, present in games, gamification, and PAGs, allowing us to develop inventive methodologies and pedagogical interventionist, aggregative, and gamified practices in the educational context.

Thus, relating the cartographic method of controversies and, consequently, elements present in the ANT with the cartographic method of intervention research and with the perspective of enactive cognition, in its two senses of action—corporate cognition and inventive cognition—allows us to understand that the process of invention or innovation, both in development and research, advances from multiple breakdowns, deviations, compositions, associations, and reassociations that occur in the empirical movement. In this way, understanding an invention, an innovation, implies tracking the traces and retracing the whole network of deviations and compositions, associations and reassociations that constitute the path.

From this context questions that inspire future investigations arise:

How can Latour’s concept of symmetry or flat ontology, in which HA and NHA are on the same plane (which eliminates the anthropocentric view of cognition), contribute to research in education?

How can the Latour cosmogram contribute to the intervention research cartographic method as a means of understanding the paths made by the different actors, as well as the controversies established in a network?

How can the intervention research cartographic method, comprising cognition as the invention of problems, associated with the cartography of controversies, be part of the methodology to help us understand the place of the human in the context of intelligent cities?

To sum up, “we go, we hear, we learn, we practice, we become competent, we change our minds. Very simple indeed: this is called research. Good research always produces copious new descriptions . . . There is no in-formation, just trans-formation” (Latour, 2012 , pp. 212–216).

• Alvarez, J. , & Passos, E. (2009). Cartografar é habitar um território existencial. In E. Passos , V. Kastrup , & L. Escóssia (Eds.), Pistas do método da cartografia: Pesquisa-intervenção e produção de subjetividade (pp. 131–149). Porto Alegre, Brazil: Sulina.
• Barros, L. P. de , & Kastrup, V. (2009). Cartografar é acompanhar processos. In E. Passos , V. Kastrup , & L. Escóssia (Eds.), Pistas do método da cartografia: Pesquisa-intervenção e produção de subjetividade (pp. 52–75). Porto Alegre, Brazil: Sulina.
• Bergson, H. (1934). La pensée et le mouvant: Essais et conférences. In Œuvres: Édition du Centenaire . Paris, France: Presses Universitaires de France.
• Caiafa, J. (2007). Aventura das cidades . Rio de Janeiro, Brazil: Ed. FGV.
• Deleuze, G. , & Guattari, F. (1995). Mil platôs–capitalismo e esquizofrenia , Vol. 1 ( A. Guerra Neto and C. Pinto Costa , Trans.). Rio de Janeiro, Brazil: Editora 34.
• Despret, V. (2002). Quand le loup habitera avec l’agneau . Paris, France: Les empecheurs de penser em rond.
• Dewey, J. (1938). Experience and education . New York, NY: Macmillan.
• Di Felice, M. (2009). Paisagens pós-urbanas: o fim da experiência urbana e as formas comunicativas do habitar . São Paulo, Brazil: Annablume.
• Escóssia, L. , & Tedesco, S. (2009). O coletivo de forças como plano de experiência cartográfica. In E. Passos , V. Kastrup , & L. Escóssia (Eds.), Pistas do método da cartografia: Pesquisa-intervenção e produção de subjetividade . Porto Alegre, Brazil: Sulina.
• Foucault, M. (1979). Microfísica do poder (Organized and translated by Roberto Machado ). Rio de Janeiro, Brazil: Edições Graal.
• Kastrup, V. (2007). O funcionamento da atenção no trabalho do cartógrafo . Psicologia & Sociedade , 19 (1), 15–22.
• Kastrup, V. (2008). O método cartográfico e os quatro níveis da pesquisa-intervenção. In L. R. Castro & V. Besset (Eds.), Pesquisa-intervenção na infância e adolescência . Rio de Janeiro, Brazil: Nau Editora.
• Kastrup, V. (2009). O funcionamento da atenção no trabalho do cartógrafo. In E. Passos , V. Kastrup , & L. Escóssia (Eds.), Pistas do método da cartografia: Pesquisa-intervenção e produção de subjetividade . Porto Alegre, Brazil: Sulina.
• Kastrup, V. (2015). A Cognição Contemporânea e a Aprendizagem Inventiva. In V. Kastrup , S. Tedesco , & E. Passos , Políticas da Cognição . Porto Alegre, Brazil: Sulina.
• Kastrup, V. , & Barros, R. B. (2009). Movimentos-funções do dispositivo na prática da cartografia. In E. Passos , V. Kastrup , & L. Escóssia (Eds.), Pistas do método da cartografia: Pesquisa-intervenção e produção de subjetividade . Porto Alegre, Brazil: Sulina.
• Kastrup, V. , Tedesco, S. , & Passos, E. (2015). Políticas da Cognição . Porto Alegre, Brazil: Sulina.
• Latour, B. (1994). Jamais fomos modernos . Rio de Janeiro, Brazil: Editora 34.
• Latour, B. (2012). Reagregando o social: Uma introdução à teoria do ator-rede . São Paulo, Brazil: EDUSC.
• Latour, B. (2016). Cogitamus: Seis cartas sobre as humanidades científicas . São Paulo, Brazil: Editora 34.
• Lemos, A. (2013). A comunicação das coisas: Teoria ator-rede e cibercultura . São Paulo, Brazil: Annablume.
• Lopes, D. Q. , & Schlemmer, E. (2017). Considerações éticas, epistemológicas e metodológicas sobre o fazer pesquisa em educação e cultura digital. Revista Edapeci: Educação a Distância e Práticas Educativas Comunicacionais e Interculturais , 17 , 1–15.
• Lopes, D. Q. , Schlemmer, E. , & Molina, R. (2014a). Atenção Cartográfica em Pesquisas Online sobre Políticas de Inclusão Digital. Polise Psique , 4 , 150–169.
• Lopes, D. Q. , Schlemmer, E. , & Molina, R. (2014b). Cartography mediated by digital technologies: New perspectives for ethnographic research. In P. Landri , A. Maccarini , & R. DeRosa (Eds.), Networked together: Designing participatory research in online ethnography (pp. 24–38). Naples: University of Naples.
• Lopes, D. Q. , & Valentini, C. B. (2012). Mídias locativas e realidade mixada: A produção de sen dos sobre o digital-virtual a par r da cartografia com suporte das tecnologias digitais. Educação Unisinos , 16 (3), 205–214.
• Maturana, H. R. (1993a). As bases biológicas do aprendizado. Dois Pontos, Belo Horizonte , 2 (16), 64–70.
• Maturana, H. R. (1993b). Uma nova concepção de aprendizagem. Dois Pontos, Belo Horizonte , 2 (15), 28–35.
• Maturana, H. R. (1998). Da Biologia à Psicologia . Porto Alegre, Brazil: Artes Médicas.
• Maturana, H. R. , & Mpodozis, J. (1992). Origen de las especies por medio de la deriva natural . Publicacion ocasional No 46/1992. Museo Natural de Historia Natural. Santiago, Direccion de Bibliotecas. Archivos y Museos.
• Maturana H. , & Rezepka, S. N., de . (2000). Formação Humana e Capacitação . Petrópolis, Brazil: Vozes.
• Maturana, H. R. , & Varela, F. J. G. (1997). De Máquinas e Seres Vivos: Autopoiese—a Organização do Vivo . Porto Alegre, Brazil: Artes Médicas.
• Maturana, H. R. , & Varela, F. J. (2001). A Árvore do Conhecimento: As bases biológicas da compreensão humana . São Paulo, Brazil: Pala Athenas.
• Melo, M. F. A. Q. (2011). Discutindo a aprendizagem sob a perspectiva da teoria ator-rede. In Educar em Revista (vol. 39, pp. 177–190). Curitiba, Brazil: UFPR.
• Passos, E. (2015). Pensando a subjetividade com conceitos híbridos: A psicologia em interface com a filosofia e a biologia. In V. Kastrup , S. Tedesco , & E. Passos , Políticas da Cognição . Porto Alegre, Brazil: Sulina.
• Passos, E. , & Barros, R. B. (2009). Por uma política da narratividade. In E. Passos , V. Kastrup , & L. Escóssia (Eds.), Pistas do método da cartografia: Pesquisa-intervenção e produção de subjetividade . Porto Alegre, Brazil: Sulina.
• Passos, E. , & Eirado, A. (2009). Cartografia como dissolução do ponto de vista do observador. In E. Passos , V. Kastrup , & L. Escóssia (Eds.), Pistas do método da cartografia: Pesquisa-intervenção e produção de subjetividade . Porto Alegre, Brazil: Sulina.
• Passos, E. , Kastrup, V. , & Escóssia, L. (Eds.). (2009). Pistas do método da cartografia: Pesquisa-intervenção e produção de subjetividade . Porto Alegre, Brazil: Sulina.
• Passos, E. , Kastrup, V. , & Tedesco, S. (2014). Pistas do método da cartografia: A experiência da pesquisa e o plano comum . Porto Alegre, Brazil: Sulina.
• Piaget, J. (1976). A Equilibração das Estruturas Cognitivas . Rio de Janeiro, Brazil: Zahar Editores.
• Piaget, J. (1978a). Fazer e compreender . São Paulo, Brazil: Melhoramentos.
• Piaget, J. (1978b). A Tomada de consciência . São Paulo, Brazil: Melhoramentos.
• Piaget, J. (1995). Abstração reflexionante: Relações lógico-aritméticas e ordem das relações espaciais . Porto Alegre, Brazil: Artes Médicas.

• Schlemmer, E. (2013). Anatomia no metaverso Second Life: Uma proposta em i-Learning . Relatório de Pesquisa. São Leopoldo, Brazil: Unisinos.
• Schlemmer, E. (2014). Gamificação em espaços de convivência híbridos e multimodais: Design e cognição em discussão. Revista da FAEEBA-Educação e Contemporaneidade , 23 (42), 73–89.
• Schlemmer, E. (2015). Mídia social em contexto de hibridismo e multimodalidade: O percurso da experiência na formação de mestres e doutores. Revista Diálogo Educacional (PUCPR. Impresso) , 15 , 399–421.
• Schlemmer, E. (2016a). Games e Gamificação: Uma alternativa aos modelos de EaD. Revista Iberoamericana de Educación a Distancia , 19 , 1–12.
• Schlemmer, E. (2016b). Gamificação em Espaços de Convivência Híbridos e Multimodais: Uma experiência no ensino superior . Research Report. Processo v. 408336, pp. 2013–2027.
• Schlemmer, E. (2016c). Hibridismo, Multimodalidade e Nomadismo: Codeterminação e coexistência para uma Educação em contexto de ubiquidade. In D. Mill & A. Reali (Eds.), Educação a distância, qualidade e convergências: sujeitos, conhecimentos, práticas e tecnologias (Vol. 1 (pp. 1–24). São Carlos, Brazil: EdUFSCar.
• Schlemmer, E. (2017). Gamificação em espaços de convivência híbridos e multimodais: a educação na cultura digital . Research Report. São Leopoldo, Brazil: Unisinos.
• Schlemmer, E. (2018). Projetos de aprendizagem gamificados: Uma metodologia inventiva para a educação na cultura híbrida e multimodal. MOMENTO–Diálogos em Educação , 27 , 41–69.
• Schlemmer, E. , Chagas, W. S. , & Schuster, B. E. (2015). Games e Gamificação na modalidade EAD: Da prática pedagógica na formação Inicial em Pedagogia à prática pedagógica no Ensino Fundamental. In IV Seminário Web Currículo e XII Encontro de Pesquisadores em Currículo (vol. 1, pp. 728–736). São Paulo, Brazil: PUC-SP.
• Schlemmer, E. , & Lopes, D. Q. (2016). Avaliação da Aprendizagem em Processos Gamificados: Desafios para Apropriação do Método Cartográfico. In L. Alves & I. de J. Coutinho (Eds.), Jogos Digitais e Aprendizagem (Vol. 1, pp. 179–208). Campinas, Brazil: Papirus Editora.
• Schlemmer, E. , Lopes, D. Q. , & Molina, R. (2012). Epistemological and methodological challenges in the field of research in education and digital culture. In Rethinking educational ethnography: Researching online communities and interactions Proceedings of the 2nd Annual Rethinking Educational Ethnography conference in Barcelona (Vol. 1, pp. 74–81). Barcelona, Spain: Universitat de Barcelona.
• Varela, F. (1988). Conhecer: As ciências cognitivas tendências e perspectivas . Lisbon, Portugal: Instituto Piaget.
• Varela, F. (1990). Conhecer: As ciências cognitivas: Tendências e perspectivas . Lisbon, Portugal: Instituto Piaget.
• Varela, F. , Thompson, E. , & Rosch, E. (2003). A mente Incorporada: Ciências Cognitivas e Experiência Humana . Porto Alegre, Brazil: Artmed.
• Venturini, T. (2010). Diving in magma . Public Understanding of Science , 19 (3), 258–273.
• Venturini, T. (2012). Building on faults: How to represent controversies with digital methods . Public Understanding of Science , 21 (7), 796–812.

1. A group composed of researchers from the Universidade Federal Fluminense and Universidade Federal do Rio de Janeiro.

2. The idea of an apparatus (dispositif) is based on Foucault ( 1979 ), who understood it as a decidedly heterogeneous agglomeration involving discourses, institutions, architectural organizations, regulatory decisions, laws, administrative measures, scientific statements, and philosophical, moral, and philanthropic propositions. In short, the apparatus is made up of that which is said and unsaid. The apparatus is the network that can be established among these elements (p. 244). The relationship among these elements indicates the existence of change of positions and modification of functions. An apparatus always responds to an urgent need, made clear by its strategic or dominant function.

3. Passos, Kastrup, and Escóssia ( 2009 ) understand politics in a broad sense as the form of human activity that, linked to power, relates to subjects, articulating them according to rules or norms that are not necessarily only legal in nature. Politics is also done through local arrangements, that is, micro relations, indicating this micropolitical dimension of power relations (Foucault, 1979 ).

4. See conception of non human actor, present in Latour’s ANT.

5. It is important to emphasize that the perspective of “doing and understanding” is also described as a theory in the work of Jean Piaget ( 1978a ). It is, however, necessary to establish differences and similarities.

6. ANT emerged from an interdisciplinary perspective, with contributions from different areas, and is still defining itself as a methodological tool.

7. The author of this article.

8. In the sense put forth by Dewey ( 1938 ) and taken up by Latour ( 2016 ).

9. In gamer lingo, achievements are goals that a subject can complete during the game. They can be explicit or secret, that is, that the subject discovers during the process of play.

Printed from Oxford Research Encyclopedias, Education. Under the terms of the licence agreement, an individual user may print out a single article for personal use (for details see Privacy Policy and Legal Notice).

date: 18 May 2024

• Cookie Policy
• Privacy Policy
• Legal Notice
• Accessibility
• [66.249.64.20|81.177.180.204]
• 81.177.180.204

Character limit 500 /500

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• J Med Internet Res
• v.22(8); 2020 Aug

Digital Inequality During a Pandemic: Quantitative Study of Differences in COVID-19–Related Internet Uses and Outcomes Among the General Population

Alexander jam van deursen.

1 University of Twente, Enschede, Netherlands

The World Health Organization considers coronavirus disease (COVID-19) to be a public emergency threatening global health. During the crisis, the public’s need for web-based information and communication is a subject of focus. Digital inequality research has shown that internet access is not evenly distributed among the general population.

The aim of this study was to provide a timely understanding of how different people use the internet to meet their information and communication needs and the outcomes they gain from their internet use in relation to the COVID-19 pandemic. We also sought to reveal the extent to which gender, age, personality, health, literacy, education, economic and social resources, internet attitude, material access, internet access, and internet skills remain important factors in obtaining internet outcomes after people engage in the corresponding uses.

We used a web-based survey to draw upon a sample collected in the Netherlands. We obtained a dataset with 1733 respondents older than 18 years.

Men are more likely to engage in COVID-19–related communication uses. Age is positively related to COVID-19–related information uses and negatively related to information and communication outcomes. Agreeableness is negatively related to both outcomes and to information uses. Neuroticism is positively related to both uses and to communication outcomes. Conscientiousness is not related to any of the uses or outcomes. Introversion is negatively related to communication outcomes. Finally, openness relates positively to all information uses and to both outcomes. Physical health has negative relationships with both outcomes. Health perception contributes positively to information uses and both outcomes. Traditional literacy has a positive relationship with information uses and both outcomes. Education has a positive relationship with information and communication uses. Economic and social resources played no roles. Internet attitude is positively related to information uses and outcomes but negatively related to communication uses and outcomes. Material access and internet access contributed to all uses and outcomes. Finally, several of the indicators and outcomes became insignificant after accounting for engagement in internet uses.

Conclusions

Digital inequality is a major concern among national and international scholars and policy makers. This contribution aimed to provide a broader understanding in the case of a major health pandemic by using the ongoing COVID-19 crisis as a context for empirical work. Several groups of people were identified as vulnerable, such as older people, less educated people, and people with physical health problems, low literacy levels, or low levels of internet skills. Generally, people who are already relatively advantaged are more likely to use the information and communication opportunities provided by the internet to their benefit in a health pandemic, while less advantaged individuals are less likely to benefit. Therefore, the COVID-19 crisis is also enforcing existing inequalities.

Introduction

The World Health Organization considers coronavirus disease (COVID-19) to be a public emergency threatening global health [ 1 ]. Governments worldwide have taken stringent action, including requiring social distancing, closing public services, schools and universities, and canceling cultural events [ 2 , 3 ]. People are being advised or ordered to stay at home and socially isolate themselves to avoid being infected [ 4 ]. The ongoing pandemic represents an outbreak of an unparalleled scale, and it has induced widespread fear and uncertainty.

In this paper, we focus on the role of the internet during the crisis. The internet has become a crucial source for the general public, as it provides access to general information, the latest national and international developments, and guidelines on behavioral norms during the crisis. In this respect, the internet plays an important role in the great challenges facing governments regarding the transfer of knowledge and guidelines to the population at large. When individuals understand the need and rationale behind government-enforced measures, they are more motivated to comply and even adopt measures voluntarily [ 5 , 6 ]. In addition to informational purposes, the internet enables individuals to share news and experiences with people they cannot meet face-to-face, remain in contact with friends and family, seek support, and ask questions of official agencies, including health agencies. Further, the internet enables people to take initiatives such as raising money or preparing packaged meals for people in need, such as health workers or people who have lost their jobs. In sum, the internet plays a vital role for people of all social strata and backgrounds during a time of worldwide crisis. All people should thus be able to use the internet as a source of information and communication.

However, digital inequality research has shown that internet access is not evenly distributed among the general population [ 7 , 8 ]. The basic idea of digital inequality stems from a comparative perspective of social and information inequality, as there are benefits associated with internet access and negative consequences of lack of access [ 9 ]. Calamities are often a story of inequality [ 10 ]; therefore, in this paper, we aimed to gain a deeper and broader understanding of the differences in how people use the internet to cope during the COVID-19 crisis. Van Dijk’s resources and appropriation theory [ 8 ] explains differences or inequalities of internet access by considering personal and positional categories of individuals and the individuals’ resources. Internet access itself is considered to be a process of appropriation involving attitudinal access, material access, skills access, and in the final stage, usage access. The latter entails differences in the type of activities that people perform on the internet. The consequences of the process are the outcomes of internet use. These outcomes in turn reinforce personal and positional inequalities and an unequal distribution of resources [ 8 ] ( Figure 1 ). The first goal of this paper is to provide a timely understanding of how different people use the internet and the outcomes they gain from it in relation to the COVID-19 pandemic.

Simplified model of the resources and appropriation theory [ 8 ].

Internet use and outcome differences between groups of people are likely to have profound consequences on how people manage a crisis. For example, older people are most in danger of being infected with the virus and most likely to die from the infection [ 11 ], and they also use the internet less and have the fewest internet outcomes [ 12 ]. The latter may further endanger their peculiar situation, as limited internet use and outcomes may result in a lack of critical information or necessary support.

COVID-19–Related Internet Uses and Outcomes

To study differences in internet uses and outcomes during the COVID-19 pandemic, it is necessary to understand the types of uses and outcomes that are at play. Typically, uses and outcomes are studied by following conceptual classifications that distinguish different domains, such as economic, social, cultural, or personal domains [ 13 ]. Here, we take the COVID-19 pandemic as the domain of interest. Within this domain, we consider two main and conceptually different types of uses and outcomes: information and communication [ 14 , 15 ]. Information internet uses involve searching for information on all aspects of COVID-19. Potential information outcomes include becoming better informed about the disease, understanding why certain measures are necessary, and limiting the risk of becoming infected by developing greater awareness of one’s own behavior. Communication internet uses include talking to friends about the crisis, asking questions on social media or online fora, giving advice, or offering support to others. Communication outcomes include finding people on the internet who can offer support or share concern, being less lonely, and protecting others from potential COVID-19 risks. Studying both types of uses and outcomes is important, as prior research has shown that communication uses can compensate for information uses to attain beneficial internet outcomes [ 16 ].

Determinants of COVID-19–Related Internet Uses and Outcomes

Digital inequality research suggests that the vast amount of web-based information and communication possibilities around the COVID-19 pandemic are likely to be difficult to grasp and conceptualize for sections of the general population [ 7 ]. Some frequently observed personal categorical inequalities are gender, age, personality, and health [ 7 ]. Earlier research revealed that men and women differ in their internet activities; women are more likely to use email and social media, whereas men are more likely to use the internet to obtain information [ 17 , 18 ]. Age in general has a negative impact on all types of internet uses and outcomes [ 7 ]. In the COVID-19 crisis, older people are especially vulnerable; therefore, it is very important for them to know how to behave and be safe. We hypothesize that (H1) men are more likely to be involved in information-related uses and outcomes while women are more likely to be involved in communication-related internet uses and outcomes regarding COVID-19-related internet uses and outcomes. We also hypothesize that (H2) age contributes negatively to COVID-19–related internet uses and outcomes.

An individual’s personality may hinder or stimulate their engagement in certain COVID-19–related activities. Cognitive appraisal theory suggests that individuals complete two types of cognitive appraisal processes in a crisis [ 19 ]. The process starts with an evaluation of the crisis as a potential source of danger or life disruption. If the crisis is not determined to be dangerous, it is not considered a stressor and does not require intervention. If the crisis is determined to be relevant, it is considered a stressor and must be evaluated further by balancing the demands of the crisis and the person’s resources [ 20 ]. At this point, personality enters the equation [ 20 ]. There is a general consensus regarding the Big Five model when personality traits are studied. This model proposes five personality traits of agreeableness, neuroticism, conscientiousness, introversion, and openness [ 21 ]. However, there is no agreement as to whether these traits contribute to or detract from resisting disturbance [ 20 ]. There is also no consensus on how the Big Five personality traits relate to internet use [ 7 , 22 ]. For example, conscientiousness relates to people who abide by rules. On one hand, one might argue that this would result in a greater need for information on how to behave. On the other hand, the internet is unstructured, and rules and policies are absent to a large extent. When linking personality traits to internet use for psychological adjustments to the COVID-19 crisis, it is not evident whether these traits will support or hinder COVID-19–related internet uses and outcomes. We hypothesize that (H3a) agreeableness, (H3b) neuroticism, (H3c) contentiousness, (H3d) introversion, and (H3e) openness are related to COVID-19–related internet uses and outcomes.

An individual’s health may play an important role in how they approach COVID-19. To gain an elaborate understanding of how health relates to COVID-19–related internet uses and outcomes, we followed earlier research that distinguishes between different health aspects [ 23 ]: A person’s physical functioning or the degree to which their health currently interferes with activities such as sports, carrying groceries, climbing stairs, and walking, their mental health or psychological distress and well-being, and their health perception concerning their own health rating in general. During a crisis, we expect that people with health issues are more likely to turn to the internet for comfort and reassurance. We hypothesize that (H4a) physical functioning, (H4b) mental health, and (H4c) health perception contribute negatively to COVID-19–related internet uses and outcomes.

The final type of personal inequality considered in this study is traditional literacy, which is known to have a substantial impact on how the internet is used [ 24 , 25 ]. We consider literacy to be the ability to read, write, and understand text, which is also framed under the umbrella terms functional literacy or fundamental literacy [ 24 ]. Functional or traditional literacy can be considered as the basic dimension of all literacy concepts [ 26 ]. Considering the crucial role the internet is playing in the COVID-19 crisis, a low level of literacy is a potentially large inhibitor of understanding information and being involved in web-based communication. We hypothesize that (H5) traditional literacy contributes positively to COVID-19–related internet uses and outcomes.

Education is the most observed positional categorial inequality in digital divide research, and it is likely to play a role in the current context. People with higher levels of education are better equipped to comprehend web-based information and benefit from internet use [ 7 ]. We hypothesize that (H6) education contributes positively to COVID-19–related internet uses and outcomes.

When studying differences in internet uses and outcomes, the resources people can access are often derived from Pierre Bourdieu’s capital theory [ 27 ], which stresses the importance of including not only economic but also social and cultural resources to determine one’s status and position in society. In the COVID-19 pandemic, economic and social resources are likely to be important, as earlier research has shown that people with greater economic resources—mostly operationalized as income in digital inequality research—are known to use the internet more efficaciously and productively [ 7 , 28 ]. People with more social resources are more likely to have access to family, friends, or other contacts on the internet [ 29 ]. We hypothesize that (H7a) economic and (H7b) social resources contribute positively to COVID-19–related internet uses and outcomes.

The Internet Appropriation Process

The core of the resources and appropriation theory is access to technology, which is considered as a process of appropriation involving attitudinal, material, skills, and usage access. Attitudinal access concerns a person’s attitude towards the internet; according to theories of technology adoption, this type of access is crucial for using the internet [ 30 ]. Material access can be defined in terms of the different devices that people use to access the internet and all other web-based resources, including desktop computers, laptop computers, tablets, smartphones, game consoles, and interactive televisions [ 31 ]. Skills access concerns the skills necessary to use the internet, ranging from operational and information skills to social and content creation skills [ 32 ]. Prior research has revealed that all three types of internet access directly affect internet uses and outcomes [ 16 ]. We hypothesize that (H8a) attitudinal internet access, (H8b) material internet access, and (H8c) skills internet access contribute positively to COVID-19–related internet uses and outcomes.

The Effects of COVID-19–Related Internet Uses on Their Corresponding Outcomes

A recent multifaceted consideration of digital inequality revealed a strong effect of internet uses on outcomes [ 12 ]. Further, people’s internet activities appeared to be more important than their personal characteristics with regard to inequalities in outcomes of internet use. This suggests that the variables discussed in the prior sections will become less important for obtaining information outcomes when people are involved in COVID-19–related internet information uses. This is also true for COVID-19–related communication uses and outcomes. The second goal of this paper is to reveal the extent to which the indicators discussed remain important for obtaining internet outcomes after people are involved in the corresponding uses.

Recruitment

This study used a web-based survey and drew upon a sample collected in the Netherlands. To obtain a representative sample of the population, we used PanelClix, a professional organization for market research, to provide a panel of approximately 110,000 people. Members of the panel received a small incentive for every survey they completed. In the Netherlands, 98% of the population uses the internet; therefore, the internet user population is very closely representative of the general population in terms of its sociodemographic makeup. The panel included novice and advanced internet users. In total, we aimed to obtain a dataset with approximately 1700 respondents over the age of 18. Eventually, this resulted in the collection of 1733 responses over a 1-week period in April 2020. During the data collection period, three amendments to the sampling frame were made to ensure the representativity of the Dutch population. Accordingly, the analyses revealed that the gender, age, and formal education of our respondents largely matched official census data. As a result, only very small post hoc corrections were needed.

The web-based survey used software that checked for missing responses and prompted users to respond. The survey was pilot-tested with 10 internet users over two rounds. Amendments were made based on the feedback provided. No major comments were provided in the second round. The average time required to complete the survey was 20 minutes.

We initially developed 11 survey items pertaining to COVID-19–related internet use. Respondents were asked to indicate the extent to which they used the internet for various activities in the past month using a 5-point scale (“not” to “multiple times a day”) as an ordinal-level measure. Principal component analysis with varimax rotation was used to determine two underlying usage clusters, one related to information and one to communication. Factor loadings were employed at 0.4 and above for each item [ 33 ]. In total, 8 items (3 for information and 5 for communication) were retained in a two-factor structure with eigenvalues over 1.0, together accounting for 76% of the total variance.

For COVID-19-related information and communication internet outcomes, we developed 14 items mapped onto the use items. A 5-point agreement scale as an ordinal level measure was used. Principal component analysis with varimax rotation resulted in a structure that matched the conceptual definition of information outcomes (4 items) and communication outcomes (4 items). The two factors showed eigenvalues over 1.0 and explained 65% of the variance.

Gender was included as a dichotomous variable, and age was directly asked (mean 50.2, SD 17.0).

Personality was measured with the Quick Big Five personality questionnaire [ 34 ], which consists of 30 adjectives reflecting a valid and reliable measure of the Big Five traits. Participants were asked to rate the extent to which a particular adjective applied to them on a 7-point scale, ranging from completely untrue to completely true. The Cronbach α values for the five traits were .89 for agreeableness, .88 for neuroticism, .88 for conscientiousness, .87 for introversion, and .81 for openness.

Physical health, mental health, and health perception were measured with the Dutch version of the Medical Outcomes Study (MOS) Short-Form General Health Survey (SF-20) [ 35 ]. This instrument enables respondents to assess their general health and generates composite summary scores representing different types of health. We normalized the scales, with higher scores representing better functioning. Physical health was measured with 5 items (2-point scale; α=.89; mean 1.75, SD 0.34), mental health with 5 items (5-point scale; α=.85; mean 3.65, SD 0.77), and health perception with 5 items (5-point scale; α=.86; mean 3.39, SD 0.85).

To measure traditional literacy, we used the validated 11-item Diagnostic Illiteracy Scale [ 36 ]. Sample items included “I have difficulties with reading and understanding information from my municipality” and “I find it difficult to read and understand my telephone bill.” A 5-point agreement scale was used. Scores on the scale exhibited high internal consistency. Items were recoded so that higher scores corresponded with higher levels of literacy (α=.94; mean 4.33, SD 0.71).

To assess education, data regarding degrees earned were collected and used to create three groups: low (primary), middle (secondary), and high (tertiary) educational achievement.

Economic resources were objectively measured by seeking the annual family income in the last 12 months. Twelve categories were recoded into three categories of low for <€30,000 (US $35,503.50), middle for €30,000 to €70,000 (US $35,503.50 to $82841.50), and high for >€70,000 (>US $82841.50). For social resources, we used the MOS Social Support Survey [ 37 ]. Respondents completed 18 items covering emotional support (eg, “Someone you can count on to listen when you need to talk”), informational support (eg, “Someone to give you good advice about a crisis”), and tangible support (eg, “Someone to help you if you were confined to bed”). All items were rated on a 5-point Likert scale with anchors of none of the time (1) and most of the time (5). We computed an aggregate measure of support availability (α=.96; mean 3.83, SD 0.85).

Attitudinal internet access was measured by three items adapted from the Digital Motivation Scale [ 38 ]. A 5-point agreement scale was used, and all items were balanced for the direction of response (α=.74; mean 4.10, SD 0.70). An example statement is “Technologies such as the internet and mobile phones make life easier.” To measure material internet access, we considered 7 devices used to connect to the internet (mean 3.43, SD 1.53). Included were desktop computer, laptop computer, tablet, smartphone, smart TV, game console, and smart device (eg, activity tracker). Finally, skills internet access was adapted from the conceptual idea behind the Internet Skills Scale [ 32 ]. We proposed 30 items reflecting operational, information navigation, social, and creative internet skills. A 20-item single skills construct resulted from the principal component analysis. All items were scored on a 5-point scale that ranged from “not at all true of me” to “very true of me” and exhibited high internal consistency (α=.96; mean 3.67, SD 0.97). Example items are “I know how to open downloaded files,” “I find it hard to decide what the best keywords are to use for online searches,” and “I know which information I should and shouldn’t share online.”

Statistical Analysis

To test the hypotheses and account for the sequentiality between COVID-19–related internet uses and outcomes, hierarchical regression analyses were used. In the first model, we tested our hypotheses by analyzing the significant determinants for the two types of COVID-19–related internet uses and the two corresponding outcomes. In the second model, we sought to determine the changes in the significance of the determinants after the internet uses were added to the models.

Table 1 provides an overview of the sample of people surveyed in the study.

Demographic profile of the Dutch internet user sample (N=1733), n (%).

a Low: primary; middle: secondary; high: tertiary.

Table 2 shows the mean scores of the survey questions related to internet uses and internet outcomes.

Survey questions and responses on the 5-point Likert scale.

a COVID-19: coronavirus disease.

b RIVM: Rijksinstituut voor Volksgezondheid en Milieu.

The first goal of this paper was addressed in the first model, as presented in Table 3 , where several significant determinants for COVID-19 uses and outcomes are revealed.

Hierarchical regression analysis summary for coronavirus disease–related internet uses and outcomes (Model 1).

Table 3 shows that men are more likely to be involved in COVID-19–related communication uses. Age is positively related to COVID-19–related information uses and negatively related to COVID-19 communication uses and outcomes. Concerning personality traits, agreeableness is negatively related to COVID-19–related information and communication uses and to communication outcomes. Neuroticism is positively related to both uses and to communication outcomes.

Conscientiousness is not related to any of the uses or outcomes. Introversion is negatively related to COVID-19–related communication uses and outcomes, suggesting that this is performed more by extraverted people. Finally, openness relates positively to information uses and to both outcomes.

The results further show that concerning the three health indicators, physical health is negatively related to communication uses and outcomes. Mental health did not contribute to any uses or outcomes. Health perception contributes positively to information uses and to both outcomes.

Traditional literacy has a positive relationship with information-type uses and with both outcomes, and education has a positive relationship with COVID-19–related information and communication uses. Economic and social resources were not related to any COVID-19 uses or outcomes.

Attitudinal internet access is positively related to information uses and outcomes but is negatively related to communication uses and outcomes. Material internet access contributes positively to all uses and outcomes, and skills access has a positive relationship with all uses and outcomes. Table 4 provides an overview of the hypotheses.

Overview of the hypotheses.

a ns: no significant contribution.

b –: significant negative contribution.

c R: reject.

d +: significant positive contribution.

e PS: partial support.

f S: support.

Finally, to address the second goal of the study, we tested what would happen to the contribution of the outcome determinants when the corresponding uses were added to the analyses (Model 2: see Tables 5 and ​ and6). 6 ). Adding the uses significantly increased the explained variance; also, several of the relationships between personal and positional categories and between resources and outcomes became insignificant. The relationships that remained significant for information outcomes were age, health perception, and traditional literacy. Furthermore, attitudinal internet access remained significant. For communication outcomes, the relationships that remained significant were age, openness, and traditional literacy.

Hierarchical regression analysis summary for coronavirus disease–related internet outcomes (Model 2).

a N/A: not applicable.

Changes in the significance of the determinants after internet uses were added to the models ( P <.001).

Principal Results

This paper aims to provide a comprehensive examination of digital inequality in the case of an unprecedented health pandemic. The first goal of the study was to reveal how inequality manifests itself in COVID-19–related internet information and communication uses and outcomes. The findings revealed several relationships between the background variables and the two types of internet uses and outcomes.

Older people were found to be less equipped to use the internet for information and communication during a time of crisis. However, they were more likely to engage in information-type COVID-19–related internet uses, possibly because they are at greatest risk from the disease [ 11 ]. This did not result in more beneficial information outcomes. Internet skills play an important role in translating internet uses into beneficial internet outcomes [ 39 ], and prior research has shown that older people have lower internet skill levels in general [ 32 ]. The finding that older people are less likely to perform communication activities or obtain communication-related outcomes is in line with prior studies [ 15 ]; however, these outcomes are important, as older people are more at risk of having severe complications when diagnosed with COVID-19. Regarding gender, contrary to general internet use, men were found to be more likely to engage in communication-type COVID-19–related internet uses during the crisis than women. A possible explanation is that men and women may respond to crisis news in different ways [ 40 ].

The positive effect of neuroticism suggests that a tendency to experience negative emotions such as anger, anxiety, or depression fuels the need to turn to the internet for COVID-19–related information and communication. People who score higher on the neuroticism scale may be more in need of guidelines on how to mitigate risks or may need more support from others to be comforted. Also, the openness trait supports both information and communication internet use and outcomes. A possible explanation is that a major crisis triggers adventure, unconventional ideas, imagination, awareness of feelings, curiosity, or a variety of experiences, all of which are aspects linked to high openness [ 21 ]. The negative contribution of agreeableness raises questions. A possible explanation is that agreeable people are less frequently sought out for communication activities. However, the internet may also be a very inviting environment for less agreeable people. Conscientiousness did not appear to be a significant determinant. People who are more stubborn and focused or more flexible and spontaneous both appear to be involved in information- and communication-type COVID-19–related internet uses and outcomes. Extroversion emerged as a trait that supports using the internet for communication uses and outcomes; this can be expected, as extroversion is marked by pronounced engagement with the external world [ 21 ].

Although we expected that psychological distress would play a role in the current context, as there would be a relatively high need for information and support from others, mental health did not surface as a significant contributor. Furthermore, we did find that physical health problems appear to encourage web-based COVID-19–related communication uses and outcomes. The most likely explanation is that people with underlying health problems are more at risk (and thus more bound to their homes) and thus have higher needs for communication with friends and family. A possible reason for the positive effect of health perception is that people who believe their personal health to be good may feel better equipped to support others during the COVID-19 pandemic.

As expected, traditional literacy played an important role. A lack of general ability to read, write, and understand text further disadvantages individuals in the case of the COVID-19 pandemic, as they have less access to information and communication sources. COVID-19 is a new, unknown, and complicated disease with characteristics that are often described in difficult medical language that is not easy to read. Similar findings were found for educational attainment. Research has long shown that education is one of the most prominent positional variables in digital divide research [ 7 ]. However, our results suggest that when less educated individuals are involved in information and communication internet uses, they are as likely to achieve the corresponding outcomes as people with higher levels of education. This is an important finding for designing interventions for those of lower levels of education.

An effect of economic resources did not emerge in relation to COVID-19–related internet uses and outcomes. The participants’ income did not make a difference in obtaining information and communication COVID-19–related internet outcomes. Earlier research often showed that income is especially important to consumptive and work-related internet uses [ 17 ], topics that are not considered here. Unexpectedly, social resources were not found to be influential. Apparently, a person who has an offline support network will not necessarily turn more to web-based information and communication support during a crisis.

Concerning internet access, we can first conclude that a person’s internet attitude is important for engaging in information uses and gaining information outcomes. Unexpectedly, there was a negative contribution of internet attitude to communication uses and outcomes, suggesting that individuals who have a negative evaluation of the internet in general are more likely to engage in communication uses in the event of a major crisis. Both material and skills internet access played important roles in achieving all uses and outcomes. Using a higher diversity of devices was related to higher COVID-19–related internet use and to more outcomes. The opportunities devices offer are known to be related to inequalities in internet uses and outcomes. As each device offers its own specific characteristics and advantages, a higher diversity of devices supports a larger range of use activities and outcomes [ 31 ]. Furthermore, internet skills play a fundamental role in COVID-19–related uses and in obtaining beneficial outcomes [ 12 ].

In this paper, several indicators surfaced for people’s web-based COVID-19–related uses and outcomes. The variety of important indicators raises the question of whether general policies to address digital inequalities in a time of crisis will be effective. The complex relationships between the different indicators on one hand and internet uses and outcomes on the other hand demand more focused policies, such as those related to health indicators and the need for information to enhance health outcomes. This study reveals that the greater an individual’s existing advantages, the more they benefit from the internet at a time of crisis; the converse is true as well. Marginalized people are likely to have fewer types of access available to take actions, behave as requested, or be comforted by help, creating a vicious cycle where already marginalized groups are further marginalized in a time of crisis.

To end on a positive note, the situation may become slightly less complex when we address the second goal of this paper. When people engage in information and communication internet uses in a crisis situation, their personal characteristics become less important to achieving the corresponding outcomes. This suggests that to achieve information and communication outcomes, policy or research should especially focus on encouraging people to engage in the corresponding internet uses, as we can assume to some extent that engagement with information and communication-related COVID-19 uses is the best way to achieve beneficial outcomes at a time when they are most needed.

Limitations

The current study was conducted in the Netherlands, a country whose citizens have very high household internet penetration and high levels of educational attainment. Although differences in educational background and income are present and were taken into consideration, the observed inequalities may be even stronger in countries with a less homogeneous population. Given that the greatest burden of deaths has been in countries with very diverse populations, race and associated factors are likely to play a major role.

The aim of this study was to provide a broader picture of inequality in relation to how the internet is used in the case of a major global health crisis. A broad range of determinants was considered, and the relative importance of these indicators was revealed. However, a deeper understanding and further investigation to reveal the exact underlying mechanisms that cause these indicators to play a role would provide additional explanations. This suggests that further qualitative research is needed not only to obtain in-depth understanding of the mechanisms but also to understand the consequences of the observed inequalities to complement the findings of the current quantitative approach.

Digital inequality is a major concern among national and international scholars and policy makers. In this paper, we aimed to provide a broader understanding in the case of a major health pandemic by using the ongoing COVID-19 crisis as a context for empirical work. Several groups of people were identified as vulnerable, such as older people and people with lower levels of education, physical health problems, higher levels of neuroticism, low literacy levels, and low levels of trust. The general conclusion is that people who are already relatively advantaged are more likely to use the information and communication opportunities provided by the internet to their benefit in a health pandemic, while more disadvantaged individuals are less likely to benefit. Therefore, the COVID-19 crisis is also an enforcer of existing inequalities.

Abbreviations

Conflicts of Interest: None declared.

Advertisement

Quantitative-comparative research on digital competence in students, graduates and professors of faculty education: an analysis with ANOVA

• Published: 28 March 2020
• Volume 25 , pages 4157–4174, ( 2020 )

Cite this article

• Francisco D. Guillén-Gámez 1 &
• Mª José Mayorga-Fernández 2  

3805 Accesses

27 Citations

Explore all metrics

Currently, the figure of the teacher is a key element to train students in the use of new information and communication technologies (ICT), which will positively influence the entire teaching-learning process. Therefore, it is an indispensable requirement in the initial training of the teacher the development of the digital teaching competence, understanding this construct as the set of knowledge, use and attitudes towards digital technologies. However, in the initial teacher training, the development of digital teaching competence may vary depending on the educational stage in which they carry out their teaching. For this work, the level of development of digital teaching competence of university teaching staff of the Faculty of Education, graduates of education, and students (future teacher of primary and child education) has been taken into consideration. Therefore, in this work we have tried: (1) to analyse the level of competence of the teachers regarding to three components which structure the term digital teaching competence; and (2) to compare the level of competence between the different types of teachers in each of the dimensions that make up the term digital teaching competence. For this purpose, an ex post facto investigation has been carried out with 715 participants, using one-way ANOVA technique by multiple comparisons. The results show that there are significant differences in attitudes towards digital technologies among students and graduates with respect to the university teaching staff. In relation to the knowledge and use dimensions, there are no differences between graduates and university teaching staff, but there are differences between both groups with the students.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Similar content being viewed by others

Impacts of digital technologies on education and factors influencing schools' digital capacity and transformation: A literature review

From digital literacy to digital competence: the teacher digital competency (TDC) framework

Effects of higher education institutes’ artificial intelligence capability on students' self-efficacy, creativity and learning performance

Alexander, B., Ashford-Rowe, K., Barajas-Murph, N., Dobbin, G., Knott, J., McCormack, M., ... & Weber, N. (2019). EDUCAUSE Horizon Report 2019 Higher Education Edition (pp. 3–41). EDU19.

Agreda Montoro, M., Hinojo Lucena, M. A., & Sola Reche, J. M. (2016). Design and validation of an instrument for assess digital skills of teachers in Spanish higher education. Pixel-Bit-Revista De Medios Y Educacion, 49 , 39–56.

Article   Google Scholar  

Agufana, P. B., Too, J. K., & Mukwa, C. W. (2018). Assessment of perceived ease of use and instructional use of ICT by lecturers in technical training institutions in Kenya. African Journal of Education, Science and Technology, 5 (1), 150–154.

Google Scholar  

Alfonso, R. D., Mendo, A. H., & Merino, E. C. (2018). Development and validation of a questionnaire for evaluating the impact of ICT in secondary schools. Digital Education Review, 34 , 1–26.

Bamigboye, O. B., Bankole, O. M., Ajiboye, B. A., & George, A. E. (2013). Teachers’ attitude and competence towards the use of ICT resources: A case study of university of agriculture lecturers, Abeokuta Ogun state, Nigeria. Information Manager (The), 13 (1–2), 10–15.

Barrio, T. V., Valle, M. S., & de Frutos Torres, B. (2017). Autopercepción de la competencia digital de profesores de educación primaria y secundaria de la Comunidad de Madrid e identificación del uso de las TIC en su práctica docente. In Del verbo al bit (pp. 1208-1226). Sociedad Latina de Comunicación Social.

Baturay, M. H., Gökçearslan, Ş., & Ke, F. (2017). The relationship among pre-service teachers' computer competence, attitude towards computer-assisted education, and intention of technology acceptance. International Journal of Technology Enhanced Learning, 9 (1), 1–13. https://doi.org/10.1504/IJTEL.2017.084084 .

Bisquerra, R. (2004). Metodología de la investigación educativa . Madrid: Plaza.

Buarki, H. (2016). ICT skills evaluation of faculty members in Kuwait; preliminary findings. Information Development, 32 (4), 777–798. https://doi.org/10.1177/0266666914568796 .

Bugawa, A. M., & Mirzal, A. (2018). The impact of web 2.0 technologies on the learning experience of students in higher education: A review. International Journal of Web-Based Learning and Teaching Technologies (IJWLTT), 13 (3), 1–17. https://doi.org/10.4018/IJWLTT.2018070101 .

Cabero-Almenara, J., & Barroso-Osuna, J. (2016). ICT teacher training: A view of the TPACK model. Cultura y Educación, 28 (3), 633–663. https://doi.org/10.1080/11356405.2016.1203526 .

Cabezas Gonzalez, M., & Casillas Martín, S. (2018). Social educators: A study of digital competence from a gender differences perspectivezlika. Croatian Journal of Education: Hrvatski časopis za odgoj i obrazovanje , 20 (1), 11–42. https://doi.org/10.15516/cje.v20i1.2632 .

Capilla, M. M., Torres, J. M. T., & Sánchez, F. R. (2016). Percepción del profesorado y alumnado universitario ante las posibilidades que ofrecen las TIC en su integración en el proceso educativo: reflexiones, experiencias e investigación en la Facultad de educación de Granada. EDMETIC, 5 (1), 113–142. https://doi.org/10.21071/edmetic.v5i1.4019 .

Charris Franco, M. M. (2016). Conocimientos, usos y actitudes de los estudiantes de 3° y 4° de Pedagogía de la facultad de Educación de la USAL sobre las TIC. Tesis Doctoral. Universidad de Salamanca (USAL).

Chouit, D., Nfissi, A., & Laabidi, H. (2017). Exploring the correlation between professors’ use of ICT in teaching and the levels of institutional support. Journal of English Language Teaching and Linguistics, 2 (1), 47–63. https://doi.org/10.21462/jeltl.v2i1.39 .

EC (European Commission). (2007). The Key Competences for Lifelong Learning – A European Framework. Retrieved from https://www.erasmusplus.org.uk/file/272/download

EC (European Commission). (2013). DIGCOMP: A Framework for Developing and Understanding Digital Competence in Europe. Retrieved from https://publications.jrc.ec.europa.eu/repository/bitstream/JRC83167/lb-na-26035-enn.pdf

EC (European Commission). (2017). Digital Competence Framework for Educators (DigCompEdu). Retrieved from https://ec.europa.eu/jrc/en/digcompedu

EC (European Commission). (2018). Key competences for lifelong learning. Retrieved from https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:32018H0604(01)&from=EN

Eickelmann, B., & Vennemann, M. (2017). Teachers ‘attitudes and beliefs regarding ICT in teaching and learning in European countries. European Educational Research Journal, 16 (6), 733–761. https://doi.org/10.1177/1474904117725899 .

Esteve-Mon, F. M., Gisbert-Cervera, M., & Lázaro-Cantabrana, J. L. (2016). La competencia digital de los futuros docentes: ¿Cómo se ven los actuales estudiantes de educación? Perspectiva Educacional, Formación de Profesores, 55 (2), 38–54.

Fernández Batanero, J. M., & Rodríguez Martín, A. (2017). TIC y diversidad funcional: Conocimiento del profesorado. EJIHPE. European Journal of Investigation in Health, Psychology and Education, 7 (3), 157–175. https://doi.org/10.30552/ejihpe.v7i3.204 .

Fernández-Sanz, L., Gómez-Pérez, J., & Castillo-Martínez, A. (2017). E-skills match: A framework for mapping and integrating the main skills, knowledge and competence standards and models for ICT occupations. Computer Standards & Interfaces, 51 , 30–42. https://doi.org/10.1016/j.csi.2016.11.004 .

From, J. (2017). Pedagogical digital competence--between values, knowledge and skills. Higher Education Studies, 7 (2), 43–50. https://doi.org/10.5539/hes.v7n2p43 .

García-Valcarcel, A. (2011). Integración de las TIC en la docencia universitaria. Netbiblo.

Gisbert, M., & Esteve, F. (2016). Digital Leaners: la competencia digital de los estudiantes universitarios. La cuestión universitaria, 7 , 48–59.

González Martínez, J., Esteve-Mon, F. M., Larraz Rada, V., Espuny Vidal, C., & Gisbert Cervera, M. (2018). INCOTIC 2.0: una nueva herramienta para la autoevaluación de la competencia digital del alumnado universitario= INCOTIC 2.0: A new self-assessment tool for digital competences at the university studies. Profesorado: revista de currículum y formación del profesorado, 2018, vol. 22, núm. 4, p. 133-152 . Doi: 10.30827/profesorado.v22i4.8401.

Guillén-Gámez, F. D., & Perrino, M. P. (2020). Análisis Univariante de la Competencia Digital en Educación Física: un estudio empírico. Retos, 37 (37), 326–332.

Guillén-Gámez, F. D., Lugones, A., & Mayorga-Fernández, M. J. (2019). ICT use by pre-service foreign languages teachers according to gender, age and motivation. Cogent Education , 1–17. https://doi.org/10.1080/2331186X.2019.1574693 .

Guillén-Gámez, F. D., Mayorga-Fernández, M. J., & Álvarez-García, F. J. (2018). A study on the actual use of digital competence in the practicum of education degree. Technology, Knowledge and Learning , 1–18. https://doi.org/10.1007/s10758-018-9390-z .

Gutiérrez Castillo, J. J., & Cabero Almenara, J. (2016). Estudio de caso sobre la autopercepción de la competencia digital del estudiante universitario de las titulaciones de grado de Educación Infantil y Primaria. Profesorado. Revista de currículum y formación del profesorado, 20 (2), 180–199.

Gutiérrez Porlán, I. (2014). Perfil del profesor universitario español en torno a las competencias en tecnologías de la información y la comunicación. Pixel-Bit. Revista de Medios y Educación, 44 , 51–65.

Gutiérrez, I., Prendes, M. P., & Castañeda, L. (2015). Aprendices y competencia digital. J. Cabero y J. Barroso (Coords.). Nuevos retos en tecnología educativa , 239-256.

Hatlevik, O. E., & Christophersen, K. A. (2013). Digital competence at the beginning of upper secondary school: Identifying factors explaining digital inclusion. Computers & Education, 63 , 240–247. https://doi.org/10.1016/j.compedu.2012.11.015 .

Horizon. (2017). The NMC Horizon Report > 2017 Higher Education Edition. Available in http://cdn.nmc.org/media/2017-nmc-horizon-report-he-EN.pdff

INTEF (2017). Marco Común de Competencia Digital Docente. Instituto Nacional de Tecnologías Educativas y de Formación del Profesorado. Retrieved from https://intef.es/Noticias/marco-comun-de-competencia-digital-docente-2017-intef/

Janssen, J., Stoyanov, S., Ferrari, A., Punie, Y., Pannekeet, K., & Sloep, P. (2013). Experts' views on digital competence: Commonalities and differences. Computers & Education, 68 , 473–481. https://doi.org/10.1016/j.compedu.2013.06.008 .

Joshua Chukwuemeka, E., & Iscioglu, E. (2016). An examination of lecturers’ technological pedagogical content knowledge perceptions at the Faculty of Education in EMU in Cyprus. Croatian Journal of Education: Hrvatski časopis za odgoj i obrazovanje, 18 (4), 999–1034. https://doi.org/10.15516/cje.v18i4.1845 .

Kale, U., & Goh, D. (2014). Teaching style, ICT experience and teachers’ attitudes toward teaching with web 2.0. Education and Information Technologies, 19 (1), 41–60. https://doi.org/10.1007/s10639-012-9210-3 .

Kerlinger, F. N. L., Howard, B., Pineda, L. E., & Mora Magaña, I. (2002). Investigación del comportamiento . México: McGraw Hill.

Krumsvik, R. J. (2011). Digital competence in the Norwegian teacher education and schools. Högre utbildning, 1 (1), 39–51.

Kunda, D., Chembe, C., & Mukupa, G. (2018). Factors that influence Zambian higher education lecturer’s attitude towards integrating ICTs in teaching and research. JOTSE, 8 (4), 360–384. https://doi.org/10.3926/jotse.338 .

Lázaro-Cantabrana, J., Usart-Rodríguez, M., & Gisbert-Cervera, M. (2019). Assessing teacher digital competence: The construction of an instrument for measuring the knowledge of pre-service teachers. Journal of New Approaches in Educational Research (NAER Journal), 8 (1), 73–78. https://doi.org/10.7821/naer.2019.1.370 .

Llamas-Salguero, F., & Gomez, E. (2018). Formación inicial de docentes en educación básica para la generación de conocimiento con las Tecnologías de la Información y la Comunicación. Revista Complutense De Educación, 29 (2), 577–593. https://doi.org/10.5209/RCED.53520 .

López, M. J. A. (2018). Estudio de un caso de la formación del profesorado en las tecnologías de la información y la comunicación en los centros de educación infantil y primaria. DIM: Didáctica, Innovación y Multimedia, 36 , 1–19.

López, Z. C., Aristizabal, S. S., Guerrero, S. V. A., Oviedo, G. M., Palencia, A. F. C., Rodríguez, M. D. D., et al. (2017). Actitud, conocimiento y uso de las tecnologías de la información y la comunicación (tic) para la enseñanza de las ciencias naturales en las instituciones educativas públicas del municipio de NEIVA: un estudio diagnóstico. Bio-grafía Escritos sobre la biología y su enseñanza , 1211–1220. https://doi.org/10.17227/bio-grafia.extra2017-7292 .

Martín, S. C., & González, M. C. (2018). Estudio psicométrico de un cuestionario para medir la competencia digital de estudiantes universitarios (CODIEU). Education in the Knowledge Society, 19 (3), 69–81. https://doi.org/10.14201/eks20181936981 .

Mefp (2018). Ministerio de Educación y Formación Profesional del Gobierno de España. Accedido el 27 de diciembre de 2018. Disponible en: http://www.educacionyfp.gob.es/educacion/mc/lomce/el-curriculo/curriculo-primaria-eso-bachillerato/competencias-clave/competencias-clave/digital.html

Mercader, C., & Sallán, J. G. (2017). ¿Cómo utiliza el profesorado universitario las tecnologías digitales en sus aulas?. REDU. Revista de Docencia Universitaria , 15 (2), 257-274. Doi: 10.4995/redu.2017.7635.

Mirete Ruiz, A. B. (2016). El profesorado universitario y las tic. análisis de su competencia digital. Ensayos: revista de la facultad de educacion de albacete, 31 (1), 133–147.

Muñoz-Repiso, A. G. V., Gómez-Pablos, V. B., González, M. C., Martín, S. C., Rodero, L. G., Martín, A. H., & Marcos, J. J. M. (2015). Training of university lecturers in information and communication Technology at the University of Salamanca. Revista Latinoamericana de Tecnología Educativa-RELATEC, 14 (1), 75–88. https://doi.org/10.17398/1695-288X.14.1.75 .

Pérez Escoda, A., & Rodríguez Conde, M. J. (2016). Evaluación de las competencias digitales autopercibidas del profesorado de Educación Primaria en Castilla y León (España). Revista de Investigación Educativa, 34 (2), 399–415. https://doi.org/10.6018/rie.34.2.215121 .

Peciuliauskiene, P., & Barkauskaite, M. (2007). Would-be teachers' competence in applying ICT: Exposition and preconditions for development. Informatics in Education-International Journal, 6 , 397–410.

Porlan, I. G., & Sanchez, J. S. (2016). Evaluation and development of digital competence in future primary school teachers at the University of Murcia. Journal of New Approaches in Educational Research (NAER Journal), 5 (1), 51–56. https://doi.org/10.7821/naer.2016.1.152 .

Richardson, J. T. (2011). Eta squared and partial eta squared as measures of effect size in educational research. Educational Research Review, 6 (2), 135–147. https://doi.org/10.1016/j.edurev.2010.12.001 .

Rivera-Laylle, L. I., Fernández-Morales, K., Guzmán-Games, F. J., & Eduardo-Pulido, J. (2017). ICT acceptance by university professors: Knowledge, attitude, and practicality. Revista Electrónica Educare, 21 (3), 99–116. https://doi.org/10.15359/ree.21-3.6 .

Roblizo Colmenero, M. J., & Cózar Gutiérrez, R. (2015). Usos y competencias en TIC en los futuros maestros de Educación Infantil y Primaria: hacia una alfabetización tecnológica real para docentes. Pixel-Bit. Revista de Medios y Educación, 47 , 23–39.

Rodríguez, M. D. M., Méndez, V. G., & Martín, A. M. R. M. R. (2018). Alfabetización informacional y competencia digital en estudiantes de magisterio. Profesorado, Revista de Currículum y Formación del Profesorado, 22 (3), 253–270.

Roig-Vila, R., Luna, P., & María, A. (2012). The digital competences of future teachers. An analysis with pre-school education teaching degree students at the University of Alicante. @tic. Revista d’innovació educativa, 9, 53-60. Doi: 10.7203/attic.9.1958

Røkenes, F. M., & Krumsvik, R. J. (2014). Development of student teachers’ digital competence in teacher education-a literature review. Nordic Journal of Digital Literacy, 9 (04), 250–280.

Rubio, J. C. C., Serrano, J. S., & Martínez, J. C. B. (2018). Competencia digital en futuros docentes de Ciencias Sociales en Educación Primaria: análisis desde el modelo TPACK. Educatio Siglo XXI, 36 (1), 107–128. https://doi.org/10.6018/j/324191 .

Ruiz, A. B. M. (2015). ACUTIC: cuestionario para el estudio de la actitud, el conocimiento y el uso de TIC en profesores de educación superior. In Investigar con y para la sociedad (pp. 1733-1744).

Ruiz, A. B. M., Sánchez, F. A. G., & Pina, F. H. (2015). Cuestionario para el estudio de la actitud, el conocimiento y el uso de TIC (ACUTIC) en Educación Superior. Estudio de fiabilidad y validez. Revista interuniversitaria de formación del profesorado , (83), 75-89.

Skryabin, M., Zhang, J., Liu, L., & Zhang, D. (2015). How the ICT development level and usage influence student achievement in reading, mathematics, and science. Computers & Education, 85 , 49–58.

Tondeur, J., Forkosh-Baruch, A., Prestridge, S., Albion, P., & Edirisinghe, S. (2016). Responding to challenges in teacher professional development for ICT integration in education. Educational Technology and Society, 19 (3), 110–120 Retrieved from http://www.jstor.org/stable/jeductechsoci.19.3.110 .

Upsa. (2018). Training actions 2017–2018. accessed January 24, 2020, Available at: https://www.upsa.es/investigacion-y-docencia/plan-de-investigacion/plan-formacion-profesorado-17-18.php

Van Laar, E., van Deursen, A. J., van Dijk, J. A., & de Haan, J. (2017). The relation between 21st-century skills and digital skills: A systematic literature review. Computers in Human Behavior, 72 , 577–588. https://doi.org/10.1016/j.chb.2017.03.010 .

Zhang, J., Yang, J., Chang, M., & Chang, T. (Eds.). (2016). ICT in education in global context: The best practices in K-12 schools. Springer.

Download references

Author information

Authors and affiliations.

Department of Didactics and School Organization, Faculty of Education, University of Almería (UAL), Almería, Spain

Francisco D. Guillén-Gámez

Department of Didactics and School Organization, Faculty of Education Sciences of the University of Malaga (UMA), Málaga, Spain

Mª José Mayorga-Fernández

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Francisco D. Guillén-Gámez .

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Guillén-Gámez, F.D., Mayorga-Fernández, M.J. Quantitative-comparative research on digital competence in students, graduates and professors of faculty education: an analysis with ANOVA. Educ Inf Technol 25 , 4157–4174 (2020). https://doi.org/10.1007/s10639-020-10160-0

Download citation

Received : 04 January 2020

Accepted : 12 March 2020

Published : 28 March 2020

Issue Date : September 2020

DOI : https://doi.org/10.1007/s10639-020-10160-0

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Digital teaching competence
• Digital technologies
• University teaching staff
• Find a journal
• Publish with us
• Track your research

Setting a new bar for online higher education

The education sector was among the hardest hit  by the COVID-19 pandemic. Schools across the globe were forced to shutter their campuses in the spring of 2020 and rapidly shift to online instruction. For many higher education institutions, this meant delivering standard courses and the “traditional” classroom experience through videoconferencing and various connectivity tools.

The approach worked to support students through a period of acute crisis but stands in contrast to the offerings of online education pioneers. These institutions use AI and advanced analytics to provide personalized learning and on-demand student support, and to accommodate student preferences for varying digital formats.

Colleges and universities can take a cue from the early adopters of online education, those companies and institutions that have been refining their online teaching models for more than a decade, as well as the edtechs that have entered the sector more recently. The latter organizations use educational technology to deliver online education services.

To better understand what these institutions are doing well, we surveyed academic research as well as the reported practices of more than 30 institutions, including both regulated degree-granting universities and nonregulated lifelong education providers. We also conducted ethnographic market research, during which we followed the learning journeys of 29 students in the United States and in Brazil, two of the largest online higher education markets in the world, with more than 3.3 million 1 Integrated Postsecondary Education Data System, 2018, nces.ed.gov. and 2.3 million 2 School Census, Censo Escolar-INEP, 2019, ensobasico.inep.gov.br. online higher education students, respectively.

We found that, to engage most effectively with students, the leading online higher education institutions focus on eight dimensions of the learning experience. We have organized these into three overarching principles: create a seamless journey for students, adopt an engaging approach to teaching, and build a caring network (exhibit). In this article, we talk about these principles in the context of programs that are fully online, but they may be just as effective within hybrid programs in which students complete some courses online and some in person.

Create a seamless journey for students

The performance of the early adopters of online education points to the importance of a seamless journey for students, easily navigable learning platforms accessible from any device, and content that is engaging, and whenever possible, personalized. Some early adopters have even integrated their learning platforms with their institution’s other services and resources, such as libraries and financial-aid offices.

1. Build the education road map

In our conversations with students and experts, we learned that students in online programs—precisely because they are physically disconnected from traditional classroom settings—may need more direction, motivation, and discipline than students in in-person programs. The online higher education  programs that we looked at help students build their own education road map using standardized tests, digital alerts, and time-management tools to regularly reinforce students’ progress and remind them of their goals.

Brazil’s Cogna Educação, for instance, encourages students to assess their baseline knowledge at the start of the course. 3 Digital transformation: A new culture to shape our future , Kroton 2018 Sustainability Report, Kroton Educacional, cogna.com.br. Such up-front diagnostics could be helpful in highlighting knowledge gaps and pointing students to relevant tools and resources, and may be especially helpful to students who have had unequal educational opportunities. A web-based knowledge assessment allows Cogna students to confirm their mastery of certain parts of a course, which, according to our research, can potentially boost their confidence and allow them to move faster through the course material.

At the outset of a course, leaders in online higher education can help students clearly understand the format and content, how they will use what they learn, how much time and effort is required, and how prepared they are for its demands.

The University of Michigan’s online Atlas platform, for instance, gives students detailed information about courses and curricula, including profiles of past students, sample reports and evaluations, and grade distributions, so they can make informed decisions about their studies. 4 Atlas, Center for Academic Innovation, University of Michigan, umich.edu. Another provider, Pluralsight, shares movie-trailer-style overviews of its course content and offers trial options so students can get a sense of what to expect before making financial commitments.

Meanwhile, some of the online doctoral students we interviewed have access to an interactive timeline and graduation calculator for each course, which help students understand each of the milestones and requirements for completing their dissertations. Breaking up the education process into manageable tasks this way can potentially ease anxiety, according to our interviews with education experts.

2. Enable seamless connections

Students may struggle to learn if they aren’t able to connect to learning platforms. Online higher education pioneers provide a single sign-on through which students can interact with professors and classmates and gain access to critical support services. Traditional institutions considering a similar model should remember that because high-speed and reliable internet are not always available, courses and program content should be structured so they can be accessed even in low-bandwidth situations or downloaded for offline use.

The technology is just one element of creating seamless connections. Since remote students may face a range of distractions, online-course content could benefit them by being more engaging than in-person courses. Online higher education pioneers allow students to study at their own pace through a range of channels and media, anytime and anywhere—including during otherwise unproductive periods, such as while in the waiting room at the doctor’s office. Coursera, for example, invites students to log into a personalized home page where they can review the status of their coursework, complete unfinished lessons, and access recommended “next content to learn” units. Brazilian online university Ampli Pitagoras offers content optimized for mobile devices that allows students to listen to lessons, contact tutors for help, or do quizzes from wherever they happen to be.

Adopt an engaging approach to teaching

The pioneers in online higher education we researched pair the “right” course content with the “right” formats to capture students’ attention. They incorporate real-world applications into their lesson plans, use adaptive learning tools to personalize their courses, and offer easily accessible platforms for group learning.

3. Offer a range of learning formats

The online higher education programs we reviewed incorporate group activities and collaboration with classmates—important hallmarks of the higher education experience—into their mix of course formats, offering both live classes and self-guided, on-demand lessons.

The Georgia Institute of Technology, for example, augments live lessons from faculty members in its online graduate program in data analytics with a collaboration platform where students can interact outside of class, according to a student we interviewed. Instructors can provide immediate answers to students’ questions via the platform or endorse students’ responses to questions from their peers. Instructors at Zhejiang University in China use live videoconferencing and chat rooms to communicate with more than 300 participants, assign and collect homework assignments, and set goals. 5 Wu Zhaohui, “How a top Chinese university is responding to coronavirus,” World Economic Forum, March 16, 2020, weforum.org.

The element of personalization is another area in which online programs can consider upping their ante, even in large student groups. Institutions could offer customized ways of learning online, whether via digital textbook, podcast, or video, ensuring that these materials are high quality and that the cost of their production is spread among large student populations.

Some institutions have invested in bespoke tools to facilitate various learning modes. The University of Michigan’s Center for Academic Innovation embeds custom-designed software into its courses to enhance the experience for both students and professors. 6 “Our mission & principles,” University of Michigan Center for Academic Innovation, ai.umich.edu. The school’s ECoach platform helps students in large classes navigate content when one-on-one interaction with instructors is difficult because of the sheer number of students. It also sends students reminders, motivational tips, performance reviews, and exam-preparation materials. 7 University of Michigan, umich.edu. Meanwhile, Minerva University focuses on a real-time online-class model that supports higher student participation and feedback and has built a platform with a “talk time” feature that lets instructors balance class participation and engage “back-row students” who may be inclined to participate less. 8 Samad Twemlow-Carter, “Talk Time,” Minerva University, minervaproject.com.

4. Ensure captivating experiences

Delivering education on digital platforms opens the potential to turn curricula into engaging and interactive journeys, and online education leaders are investing in content whose quality is on a par with high-end entertainment. Strayer University, for example, has recruited Emmy Award–winning film producers and established an in-house production unit to create multimedia lessons. The university’s initial findings show that this investment is paying off in increased student engagement, with 85 percent of learners reporting that they watch lessons from beginning to end, and also shows a 10 percent reduction in the student dropout rate. 9 Increased student engagement and success through captivating content , Strayer Studios outcomes report, Strayer University, studios.strategiced.com.

Other educators are attracting students not only with high-production values but influential personalities. Outlier provides courses in the form of high-quality videos that feature charismatic Ivy League professors and are shot in a format that reduces eye strain. 10 Outlier online course registration for Calculus I, outlier.org. The course content follows a storyline, and each course is presented as a crucial piece in an overall learning journey.

5. Utilize adaptive learning tools

Online higher education pioneers deliver adaptive learning using AI and analytics to detect and address individual students’ needs and offer real-time feedback and support. They can also predict students’ requirements, based on individuals’ past searches and questions, and respond with relevant content. This should be conducted according to the applicable personal data privacy regulations of the country where the institution is operating.

Cogna Educação, for example, developed a system that delivers real-time, personalized tutoring to more than 500,000 online students, paired with exercises customized to address specific knowledge gaps. 11 Digital transformation , 2018. Minerva University used analytics to devise a highly personalized feedback model, which allows instructors to comment and provide feedback on students’ online learning assignments and provide access to test scores during one-on-one feedback sessions. 12 “Maybe we need to rethink our assumptions about ‘online’ learning,” Minerva University, minervaproject.com. According to our research, instructors can also access recorded lessons during one-on-one sessions and provide feedback on student participation during class.

6. Include real-world application of skills

The online higher education pioneers use virtual reality (VR) laboratories, simulations, and games for students to practice skills in real-world scenarios within controlled virtual environments. This type of hands-on instruction, our research shows, has traditionally been a challenge for online institutions.

Arizona State University, for example, has partnered with several companies to develop a biology degree that can be obtained completely online. The program leverages VR technology that gives online students in its biological-sciences program access to a state-of-the-art lab. Students can zoom in to molecules and repeat experiments as many times as needed—all from the comfort of wherever they happen to be. 13 “ASU online biology course is first to offer virtual-reality lab in Google partnership,” Arizona State University, August 23, 2018, news.asu.edu. Meanwhile, students at Universidad Peruana de Ciencias Aplicadas are using 3-D games to find innovative solutions to real-world problems—for instance, designing the post-COVID-19 campus experience. 14 Cleofé Vergara, “Learn by playing with Minecraft Education,” Innovación Educativa, July 13, 2021, innovacioneducativa.upc.edu.pe.

Some institutions have expanded the real-world experience by introducing online internships. Columbia University’s Virtual Internship Program, for example, was developed in partnership with employers across the United States and offers skills workshops and resources, as well as one-on-one career counseling. 15 Virtual Internship Program, Columbia University Center for Career Education, columbia.edu.

Create a caring network

Establishing interpersonal connections may be more difficult in online settings. Leading online education programs provide dedicated channels to help students with academic, personal, technological, administrative, and financial challenges and to provide a means for students to connect with each other for peer-to-peer support. Such programs are also using technologies to recognize signs of student distress and to extend just-in-time support.

7. Provide academic and nonacademic support

Online education pioneers combine automation and analytics with one-on-one personal interactions to give students the support they need.

Southern New Hampshire University (SNHU), for example, uses a system of alerts and communication nudges when its digital platform detects low student engagement. Meanwhile, AI-powered chatbots provide quick responses to common student requests and questions. 16 “SNHU turns student data into student success,” Southern New Hampshire University, May 2019, d2l.com. Strayer University has a virtual assistant named Irving that is accessible from every page of the university’s online campus website and offers 24/7 administrative support to students, from recommending courses to making personalized graduation projections. 17 “Meet Irving, the Strayer chatbot that saves students time,” Strayer University, October 31, 2019, strayer.edu.

Many of these pioneer institutions augment that digital assistance with human support. SNHU, for example, matches students in distress with personal coaches and tutors who can follow the students’ progress and provide regular check-ins. In this way, they can help students navigate the program and help cultivate a sense of belonging. 18 Academic advising, Southern New Hampshire University, 2021, snhu.edu. Similarly, Arizona State University pairs students with “success coaches” who give personalized guidance and counseling. 19 “Accessing your success coach,” Arizona State University, asu.edu.

8. Foster a strong community

The majority of students we interviewed have a strong sense of belonging to their academic community. Building a strong network of peers and professors, however, may be challenging in online settings.

To alleviate this challenge, leading online programs often combine virtual social events with optional in-person gatherings. Minerva University, for example, hosts exclusive online events that promote school rituals and traditions for online students, and encourages online students to visit its various locations for in-person gatherings where they can meet members of its diverse, dispersed student population. 20 “Join your extended family,” Minerva University, minerva.edu. SNHU’s Connect social gateway gives online-activity access to more than 15,000 members, and helps them interact within an exclusive university social network. Students can also join student organizations and affinity clubs virtually. 21 SNHU Connect, Southern New Hampshire University, snhuconnect.com.

Getting started: Designing the online journey

Building a distinctive online student experience requires significant time, effort, and investment. Most institutions whose practices we reviewed in this article took several years to understand student needs and refine their approaches to online education.

For those institutions in the early stages of rethinking their online offerings, the following three steps may be useful. Each will typically involve various functions within the institution, including but not necessarily limited to, academic management, IT, and marketing.

The diagnosis could be performed through a combination of focus groups and quantitative surveys, for example. It’s important that participants represent various student segments, which are likely to have different expectations, including young-adult full-time undergraduate students, working-adult part-time undergraduate students, and graduate students. The eight key dimensions outlined above may be helpful for structuring groups and surveys, in addition to self-evaluation of institution performance and potential benchmarks.

• Set a strategic vision for your online learning experience. The vision should be student-centric and link tightly to the institution’s overarching manifesto. The function leaders could evaluate the costs/benefits of each part of the online experience to ensure that the costs are realistic. The online model may vary depending on each school’s market, target audience, and tuition price point. An institution with high tuition, for example, is more likely to afford and provide one-on-one live coaching and student support, while an institution with lower tuition may need to rely more on automated tools and asynchronous interactions with students.
• Design the transformation journey. Institutions should expect a multiyear journey. Some may opt to outsource the program design and delivery to dedicated program-management companies. But in our experience, an increasing number of institutions are developing these capabilities internally, especially as online learning moves further into the mainstream and becomes a source of long-term strategic advantage.

We have found that leading organizations often begin with quick wins that significantly raise student experiences, such as stronger student support, integrated technology platforms, and structured course road maps. In parallel, they begin the incremental redesign of courses and delivery models, often focusing on key programs with the largest enrollments and tapping into advanced analytics for insights to refine these experiences.

Finally, institutions tackle key enabling factors, such as instructor onboarding and online-teaching training, robust technology infrastructure, and advanced-analytics programs that enable the institutions to understand which features of online education are performing well and generating exceptional learning experiences for their students.

The question is no longer whether the move to online will outlive the COVID-19 lockdowns but when online learning will become the dominant means for delivering higher education. As digital transformation accelerates across all industries, higher education institutions will need to consider how to develop their own online strategies.

Felipe Child is a partner in McKinsey’s Bogotá office, Marcus Frank is a senior practice expert in the São Paulo office, Mariana Lef is an associate in the Buenos Aires office, and Jimmy Sarakatsannis is a partner in the Washington, DC, office.

References to specific products, companies, or organizations are solely for information purposes and do not constitute any endorsement or recommendation.

This article was edited by Justine Jablonska, an editor in the New York office.

Explore a career with us

Related articles.

How to transform higher-education institutions for the long term

Higher education in the post-COVID world

Reimagining higher education in the United States

Data Privacy

This $90m education research project is banking on data privacy to drive insights, by nadia tamez-robledo     may 17, 2024.

Alphavector / Shutterstock

With digital education platforms generating data on how millions of students are learning, they are also sitting on veritable information gold mines for researchers who are trying to improve education.

An ethical and legal conundrum stands in the way: how to responsibly share that data without opening students up to the possibility of having their personal information exposed to outside parties .

Now a consortium of education researchers and learning platforms are developing what they hope is a solution — researchers will never see the actual data.

The project dubbed SafeInsights, helmed by OpenStax at Rice University, is supported by a $90 million grant from the National Science Foundation over five years.

The idea is for SafeInsights to serve as a bridge between its learning platform and research partners, alongside collaborators helping flesh out how the exchange will work to safeguard student privacy.

“In a normal situation, you end up taking data from learning websites and apps and giving it to researchers for them to study and for them to analyze it to learn from,” JP Slavinsky, SafeInsights executive director and OpenStax technical director, says. “Instead, we're taking the researchers’ questions to that data. This creates a safer environment for research that's easier for schools and platforms to participate in, because the data is staying where it is already.”

Deeper Insights on a Large Scale

Another way to think of SafeInsights is as a telescope, say Slavinsky and his colleague Richard Baraniuk, the founder and director of OpenStax, which publishes open access course materials . It will allow researchers to peer into the vast amount of data from learning platforms like the University of Pennsylvania’s Massive Online Open Courses and Quill.org.

Researchers would develop questions — then transform those questions into computer code that can sift through the data — to be delivered to learning platforms. After the results are generated, they would be returned to researchers without the data ever having to be directly shared.

“It is really a partnership where we have researchers coming together with schools and platforms, and we're jointly trying to solve some problems of interest,” Slavinsky says. “We are providing that telescope for others to bring their research agenda and the questions they want to answer. So we're less involved on what specifically is going to be asked and more on making as many questions as possible answerable.”

Part of why this model would be so powerful is how it would increase the scale at which education research is done, Baraniuk says. There are plenty of studies that have small sample sizes of about 50 college students, he explains, who participate as part of a psychology class.

“A lot of the studies are about freshman college kids, right? Well, that's not representative of the huge breadth of different students,” Baraniuk says. “The only way you're gonna be able to see that breadth is by doing large studies, so really the first key behind SafeInsights is partnering with these digital education websites and apps who host literally millions of students every day.”

Another aspect where he sees the project opening new doors for researchers is the diversity of the student populations represented by the learning platform partners, which include education apps for reading, writing and science along with learning management systems.

“By putting together all of these puzzle pieces, the idea is that we can — at a very large scale — get to see a more complete picture of these students,” Baraniuk says. “The big goal of ours is to try to remove as much friction as possible so that more useful research can happen, and then more research-backed pedagogies and teaching techniques can actually get applied. But while removing that friction, how do we keep everything really safeguarded?”

Creating Trust, Protecting Privacy

Before any research takes place, SafeInsights partners at the Future of Privacy Forum are helping develop the policies that will shape how the program guards students’ data.

John Verdi, the Future of Privacy Forum’s senior vice president for policy, says the goal is to have privacy protections baked into how everything operates. Part of that is helping to develop what he calls the “data enclave,” or the process by which researchers can query a learning platform’s data without having direct access. Other aspects include helping develop the review process for how research projects are selected, training researchers on privacy and publishing lessons learned about operating with privacy at the forefront.

“Even if you have great technical safeguards in place, even if you do great ethical vetting,” he says about the training aspect, “at the end of the day, researchers themselves have decisions to make about how to responsibly use the system. They need to understand how the system works.”

The protection of student data privacy in education is generally “woefully under-funded,” he says, but it’s safeguarding that information that allows students to trust learning platforms — and ultimately create research opportunities like SafeInsights.

“Tasking students and parents to protect data is the wrong place to put that responsibility,” Verdi says. “Instead, what we need to do is build digital infrastructure that is privacy respectful by default, and [that] provides assurances that information will be kept confidential and used ethically.”

Nadia Tamez-Robledo ( @nadiatamezr ) is a reporter covering K-12 education for EdSurge with focuses on student and teacher mental health and changing demographics. You can reach her at nadia [at] edsurge [dot] com.

More from EdSurge

Teacher Well-Being Depends on Workload, School Climate and Feeling Supported

By nadia tamez-robledo.

Artificial Intelligence

Los angeles school district launched a splashy ai chatbot. what exactly does it do, by jeffrey r. young.

EdSurge Podcast

‘college for what' high school students want answers before heading to campus.

Scholar Hopes to Diversify the Narrative Around Undocumented Students

Journalism that ignites your curiosity about education.

EdSurge is an editorially independent project of and

• Product Index
• Write for us
• Advertising

FOLLOW EDSURGE

© 2024 All Rights Reserved

• Copy/Paste Link Link Copied

Understanding How Digital Media Affects Child Development

Technology and digital media have become ubiquitous parts of our daily lives. Screen time among children and adolescents was high before COVID-19 emerged, and it has further risen during the pandemic, thanks in part to the lack of in-person interactions.  

In this increasingly digital world, we must strive to better understand how technology and media affect development, health outcomes, and interpersonal relationships. In fact, the fiscal year 2023 federal budget sets aside no less than $15 million within NICHD’s appropriation to investigate the effects of technology use and media consumption on infant, child, and adolescent development.

Parents may not closely oversee their children’s media use, especially as children gain independence. However, many scientific studies of child and adolescent media use have relied on parents’ recollections of how much time the children spent in front of a screen. By using software embedded within mobile devices to calculate children’s actual use, NICHD-supported researchers found that parent reports were inaccurate more often than they were on target. A little more than one-third of parents in the study underestimated their children’s usage, and nearly the same proportion overestimated it. With a recent grant award from NICHD, researchers at Baylor College of Medicine plan to overcome the limitation of relying on parental reports by using a novel technology to objectively monitor preschool-age children’s digital media use. They ultimately aim to identify the short- and long-term influences of technology and digital media use on children’s executive functioning, sleep patterns, and weight. This is one of three multi-project program grants awarded in response to NICHD’s recent funding opportunity announcement inviting proposals to examine how digital media exposure and use impact developmental trajectories and health outcomes in early childhood or adolescence. Another grant supports research to characterize the context, content, and use of digital media among children ages 1 to 8 years and to examine associations with the development of emotional regulation and social competence. A third research program seeks to better characterize the complex relationships between social media content, behaviors, brain activity, health, and well-being during adolescence.

I look forward to the findings from these ongoing projects and other studies that promise to inform guidance for technology and media use among children and adolescents. Additionally, the set-aside funding for the current fiscal year will allow us to further expand research in this area. These efforts will help us advance toward our aspirational goal to discover how technology exposure and media use affect developmental trajectories, health outcomes, and parent-child interactions.