research papers on augmented reality

• Open access
• Published: 04 February 2019

Perspectives on how to evaluate augmented reality technology tools for education: a systematic review

• Manoela M. O. da Silva   ORCID: orcid.org/0000-0001-5401-6540 1 ,
• João Marcelo X. N. Teixeira 1 , 2 ,
• Patrícia S. Cavalcante 3 &
• Veronica Teichrieb 1  

Journal of the Brazilian Computer Society volume  25 , Article number:  3 ( 2019 ) Cite this article

18k Accesses

40 Citations

Metrics details

Education has benefited from augmented reality’s (AR) potential to promote interactive experiences both inside and outside the classroom. A systematic review was conducted on how AR’s impact in the learning process has been evaluated. We selected papers from 2009 to 2017 in three databases, IEEE, ACM, and Science Direct, using an open-source crawler, and in one Brazilian Conference, SBIE. We followed the PRISMA protocol. Forty-five works were selected and used to extract data for our research. They were also analyzed according to quantitative and qualitative criteria. The results from all the papers are available in an online database. Results evidenced an increase in the number of papers evaluating the AR’s impact in education. They also showed that AR has been applied in different areas and contexts. Most papers reported positive outcomes as a result of AR insertion. However, most studies lacked the involvement of the teacher and the use of multiple metrics to evaluate educational gains.

Introduction

Augmented reality (AR) is a technology that consists of adding virtual elements to a real scene coherently so that ideal users cannot differentiate them from the real scene [ 3 ]. Although all fields of knowledge can potentially take advantage from AR, Tori et al. [ 72 ] argue that education will be particularly modified by its introduction. The coexistence of virtual and real environments allows learners to experience phenomena that otherwise would be impossible in the real world. This allows learners to visualize complex spatial relationships and abstract concepts and, therefore, develop important abilities that cannot be evolved in other technology learning environments [ 78 ].

It has been long since AR’s potential in education has been investigated. According to Kostaras et al. [ 41 ], AR can aid learning and make the overall process more interesting and pleasant. In a rapidly changing society as ours where there is a great amount of information available, it is of major importance to know how to locate information and use it efficiently. AR dramatically shifts the location and timing of education and training [ 46 ].

Billinghurst and Duenser [ 5 ] explain that unlike other computer interfaces that draw users away from the real world and onto the screen, AR interfaces enhance the real world experience as shown in Fig.  1 , which presents an AR application designed to create new museum experiences [ 2 ]. Billinghurst and Duenser [ 5 ] also highlight some reasons why AR educational experiences are different: (a) support of seamless interaction between real and virtual environments, (b) use of a tangible interface metaphor for object manipulation, and (c) ability to transition smoothly between reality and virtuality.

AR application developed to enhance museum experience

Although AR has been studied for over 40 years, only in the last decade it began to be formally evaluated [ 23 , 24 , 68 ]. One of the reasons why it took so long to have user evaluations may be a lack of knowledge on how to properly evaluate AR experiences and design experiments [ 24 ]. Dünser et al. [ 24 ] claim that there seems to be a lack of understanding regarding the need of doing studies and the right motivation for carrying them. If user evaluations are conducted out of incorrect motivation or if empirical methods are not properly applied, the findings are of limited value or can even be misleading.

Until that time, the amount of AR systems formally evaluated was rather small [ 23 ]. Swan and Gabbard [ 68 ] and Dünser et al. [ 24 ] have found that only around 8% of published AR research papers included formal evaluations. According to Dünser and Billinghurst [ 22 ], one reason for this small percentage may be the lack of suitable methods for evaluating AR interfaces. Researchers in non-conventional interface fields such as virtual reality (VR) or AR cannot rely solely on design guidelines for traditional user interfaces since new interfaces afford new forms of interactions [ 22 ]. Since then, more works address some form of user evaluation [ 6 ].

When dealing with educational AR systems, it is also important to evaluate the impact of learning applications and the feasibility of incorporating them into the classrooms. Many factors are involved in this process varying from cost to staff’s acceptance. Evaluation of technology is an important step in design instruction, which is the process by which learning products and experiences are designed, developed, and delivered Footnote 1 . Also, it is necessary to evaluate it properly so practitioners are more confident in its positive effects. It is also relevant to consider the points of view of both teachers and learners since they might differ.

In the last decade, a few papers have been published evaluating educational aspects of AR applications used for education. For instance, Balog and Pribeanu [ 4 ] have shown the same aspect can be valued differently by both teachers and learners. One survey reviewed applications intended to complement traditional curriculum materials for K–12 [ 65 ]. It performed a qualitative analysis on the design aspects and evaluation for AR Learning Environments (ARLES). Its focus was to investigate ARLES designed for kindergarten and primary and/or secondary school, as well as to explore learning theories as basis for effective learning experiences. They found out that there are three inherent AR affordances to educational settings: real-world annotation, contextual visualization, and vision-haptic visualization [ 65 ]. These affordances were supported by existing theories. Authors discovered that aside from the performance of students in pre- and post-tests, other aspects of the learning experience such as motivation and satisfaction were usually observed.

However, it can be noted that the aforementioned paper focuses only on K–12 education. Our paper will focus on different target groups of the AR applications evaluated.

As this research area matures and the use of AR in education grows, it is important to analyze its impact appropriately to have relevant and valid feedback for the stakeholders involved in the process. Thus, this paper presents a systematic review on how studies have been evaluating AR in education.

The contributions of this paper are:

The use of a robust research methodology to collect and analyze papers that perform educational evaluation of AR educational applications (“ Methodology ” section)

A classification and discussion of studies that evaluate educational aspects of such AR systems (“ Results and discussion ” section)

Guidelines to evaluate educational aspects of AR applications (“ Guidelines for educational evaluation ” section)

Methodology

Considering the complexity of the educational field, such as different learning needs and times, to name a few, and its implications for technology acceptance and use, a systematic review was conducted to investigate how researchers are evaluating their AR systems. This review followed the PRISMA protocol [ 57 ] as shown in Fig.  2 .

PRISMA protocol diagram

Research questions

Our main question was “how do researchers evaluate AR-based educational technology?”. To guide data extraction, analysis, and synthesis, sub-questions were formulated as listed below. The questions are divided into three categories: descriptive, classificatory, and relation and effect.

Descriptive questions :

What is the evolution in number and type of research from 2009 to 2017?

What institutions are most involved in performing this type of research?

Classificatory :

What are the different designs (methodologies) used in these studies?

What are the target populations used in these studies?

What are the constructs being analyzed?

What are the domains of the different applications tested?

What types of research questions are investigated?

What are the types of AR technology used?

What is the problem being analyzed?

Is the application based on any educational theory?

What technologies AR is combined with?

How was the involvement of teachers in the evaluation process?

Did the study use multiple metrics (both quantitative and qualitative)?

Did the study use multiple metrics for educational evaluation purposes?

Relation and effect :

What is the kind of impact of the tool analyzed?

Systematic review procedure

The first step was to establish the search string for paper selection. The search string was created based on our research questions. The terms were defined along with synonyms found in the literature as shown in Table  1 .

Then, the databases for the search were defined. Papers were searched automatically in three databases: ACM, IEEE Xplore, and Science Direct. Also, papers were searched in the main Brazilian Conference related to Informatics in Education, the Brazilian Symposium on Informatics in Education, SBIE. This search was performed manually in the Google Scholar platform using our search string.

The automatic search was performed in the databases using the same open-source paper crawler software that was used by Roberto et al. [ 63 ]. This crawler enabled authors to automate the process of retrieving papers. It uses only the search string as input, and it accesses the digital libraries to search in the title, abstract, and keywords of each paper. The crawler collects the papers, eliminates duplicate versions, and creates a spreadsheet containing all the works with their title, year, source, primary affiliation, abstract, and web address.

For papers to be included in the study, they must meet the following criteria:

Papers published in English with more than four pages

Papers were only considered once (in case of repetitive papers, we considered the more complete or the most recent one)

Papers published from 2009 to 2017

Papers that explicitly mentioned their evaluation methodology

The papers must have at least an AR prototype working

The AR solution must be tested with its end users

The solutions presented must be applied to learning a new concept or skill

Papers that intended to evaluate learning aspects

First, a search was performed in the databases using the search strings. Then, in the pre-selection phase, the researchers screened the papers by reading their title, abstract, and conclusion to eliminate the ones clearly not related to the research question. Later, we applied the inclusion criteria to those papers. These papers were screened to evaluate their quality concerning quantitative and qualitative aspects. In the extraction phase, we read the papers to extract relevant data concerning the research questions.

Data extraction

We extracted relevant information from the selected papers as listed below. The data was organized in a spreadsheet.

University/research group

Source (conference or journal)

Methodology design

Target population

Application domain

Type of research question

Implications for practice

Type of AR technology (tracking, display, interaction)

What constructs does it evaluate?

Is the application based on educational research?

What are the implications of the findings in research and practice?

What is the impact of the tool analyzed (positive or negative)?

Observations

Quality criteria evaluation

The QualSyst standard was used as a guideline for quality control [ 40 ]. This questionnaire consisted of 14 items evaluating study questions concerning design methodology, sample, outcomes, results outcomes, description, and conclusions. Some items were not scored due to their non-applicability in the study’s methodology (e.g., evaluator and user blinding); in these cases, we used n/a (not applicable) in the table. Other items such as interventional and random allocation were applied only in some cases. Each item was graded as it fulfilled the requirements in three categories: total, partial, and none with assigned scores of 2, 1, or 0, respectively. The total sum was divided by the maximum possible points (e.g., 10 items × 2 points = 20 points). The final score of each paper formed a grade. In case the paper did not conduct one type of research, qualitative or quantitative, a dash was used (-) to represent this situation in the spreadsheet.

Threats to valitidy

Authors are aware of the importance of considering threats to validity in order to judge the systematic review strengths and limitations. The main issues in this type of research are related to incomplete sets of relevant papers and researcher bias regarding quality analysis.

Limitations with search string, scientific databases, and search strategy can result in an incomplete set of relevant papers. As a way to mitigate the risks, the following strategies were used: first, in order to validate the search string, the terms were discussed among the authors. The authors have a different set of skills, two of them hold Ph.D. degrees in the field of Computer Science, one has a Ph.D. degree in Education, and one has a B.A. in Languages and is currently a Ph.D. candidate in Computer Science. All of them are teachers with experience in different educational levels, from early childhood education to post-graduate education. Second, the scientific databases that publish works from the most important conferences and journals in the area were selected, along with the papers published in the main Brazilian conference in the area. Third, the crawler uses a different approach to maximize the number of papers found. Instead of using the complete search string, eight different searches were performed using the combination of every term in both parts of the search string, which increases the number of papers collected [ 63 ].

The qualitative analysis of the papers was conducted by one of the authors. Since this may lead to a researcher bias, 15% of the papers were randomly selected to compose a set of control papers in order to increase credibility. The other authors examined the control papers to analyze them concerning their quality. The authors compared their results using Cohen’s Kappa coefficient, which measures the agreement between the two classifications taking into account how much agreement would be expected to be present by chance [ 11 ]. The coefficient lies between −1.0 and 1.0 in which 1.0 denotes perfect agreement, 0.0 indicates that any agreement is due to chance, and negative values present agreement less than chance. There is no consensus on what are good levels of agreement. Nevertheless, studies [ 1 ] mention that there is no agreement for negative values, poor agreement between 0.00 and 0.20, fair agreement between 0.21 and 0.40, moderate agreement between 0.41 and 0.60, good agreement between 0.61 and 0.80, and very good agreement for values higher than 0.80. In our work, the qualitative analysis Cohen’s Kappa was 0.7969, which is close to very good agreement among the authors.

Results and discussion

This section describes and discusses the results of the systematic review.

The search in the databases using the search strings returned 607 articles, and 148 papers remained after the pre-selection phase. Finally, after applying the inclusion criteria, 45 papers were eligible for this study. The results from all the papers are available in an online database, which can be collaboratively updated Footnote 2 .

Quality of report

The quantitative and qualitative assessments are available at Appendixes A.1 and A.2 , respectively.

Descriptive questions

Questions 1 and 2 are in this category. Figure  3 shows that although no research was found in 2009, the research in this field is steadily growing, reaching the highest number of papers in 2014. Although the number of papers per year has decreased compared to 2014, we observe that the interest in evaluating AR for education remains.

Papers according to the year of publication

Table  2 presents the institutions involved in the research.

Table  3 shows the venues where the studies have been published.

Figure  4 evidences that the methodology most commonly used is the experimental design, while the quasi-experimental design appeared in fourth place. The essential feature of experimental research is that the researcher deliberately controls and manipulates the conditions, which determine the events of interest [ 12 ]. Quasi-experiments are used when subjects must be allowed to choose their treatment, which is the main difference when compared to experimental designs.

Papers according to the design methodology

Questionnaires were the second most popular method among the studies. They consist in a series of questions or prompts aimed at gathering information from subjects. The questionnaires used in the papers were designed in different ways and for varied purposes. As examples, Zhang et al. [ 82 ] used a questionnaire to investigate flow experience and Wei et al. [ 77 ] to assess creative design learning motivation. In turn, Ibánez et al. [ 36 ] designed an open-ended questionnaire. Regardless of its structure and aim, Cohen et al. [ 12 ] point out that an ideal questionnaire must be clear and unambiguous.

Observations appeared in seven studies while only one work reported a case study. Merriam [ 56 ] explains that observations take place where a given phenomenon naturally occurs. She points out that the skills to be a good observer must be learned; thus, training and mental preparation is important. She highlights the need to define what to observe as well as to write careful and useful field notes.

Case study, on the other hand, is an empirical inquiry that investigates a contemporary phenomenon within its real-life context, especially when the boundaries between phenomenon and context are not clearly evident [ 81 ]. Merriam [ 56 ] points out that the most defining characteristic of a case study lies in delimiting the case to be studied. Thus, case study research uses purposive sampling rather than random sampling [ 25 ].

It is important to highlight that a high number of papers (32, total) reported a combination of methods or metrics. The most common combination is the experiment coupled with questionnaires. However, these multiple metrics usually not only evaluated education, but also other aspects such as motivation and satisfaction. The results for this question also evidenced a predominance of quantitative methods in the works.

Question 4 refers to the target population of the studies as seen in Fig.  5 .

Papers according to the target population

Figure  5 shows that the most popular target audience are undergraduate students and elementary school children. High school students appeared in seven works, thus being the third most popular audience for AR tools.

Other groups were also considered in these papers. For instance, two papers targeted general audiences of users. For instance, in Sommerauer and Müller [ 67 ], the population was an exhibition audience, which included heterogeneous genders, age groups, and educational levels. These varied target populations show that AR can expand the barriers of the school setting and achieve both formal and informal environments.

Also, parents were the audience in Cheng and Tsai [ 10 ]. This paper also targets children in both elementary and preschool. Tobar-Muñoz et al. [ 71 ] present an AR tool for children with varied ages and special needs. Finally, four papers are targeted to workers in different fields, such as engineering [ 8 ] and surgery [ 45 ]. This data evidence that AR can also be successfully used for training.

Hence, data show that although there has been a preference for undergraduate students and elementary school children, AR can be used by a variety of people, with different needs and in different contexts.

Question 5 was about the constructs evaluated in the studies as displayed in Fig.  6 .

Papers according to the constructs evaluated

Figure  6 reveals that many studies did not evaluate solely educational aspects. Twelve works evaluated more than one aspect. The majority of the papers evaluated knowledge retention or performance.

Some applications were under development or had been recently developed; thus, usability aspects, such as users’ attitudes and satisfaction, were also analyzed. Martín-Gutiérrez et al. [ 52 ] point out that the study was carried out with the beta version of the tool, which was tested with 235 students. These authors, thus, also evaluated user’s satisfaction. In turn, Tarng et al. [ 70 ] investigated the attitudes of experimental group students after using the AR system. The authors explain that the questions in their study were categorized in learning contents, interface design, and applications.

Behavior and motivation were also evaluated in eight studies. Other studies evaluated constructs related to the theories they used, such as flow experience [ 8 , 36 ] and dimensions of learning style [ 49 ].

Other aspects evaluated were creativity [ 77 ], teaching effects [ 77 ], and learner’s opinions [ 69 ]. This variety evidences that due to the complexity of the learning environment, different aspects can be the focus of educational or learning evaluation. Depending on the focus of the studies, such as training, authors would focus on more mechanical aspects such as precise skills development and time. Conversely, studies focusing on the school environment may focus their attention on the role of the teacher, flow experience, or student’s motivation to learn.

Question 6 concerns the application domains of knowledge as shown in Fig.  7 .

Papers according to their domains of knowledge

Figure  7 shows that most AR tools are related to STEM fields. STEM is an acronym that refers to the fields of science, technology, engineering, and mathematics. The second most popular domain for applications are humanities, followed by medicine and health.

Question 7 investigated the types of research questions in the works. The questions were classified according to their types as proposed by Easterbrook et al. [ 25 ]. These authors divide research questions in two types: design questions, which are usually asked by software engineers in order to better ways to do software engineering, and knowledge questions, which are described below:

Exploratory questions: are asked in the early stages of research when researchers are attempting to understand the phenomena, e.g., existence questions, description and classification, descriptive comparative

Base-rate questions: are frequently asked after having a clearer understanding of the phenomena. They might be frequency and distribution questions and descriptive process

Relationship questions: are meant to understand the relationship between two different phenomena

Causality questions: are an attempt to explain why a relationship holds and identify its cause and effect, e.g., causality questions, causality-comparative questions and causality-comparative-interaction questions.

Figure  8 presents the types of research questions found in the papers.

Papers according to their research questions

Twenty-three papers asked more than one question. The chart shows that the majority of the papers asked relationship questions; those papers aimed to describe the effect of AR compared to other resources and its relationship with different aspects (e.g., academic achievement or motivation). The second most common type of question was exploratory ones, mainly descriptive comparative (present in 19 papers). Design questions were asked by two studies and causality ones by one study. This amount of exploratory questions may indicate that research in the use AR tools for education might still be in early stages, in which researchers attempt to better understand the field and the implications of such technology in education. Also, they want to understand what are the better ways to develop their tools, as evidenced in the design questions.

AR technologies used in the studies were classified according to their tracking, display, and interaction techniques. As concerns the displays used, Fig.  9 shows that screen-based and handheld were the most frequent used displays (21 and 14 papers, respectively). Screenbased displays are known for their cost-efficiency since they require off-the-shelf hardware and standard PC equipment. They are also largely present in schools nowadays and were usually well evaluated by users.

Papers according to the display used

On the other hand, the popularization and technical advancements in smartphones make handheld displays a good option for AR applications. These devices are minimally intrusive and highly mobile [ 83 ]. They enable high flexibility, as shown in Jerry and Aaron [ 38 ], in which a context-aware AR learning solution is proposed as a scaffolding platform for outdoor field learning. Tarng et al. [ 70 ] used these displays to provide situated learning.

Two papers used head-attached displays. Although these displays provide a better field of view, fashion constraints are a common issue. For instance, Martín-Gutiérrez et al. [ 52 ] reported that the HMD use was not comfortable. The cables linking the glass and camera with the PC interfered with user’s movement.

Five studies presented spatial displays. Three studies did not provide enough information about the system evaluated; therefore, it was not possible to classify these systems in all three categories of AR [ 27 , 28 , 38 ].

As regards to tracking, 35 papers presented vision-based tracking and five papers presented sensor-based tracking as shown in Fig.  10 . Two papers presented hybrid tracking.

Papers according to the tracking technique

Vision-based tracking can be divided in two categories, marker-based and markerless as illustrated in Fig.  10 .

Marker-based was the most common type found (25 works). It is a very popular choice since there are many marker-based kits available for a low cost. Most papers presented positive outcomes regarding these tools. However, markers can be intrusive in the scene.

On the other hand, markerless systems do not require the use of markers. In this case, the environment itself acts as a marker. It allows guidance information to be superimposed on a real game board, for example. This type was chosen in ten studies.

Finally, as regards to interaction techniques, 19 papers presented a more traditional type of interaction using buttons, touch, or simply providing visualization of the augmented content. Their use was generally positive.

The second most common choice was tangible interaction (13 papers). These interfaces are promising as they take advantage of the familiarity of everyday objects to ease the interaction. Their use provided positive results.

Haptic interfaces were chosen in four papers. One paper presented collaborative interaction [ 48 ]. No papers chose hybrid interfaces.

Figure  11 shows the interaction techniques used in the papers.

Papers according to the interaction techniques

Question 11 investigated if the studies were based on any educational theory as presented in Table  4 .

Table  4 evidences that most papers mentioned educational theories. However, 19 studies did not mention any theory. The most mentioned theories were situated learning theory and the cognitive theory of multimedia learning and cognitive load theory, mentioned in three works each. The situated learning theory emphasizes the reality of learning activities; thus, the context in which the activity naturally occurs is indispensable. AR allows real-life experiences to be enhanced with virtual content, hence expanding learning horizons.

In turn, the cognitive theory of multimedia learning (CTML) states that people learn better from pictures and words rather than pictures alone. This theory is based on three assumptions: (a) people possess two channels for processing information (the auditory/verbal and visual/pictorial), (b) there is a limited amount of information each channel can process at a time, and (c) learning is an active process of selecting relevant information, organizing them into coherent mental representations, and finally integrating those representations with existing knowledge [ 54 ].

Inquiry-based learning was mentioned in two studies. As an example, Jerry and Aaron [ 38 ] mentioned this theory, which is an approach to teaching and learning that places students’ questions, ideas, and observations at the center of the learning experience [ 60 ]. Hutchings [ 34 ] adds that the process of inquiry is in the ownership of the learners; thus, inquiry-based learning is fundamentally concerned with establishing the context within which inquiry may best be stimulated and students can take charge of their learning.

Mobile learning was mentioned in two works. Mobile learning or, simply, m-learning is the didactic-pedagogical expression used to designate a new educational “paradigm” based on the use of mobile technologies [ 58 ]. Also, McGreal [ 55 ] adds that “m-learning happens in context in which it is needed and relevant and is situated within the active cognitive processes of individual and groups of learners.” Thus, it takes advantage of the widely available mobile devices to provide access to learning anywhere and anytime, which changes many paradigms of traditional education.

The learning styles theory was also found in two papers. For instance, Zhang et al. [ 82 ] was based, specifically, on the kinesthetic learning style theory. Learning styles are the general approaches used by students in learning a new subject [ 61 ]. These “overall patterns” that generally direct learning behavior are divided in dimensions, for example, the sensory preference [ 14 ]. Sensory preferences can be divided into four main areas: visual, auditory, kinesthetic (movement-oriented)—explored in Zhang et al. [ 82 ], and tactile (touch-oriented) [ 61 ].

Studio-based learning theory was found only in one study [ 77 ]. It is a learning model first developed as part of education and training and later adopted by architectural education in the 1800s [ 43 ]. This model has its roots on the notion of the apprentice in the atelier where they worked and learned skills of the master design or artist. Young apprentices did not learn in isolated schools, but were exposed to real adult world and worked on real products in the community.

Other theories were represented by one paper each. Another one was the flow theory that brings the concept of flow which is a state of complete absorption or engagement in an activity that acts as a motivating factor in daily activities such as work, sport, and education [ 16 ]. This state encourages a person to persist at an activity due to experience rewards it promises, and it fosters the growth of skills over time [ 59 ].

Most of these theories have in common a learner-centered approach, thus focusing more on student’s discovery, construction and interaction process, and the attachment to the context of learning. In this sense, AR, along with other types of technology, can expand the learning horizons. Some theories focus on understanding learning processes to provide a more effective experience for the students considering their personal needs and abilities. As shown, the trend is to look at AR instructional design from the learners’ perspective.

The following question was: “what technologies AR is combined with?”. This question inquired if AR applications were combined with other technologies and what kinds of technologies they were combined with. As can be seen in Table  5 , 37 papers did not combine AR with other types of technology. The other papers combined it with different types of technology, such as YouTube tutorial, personal blogs, digital sketching, notes and texts provided by the teacher, robotics, mobile pedestrian navigation, virtual reality and digital sketching using hybrid models (DS/HM), and web-based simulation environment. All these technologies appeared one time each. Although it is evident a preference to not combine AR with other types of technology, it is interesting to note that in the classroom environment, AR is another possibility among many others already present in that environment. It is helpful, thus, to understand how these multiple possibilities can work together to scaffold learning.

Question 13 refers to the involvement of teachers in the evaluation process as shown in Fig.  12 .

Papers according to the involvement of teachers

Most studies did not involved the teachers in the studies. Some of the studies were in different contexts, such as library instruction by Wang et al. [ 74 ]; thus, in this case, authors mentioned the role of the librarian.

Nevertheless, 13 studies reported the involvement of teachers in different ways and levels. Figure  12 evidences that the teacher may be involved in the design and evaluation process of AR educational tools in different ways. The most common way was the teacher(s), or in some cases, schools directors, working as consultants or curators. Teachers were consulted for different purposes, such as problematic contents to teach [ 82 ] or to review or modify tests [ 36 , 67 , 82 ].

Seven studies involved the teachers as evaluators of student outputs. As an example, [ 44 ] explains that “AR will be used for self-assessment and that the teacher can mark the answers and give the scores on internet web page.”

Another role was to act as a tutor (six mentions). That means the teachers had a role of explaining content to students or monitor their work. For instance, [ 73 ] mentions that “the procedure of experiment is started with teacher lectures to all students in class.”

Also, five papers reported the participation of the teachers as creators of learning experiences. Cubillo et al. [ 17 ] reports that the teachers can follow an established procedure to create content using the tool. In da Silva et al. [ 19 ] and da Silva et al. [ 20 ], teachers were not able to create the applications by themselves since the AR tool evaluated needs an authoring tool, but they were able to design the activities to be worked and programmers created the content accordingly.

Finally, in Frank and Kapila [ 30 ], the teacher was considered a confounding, as illustrated in these lines: “teacher’s feedback was prevented in the design of the experiment by having student participants tested individually, being directed to perform the activity immediately after the pre-assessment and then immediately to complete the post-assessment.”

The results for questions 14 and 15 can be seen in Fig.  13 . Q14 refers to the use of multiple metrics. We can see that 24 studies used both quantitative and qualitative metrics and that 21 did not adopt this practice. However, most papers did not use both metrics to evaluate learning gains.

Papers according to the use of multiple metrics

Papers, such as Zhang et al. [ 82 ] and Wei et al. [ 77 ] used both types of metrics to evaluate learning gains. Zhang et al. [ 82 ] investigated the application of location-based AR to astronomical observation instruction. It used both quantitative and qualitative data to investigate aspects related to learning. To gather qualitative data, the authors performed an interview with teachers to understand the limitations of traditional teaching methods as a reference for the system’s design proposed. The quantitative data assessed learning effectiveness and motivation.

On the other hand, Wei et al. [ 77 ] showed a general technical creative design teaching scheme that includes AR. It used questionnaires to assess creative design learning, motivation, and teaching efficiency. There were also tests on creative design learning motivation, teaching effects, and creativity of the output.

This is an interesting aspect since the educational aspects are very complex and only quantitative metrics are not enough to understand the nuances involved in the process.

Relation and effect questions

Question 15 was in this category. This question explores the kind of impact of the tools analyzed in the studies. As shown in Fig.  14 , 33 papers reported positive of the results. For instance, Jerry and Aaron [ 38 ] proposed a system that promoted a better relation to physics concepts.

Papers according to the AR impact in education

Ibánez et al. [ 36 ] revealed that the AR-based application was more effective than the web-based one in promoting student’s knowledge. The four teachers in Wei et al. [ 77 ] considered the creative designs produced with AR by students more novel, sophisticated, and with more practical value.

In terms of performance improvement, Yeo et al. [ 80 ] reported that the AR image overlay and laser guidance improved the training process of needle placement. The participants who trained with overlay guidance performed better even when required to do freehand insertions. Zhang et al. [ 82 ] describe that in outdoor teaching environments, altering tool factors significantly enhances performance factors.

Regarding usability aspects, Wei et al. [ 77 ] reported that students considered the teaching contents with AR relevant and so had greater satisfaction.

The systems were described as convenient/interesting in some studies. Additionally, students in Tarng et al. [ 70 ] considered the virtual scenes and butterflies very realistic, and they would like to use it again in the future.

Reduction in costs were also reported. Student’s attention was also significantly improved due to the introduction of AR technology as reported in Wei et al. [ 77 ].

AR also enabled learning formal contents in informal environments as shown in Sommerauer and Müller [ 67 ]. This study pointed out that the empirical evidence suggests that AR has the potential to be an effective tool for learning mathematics in a museum. Students also perceived AR as a valuable add-on of the exhibition.

Eleven papers reported mixed results. That means the results could be either positive or negative for one aspect and neutral for others, for example. This situation is illustrated in Martín-Gutiérrez et al. [ 52 ]. This paper reported improvement on user’s spatial skills while working on their own; the statistic results show that use of the HMD device does not provide any difference when obtaining spatial ability upgrades with respect to the PC monitor. Authors argue that this result may be caused by the fact that HMD use is not the most suitable as users stated that the glass and camera set were not comfortable.

In Wang et al. [ 74 ], the proposed librarian system was more helpful in promoting the learning performance of learners with the field-dependent cognitive style than the conventional librarian instruction, particularly for learning content associated with application and comprehension.

Chen and Tsai [ 9 ] revealed that there was no gender difference in learning. This study investigated the AR’s impact depending on student’s personal learning styles (there was an impact) and personal gaming skills (there was no impact). Chen and Tsai [ 9 ] revealed a neutral outcome.

Another example is [ 37 ], which reported positive regarding intrinsic motivation, but slightly negative (although not significantly different) regarding selflearning. Nevertheless, no paper reported only negative or neutral outcomes.

Guidelines for educational evaluation

Through this literature review, authors were able to understand the current status of AR evaluation in education. In this section, we will discuss some principles that are important to be taken into account in similar situations. These aspects have already been discussed in [ 18 ].

Many studies have pointed out the importance of multiple metrics in research design. For instance, Easterbrook et al. [ 25 ] point out its usefulness and highlight the importance of employing both quantitative and qualitative metrics as a way of compensating the weakness of each method. Cohen et al. [ 12 ] explain that there are many advantages of using multimethod approaches in social research. The authors highlight two of them:

While single observation in fields such as physics and chemistry usually yield sufficient and unambiguous information, it provides a limited view of the complexity of human behavior and interactions.

Exclusive reliance on one method may bias or distort the researcher’s picture of a particular reality he/she is investigating.

Although not all the papers used multiple metrics to evaluate educational aspects, we observed that many papers did use them in their studies.

Another important issue is technology integration into the classrooms. In order to effectively evaluate new educational technology, it is important to effectively integrate them in the schools. Dexter [ 21 ] points out two premises for effective integration and implementation of technology for K–12 classrooms, that are:

The teacher must act as an instructional designer, planning the use of technology to support learning.

Schools must support teachers in this role.

It is important for researchers and developers to have an understanding on how teachers will integrate new technologies into their lessons since this will shape student’s learning opportunities. Fitzpatrick [ 26 ] stresses the need to involve teachers in the process of adopting new technology, so the activities are integrated to their lesson plan and meaningful to the students. For instance, activity theory [ 47 ] shows that activities are culturally mediated and inserted into a given context that includes the mediation of artifacts, of the community, and of its rules and its division of labor. In the process of transforming the activity of teaching into learning, there is a whole complex of mediations involving the curriculum, the educational rules, teacher’s training, and artifacts to name a few. This complex scenario needs to be taken into account in order for researchers to understand the changes caused by the introduction of a new artifact and the changes needed to expand and adjust the system.

Hence, taking this information into account, it is possible to infer that teachers need to have a very active approach when it comes to use and evaluation of technology in education. However, the data showed that only five papers considered the teacher as a creator in their evaluation process.

Crompton [ 15 ] explains that the evaluation of a piece of technology in isolation will tend to focus on various aspects of the technology itself, such as screen design and text layout. On the other hand, the evaluation of a courseware within the course itself will allow for examination of other factors that will lead to successful integration of the product within the course. Some of these aspects are:

Educational setting

Aims and objectives of the course

Teaching approach

Learning strategies

Assessment methods

Implementation strategy

Formative evaluations as stated by Scriven are typically conducted during the development or improvement of a program, person, or product, and it is conducted with the intent to improve [ 66 ]. On the other hand, summative evaluation is typically quantitative, using numeric scores or letter grades to assess learner achievement. Thus, a comprehensive evaluation involving both types of assessment is advisable in order to have a better overview of the process and its outcome.

Final remarks

Through this research, we identified AR’s potential to be applied in learning contexts. Developments in AR technology have enabled researchers to develop and evaluate more tools in the field of education. Hence, it was evident a growing interest in evaluating its impact in the learning process.

Results have shown that most studies combined different methodologies to evaluate their tools; however, only few papers combined them to evaluate educational gains.

Most of these papers used multiple metrics but to evaluate different aspects rather than just learning, such as usability and efficiency. Merriam [ 56 ] explains that all research designs can be discussed in terms of their relative strengths and limitations. She claims that their merits are related to select the most appropriate ones to address the research problem. Cohen et al. [ 12 ] argue that there are many advantages of using multimethod approach in social research. They highlight that (a) while single observation in fields such as physics usually yield sufficient and unambiguous information, it provides a limited view of the complexity of human behavior and interactions, and (b) exclusive reliance on one method may bias or distort the researcher’s picture of a particular reality.

It was also evident that most studies did not involve the teacher as an instructional designer. However, teachers were involved in many studies in a wide range of ways from consultant to creator. Fitzpatrick [ 26 ] highlights the need to involve teachers in the process of adopting new technological tools, so activities are integrated into their lesson plans and, thus, meaningful to the students.

Although AR has been shown to be helpful for teachers, it can also be inferred that its use, in some situations, may decrease the role of the teacher as the only source of knowledge since it may enable learners to be aided by other peers, trainers, or even their parents depending on the situation.

In this review, we noticed that there are solutions being developed to different age groups and knowledge domains. However, it was noticed a lack of evaluation of AR systems aimed at very young learners.

Regarding the types of questions asked, most papers presented more than one question. These questions were mainly relationship and descriptive-comparative ones. Those papers intended to describe the effect of a given AR technology comparing it with different resources as well as its relationship with different aspects, such as academic achievement or motivation, which indicates that the field is still maturing when it comes to evaluating AR educational impacts.

The papers were also classified according to the tracking, display, and interaction techniques used. It was noticeable that this choice of technology varied deeply depending on the learning objectives of the tool. However, this choice had an impact in the possibilities and limitations of use of the applications.

We also investigated if the papers based their work in any educational theory. Most papers mentioned educational theories. However, 19 studies did not mention any theory. It is important to highlight that educational theories may help to unravel contributions of AR tools as well as its limitations. In addition, it may help to understand how AR unique features may impact in the learning setting. The theories mentioned varied considerably, but something that most of them had in common is a learner-centered approach, thus putting the focus on student’s discovery, construction, and interaction processes and the attachment to the learning context.

It is noticeable that AR can expand the learning horizons. Some of the theories focus on understanding learning processes to provide a more effective experience for students considering their personal needs and abilities. We observed the need to look at AR instructional design from the perspective and limitations of the learners themselves.

The latter question investigated the kinds of impact of the results of the studies. Most of them presented positive outcomes. AR has been proved to be a helpful tool concerning many aspects of learning. In this sense, studies presented positive outcomes regarding a wide range of aspects, such as learning, academic performance, and motivation, among others.

Neutral outcomes were also reported as in some studies; the proposed AR system generated equivalent learning performance when compared to a traditional one. However, as already discussed, in many cases, results were neutral for one aspect and positive for others.

The analysis evidenced that AR can help to promote independence and interest among students, which can lead to more student-centered approaches, in which students are the center of their own learning and may apply it in more practical ways. The use of AR also enabled students to experience more concrete situated learning experiences, and together with mobile technologies, it may help to extend learning to different environments in a contextualized way, such as museums and student’s campi.

To sum up, during this review, it was noticed that AR has unique affordances that can impact the learning experience. As technology matures, researchers are increasingly concerned with how to incorporate real classroom/learning issues into their investigation.

Thus, authors discussed some guidelines for AR educational evaluation based on the lessons learned. First, based on the literature review, we advocate for the use of multiple metrics both quantitative and qualitative in order to have a better overview of the technology inserted in the teaching context as well as its effects.

Second, although it is not always possible to have a longitudinal evaluation, it is recommended to have a comprehension of more than punctual assessments but rather understand its effect in student’s development in a longer term. Finally, as it is widely recognized that teachers play a major role in technology adoption in the schools, we advocate for the involvement of teachers in the evaluation in more active ways as possible. Moreover, it is important to have tools that are flexible enough in order to facilitate teachers’ and students’ input of content.

As for limitations, due to the limited number of databases used, authors are aware that results may not fully represent the research development in the field.

Implications of the research

As implications of this research, it was noticed the need for more authoring tools that would enable users to create their own materials independently. Moreover, it is evident the need for more research regarding the evaluation of AR, especially, long-term ones since they could provide a better overview of the process of using this technology into the learning environment.

A.1 Quantitative criteria

Table 6 shows the scores of the quantitative evaluation of each paper.

Question/objective sufficiently described?

Study design evident and appropriate?

Method of subject/comparison group selection or source of information/input variables described and appropriate?

Subject (and comparison group, if applicable) characteristics sufficiently described?

If interventional and random allocation was possible, was it described?

If interventional and blinding of investigators was possible, was it reported?

If interventional and blinding of subjects was possible, was it reported?

Outcome and (if applicable) exposure measure(s) well defined and robust to measurement/misclassification bias means of assessment reported?

Sample size appropriate?

Analytic methods described/justified and appropriate?

Some estimate of variance is reported for the main results?

Controlled for confounding?

Results reported in sufficient detail?

Conclusions supported by the results?

A.2 Qualitative criteria

Table 7 shows the scores of the quantitative evaluation of each paper.

Context for the study clear?

Connection to a theoretical framework/wider body of knowledge?

Sampling strategy described, relevant, and justified?

Data collection methods clearly described and systematic?

Data analysis clearly described and systematic?

Use of verification procedure(s) to establish credibility?

Reflexivity of the account?

https://www.instructionaldesigncentral.com/whatisinstructionaldesign

The database with the selected papers is available at: https://papercatalog.000webhostapp.com

Altman D (1990) Practical statistics for medical research. Chapman & Hall/CRC Texts in Statistical Science. Taylor & Francis, London. ISBN 9780412276309. https://books.google.com.br/books?id=v-walRnRxWQC .

Google Scholar  

Amor B (2014) Experience art through relative realities with Samsung digital gallery at Yuchengco Museum. https://goo.gl/KLygPJ . Accessed 2 Oct 2018.

Azuma RT (1997) A survey of augmented reality. Presence: Teleoper. Virtual Environ 6(4):355–385. ISSN 1054-7460. https://doi.org/10.1162/pres.1997.6.4.355 .

Balog A, Pribeanu C (2010) The role of perceived enjoyment in the students’ acceptance of an augmented reality teaching platform: a structural equation modelling approach. Stud Inform Control 19(3):319–330.

Article   Google Scholar  

Billinghurst M, Duenser A (2012) Augmented reality in the classroom. Computer 45(7):56–63. ISSN 0018-9162. https://doi.org/10.1109/MC.2012.111 .

Billinghurst M, Clark A, Lee G, et al (2015) A survey of augmented reality. Found Trends Ⓡ Hum–Comput Interact 8(2-3):73–272.

Bosque LD, Martinez R, Torres JL (2015) Decreasing failure in programming subject with augmented reality tool. Procedia Comput Sci 75:221–225. ISSN 1877-0509. https://doi.org/10.1016/j.procs.2015.12.241 . http://www.sciencedirect.com/science/article/pii/S1877050915037023 .

Chang K-E, Chang C-T, Hou H-T, Sung Y-T, Chao H-L, Lee C-M (2014) Development and behavioral pattern analysis of a mobile guide system with augmented reality for painting appreciation instruction in an art museum. Comput Educ 71:185–197. ISSN 0360-1315. https://doi.org/10.1016/j.compedu.2013.09.022 . http://www.sciencedirect.com/science/article/pii/S0360131513002868 .

Chen C-M, Tsai Y-N (2012) Interactive augmented reality system for enhancing library instruction in elementary schools. Comput Educ 59(2):638–652. ISSN 0360-1315. https://doi.org/10.1016/j.compedu.2012.03.001 . http://www.sciencedirect.com/science/article/pii/S0360131512000589 .

Cheng K-H, Tsai C-C (2014) Children and parents’ reading of an augmented reality picture book: analyses of behavioral patterns and cognitive attainment. Comput Educ 72:302–312. ISSN 0360-1315. https://doi.org/10.1016/j.compedu.2013.12.003 . http://www.sciencedirect.com/science/article/pii/S0360131513003278 .

Cohen J (1960) A coefficient of agreement for nominal scales. Educ Psychol Meas 20(1):37–46.

Cohen L, Manion L, Morrison K (2003) Research methods in education. Taylor & Francis ebook collection. Taylor & Francis. ISBN 9780203224342. https://books.google.co.uk/books?id=5twk1pHwyL8C . Accessed 2 Oct 2018.

Contero M, Gomis JM, Naya F, Albert F, Martin-Gutierrez J (2012) Development of an augmented reality based remedial course to improve the spatial ability of engineering students In: 2012 Frontiers in Education Conference Proceedings, 1–5. https://doi.org/10.1109/FIE.2012.6462312 .

Cornett CE (1983) What you should know about teaching and learning styles. Fastback Series In: Phi Delta Kappa Educational Foundation. ISBN 9780873671910. https://books.google.at/books?id=SlckAQAAMAAJ . Accessed 2 Oct 2018.

Crompton P (1996) Evaluation: a practical guide to methods In: Learn Technol Dissemination Initiative, 66.

Csikszentmihalyi M (1990) Flow: the psychology of optimal experience, 6 edn. Harper & Row, New York.

Cubillo J, Martín S, Castro M, Diaz G, Colmenar A, Boticki I (2014) A learning environment for augmented reality mobile learning In: 2014 IEEE Frontiers in Education Conference (FIE) Proceedings, 1–8. https://doi.org/10.1109/FIE.2014.7044039 .

da Silva MMO, Roberto RA, Teichrieb V, Cavalcante PS (2016) Towards the development of guidelines for educational evaluation of augmented reality tools In: 2016 IEEE Virtual Reality Workshop on K-12 Embodied Learning through Virtual Augmented Reality (KELVAR), 17–21. https://doi.org/10.1109/KELVAR.2016.7563677 .

da Silva M, Roberto R, Teichrieb V (2013) Evaluating an educational system based on projective augmented reality In: Brazilian Symposium on Computers in Education (Simpósio Brasileiro de Informática na Educação-SBIE), vol 24, 214.. Sociedade Brasileira de Computação, Porto Alegre.

da Silva MMO, Roberto R, Teichrieb V (2015) Evaluation of augmented reality technology in the English language field In: Brazilian Symposium on Computers in Education (Simpósio Brasileiro de Informática na Educação-SBIE), vol 26, 577.. Sociedade Brasileira de Computação, Porto Alegre.

Dexter S (2002) eTIPS-Educational technology integration and implementation principles. Designing Instr Technol-Enhanc Learn 24:56–70.

Dünser A, Billinghurst M (2011) Evaluating augmented reality systems, 289–307.. Springer New York, New York. ISBN 978-1-4614-0064-6. https://doi.org/10.1007/978-1-4614-0064-6_13 .

Dünser A, Grasset R, Seichter H, Billinghurst M (2007) Applying HCI principles to AR systems design. Technical report. HIT Lab NZ.

Dünser A, Grasset R, Billinghurst M (2008) A survey of evaluation techniques used in augmented reality studies In: ACM SIGGRAPH ASIA 2008 courses, SIGGRAPH Asia ’08, 1–27.. ACM, New York. https://doi.org/10.1145/1508044.1508049 . http://portal.acm.org/citation.cfm?id=1508044.1508049 .

Easterbrook S, Singer J, Storey M-A, Damian D (2008) Guide to advanced empirical software engineering, chapter Selecting Empirical Methods for Software Engineering Research. Springer London, London. ISBN 978-1-84800-044-5. https://doi.org/10.1007/978-1-84800-044-5_11 . http://doi.org/10.1007/978-1-84800-044-5_11 .

Fitzpatrick A (2004) Analytical survey information and communication technologies in the teaching and learning of foreign languages: state-of-the-art, needs and perspectives. unesco institute for information technologies in education. Technical report. UNESCO, Moscow, Russia.

Fonseca D, Villagrasa S, Vails F, Redondo E, Climent A, Vicent L (2014) Engineering teaching methods using hybrid technologies based on the motivation and assessment of student’s profiles In: 2014 IEEE Frontiers in Education Conference (FIE) Proceedings, 1–8. https://doi.org/10.1109/FIE.2014.7044209 .

Fonseca D, Villagrasa S, Valls F, Redondo E, Climent A, Vicent L (2014) Motivation assessment in engineering students using hybrid technologies for 3d visualization In: 2014 International Symposium on Computers in Education (SIIE), 111–116. https://doi.org/10.1109/SIIE.2014.7017714 .

Fonseca D, Valls F, Redondo E, Villagrasa S (2016) Informal interactions in 3D education: Citizenship participation and assessment of virtual urban proposals. Comput Hum Behav 55:504–518. ISSN 0747-5632. https://doi.org/10.1016/j.chb.2015.05.032 . http://www.sciencedirect.com/science/article/pii/S0747563215004033 .

Frank JA, Kapila V (2017) Mixed-reality learning environments: integrating mobile interfaces with laboratory test-beds. Comput Educ 110:88–104. ISSN 0360-1315. https://doi.org/10.1016/j.compedu.2017.02.009 . http://www.sciencedirect.com/science/article/pii/S0360131517300325 .

Hou L, Wang X (2013) A study on the benefits of augmented reality in retaining working memory in assembly tasks: a focus on differences in gender. Autom Constr 32:38–45. ISSN 0926-5805. https://doi.org/10.1016/j.autcon.2012.12.007 . http://www.sciencedirect.com/science/article/pii/S0926580512002464 .

Hsiao KF, Rashvand HF (2011) Body language and augmented reality learning environment In: 2011 Fifth FTRA International Conference on Multimedia and Ubiquitous Engineering, 246–250. https://doi.org/10.1109/MUE.2011.51 .

Hulin T, Schmirgel V, Yechiam E, Zimmermann UE, Preusche C, Pöhler G (2010) Evaluating exemplary training accelerators for programming-by-demonstration In: 19th International Symposium in Robot and Human Interactive Communication, 440–445. https://doi.org/10.1109/ROMAN.2010.5598611 .

Hutchings W (2007) Enquiry-based learning: definitions and rationale. Technical report, Centre for Excellence in Enquiry-Based. http://www.ceebl.manchester.ac.uk/resources/papers/hutchings2007_definingebl.pdf .

Ibánez MB, Di-Serio Á, Villarán-Molina D, Delgado-Kloos C (2015) Augmented reality-based simulators as discovery learning tools: an empirical study. IEEE Trans Educ 58(3):208–213. ISSN 0018-9359. https://doi.org/10.1109/TE.2014.2379712 .

Ibánez MB, Di Serio Á, Villarán D, Kloos CD (2014) Experimenting with electromagnetism using augmented reality: impact on flow student experience and educational effectiveness. Comput Educ 71:1–13. ISSN 0360-1315. https://doi.org/10.1016/j.compedu.2013.09.004 . http://www.sciencedirect.com/science/article/pii/S0360131513002571 .

Iwata T, Yamabe T, Nakajima T (2011) Augmented reality go: extending traditional game play with interactive self-learning support In: 2011 IEEE 17th International Conference on Embedded and Real-Time Computing Systems and Applications, vol 1, 105–114. https://doi.org/10.1109/RTCSA.2011.43 .

Jerry TFL, Aaron CCE (2010) The impact of augmented reality software with inquiry-based learning on students’ learning of kinematics graph In: 2010 2nd International Conference on Education Technology and Computer, vol 2, V2–1–V2–5. https://doi.org/10.1109/ICETC.2010.5529447 .

Joo-Nagata J, Abad FM, Giner JG-B, García-Peñalvo FJ (2017) Augmented reality and pedestrian navigation through its implementation in m-learning and e-learning: evaluation of an educational program in Chile. Comput Educ 111:1–17. ISSN 0360-1315. https://doi.org/10.1016/j.compedu.2017.04.003 . http://www.sciencedirect.com/science/article/pii/S0360131517300751 .

Kmet LM, Lee RC, AHFMR, Cook LS (2004) Standard quality assessment criteria for evaluating primary research papers from a variety of fields In: HTA initiative. Alberta Heritage Foundation for Medical Research. ISBN 9781896956770. https://books.google.at/books?id=UYE8HQAACAAJ . Accessed 2 Oct 2018.

Kostaras NN, Xenos MN, et al (2009) Assessing the usability of augmented reality systems In: Proceedings of the 13th Panhellenic Conference on Informatics, 197–201, Alberta Heritage Foundation for Medical Research (AHFMR). AHFMR-HTA Initiative.

Kraut B, Jeknić J (2015) Improving education experience with augmented reality (AR) In: 2015 38th International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO), 755–760. https://doi.org/10.1109/MIPRO.2015.7160372 .

Lackney JA (1999) A history of the studio-based learning model. http://edi.msstate.edu/work/pdf/history_studio_based_learning.pdf . Accessed 2 Oct 2018.

Lai ASY, Wong CYK, Lo OCH (2015) Applying augmented reality technology to book publication business In: 2015 IEEE 12th International Conference on e-Business Engineering, 281–286. https://doi.org/10.1109/ICEBE.2015.55 .

Leblanc F, Champagne BJ, Augestad KM, Neary PC, Senagore AJ, Ellis CN, Delaney CP (2010) A comparison of human cadaver and augmented reality simulator models for straight laparoscopic colorectal skills acquisition training. J Am Coll Surg 211(2):250–255. ISSN 1072-7515. https://doi.org/http://doi.org/10.1016/j.jamcollsurg.2010.04.002 . http://www.sciencedirect.com/science/article/pii/S1072751510002735 .

Lee K (2012) Augmented reality in education and training. TechTrends 56(2):13–21. ISSN 1559-7075. https://doi.org/10.1007/s11528-012-0559-3 . http://doi.org/10.1007/s11528-012-0559-3 .

Leont’ev AN (1978) Activity, consciousness, and personality. Prentice-Hall, Englewood Cliffs. ISBN 9780130035332. http://lchc.ucsd.edu/MCA/Paper/leontev/ .

Lin T-J, Duh HB-L, Li N, Wang H-Y, Tsai C-C (2013) An investigation of learners’ collaborative knowledge construction performances and behavior patterns in an augmented reality simulation system. Comput Educ 68:314–321. ISSN 0360-1315. https://doi.org/10.1016/j.compedu.2013.05.011 . http://www.sciencedirect.com/science/article/pii/S0360131513001322 .

Mahmoudi MT, Badie K, Valipour M (2015) Assessing the role of AR-based content in improving learning performance considering Felder-Silverman learning style In: 2015 International Conference on Interactive Collaborative Learning (ICL), 838–843. https://doi.org/10.1109/ICL.2015.7318137 .

Martín-Gutiérrez J (2011) Proposal of methodology for learning of standard mechanical elements using augmented reality In: 2011 Frontiers in Education Conference (FIE), 1–6. https://doi.org/10.1109/FIE.2011.6142708 .

Martín-Gutiérrez J, Saorin JL, Contero M, Alcaniz M (2010) Ar_Dehaes: an educational toolkit based on augmented reality technology for learning engineering graphics In: 2010 10th IEEE International Conference on Advanced Learning Technologies, 133–137. https://doi.org/10.1109/ICALT.2010.45 .

Martín-Gutiérrez J, Navarro RE, González MA (2011) Mixed reality for development of spatial skills of first-year engineering students In: 2011 Frontiers in Education Conference (FIE), T2D–1–T2D–6. https://doi.org/10.1109/FIE.2011.6142707 .

Martínez AA, Benito JRL, González EA, Ajuria EB (2017) An experience of the application of augmented reality to learn english in infant education In: 2017 International Symposium on Computers in Education (SIIE), 1–6. https://doi.org/10.1109/SIIE.2017.8259645 .

Mayer RE (2009) Multimedia learning, 2 edn. Cambridge University Press, Cambridge.

Book   Google Scholar  

McGreal R (2009) Mobile devices and the future of free education. http://www.ou.nl/Docs/Campagnes/ICDE2009/Papers/Final_paper_252mcgreal.pdf . Accessed 07 Oct 2018.

Merriam SB (2009) Qualitative research: a guide to design and implementation. Jossey-Bass higher and adult education series. John Wiley & Sons. ISBN 9780470283547. https://books.google.at/books?id=tvFICrgcuSIC . Accessed 2 Oct 2018.

Moher D, Liberati A, Tetzlaff J, Altman DG (2010) Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Int J Surg 8(5):336–341. ISSN 1743-9191. https://doi.org/10.1016/j.ijsu.2010.02.007 . http://www.sciencedirect.com/science/article/pii/S1743919110000403 .

Moura AMC (2010) Apropriação do Telemóvel como Ferramenta de Mediação em Mobile Learning: Estudos de Caso em Contexto Educativo. PhD thesis. Universidade do Minho. http://repositorium.sdum.uminho.pt/handle/1822/13183 . Accessed 2 Oct 2018.

Nakamura J, Csikszentmihalyi M (2014) The concept of flow. Springer Netherlands, Dordrecht. ISBN 978-94-017-9088-8. https://doi.org/10.1007/978-94-017-9088-8_16 .

Ontario Ministry of Education (2013) Inquiry-based learning. Technical report, Ontario Ministry of Education. http://www.edu.gov.on.ca/eng/literacynumeracy/inspire/research/CBS_InquiryBased.pdf .

Oxford RL (2001) Learning styles and strategies. In: Celce-Murcia M (ed)Teaching English as a Second or Foreign Language, 359–366.. Heinle Cengage Learning, Boston.

Ramírez H, Mendoza E, Mendoza M, González E (2015) Application of augmented reality in statistical process control, to increment the productivity in manufacture. Procedia Comput Sci 75:213–220. ISSN 1877-0509. https://doi.org/10.1016/j.procs.2015.12.240 . http://www.sciencedirect.com/science/article/pii/S1877050915037011 .

Roberto R, Lima JP, Teichrieb V (2016) ISSN 0097-8493. https://doi.org/10.1016/j.cag.2016.02.002 . http://www.sciencedirect.com/science/article/pii/S0097849316300115 . Comput Graph 56:20–30.

Salazar M, Gaviria J, Laorden C, Bringas PG (2013) Enhancing cybersecurity learning through an augmented reality-based serious game In: 2013 IEEE Global Engineering Education Conference (EDUCON), 602–607. https://doi.org/10.1109/EduCon.2013.6530167 .

Santos MEC, Chen A, Taketomi T, Yamamoto G, Miyazaki J, Kato H (2014) Augmented reality learning experiences: survey of prototype design and evaluation. IEEE Trans Learn Technol 7(1):38–56. ISSN 1939-1382. https://doi.org/10.1109/TLT.2013.37 .

Scriven M (1991) Beyond formative and summative evaluation In: Evaluation and Education: A Quarter Century.

Sommerauer P, Müller O (2014) Augmented reality in informal learning environments: a field experiment in a mathematics exhibition. Comput Educ 79:59–68. ISSN 0360-1315. https://doi.org/10.1016/j.compedu.2014.07.013 . http://www.sciencedirect.com/science/article/pii/S036013151400164X .

Swan JE, Gabbard JL (2009) Survey of user-based experimentation in augmented reality In: 1st International Conference on Virtual Reality, HCI International, 1–9. http://people.cs.vt.edu/jgabbard/publications/hcivr05.pdf . Accessed 2 Oct 2018.

Tarng W, Ou KL (2012) A study of campus butterfly ecology learning system based on augmented reality and mobile learning In: 2012 IEEE Seventh International Conference on Wireless, Mobile and Ubiquitous Technology in Education, 62–66. https://doi.org/10.1109/WMUTE.2012.17 .

Tarng W, Yu CS, Liou FL, Liou HH (2013) Development of a virtual butterfly ecological system based on augmented reality and mobile learning technologies In: 2013 9th International Wireless Communications and Mobile Computing Conference (IWCMC), 674–679. https://doi.org/10.1109/IWCMC.2013.6583638 .

Tobar-Muñoz H, Fabregat R, Baldiris S (2014) Using a videogame with augmented reality for an inclusive logical skills learning session In: 2014 International Symposium on Computers in Education (SIIE), 189–194. https://doi.org/10.1109/SIIE.2014.7017728 .

Tori R, Kirner C, Siscoutto RA (2006) Fundamentos e tecnologia de realidade virtual e aumentada. Editora SBC. ISBN 9788576690689. https://books.google.com.br/books?id=h1H7PgAACAAJ . Accessed 2 Oct 2018.

Tsai CH, Huang JY (2014) A mobile augmented reality based scaffolding platform for outdoor fieldtrip learning In: 2014 IIAI 3rd International Conference on Advanced Applied Informatics, 307–312. https://doi.org/10.1109/IIAI-AAI.2014.70 .

Wang YS, Chen CM, Hong CM, Tsai YN (2013) Interactive augmented reality game for enhancing library instruction in elementary schools In: 2013 IEEE 37th Annual Computer Software and Applications Conference Workshops, 391–396. https://doi.org/10.1109/COMPSACW.2013.128 .

Wang Y-H (2017) Exploring the effectiveness of integrating augmented reality-based materials to support writing activities. Comput Educ 113:162–176. ISSN 0360-1315. https://doi.org/10.1016/j.compedu.2017.04.013 . http://www.sciencedirect.com/science/article/pii/S0360131517300969 .

Wei L, Najdovski Z, Abdelrahman W, Nahavandi S, Weisinger H (2012) Augmented optometry training simulator with multi-point haptics In: 2012 IEEE International Conference on Systems, Man, and Cybernetics (SMC), 2991–2997. https://doi.org/10.1109/ICSMC.2012.6378250 .

Wei X, Weng D, Liu Y, Wang Y (2015) Teaching based on augmented reality for a technical creative design course. Comput Educ 81:221–234. ISSN 0360-1315. https://doi.org/10.1016/j.compedu.2014.10.017 . http://www.sciencedirect.com/science/article/pii/S0360131514002371 .

Wu H-K, Lee SW-Y, Chang H-Y, Liang J-C (2013) Current status, opportunities and challenges of augmented reality in education. Comput Educ 62(0):41–49. ISSN 0360-1315. https://doi.org/10.1016/j.compedu.2012.10.024 . http://www.sciencedirect.com/science/article/pii/S0360131512002527 .

Yen J-C, Tsai C-H, Wu M (2013) Augmented reality in the higher education: students’ science concept learning and academic achievement in astronomy. Procedia-Soc Behav Sci 103:165–173. ISSN 1877-0428. https://doi.org/10.1016/j.sbspro.2013.10.322 . http://www.sciencedirect.com/science/article/pii/S1877042813037671 .

Yeo CT, Ungi T, U-Thainual P, Lasso A, McGraw RC, Fichtinger G (2011) The effect of augmented reality training on percutaneous needle placement in spinal facet joint injections. IEEE Trans Biomed Eng 58(7):2031–2037. ISSN 0018-9294. https://doi.org/10.1109/TBME.2011.2132131 .

Yin RK (2009) Case study research: design and methods. Applied Social Research Methods. SAGE Publications. ISBN 9781412960991. https://books.google.com.br/books?id=FzawIAdilHkC .

Zhang J, Sung Y-T, Hou H-T, Chang K-E (2014) The development and evaluation of an augmented reality-based armillary sphere for astronomical observation instruction. Comput Educ 73:178–188. ISSN 0360-1315. https://doi.org/10.1016/j.compedu.2014.01.003 . http://www.sciencedirect.com/science/article/pii/S036013151400013X . Accessed 2 Oct 2018.

Zhou F, Duh HB-L, Billinghurst M (2008) Trends in augmented reality tracking, interaction and display: a review of ten years of ISMAR In: Proceedings of the 7th IEEE/ACM International Symposium on Mixed and Augmented Reality, 193–202.. IEEE Computer Society, Cambridge.

Download references

Acknowledgements

The authors would like to thank Rafael Roberto for his valuable contributions.

The authors would like to thank Fundação de Amparo a Ciência e Tecnologia de Pernambuco (FACEPE) (processes IBPG-0605-1.03/15) for partially funding this research.

Availability of data and materials

The datasets supporting the conclusions of this article are available at https://papercatalog.000webhostapp.com/ .

Author information

Authors and affiliations.

Voxar Labs, Centro de Informática, Universidade Federal de Pernambuco, Recife, Brazil

Manoela M. O. da Silva, João Marcelo X. N. Teixeira & Veronica Teichrieb

Departamento de Eletrônica e Sistemas, Universidade Federal de Pernambuco, Recife, Brazil

João Marcelo X. N. Teixeira

EDUMATEC, Centro de Educação, Universidade Federal de Pernambuco, Recife, Brazil

Patrícia S. Cavalcante

You can also search for this author in PubMed   Google Scholar

Contributions

MMOS contributed to the conceiving and designing of the research protocol as well as acquiring, analyzing, and interpreting the data and drafting the manuscript. JMXNT contributed to the conceiving and designing the research protocol, analyzing and interpreting the data, and drafting and revising the manuscript. VT and PSC contributed to the conceiving and designing of the research protocol, revising the manuscript, and coordinating the research. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Manoela M. O. da Silva .

Ethics declarations

Competing interests.

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Additional information

Authors’ information.

MMOS is a Ph.D. candidate in Computer Science at the Federal University of Pernambuco and a researcher at Voxar Labs. Her research interests include interactive media, evaluation of educational tools, augmented reality for education and teaching and learning of mother and foreign languages.JMXNT is an assistant professor at Electronics and Systems Department of Federal University of Pernambuco and senior scientist at Voxar Labs. His research interests include 3D tracking, augmented reality, computer vision, computer graphics and embedded systems. VT is an associate professor at the Federal University of Pernambuco and head of the Voxar Labs research group. Her research interests include augmented reality, visualization, tracking and interaction. PSC is an associate professor at the Federal University of Pernambuco and head of the GENTE - New Technologies and Education research group. Her research interests include MOOCS and teachers’ long life formations, Learning Spaces and Learning Scenarios.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Reprints and permissions

About this article

Cite this article.

da Silva, M., Teixeira, J., Cavalcante, P. et al. Perspectives on how to evaluate augmented reality technology tools for education: a systematic review. J Braz Comput Soc 25 , 3 (2019). https://doi.org/10.1186/s13173-019-0084-8

Download citation

Received : 11 April 2018

Accepted : 10 January 2019

Published : 04 February 2019

DOI : https://doi.org/10.1186/s13173-019-0084-8

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Augmented reality
• Educational systems

Open Access is an initiative that aims to make scientific research freely available to all. To date our community has made over 100 million downloads. It’s based on principles of collaboration, unobstructed discovery, and, most importantly, scientific progression. As PhD students, we found it difficult to access the research we needed, so we decided to create a new Open Access publisher that levels the playing field for scientists across the world. How? By making research easy to access, and puts the academic needs of the researchers before the business interests of publishers.

We are a community of more than 103,000 authors and editors from 3,291 institutions spanning 160 countries, including Nobel Prize winners and some of the world’s most-cited researchers. Publishing on IntechOpen allows authors to earn citations and find new collaborators, meaning more people see your work not only from your own field of study, but from other related fields too.

Brief introduction to this section that descibes Open Access especially from an IntechOpen perspective

Want to get in touch? Contact our London head office or media team here

Our team is growing all the time, so we’re always on the lookout for smart people who want to help us reshape the world of scientific publishing.

Home > Books > Augmented Reality and Its Application

Augmented Reality Research and Applications in Education

Submitted: 08 July 2021 Reviewed: 09 July 2021 Published: 24 August 2021

DOI: 10.5772/intechopen.99356

Cite this chapter

There are two ways to cite this chapter:

From the Edited Volume

Augmented Reality and Its Application

Edited by Dragan Cvetković

To purchase hard copies of this book, please contact the representative in India: CBS Publishers & Distributors Pvt. Ltd. www.cbspd.com | [email protected]

Chapter metrics overview

1,077 Chapter Downloads

Impact of this chapter

Total Chapter Downloads on intechopen.com

Total Chapter Views on intechopen.com

Augmented reality is defined as the technology in which virtual objects are blended with the real world and also interact with each other. Although augmented reality applications are used in many areas, the most important of these areas is the field of education. AR technology allows the combination of real objects and virtual information in order to increase students’ interaction with physical environments and facilitate their learning. Developing technology enables students to learn complex topics in a fun and easy way through virtual reality devices. Students interact with objects in the virtual environment and can learn more about it. For example; by organizing digital tours to a museum or zoo in a completely different country, lessons can be taught in the company of a teacher as if they were there at that moment. In the light of all these, this study is a compilation study. In this context, augmented reality technologies were introduced and attention was drawn to their use in different fields of education with their examples. As a suggestion at the end of the study, it was emphasized that the prepared sections should be carefully read by the educators and put into practice in their lessons. In addition it was also pointed out that it should be preferred in order to communicate effectively with students by interacting in real time, especially during the pandemic process.

• augmented reality research and applications
• field of education
• pandemic process
• digital transformation
• virtual environment

Author Information

Ezgi pelin yildiz *.

• Department of Computer Programming, Kazim Karabekir Vocational School of Technical Sciences, Kafkas University, Kars, Turkey

*Address all correspondence to: [email protected]

1. Introduction

Today, rapid changes and advances in science and technology affect and change the lifestyle of individuals. Apart from individuals, it is not possible for the education process and educational environments not to be affected by this change [ 1 ]. When the technologies used in educational environments from the past to the present are examined, it is seen that there is a transformation from blackboard and chalk to the computer and internet world, even to smart technologies with artificial intelligence. Especially in recent years, computer and internet technologies have had such a wide area of use in our lives that it was unthinkable for education services to be left out of the field [ 2 ].

The definition of today’s learners as Z generation and/or digital generation and their characteristics require educators to follow technological developments and use the most appropriate technological tools in learning environments. One of these new technologies is augmented reality applications in education. When the literature is examined, there are many definitions of the concept of augmented reality made by researchers. Some of these definitions:

Augmented reality according to Milgram and Kishino [ 3 ]; “it is a reality environment where digital media products are used instead of real world objects” appears to be the most general definition. According to Azuma [ 4 ], augmented reality is a derivative of virtual reality. According to this definition, augmented reality is virtual environments in which existing reality is supported, not created from scratch. In this context virtual and real objects in augmented reality environments offered to users in harmony. Augmented reality creates the interactive environment between the virtual and real world. Augmented reality is used to achieve this [ 5 , 6 ]. When the definitions in the literature are examined, as a common definition; augmented reality can be defined as real worlds enriched using virtual objects.

Game and Video

Art and Museums

Device Maintenance/Support

With the rapid development of Augmented Reality applications day by day, usage areas in many sectors are starting to increase. Major brands have started to give importance to providing a more realistic and embodied experience to their customers by using Augmented Reality (AR). This technology, which appears in many fields such as cosmetics, automobiles, construction, food, combines the virtual world with real life. Identifying target audiences, tracking and using technology in brand awareness and sustainable marketing is now vital for companies. The most importantly, companies from the public or private sector invest on enhanced technology in order to better promote or market their services/products and need talented people/firms in this field. In this context, augmented reality applications offer these services to businesses with technology support.

to provide students with more flexible and interesting learning environments,

to experience an excitement they have never experienced before,

to increase their willingness and motivation to learn,

to help students make active observations during their learning processes and to form hypotheses as a result of these observations,

to increasing students’ learning performance and helping them establish social interactions within the group,

to bridging formal and informal learning and encouraging students to learn collaboratively,

AR technology; it gives a feeling of independence from the place, freedom and personal,

to creating new opportunities in education by promoting learning.

it is possible to rank as.

When the augmented reality technologies, which are frequently used in the field of education, are examined, wearable technologies draw attention. Wearables are loaded with smart sensors that track body movements. Usually these products use bluetooth, Wi-Fi and mobile internet connection to sync with smartphone wirelessly. Users are connected to wearable devices with the help of sensors. Wearable technology products that are always with the user; it provides important services in many areas, especially in entertainment, health, work, information, education, socialization and security.

Wearable technologies in the field of education are used in learning-teaching environments. Modern visualization techniques help students explore existing educational resources and new knowledge ( Figure 1 ) [ 12 ].

Wearable technologies the past and present and future.

Internet of things

Smart watches

Google – Glass Project

HoloLens – Microsoft:

Oculus Rift – Facebook

Bracelets, Rings and Necklaces

Smart Clothing and Tattoos

These tools, which can also be named as wearable computers in the literature, reveal a commensalistic relationship between human and computer however, the daily life of the individual has a structure that enriches their experience [ 13 ]. From smart watches to wristbands, sensor accessories such as rings and necklaces, virtual reality glasses, Google Glass project and derivative smart glasses, as well as smart optical lenses and headphones, many things can be shown among wearable technologies [ 14 ].

Augment – 3B

Google Translate

LifePrint Photos

In the light of all this information, the purpose of this chapter; the use of augmented reality environments and applications in the field of education, the programs and technologies used in this context, and the researches are discussed in detail.

The new normal situation, especially with the pandemic process, also creates an opportunity for more educators to try new generation technologies (VR and AR technologies) beyond video and teleconferencing applications. It is predicted that such research studies will be important so that educators realize the benefits of these technologies and use them actively in learning environments.

2. Conceptual framework

Augmented reality (AR) has been slowly but surely following its predecessor virtual reality in changing the education sector—digitizing classroom learning, and making training more diverse and interactive. In this section, current studies in the literature in recent years on the integration of augmented reality applications into education are given. When these studies are examined;

Çetin [ 15 ], investigated the effect of augmented reality-based stories on reading skills in his research. In the research, augmented reality based story text samples were presented to primary school 3rd grade students ( Figure 2 ).

Augmented reality based story text samples.

A scoring key was developed for the answers given to the questions prepared by the researcher to measure the skills of expressing what they read in writing. As a result of the research, it was observed that the augmented reality-based stories did not have a significant effect on the reading motivation and reading comprehension skill levels of the students, but they created a positive significant difference on their ability to tell what they read in written and verbal form. In addition, as a result of the research, it was observed that the reactions of the students towards the texts increased.

As a similar study Baysan and Uluyol [ 16 ], the effect of the use of augmented reality books (AR-books) on the academic success of the students and the students’ opinions about the environment were investigated in his study. The AR-based teaching material developed by the HITLibHZ-BuildAR program was used in the laboratory environment for the experimental group of 22 people and the course was taught by the researcher. As a result; according to the qualitative data obtained from the students, AR is a promising technology. Educational AR applications should be used in areas that require 3D spatial visualization such as Geometry and Geography rather than technology education. Participants support the use of AR in Computer Hardware training, with better developed platforms and more professional designs ( Figure 3 ).

Augmented reality application book sample.

Almusawi et al. [ 17 ], in their study, they discussed innovation in physical education: teachers’ perspectives on readiness for wearable technology integration. The study is a case study and includes semi-structured interviews with 38 public school physical education teachers. The following scheme was used in the study ( Figure 4 ).

The findings show that physical education teachers have concerns about the design aspects of wearable technologies in terms of material design and device suitability for physical education. To eliminate these concerns, it is proposed to provide innovative learning environments that impact technology through collaborative, competitive, engaging and evidence-based learning experiences through wearable technologies that provide comfort, enhanced wearability and injury prevention in physical education.

It is understood from the existence of studies in the literature that augmented reality technologies have been used frequently in medical education recently. When the relevant studies in the literature are examined ( Figure 5 ).

Use of augmented reality technologies in medical education.

Kucuk et al. [ 18 ], a new perspective in medical education multimedia applications: augmented reality has been studied in their research. As a result, it is difficult to understand the subjects including the structure of the brain and vessels such as neuroanatomy in medical courses, in this direction, it was emphasized that AR applications could be developed to facilitate the learning processes of students in such subjects. Considering the characteristics of today’s students in the digital citizen group, it has been suggested in the study that students should be supported with various technological solutions in this process, at this point, the dissemination of medical augmented reality applications that are based on the learning approach anytime and anywhere and support individual learning.

3. Augmented reality applications used in education

Augmented reality, a concept that has been frequently encountered recently, promises a future where we can get away from the world we live in, create a new worlds and enter ‘inside’ our imagination. By adding this technology with which we can ‘beautify’ the world we live in, make brand new additions to our world and bring our imagination to the place we live in, we started to manipulate our real world at the same time, while constructing mixed reality virtual worlds that we use together. It has become compulsory to benefit from these privileges and advantages that augmented reality offers to our lives, especially in terms of education, on behalf of the Z generation youth.

It is now possible to use these technologies in learning and teaching environments by making use of the ready-made programs of augmented reality. When the literature is examined, the frequently used programs and application areas are below:

3.1 Augment: 3B

Augment is an ARCore-based mobile app to visualize 3D models in Augmented Reality, integrated in real time in their actual size and environment. Balak and Kısa [ 19 ] investigated the effects of this application on technical drawing education in their studies. The data obtained as a result of the use of Augmented Reality technology in the technical drawing course of the 2015–2016 period were examined. As a result; the result of the survey made with the pre- and post-tests applied; it has been determined that the students understand and adopt the Augmented Reality technology, which is a modern education tool, and this technology increases their interest in the lesson ( Figure 6 ).

Technical drawing with 3D modeling with AR technologies.

3.2 Google translate

According to Google, the Translate app currently supports text translations between 103 languages, offline translations for 52 languages and Word Lens-based augmented reality translations for 30 languages. Aiming to make life easier for users with its mobile translation application, Google offers Instant camera translation; It started to support a total of 88 languages with the addition of 60 new languages such as Arabic, Hindi, Malaysian, Thai and Vietnamese etc. ( Figure 7 ).

Augmented reality-based Google translate app.

3.3 SketchAR

SketchAR, which is an application that combines augmented reality and drawing, is among the applications frequently preferred by artists recently. SketchAR, which is basically a drawing application made available to artists, confirms that digital works created by artists are unique and original, making them accepted as NFT (data unit). SketchAR, an initiative founded in 2017 by Aleksandr Danilin, Alexander Danilin and Andrey Drobitko in Lithuania, offers its users a different drawing experience by combining augmented reality technology with drawing, together with artificial intelligence support ( Figure 8 ).

Drawing courses with SketchAR.

3.4 Wikitude

Wikitude initially focused on providing location-based augmented reality experiences through the Wikitude World Browser App. In 2012, the company restructured its proposition by launching the Wikitude SDK, a development framework utilizing image recognition and tracking, and geolocation technologies. Wikitude initially entered the market with its geo location AR app. The Wikitude app was the first publicly available application that used a location-based approach to augmented reality ( Figure 9 ).

Wikitude world browser app.

It is supported by studies in the literature that this application is also used in geography education. Wikitude; it is a complete AR development platform used by major brands, travel catalogs, retailers and publishers to deliver a variety of engaging solutions.

3.5 LifePrint photos

Life Print is an Android and iPhone photo and video printer. The Life Print program uses augmented reality to magically bring photos to life ( Figure 10 ).

Augmented reality app: LifePrint photos.

3.6 Smartify

The application starts with permission from users to access camera and location. With camera access, the artwork is scanned, and according to the location, it provides the opportunity to get information about which museums are and how far, how many artworks of art they are, open and closed hours, and to see some of the artworks in the museum. The application has three basic directions; scan , profile and explore ( Figure 11 ).

Augmented reality app: Smartify.

3.7 Spyglass

Spyglass app is a program that allows users to turn their smartphones into a compass, gyroscope, star tracker and more ( Figure 12 ).

Locating with spyglass technologies.

3.8 Blippar

Blippar uses augmented reality, artificial intelligence and computer vision to provide you with information about what you find around you. It is quite successful with its advanced image recognition algorithms that find out what the objects are and bring the relevant information. Blippar will introduce the feature that will allow its users to create their own profiles very soon, but it will be possible to get detailed information about a person with the innovation called Augmented Reality Face Profiles ( Figure 13 ).

Unlock augmented reality of everyday objects and places with the Blippar app.

3.9 Aurasma

by creating animated and interactive boards

prepare interactive lecture notes or handouts

interactive presentation of albums or details about activities such as observation projects, experiments ( Figure 14 ).

Educational use of Aurasma app.

According to Onder [ 21 ], the Aurasma application draws attention with its ability to provide AR environments and opportunities to teachers and students, ease of use, support for distance education, creating individualized learning environments and being used as an evaluation tool.

This research is an example of a literature review. A literature review is a search and evaluation of the available literature in your given subject or chosen topic area [ 22 ]. At the end of the study, it was emphasized that the prepared sections should be carefully read by the educators and put into practice in their lessons. In addition it was also pointed out that it should be preferred in order to communicate effectively with students by interacting in real time, especially during the pandemic process.

5. Conclusion and suggestions

In this research, a detailed analysis of the augmented reality environments and applications that are frequently used in the design of learning and teaching environments in the education sector with the digitalization process is included. As the general results of the research; today, with the introduction of technologies into educational environments, different tools and materials have begun to be used in teaching methods. In this context, it is seen that the inclusion of mobile tools and mobile applications in learning environments has become widespread recently. With this rapid development in mobile technologies, new media environments, in which interactivity increases, offer an increasing number of services to the user. One of the environments where this interaction is provided and which can integrate objects in virtual environments with real objects is technologies that offer “Augmented Reality (AR)”. These technologies allow virtual objects to be superimposed on real images. AR tools consist of camera, computer infrastructure, a marker and tangible objects.

One of the most important sectors in which augmented reality technologies are used is the education area. Augmented reality applications help students understand abstract concepts in the learning and teaching process; it provides environments where students can share information within the group. In addition, it has been supported by studies in the literature that these environments significantly increase students’ learning. In addition, it was emphasized that augmented reality increases the interests, motivations and experiences of students in the field of education and plays a role in transferring the knowledge and skills gained in the virtual environment to real environments.

In all this context; increasing the use of learning environments of augmented reality environments and applications, where the effectiveness of its use in education has been determined to this degree, in different levels and course contents is the most important suggestions of this research.

• 1. Akkoyunlu, B. (2002). Eğitimde İnternet Kullanımı. İstanbul: Ceren and BİTAV.
• 2. Bulun, M., Gülnar, B., and Güran, M. S. (2004). Eğitimde Mobil Teknolojiler. The Turkish Online Journal of Educational Technology, 3(2), 165-169.
• 3. Milgram, P., Takemura, H., Utsumi, A. and Kishino, F. (1994). Augmented reality: A class of displays on the reality-virtuality continuum. Telemanipulator and Telepresence Technologies, SPIE, 2351, 282-292. DOI: 10.1117/12.197321
• 4. Azuma, R. (1997). A survey of augmented reality. Presence: Teleoperators and Virtual Environments, 6, 355-385. DOI: 10.1162/pres.1997.6.4.355
• 5. Bronack, S., Sanders, R., Cheney, A., Riedl, R., Tashner, J., and Matzen, N. (2008). Presence pedagogy: Teaching and learning in a 3D immersive world. International Journal of Teaching and Learning in Higher Education, 20(1), 59-69
• 6. Klopfer, E. and Sheldon, J. (2010). Augmenting your own reality: Student authoring of science-based augmented reality games. New directions for youth development, 128, 85-94.
• 7. Erbas, C. and Demirer, V. (2015). Eğitimde Sanal ve Arttirilmiş Gerçeklik Uygulamaları. In book: Eğitim Teknolojileri Okumaları 2015 Chapter: 7
• 8. Chiang, T.H.C., Yang, S.J.H. and Hwang, G.J. (2014). An augmented reality-based Mobile learning system to improve students’ learning achievements and motivations in Natural science inquiry activities, Educational Technology and Society, 17(4), 352-365.
• 9. Ibáñez, M. B., DiSerio, Á., Villarána, D. and Kloosa, C. (2014). Experimenting with electromagnetism Using augmented reality: Impact on flow student experience and educational effectiveness, Computers and Education, 71, 1-13.
• 10. Lin, T. J., Duh, H. B. L., Li, N., Wang, H. Y. and Tsai, C. C. (2013). An investigation of Learners’ collaborative knowledge construction performances and behavior patterns In an augmented reality simulation system, Computers and Education, 68, 314-321. 7
• 11. Wu, H. K., Lee, S. W. Y., Chang, H. Y. and Liang, J. C. (2013). Current status, opportunities and challenges of augmented reality in education, Computers and Education, 62, 41-49.
• 12. Information Technology and Communication Agency Report (2020). Retreived from Information and Communication Technology in Special Needs Education report | European Agency for Special Needs and Inclusive Education ( european-agency.org )
• 13. Sezgin, S. (2016), “Eğitimde Giyilebilir Teknolojiler: Fırsatlar ve Eğilimler”, Education Faculty Journal of Mehmet Akif Ersoy, 40, 405-418.
• 14. MacLean, D. (2013). MoodWings: A Wearable Biofeedback Device for Real-Time Stress Intervention. Proceedings of the 6th International Conference on Pervasive Technologies Related to Assistive Environments (p. 66). ACM.
• 15. Çetin, S. (2020). Artırılmış gerçeklik uygulamalarının teknik resim dersinde ortaöğretim öğrencilerinin akademik başarıları, tutumları ve uzamsal görselleştirme becerilerine etkisi. University of Bursa Uludag Turkey.
• 16. Baysan, E, Uluyol, Ç. (2016). Arttırılmış Gerçeklik Kitabının (AG-KİTAP) Öğrencilerin Akademik Başarılarına Etkisi ve Eğitim Ortamlarında Kullanımı Hakkında Öğrenci Görüşleri. Journal of Education and Humanities: Theory and Practice, 7(14), 55-78.
• 17. Almusawi HA, Durugbo, C., Afaf, M. (2021). Innovation in physical education: Teachers’ perspectives on readiness for wearable technology integration. Computers and Education 167, DOI: 10.1016/j.compedu.2021.104185
• 18. Kucuk, S., Kapakin, S. ve Goktas, Y. (2015). Tıp fakültesi öğrencilerinin mobil artırılmış gerçeklikle anatomi öğrenimine yönelik görüşleri. Journal of Higher Education and Science, 5(3), 316-323.
• 19. Balak, MV, Kısa, M. (2016). Artırılmış Gerçeklik Teknolojisinin Teknik Resim Eğitimi Üzerindeki Etkilerinin Arastırılması, Harran University Journal of Engineering, 2
• 20. Squire, K. D. and Jan, M. (2007). Mad City mystery: Developing scientific argumentation skills with a place-based augmented reality game on handheld computers. Journal of science education and technology, 16(1), 5-29.
• 21. Onder, R. Eğitimde artırılmış gerçeklik uygulamaları: Aurasma ve Color Mix. (2017) Retreived from https://docplayer.biz.tr/105856887-Egitimde-artirilmis-gerceklik-uygulamalari-aurasma-ve-color-mix.html
• 22. Arshed, N., and Danson, M. (2015). The literature review. In R. MacIntosh, and K. D. O’Gorman (Eds.), Research Methods for Business and Management: A Guide to to Writing your Dissertation (Pp. 31-49). Goodfellow Publishers.

© 2021 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution 3.0 License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Continue reading from the same book

Published: 16 March 2022

By Olika Moila, Andile Mji and Sibongile Simelane-Mni...

283 downloads

By Sarita Ahlawat, Purnima Sharma, Ankita Pandey, Dur...

410 downloads

By George Nehmetallah, Logan Williams and Thanh Nguye...

868 downloads

IntechOpen Author/Editor? To get your discount, log in .

Discounts available on purchase of multiple copies. View rates

Local taxes (VAT) are calculated in later steps, if applicable.

Support: [email protected]

Enhancing educational resources through augmented reality-based assessment materials: a comprehensive study

New citation alert added.

This alert has been successfully added and will be sent to:

You will be notified whenever a record that you have chosen has been cited.

To manage your alert preferences, click on the button below.

New Citation Alert!

Please log in to your account

Information & Contributors

Bibliometrics & citations, index terms.

Computing methodologies

Computer graphics

Graphics systems and interfaces

Mixed / augmented reality

Recommendations

Multimodal augmented reality: the norm rather than the exception.

Augmented reality (AR) is commonly seen as a technology that overlays virtual imagery onto a participant's view of the world. In line with this, most AR research is focused on what we see . In this paper, we challenge this focus on vision and make a case ...

Creating interactive physics education books with augmented reality

Augmented Books show three-dimensional animated educational content and provide a means for students to interact with this content in an engaging learning experience. In this paper we present a framework for creating educational Augmented Reality (AR) ...

Haptics in Augmented Reality

An augmented reality system merges synthetic sensory information into a user's perception of a three-dimensional environment. An important performance goal for an augmented reality system is that the user perceives a single seamless environment. In most ...

Information

Published in.

Association for Computing Machinery

New York, NY, United States

Publication History

Permissions, check for updates, author tags.

• assessment materials
• augmented reality
• educational resource
• interaction
• Research-article
• Refereed limited

Acceptance Rates

Contributors, other metrics, bibliometrics, article metrics.

• 0 Total Citations
• 5 Total Downloads
• Downloads (Last 12 months) 5
• Downloads (Last 6 weeks) 5

View Options

Login options.

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

View options.

View or Download as a PDF file.

View online with eReader .

HTML Format

View this article in HTML Format.

Share this Publication link

Copying failed.

Share on social media

Affiliations, export citations.

• Please download or close your previous search result export first before starting a new bulk export. Preview is not available. By clicking download, a status dialog will open to start the export process. The process may take a few minutes but once it finishes a file will be downloadable from your browser. You may continue to browse the DL while the export process is in progress. Download
• Download citation
• Copy citation

We are preparing your search results for download ...

We will inform you here when the file is ready.

Your file of search results citations is now ready.

Your search export query has expired. Please try again.

IEEE Account

• Change Username/Password
• Update Address

Purchase Details

• Payment Options
• Order History
• View Purchased Documents

Profile Information

• Communications Preferences
• Profession and Education
• Technical Interests
• US & Canada: +1 800 678 4333
• Worldwide: +1 732 981 0060
• Contact & Support
• About IEEE Xplore
• Accessibility
• Terms of Use
• Nondiscrimination Policy
• Privacy & Opting Out of Cookies

A not-for-profit organization, IEEE is the world's largest technical professional organization dedicated to advancing technology for the benefit of humanity. © Copyright 2024 IEEE - All rights reserved. Use of this web site signifies your agreement to the terms and conditions.

Academia.edu no longer supports Internet Explorer.

To browse Academia.edu and the wider internet faster and more securely, please take a few seconds to  upgrade your browser .

•  We're Hiring!
•  Help Center

Augmented Reality

• Most Cited Papers
• Most Downloaded Papers
• Newest Papers
• Last »
• Virtual Reality (Computer Graphics) Follow Following
• Human Computer Interaction Follow Following
• Mobile Augmented Reality Follow Following
• Interaction Design Follow Following
• Computer Graphics Follow Following
• Visualization Follow Following
• Virtual Worlds Follow Following
• Augmented Reality Art Follow Following
• Computer Vision Follow Following
• Digital Humanities Follow Following

Enter the email address you signed up with and we'll email you a reset link.

• Academia.edu Journals
•   We're Hiring!
•   Help Center
• Find new research papers in:
• Health Sciences
• Earth Sciences
• Cognitive Science
• Mathematics
• Computer Science
• Academia ©2024

augmented reality Recently Published Documents

Total documents.

• Latest Documents
• Most Cited Documents
• Contributed Authors
• Related Sources
• Related Keywords

8 Wildlife Tourism in 2150: Uplifted Animals, Virtual and Augmented Reality and Everything In-between

Augmented reality in vocational training: a systematic review of research and applications, i want it my way the effect of perceptions of personalization through augmented reality and online shopping on customer intentions to co-create value, integration of augmented reality smart glasses as assembly support: a framework implementation in a quick evaluation tool, ‘what lies behind the filter’ uncovering the motivations for using augmented reality (ar) face filters on social media and their effect on well-being, a survey on haptic technologies for mobile augmented reality.

Augmented reality (AR) applications have gained much research and industry attention. Moreover, the mobile counterpart—mobile augmented reality (MAR) is one of the most explosive growth areas for AR applications in the mobile environment (e.g., smartphones). The technical improvements in the hardware of smartphones, tablets, and smart-glasses provide an advantage for the wide use of mobile AR in the real world and experience these AR applications anywhere. However, the mobile nature of MAR applications can limit users’ interaction capabilities, such as input and haptic feedback. In this survey, we analyze current research issues in the area of human-computer interaction for haptic technologies in MAR scenarios. The survey first presents human sensing capabilities and their applicability in AR applications. We classify haptic devices into two groups according to the triggered sense: cutaneous/tactile : touch, active surfaces, and mid-air; kinesthetic : manipulandum, grasp, and exoskeleton. Due to MAR applications’ mobile capabilities, we mainly focus our study on wearable haptic devices for each category and their AR possibilities. To conclude, we discuss the future paths that haptic feedback should follow for MAR applications and their challenges.

Towards Augmented Reality Driven Human-City Interaction: Current Research on Mobile Headsets and Future Challenges

Interaction design for Augmented Reality (AR) is gaining attention from both academia and industry. This survey discusses 260 articles (68.8% of articles published between 2015–2019) to review the field of human interaction in connected cities with emphasis on augmented reality-driven interaction. We provide an overview of Human-City Interaction and related technological approaches, followed by reviewing the latest trends of information visualization, constrained interfaces, and embodied interaction for AR headsets. We highlight under-explored issues in interface design and input techniques that warrant further research and conjecture that AR with complementary Conversational User Interfaces (CUIs) is a crucial enabler for ubiquitous interaction with immersive systems in smart cities. Our work helps researchers understand the current potential and future needs of AR in Human-City Interaction.

Edge-assisted Collaborative Image Recognition for Mobile Augmented Reality

Mobile Augmented Reality (AR), which overlays digital content on the real-world scenes surrounding a user, is bringing immersive interactive experiences where the real and virtual worlds are tightly coupled. To enable seamless and precise AR experiences, an image recognition system that can accurately recognize the object in the camera view with low system latency is required. However, due to the pervasiveness and severity of image distortions, an effective and robust image recognition solution for “in the wild” mobile AR is still elusive. In this article, we present CollabAR, an edge-assisted system that provides distortion-tolerant image recognition for mobile AR with imperceptible system latency . CollabAR incorporates both distortion-tolerant and collaborative image recognition modules in its design. The former enables distortion-adaptive image recognition to improve the robustness against image distortions, while the latter exploits the spatial-temporal correlation among mobile AR users to improve recognition accuracy. Moreover, as it is difficult to collect a large-scale image distortion dataset, we propose a Cycle-Consistent Generative Adversarial Network-based data augmentation method to synthesize realistic image distortion. Our evaluation demonstrates that CollabAR achieves over 85% recognition accuracy for “in the wild” images with severe distortions, while reducing the end-to-end system latency to as low as 18.2 ms.

Extending the technology acceptance model to explain how perceived augmented reality affects consumers' perceptions

Learning with augmented reality: impact of dimensionality and spatial abilities, export citation format, share document.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Springer Nature - PMC COVID-19 Collection

Trends in the Use of Augmented Reality, Virtual Reality, and Mixed Reality in Surgical Research: a Global Bibliometric and Visualized Analysis

1 Orthopedic Department, Second Hospital of Shanxi Medical University, Taiyuan, 030001 China

2 Cardiology Department, Second Hospital of Shanxi Medical University, Taiyuan, China

Associated Data

No additional data are available.

There have been many major developments in the use of augmented reality (AR), virtual reality (VR), and mixed reality (MR) technologies in the context of global surgical research, yet few reports on the trends in this field have been published to date. This study was therefore designed to explore these worldwide trends in this clinically important field. Relevant studies published from 1 January 2009 through 13 October 2020 were retrieved from the Science Citation Index-Expanded (SCI-E) tool of the Web of Science database. Bibliometric techniques were then used to analyze the resultant data, with visual bibliographic coupling, co-authorship, co-citation, co-occurrence, and publication trend analyses subsequently being conducted with GraphPad Prism 8 and with the visualization of similarities (VOS) software tool. There is no patient and public involved. In total, 6221 relevant studies were incorporated into this analysis. At a high level, clear global annual increases in the number of publications in this field were observed. The USA made the greatest contributions to this field over the studied period, with the highest H-index value, the most citations, and the greatest total link strength for analyzed publications. The country with the highest number of average citations per publication was Scotland. The Surgical Endoscopy And Other Interventional Techniques journal contributed the greatest number of publications in this field. The University of London was the institution that produced the greatest volume of research in this field. Overall, studies could be broadly classified into five clusters: Neurological Research, Surgical Techniques, Technological Products, Rehabilitative Medicine, and Clinical Therapy. The trends detected in the present analysis suggest that the number of global publications pertaining to the use of AR, VR, and MR techniques in surgical research is likely to increase in the coming years. Particular attention should be paid to emerging trends in related fields including MR, extended reality, head-mounted displays, navigation, and holographic images.

Introduction

Virtual reality (VR) adds some substances and elements of the real world to the virtual space to make people feel the existence of the real world. Augmented reality (AR) adds a certain amount of virtual components and elements in the real space to make people feel that they have entered the virtual world. Mixed reality (MR) is the fusion of real space and virtual space, and artificial interaction in the real world and virtual world. The concept of VR first emerged in the 1960s, when Tom Furness developed technology that enabled jet pilots to access three-dimensional (3D) avionic data [ 1 ]. Many recent advances in VR and the related fields of AR and MR have been made in commercial contexts, with a range of VR systems harboring integrated haptic feedback peripherals having first been marketed to consumers in the 1980s and 1990s, although these platforms were expensive and suffered from relatively poor performance [ 2 ]. The limited commercial success of these VR/AR technologies led to their initial utilization primarily in academic or corporate contexts, leading to increased interest in their medical application [ 2 ]. The emergence of MR platforms has been particularly valuable in this space, as this hybrid of AR and VR technologies overcomes the respective limitations of these technologies by enabling users to interact with 3D data packets without excluding the surrounding real-world environment [ 3 ]. This hybrid MR approach thus enables the simultaneous manipulation of both real and virtual environments [ 4 , 5 ]. Sustained advances in the computing power underlying these altered reality-based technologies can subvert many healthcare-related fields, with surgery being a particularly promising context for the application of these novel tools [ 6 ]. Indeed, the combination of AR, VR, or MR techniques with robotic-assisted surgery has the potential to greatly improve surgical precision while decreasing operative durations and alleviating surgeon fatigue [ 3 ]. These technologies also offer clear value in contexts including medical education, doctor-patient communication, preoperative planning, intraoperative guidance and navigation, teleconsultation, and other surgery-related fields [ 7 ]. Consistent with such value, total spending on AR and VR products is forecast to exceed $215 billion in 2021, and the global healthcare AR and VR market is forecast to reach $5.1 billion by 2025 [ 2 ]. Many major medical and technological companies are thus seeking to develop novel approaches to combining surgical techniques and computer imaging modalities to achieve key breakthroughs, making a timely analysis of the application of VR, AR, and MR technologies in the field of surgical research of key importance. Few such studies have been published to date, highlighting the need to examine current and future trends in this research space in an effort to both understand and guide the directions of studies in this innovative and exciting field.

Peer-reviewed publications are a primary output associated with scientific research, and can be readily analyzed to gauge contributions to a given research field [ 8 ]. Bibliometric analyses and corresponding visual mapping techniques have been routinely utilized in recent years to assess scientific progress at a high level [ 9 ]. These analyses focus on the quantitative and qualitative evaluation of trends in a given research community over time based on data obtained from key online databases [ 10 ], comparing the relative contributions of different scholars, journals, institutions, and countries to a field of interest [ 11 ]. Bibliometric analyses have previously been leveraged to guide health policy and clinical guideline development, and to improve understanding of a range of conditions and technologies [ 12 ], including Middle East respiratory syndrome coronavirus (MERS-CoV) [ 13 ], exosomes [ 14 ], retinal regeneration [ 15 ], and spinal ultrasound [ 16 ]. The present study was therefore designed to assess current and future trends pertaining to the use of AR, VR, and MR technologies in the field of surgery and to identify emerging topics in this research space that are the focus of growing scholarly investigation.

For this analysis, VR was defined as a computer-generated 3D-simulated environment in which users are able to fully immerse themselves, while AR was defined as the projection of computer-generated images onto real-world entities, and MR was defined as the AR-like projection of virtual objects onto real-world entities such that these objects were responsive and spatially aware such that users were able to interact with these 3D projections [ 6 , 7 ] (Fig.  1 ).

A map highlighting numbers of studies pertaining to the use of these technologies in surgery that have been published in different counties, with color depth corresponding to the numbers of publications per country

Materials and Methods

Data sources.

Bibliometric analyses were conducted by using the Science Citation Index-Expanded of the Web of Science (WOS), as this is the standard database used for such analyses [ 17 ]. No ethical oversight was required for this study, as it did not involve human subjects or preclinical animal models.

Search Strategy

The WOS database was searched for relevant studies published from January 2009 to October 2020 using the following search terms: (TS = “virtual reality” OR “augmented reality” OR “mixed reality”) AND (Language = English) AND (Document type = Articles OR Reviews). Key publication–related information including countries or regions of origin, institutions, funding sources, and research directions were then obtained from the WOS database. Studies were excluded from this analysis if they were unrelated to Surgery, including studies of Chemistry, Computer Theory, and Electrical Engineering.

Data Collection

Records for all studies identified through the above search strategy including title, year of publication, author names, institutional affiliations, nationality, keywords, abstracts, and publishing journals were saved in a.txt file format using the WOS and were opened in Microsoft Excel 2019 for analysis. Two co-authors (ZJ and YN) independently extracted data from these studies, with any discrepancies in extracted data being resolved through discussion or by contacting relevant experts.

Bibliometric Analysis

Bibliometric analyses leverage bibliometric theory to analyze relevant studies through statistical and mathematical approaches, thereby offering high-level insights into specific areas of scientific interest [ 18 , 19 ]. Intrinsic WOS tools can be used to assess the characteristics of studies included in bibliometric analyses. H-index values, for example, serve as key bibliometric indicators of scientific impact. H-values used to compute the H-index are based upon the number of times that publications from a given institution or researcher have been cited by other publications, thereby reflecting both the number of publications and the number of citations per published article [ 20 ]. The impact factor (IF) values for all journals in the present analysis were derived from the 2019 Journal Citation Reports.

Visualized Analysis

Past and predicted changes in publication volume over time were evaluated using a curve fitting model ( Y  =  AX  −  B ). Data pertaining to time-dependent trends in key bibliometric parameters were analyzed using GraphPad Prism (GraphPad Software Inc., CA, USA) [ 21 ]. VOS viewer (Leiden University, Leiden, Netherlands) was used for visual analyses of the bibliometric data and to conduct bibliographic coupling, co-authorship, co-citation, and co-occurrence analyses.

Global Publication Trends

Numbers of publications.

The number of scientific articles published in a particular field can offer key insight into current research trends in a given area. In total, we identified 6221 relevant studies in the WOS database published from 1 January 2009 to 13 October 2020 that met our search criteria, of which the majority (3559, 57.2%) were published within the last 5 years. We detected an overall upward trend in publications in this field over the past 12 years, suggesting that scholarly focus regarding the use of VR, AR, and MR in surgical contexts will continue to grow rapidly for the foreseeable future.

Country-Specific Contributions

The top 20 nations to have contributed to this field of research were next identified (Fig.  2 ). The greatest number of studies and reviews included in this analysis was published by research groups from the USA (1883, 30.27%), followed by those from England (677, 9.82%), Canada (600, 9.64%), Germany (509, 8.18%), and Italy 444, 7.14%) (Figs. ​ (Figs.3, 3 , ​ ,4, 4 , and ​ and5 5 ).

The total number of publications analyzing the use of AR, VR, and MR in surgical contexts from the top most prolific nations

Model fitting curves were used to track trends in the future publication of studies pertaining to this topic of interest

Annual numbers of publications related to the use of VR, AR, and MR in surgery published over the past 11 years. Institutional, research orientation, funding source, author, and journal contributions to this field of research

Institutions publishing the most studies pertaining to the use of AR, VR, and MR in surgery ranked according to number of publications

Institutional Contributions

The top 20 institutions and organizations to have engaged in research regarding the use of AR, VR, and MR in surgery are shown in Fig.  6 . Of these, the University of London has published the greatest number of studies (188), followed by the University of Toronto (44), and Imperial College London (35).

Research orientations publishing the most studies pertaining to this field of interest ranked according to numbers of publicationss

Contributions of Research Disciplines

The top three research areas that explored the use of AR, VR, and MR in surgery were the Surgery, Neuroscience, and Rehabilitation fields (Fig.  7 ).

The most cited funding sources for studies pertaining to this field of interest ranked according to numbers of citing publications

Funding Source Contributions

The top 4 most commonly noted funding sources for studies included in this analysis were the US Department of Health and Human Services (413), the National Institutes of Health (NIH, USA) (394), the National Natural Science Foundation of China (175), and the National Institute for Health Research (NIHR)(112), as shown in Fig.  8 .

The authors publishing the most studies pertaining to this field of interest ranked according to numbers of publications

Contributions of Authors

The top 3 authors in this field were Aggarwal R, Konge L, and Darzi A, who respectively published 59, 57, and 50 articles in this research space (Fig.  9 ).

Journals publishing the most studies pertaining to this field of interest ranked according to numbers of publications. Citation frequencies and H-index values for individual nations

Journal Contributions

The top 5 journals publishing the most studies pertaining to the use of AR, VR, and MR in surgery identified in the present analysis were Surgical Endoscopy And Other Interventional Techniques ( n  = 214), International Journal Of Computer Assisted Radiology And Surgery ( n  = 137), Frontiers In Human Neuroscience ( n  = 133), IEEE Transactions On Neural Systems And Rehabilitation Engineering ( n  = 72), and the International Journal Of Medical Robotics And Computer Assisted Surgery ( n  = 71) (Fig.  10 , Tables ​ Tables1 1 and ​ and2 2 ).

Total numbers of citations for studies pertaining to AR, VR, and MR use in surgery from the indicated countries

The number of publications in the top 20 countries or regions

The number of publications of the top 20 institutions

Quality of Publications from Individual Countries

Total citation frequency.

Studies from the USA had the greatest number of citations (36,577), followed by those from England (13,969), Canada (12,446), Australia (8290), and the Netherlands (8187) (Fig.  11 ).

The average numbers of citations per publication for the indicated countries

Average Citation Frequency

Publications from Scotland exhibited the greatest average number of citations per study (33.08), followed by those from Austria (32.91), the Netherlands (80.74), Australia (22.59), and Sweden (22.52) (Fig.  12 ).

H-index values for publications from the indicated countries. Bibliographic coupling analysis of global research trends pertaining to studies of the use of AR, VR, and MR technologies in surgery.

H-Index Values

Publications included in this study from the USA had the highest H-index value (81), followed by those from England and Canada (55), Australia (41), the Netherlands (41), and Italy (40) (Fig.  13 ).

The mapping of 308 identified journals that have published studies in this research field

Bibliographic Coupling Analysis

Bibliographic coupling is a metric that can be used to detect similarity relationships between studies based upon their citations. When two studies both cite another study, this suggests that they share a related research focus. Bibliographic coupling analyses pertaining to journals, institutions, and countries were associated with relevant studies using VOS viewer, with individual spheres in the resultant mapping diagrams corresponding to the corresponding journals/institutions/countries, and with sphere size corresponding to total linkage strength (TLS) between individual elements, with greater numbers of links being indicative of stronger relationships between pairs of elements [ 22 ]. This approach can also be employed to conduct co-authorship, co-citation, and co-occurrence analyses.

The TLS for 308 journals publishing relevant articles pertaining to the use of VR, AR, and MR in surgery are shown in Fig.  14 . The journals with the greatest TLS values were Surgical Endoscopy And Other Interventional Techniques (Impact Factor, IF = 3.149, 2019; TLS = 88,700), Journal Of Neuroengineering and Rehabilitation (IF = 3.519, 2019; TLS = 48.242), Frontiers In Human Neuroscience (IF = 2.673, 2019; TLS = 30.797), American Journal of Surgery (IF = 2.125, 2019; TLS = 25,893), and Annals of Surgery (IF = 10.13, 2019; TLS = 25.739).

The mapping of 64 countries associated with publications in this field. Lines between two points indicate a relationship between the two connected items, with thicker lines corresponding to a closer link between these journals, institutions, or countries. Co-authorship analysis of global studies pertaining to the use of AR, VR, and MR in surgery

Institutions

Data from 676 institutions with 5 or more relevant publications were visualized using VOS viewer (Fig.  15 ), revealing the institutions with the greatest TLS values to be University of Toronto (TLS = 182,489); Imperial College of Science, Technology, and Medicine (TLS = 120,917); McGill University (TLS = 100,914); University of Washington (TLS = 83,718); and Technische Universiteit Delft (TLS = 60,754).

The mapping of 676 institutions that have published research in this field

Papers from 64 countries were assessed with VOS viewer (Fig.  16 ), revealing the countries with the greatest TLS values to be the USA (TLS = 686,681), England (TLS = 389,195), Canada (TLS = 343,385), Italy (TLS = 217,182), and Australia (TLS = 201,596).

Co-authorship mapping for 581 authors associated with studies in this field of research

Co-authorship Analyses

Co-authorship analyses offer a means of assessing the relatedness of particular researchers or other items based upon numbers of co-authored publications, providing insights into collaboration among researchers, institutions, and countries in a given field of interest [ 21 ]. These analyses offer value as a means of providing researchers with valuable insights regarding potential collaborators, collaborative research group developments, and academic exchanges [ 22 ].

In total, we analyzed 581 different authors with a minimum of 5 publications using VOS viewer (Fig.  17 ). The authors with the greatest TLS were Calabro Rocco Salvatore (TLS = 133), De Luca Rosaria (TLS = 104), Ahmed Kamran (TLS = 94), Darzi Ara (TLS = 93), and Bramanti Placido (TLS = 89).

Co-authorship mapping was conducted for 676 institutions with studies in this field of research

In total, VOS viewer was used to analyze publications from 676 institutions (Fig.  18 ). The institutions with the greatest TLS values were the University of Toronto (TLS = 268), McGill University (TLS = 180), Harvard University (TLS = 138), Tel Aviv University (TLS = 129), and Johns Hopkins University (TLS = 128).

Co-authorship mapping was conducted for 99 countries with studies in this field of research. Point sizes correspond to co-authorship frequencies, with lines between points indicating an established collaboration between authors, institutions, or countries. Thicker lines correspond to closer collaborations between any two given authors, institutions, or countries. Co-citation analysis of global research trends pertaining to the use of AR, VR, and MR in surgery

In total, 99 different countries were analyzed with VOS viewer (Fig.  19 ). The countries with the greatest TLS values were the USA (TLS = 928), England (TLS = 645), Italy (TLS = 446), Germany (TLS = 445), and Canada (TLS = 418).

An author co-citation analysis was performed for studies in this research space, with points of a given color corresponding to shared research directions

Co-citation Analysis

Co-citation analyses are conducted by evaluating the number of times studies are cited together, thereby offering insights into correlations and similarities between articles and providing a firm knowledge base for a given field of interest.

In total, 922 authors with greater than 10 publications were next analyzed (Fig.  20 ), revealing the authors with the greatest TLS values to be Aggarwal R (TLS = 21.787), Gallagher AG (TLS = 15.005), Hoffman HG (TLS = 13.695), Grantcharov TP (TLS = 13.070), and Stefanidis D (TLS = 12.829).

Co-cited journals in this research space were mapped, with point size corresponding to citation frequency. A line between two points indicates that both were cited in a given paper or journal, with shorter lines indicating a closer link between two given papers or journals. Co-occurrence analysis of global trends pertaining to the use of AR, VR, and MR technologies in the field of surgical research

Only journals with a least 20 citations were analyzed. The TLS values for these 1476 journals are shown in Fig.  21 . Among these top journals, the largest TLS values are Surgical Endoscopy And Other Interventional Techniques (TLS = 202,529), Archives Of Physical Medicine And Rehabilitation (TLS = 164,092), PLOS One (TLS = 147,507), International Journal Of Stroke (TLS = 131,524), and Neurorehabilitation And Neural Repair (TLS = 130,230).

Keywords relevant to this analysis were mapped, with the size of individual points corresponding to frequency values. Overall, ;these keywords clustered into five categories: Neurological Research (red; upper left), Surgical Techniques (green; upper right), Technological Products (yellow; lower right), rehabilitative medicine (blue, lower left), and clinical therapy (purple, lower left)

Co-occurrence Analysis

Co-occurrence analyses offer a means of summarizing large numbers of research articles and predicting future trending topics in a given field, thereby elucidating and guiding scientific progress. To perform such a co-occurrence analysis, we used VOS viewer to visualize keywords used 10 or more times in the abstracts and titles of identified publications (Fig.  22 ). Overall, we identified 952 keywords that were clustered into five rough clusters pertaining to Neurological Research, Surgical Techniques, Technological Products, Rehabilitative Medicine, and Clinical Therapy, suggesting these to be the most prominent topics in this research field to date. The most frequently utilized keywords in the “Neurological Research” cluster included virtual reality, Alzheimer’s disease, Parkinson’s disease, brain, and movement. In the “Surgical Technique” cluster, the most commonly utilized keywords were performance, validation, simulation, surgery, and education, while in the “Technological Products” cluster, they were augmented reality, system, visualization, accuracy, and guidance. Similarly, the primary keywords for the “Rehabilitative Medicine” cluster were virtual reality, rehabilitation, balance, stroke, and exercise, while for the “Clinical Therapy” cluster, they were children, therapy, video games, distraction, and interventions.

Keyword distributions based upon mean appearance frequency, with keywords in blue appearing earlier than keywords in red or yellow

VOS viewer was then used to color code these keywords based upon the frequency with which they appeared in the studies included in this analysis (Fig.  23 ), with blue indicating that keyword appeared at earlier time points and red indicating a later appearance. Prior to 2005, more studies focused on keywords including “Virtual Reality,” “Augmented Reality,” “Simulation,” “Rehabilitation,” and “Performance,” whereas rising trends suggest that “Mixed Reality,” “Extended Reality,” “Head-Mounted Display,” “Navigation,” and “Holographic Image” keywords are likely to be more frequently represented in future studies. A density visualization map was then generated for keywords included in this analysis, with the color corresponding to each keyword corresponding to the number of times it occurred in analyzed studies such that warmer colors indicate more frequent keyword utilization [ 23 ].

A density visualization map was generated based upon the times a given keyword occurred, which defined the area for a given term, with warmer colors corresponding to greater numbers of occurrences

Trends in VR, AR, and MR-Related Surgery Research

Bibliometric and visualized analyses offer opportunities to assess the current status of a given research field and to gauge future trends. There have been significant increases in the number of studies assessing the surgical applications of VR, AR, and MR over the last 5 years. Researchers from 99 different countries have published studies in this research space over the past decade, and our analysis suggests that this is an expanding area of interest that will be associated with the publication of rising numbers of high-quality articles in the coming years.

Global Publication Quality and Status

A global analysis of publications in this field revealed that the USA is responsible for the highest overall number of published papers, with the US Department of Health and Human Services being one of the most prominent funding sources in this research space. Perhaps unsurprisingly, surgical medicine was the most heavily researched branch of this field, and the USA exhibited the greatest contribution to this field with respect to total citations, H-index values, and publication numbers. England additionally exhibited the highest total citation counts of any countries contributing to this field, highlighting the USA and England as global leaders in the use of VR, AR, and MR in surgical medicine. Canada ranked third in terms of publication volume, H-index, and total citations in this field, while China was the sixth most published country but was only 16th with respect to the average number of citations per publication. This may be attributable to the historical emphasis on quantity rather than quality in the Chinese academic sector. However, the continued expansion of Chinese government-led medical research funding is likely to improve average study quality, helping China establish itself as a global research leader in this field.

When we conducted bibliographic coupling analyses of journals, institutions, and countries in this research space, we identified Surgical Endoscopy And Other Interventional Techniques , International Orthopaedics , International Journal Of Computer Assisted Radiology And Surgery , and Frontiers In Human Neuroscience as popular journals for the publication of studies evaluating the use of VR, AR, and MR in surgical contexts. Monitoring publications in these journals may thus be an effective means of tracking research trends in this field. We additionally found that the University of London was a leader in this field of research, while Imperial College London ranked third overall, consistent with the strong contributions of England to this research area. Harvard University and the University of California System were the top research institutions in the USA, while the University of Toronto in Canada ranked second in an institutional coupling analysis. These findings emphasize the close relationship between top-ranked universities and the overall academic level of a given country. We found that Aggarwal R, Konge L, and Darzi A were the researchers who have made the greatest contributions to this field, suggesting that monitoring their future publications will highlight key trends for future development in this space.

These analyses suggest that the University of Toronto, McGill University, and Harvard University represent optimal collaborative partners and co-authors for individuals conducting research on the use of VR, AR, and MR in surgery. Surgical Endoscopy And Other Interventional Techniques , Archives Of Physical Medicine And Rehabilitation , and PLOS One were the three most frequently cited journals in the field, while Aggarwal R, Gallagher AG, and Hoffman HG were the most highly cited and widely recognized researchers conducting these studies.

VR, AR, and MR Surgical Research Focus Areas

A co-occurrence network diagram generated herein highlighted five key research trends in this field: neurological research, surgical techniques, technological products, rehabilitative medicine, and clinical therapy.

At a more focused level, MR technologies have been deployed in the context of surgical education, enabling surgeons to more rapidly expand their skillsets to ensure career growth [ 24 ]. New AR, VR, and MR tools are also used with increasing frequency to ensure that patients and their families understand pertinent surgical procedures, reducing the potential for doctor-patient miscommunication [ 25 , 26 ]. Preoperative imaging technologies offer a clear view of the surgical site and surrounding tissues, enabling safer and more precise surgical planning [ 27 , 28 ], and VR, AR, and MR tools are therefore being used with increasing frequency in clinical surgery in disciplines such as neurosurgery, orthopedics, plastic surgery, otorhinolaryngology, thoracic surgery, and urology [ 29 , 30 ]. These approaches can reduce operative durations, radiation doses, and incision sizes, thereby providing patients with increasingly high-quality care. VR-based clinical rehabilitation platforms have also been built and applied in the field of orthopedics [ 31 ], while AR-based developments have improved teleconsultation quality and medical resource sharing [ 32 ].

At a high level, approaches to combining AR and 3D printing technologies appear to be a growing area of interest in surgical medicine [ 33 ], particularly in neurosurgery and orthopedics. At the same time, image-based surgical navigation strategies are being increasingly employed in the contexts of vascular surgery, brain pacemakers, and epilepsy electrode implantation [ 34 ]. AR-based head-mounted display (HMD) units leveraging holographic imaging technology are also being employed in top operating rooms throughout the world [ 35 ]. Growing trends suggest that the development of AR-based semi- or fully automated robots with integrated scanners and surgical arms is likely to dramatically influence the surgical field in the coming years [ 36 ]. Novel extended reality (XR) technologies incorporating elements of VR, AR, or MR are forecast to be used with rising frequency by surgeons as these tools undergo further development [ 3 , 37 ], and the integration of cloud-based services and healthcare data will ensure that these XR platforms are fully optimized to facilitate high-quality patient care. The results of this bibliometric analysis emphasize the crucial need for high-quality studies in this research space, highlighting key areas for additional experimental or clinical research and funding infusion as appropriate.

Colors in the generated density visualization map correspond to the frequencies of occurrence for particular keywords. This analysis suggested that Surgical Techniques and Technological Products are likely to be popular research subjects in this field, with many studies pertaining to extended reality, navigation, and holographic images having emerged in recent years.

Strengths and Limitations of This Study

By employing bibliometric and visualization techniques to highlight trends in the use of AR, VR, and MR technologies in the field of surgical research, we have herein provided a robust overview of this clinically important field. It is worth mentioning that, through the processing of GraphPad Prism and VOSviewer, the content and results of the above research can be presented to readers more intuitively and vividly.

Even so, there are certain limitations to our analyses. For one, we excluded studies not published in English, potentially introducing a language bias into our findings. In addition, we only analyzed studies published as of 13 October 2020, and our analyses thus fail to reflect more recent studies in this field. Future analyses should therefore be extended to the non-English literature to better understand global trends in this research field. At the same time, the literature we selected was all articles or reviews, and did not include conferences, books, and other forms, which would make our research results lack certain comprehensiveness. Similarly, when we searched, we selected the Web of Science database and did not search other databases, which would also lead to certain deviation in the inclusion of literature. We should avoid the above problems in the future research.

Bias Analysis

In the selection of research content, this paper did not include other research results other than treatises and reviews, and only English articles were selected instead of articles in other languages, and only the WOS database was referred to in the selection of database, so it was suspected of selection bias. Because the data extraction and statistics of the paper were completed by the two authors, the inherent investigator bias was suspected.

Conclusions

Overall, our study provides high-level overview of current trends in the application of AR, VR, and MR technologies in surgical research. While the USA remains the clear leader in this field, followed by England, Australia is actively growing as a major in this field that prioritizes academic quality and is thereby actively influencing associated scientific norms. Key areas of future research are forecast to include extended reality, navigation, and holographic image–related technologies. Through such analyses and the predicted overall expansion in the number of studies in this field, we predict that these technologies will continue to benefit medicine and to thereby improve the available quality of care for all patients.

Abbreviations

Author Contribution

Xin Lv and Bin Wang conceived the research; Jing Zhang and Na Yu collected the data and prepared the manuscript; Jing Zhang and Na Yu analyzed the data; Na Yu provided techniques and advice.

This study was supported by a grant from National Nature Science Foundation of China (No. 81802204).

Data Sharing

Declarations.

Ethical approval of the study was not necessary.

The authors declare no competing interests.

No patient involved. This review was not registered. The protocol was not prepared.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Jing Zhang and Na Yu contributed equally to this work.

Researchers are using AI to create better virtual reality experiences

Virtual and augmented reality headsets are designed to place wearers directly into other environments, worlds and experiences.

While the technology is already popular among consumers for its immersive quality, there could be a future where the holographic displays look even more like real life.

In their own pursuit of these better displays, the Stanford Computational Imaging Lab has combined their expertise in optics and artificial intelligence. Their most recent advances in this area are detailed in a paper published Nov. 12 in Science Advances and work that will be presented at SIGGRAPH ASIA 2021 in December.

At its core, this research confronts the fact that current augmented and virtual reality displays only show 2D images to each of the viewer’s eyes, instead of 3D – or holographic – images like we see in the real world.

“They are not perceptually realistic,” explained Gordon Wetzstein , associate professor of electrical engineering and leader of the Stanford Computational Imaging Lab. Wetzstein and his colleagues are working to come up with solutions to bridge this gap between simulation and reality while creating displays that are more visually appealing and easier on the eyes.

The research published in Science Advances details a technique for reducing a speckling distortion often seen in regular laser-based holographic displays, while the SIGGRAPH Asia paper proposes a technique to more realistically represent the physics that would apply to the 3D scene if it existed in the real world.

Bridging simulation and reality

In the past decades, image quality for existing holographic displays has been limited. As Wetzstein explains it, researchers have been faced with the challenge of getting a holographic display to look as good as an LCD display.

One problem is that it is difficult to control the shape of light waves at the resolution of a hologram. The other major challenge hindering the creation of high-quality holographic displays is overcoming the gap between what is going on in the simulation versus what the same scene would look like in a real environment.

Previously, scientists have attempted to create algorithms to address both of these problems. Wetzstein and his colleagues also developed algorithms but did so using neural networks, a form of artificial intelligence that attempts to mimic the way the human brain learns information. They call this “ neural holography .”

“Artificial intelligence has revolutionized pretty much all aspects of engineering and beyond,” said Wetzstein. “But in this specific area of holographic displays or computer-generated holography, people have only just started to explore AI techniques.”

Yifan Peng, a postdoctoral research fellow in the Stanford Computational Imaging Lab, is using his interdisciplinary background in both optics and computer science to help design the optical engine to go into the holographic displays.

“Only recently, with the emerging machine intelligence innovations, have we had access to the powerful tools and capabilities to make use of the advances in computer technology,” said Peng, who is co-lead author of the Science Advances paper and a co-author of the SIGGRAPH paper.

The neural holographic display that these researchers have created involved training a neural network to mimic the real-world physics of what was happening in the display and achieved real-time images. They then paired this with a “camera-in-the-loop” calibration strategy that provides near-instantaneous feedback to inform adjustments and improvements. By creating an algorithm and calibration technique, which run in real time with the image seen, the researchers were able to create more realistic-looking visuals with better color, contrast and clarity.

The new SIGGRAPH Asia paper highlights the lab’s first application of their neural holography system to 3D scenes . This system produces high-quality, realistic representation of scenes that contain visual depth, even when parts of the scenes are intentionally depicted as far away or out-of-focus.

The Science Advances work uses the same camera-in-the-loop optimization strategy, paired with an artificial intelligence-inspired algorithm, to provide an improved system for holographic displays that use partially coherent light sources – LEDs and SLEDs. These light sources are attractive for their cost, size and energy requirements and they also have the potential to avoid the speckled appearance of images produced by systems that rely on coherent light sources, like lasers. But the same characteristics that help partially coherent source systems avoid speckling tend to result in blurred images with a lack of contrast. By building an algorithm specific to the physics of partially coherent light sources, the researchers have produced the first high-quality and speckle-free holographic 2D and 3D images using LEDs and SLEDs.

Transformative potential

Wetzstein and Peng believe this coupling of emerging artificial intelligence techniques along with virtual and augmented reality will become increasingly ubiquitous in a number of industries in the coming years.

“I’m a big believer in the future of wearable computing systems and AR and VR in general, I think they’re going to have a transformative impact on people’s lives,” said Wetzstein. It might not be for the next few years, he said, but Wetzstein believes that augmented reality is the “big future.”

Though augmented virtual reality is primarily associated with gaming right now, it and augmented reality have potential use in a variety of fields, including medicine. Medical students can use augmented reality for training as well as for overlaying medical data from CT scans and MRIs directly onto the patients.

“These types of technologies are already in use for thousands of surgeries, per year,” said Wetzstein. “We envision that head-worn displays that are smaller, lighter weight and just more visually comfortable are a big part of the future of surgery planning.”

“It is very exciting to see how the computation can improve the display quality with the same hardware setup,” said Jonghyun Kim, a visiting scholar from Nvidia and co-author of both papers. “Better computation can make a better display, which can be a game changer for the display industry.”

Related:  Gordon Wetzstein , associate professor of electrical engineering and leader of the Stanford Computational Imaging Lab

Related Departments

The future of Russia and Ukraine

Turning carbon pollution into ethanol

New gels could protect buildings during wildfires

• Computer Science and Engineering
• Computer Graphics and Animation
• Augmented Reality

Introduction to augmented reality

• January 2003

• Federal University of Juiz de Fora

• Laboratório Nacional de Computação Científica
• This person is not on ResearchGate, or hasn't claimed this research yet.

Abstract and Figures

Discover the world's research

• 25+ million members
• 160+ million publication pages
• 2.3+ billion citations

• Fritz Steuer

• Sarthak Jain
• Shaveta Arora
• Sugeng Sungkono
• Vepi Apiati
• Satya Santika
• Adrian Indra Arrohman
• Ahmad Zakki Falani

• Abu Na'im Rohman

• Lisa Maresch
• Jakob Kindl
• Markus Exler

• Samuel Boivin

• Erik Eckstein

• Proceedings of SPIE

• Paul S. Schenker
• Antal K. Bejczy
• Stanford Ollendorf
• Proc Hum Factors Ergon Soc Annu Meet

• Ronald Azuma

• COMPUT GRAPH-UK
• Anton Fuhrmann

• Recruit researchers
• Join for free
• Login Email Tip: Most researchers use their institutional email address as their ResearchGate login Password Forgot password? Keep me logged in Log in or Continue with Google Welcome back! Please log in. Email · Hint Tip: Most researchers use their institutional email address as their ResearchGate login Password Forgot password? Keep me logged in Log in or Continue with Google No account? Sign up

AI and holography bring 3D augmented reality to regular glasses

Researchers in the emerging field of spatial computing have developed a prototype augmented reality headset that uses holographic imaging to overlay full-color, 3D moving images on the lenses of what would appear to be an ordinary pair of glasses. Unlike the bulky headsets of present-day augmented reality systems, the new approach delivers a visually satisfying 3D viewing experience in a compact, comfortable, and attractive form factor suitable for all-day wear.

“Our headset appears to the outside world just like an everyday pair of glasses, but what the wearer sees through the lenses is an enriched world overlaid with vibrant, full-color 3D computed imagery,” said  Gordon Wetzstein , an associate professor of electrical engineering and an expert in the fast-emerging field of spatial computing. Wetzstein and a team of engineers introduce their device in a new paper in the journal  Nature . Additional information about this advance is available at  this website , created by the research team.

Though only a prototype now, such a technology, they say, could transform fields stretching from gaming and entertainment to training and education – anywhere computed imagery might enhance or inform the wearer’s understanding of the world around them.

“One could imagine a surgeon wearing such glasses to plan a delicate or complex surgery or airplane mechanic using them to learn to work on the latest jet engine,”  Manu Gopakumar , a doctoral student in the Wetzstein-led  Stanford Computational Imaging lab  and co-first author of the paper said.

Barriers overcome

The new approach is the first to thread a complex maze of engineering requirements that have so far produced either ungainly headsets or less-than-satisfying 3D visual experiences that can leave the wearer visually fatigued, or even a bit nauseous at times.

“There is no other augmented reality system out there now with comparable compact form factor or that matches our 3D image quality,” said  Gun-Yeal Lee , a postdoctoral researcher in the Stanford Computational Imaging lab and co-first author of the paper.

To succeed, the researchers have overcome technical barriers through a combination of AI-enhanced holographic imaging and new nanophotonic device approaches. The first hurdle was that the techniques for displaying augmented reality imagery often require the use of complex optical systems. In these systems, the user does not actually see the real world through the lenses of the headset. Instead, cameras mounted on the exterior of the headset capture the world in real time and combine that imagery with computed imagery. The resulting blended image is then projected to the user’s eye stereoscopically.

“The user sees a digitized approximation of the real world with computed imagery overlaid. It’s sort of augmented virtual reality, not true augmented reality,” explained Lee.

These systems, Wetzstein explains, are necessarily bulky because they use magnifying lenses between the wearer’s eye and the projection screens that require a minimum distance between the eye, the lenses, and the screens, leading to additional size.

“Beyond bulkiness, these limitations can also lead to unsatisfactory perceptual realism and, often, visual discomfort,” said  Suyeon Choi , a doctoral student in the Stanford Computational Imaging lab and co-author of the paper.

To produce more visually satisfying 3D images, Wetzstein leapfrogged traditional stereoscopic approaches in favor of holography, a Nobel-winning visual technique developed in the late-1940s. Despite great promise in 3D imaging, more widespread adoption of holography has been limited by an inability to portray accurate 3D depth cues, leading to an underwhelming, sometimes nausea-inducing, visual experience.

The Wetzstein team used AI to improve the depth cues in the holographic images. Then, using advances in nanophotonics and waveguide display technologies, the researchers were able to project computed holograms onto the lenses of the glasses without relying on bulky additional optics.

A waveguide is constructed by etching nanometer-scale patterns onto the lens surface. Small holographic displays mounted at each temple project the computed imagery through the etched patterns which bounce the light within the lens before it is delivered directly to the viewer’s eye. Looking through the glasses’ lenses, the user sees both the real world and the full-color, 3D computed images displayed on top.

Life-like quality

The 3D effect is enhanced because it is created both stereoscopically, in the sense that each eye gets to see a slightly different image as they would in traditional 3D imaging, and holographically.

“With holography, you also get the full 3D volume in front of each eye increasing the life-like 3D image quality,” said  Brian Chao , a doctoral student in the Stanford Computational Imaging lab and also co-author of the paper.

The ultimate outcome of the new waveguide display techniques and the improvement in holographic imaging is a true-to-life 3D visual experience that is both visually satisfying to the user without the fatigue that has challenged earlier approaches.

“Holographic displays have long been considered the ultimate 3D technique, but it’s never quite achieved that big commercial breakthrough,” Wetzstein said. “Maybe now they have the killer app they’ve been waiting for all these years.”

Additional authors are from The University of Hong Kong and NVIDIA. Wetzstein is also member of  Stanford Bio-X , the  Wu Tsai Human Performance Alliance , and the  Wu Tsai Neurosciences Institute .

This research was funded by a Stanford Graduate Fellowship in Science and Engineering, the National Research Foundation of Korea (NRF) funded by the Ministry of Education, a Kwanjeong Scholarship, a Meta Research PhD Fellowship, the ARO PECASE Award, Samsung, and the Sony Research Award Program. Part of this work was performed at the  Stanford Nano Shared Facilities (SNSF)  and  Stanford Nanofabrication Facility (SNF) , supported by the National Science Foundation and the National Nanotechnology Coordinated Infrastructure.

Related : Gordon Wetzstein , associate professor of electrical engineering 

Related Departments

The future of Russia and Ukraine

Turning carbon pollution into ethanol

New gels could protect buildings during wildfires