technical training research paper

• Open access
• Published: 20 December 2013

Technical and vocational education and training (TVET) for young people in low- and middle-income countries: a systematic review and meta-analysis

• Janice S Tripney 1 &
• Jorge G Hombrados 2  

Empirical Research in Vocational Education and Training volume  5 , Article number:  3 ( 2013 ) Cite this article

46k Accesses

23 Citations

2 Altmetric

Metrics details

The world is facing a worsening youth employment crisis. In response, technical and vocational education and training (TVET) is back on the development agenda after years of neglect. This systematic review examined the evidence from studies evaluating the impacts of TVET interventions for young people in low- and middle-income countries (LMICs). The 26 included studies evaluated 20 different interventions, predominantly from Latin America. Meta-analyses of the effectiveness of TVET on five outcome measure categories were conducted. The overall mean effects on overall paid employment, formal employment, and monthly earnings were small, positive, and significant; however, significant heterogeneity was observed. Moderator analysis was performed in an attempt to explain between-study differences in effects. The overall paucity of research in this area, together with specific gaps and methodological limitations, affirm the need for strengthening the evidence base. Implications for policy, practice and research are discussed.

Introduction

The global youth unemployment rate rose sharply between 2008 and 2009, from 11.8 to 12.7 per cent, reversing the pre-crisis trend of declining youth unemployment rates since 2002 (International Labour Organization 2011 ). By 2011, 74.8 million young people were unemployed. In some regions, young people are nearly three times as likely as adults to be unemployed; they are also more likely than older workers to be underemployed or work in the informal labour market, in poor quality jobs that require low levels of skills and offer limited socio-economic security, training opportunities, and working conditions (International Labour Organization 2012 ). Significant regional variation in youth unemployment exists; and, in many countries, young women are much more likely to be un/underemployed than young men (United Nations 2012 ). Over 40 per cent of all young people live on less than two US dollars a day, with youth in developing countries disproportionately among the working poor (International Labour Organization 2012 ). This enormous unlocked potential represents a substantial loss of opportunity for both individuals and society.

One billion young people, the majority from LMICs, are predicted to reach employment age within the next decade (International Labour Organization 2012 ), compounding what are already severely limited opportunities for integrating youth into the labour market. With increasing policy importance now placed on higher-order skills and their central role in the global knowledge-based economy, comes a renewed focus on the potential of TVET to equip youths with the abilities to seize available work opportunities, and, in turn, impact upon global poverty and social stability (African Union 2007 ; King and Palmer 2010 ; Péano et al. 2008 ; United Nations Educational, Scientific and Cultural Organization Scientific and Cultural Organization 2010 , 2012 ). Neglected by the World Bank and other donor agencies during much of the 1980s and 1990s, there has been growing investment in TVET since 2000. Deciding which programmes to implement requires an understanding not only of which models are effective, but for whom they are effective. Collecting evidence from studies that have analysed these issues is crucial for purposes of policy-making.

Technical and vocational education and training (TVET)

The term TVET, as used in this systematic review, follows the definition used by UNESCO as “those aspects of the educational process involving, in addition to general education, the study of technologies and related sciences and the acquisition of practical skills, attitudes, understanding and knowledge relating to occupation in various sectors of economic life”. It incorporates: technical education, vocational education, vocational training, on-the-job training, and apprenticeship training (or any combination thereof).

Prior reviews of TVET

Although a number of prior reviews have summarised the existing research in this area (see, for example, Angel-Urdinola et al. 2010 ; Betcherman et al. 2004 ; Betcherman et al. 2007 ; Card et al. 2010 ; Fares and Puerto 2009 ; Greenberg et al. 2003 ; Katz 2008 ; Kluve 2006 ; Kluve and Schmidt 2002 ; Puerto 2007 ; Stavreska 2006 ; Van Adams 2007 ), there are a number of limitations to these reviews. Some contain evidence from high-income countries only; others, whilst broader in geographical scope, do not focus specifically on developing countries, TVET, or young people. Very few existing reviews are based on a comprehensive, systematic search for literature, and almost all take either a traditional narrative approach to synthesis or use vote-counting methods where conclusions regarding effective interventions are made based on the number of studies found to demonstrate significant positive results. None of the previous reviews that specifically focus on TVET for youth in LMICs have computed effect sizes for the included studies or examined variability in effects by study, participant or intervention characteristics. These are problems that this review aims to remedy, thereby adding value to the existing body of research on this topic.

Purpose of the present study

This review sought to improve upon prior work by systematically examining the evidence base to provide a clearer picture of the types of TVET interventions being used to improve employment prospects for youth in LMICs, to identify the overall effectiveness of TVET, to examine whether study, participant and intervention characteristics help to explain variability in intervention effects, and to identify areas in which more research needs to be conducted. An extended version of this article is available as a technical report published in the Campbell Collaboration library (Tripney et al. 2013 ).

Systematic review methodology was utilised for all aspects of the search, selection and coding of studies. The review was conducted in accordance with Campbell Collaboration procedures and guidelines on systematic review methods, available at http://campbellcollaboration.org/ . Full details about the methods for this review can be found in the technical report, and review protocol (Tripney et al. 2012 ).

Eligibility criteria

Eligibility was restricted to studies that: were conducted in LMICs; were reported between 2000 and 2011; evaluated the impact of a TVET intervention; investigated outcomes for youth aged 15–24 years; included at least one quantifiable measure of employment or employability (broadly defined); and used an experimental or rigorous quasi-experimental design (such as the use of propensity scores to match on pre-tests and/or relevant demographic characteristics or regression analysis to control for differences between groups). Single group pre-post test studies were not eligible. No language or publication status restrictions were applied.

The conceptual definition of TVET used in this review cuts across education level, type of learning arrangement, mode of delivery, setting, and type of provider/regulator. It includes provision of (i) initial training for young people from the age of 15/16 years after compulsory school, but prior to entering work; (ii) continuing education and training for adults in the labour market leading to personal, flexible and/or vocational competencies; and (iii) training for unemployed persons currently available for and seeking work (including retraining for those made redundant). Single- and multi-service TVET interventions were eligible for inclusion in the review, as were interventions delivered for any length of time or frequency. LMICs were defined according to World Bank classification of economies (in effect 1 July 2011 until 30 June 2012).

Literature search and strategy

The review recognised the importance of using a comprehensive and diverse search strategy to locate all qualifying published and unpublished studies (Papaioannou et al. 2009 ; Schucan Bird and Tripney 2011 ). First, 10 major bibliographic databases were electronically searched (including ASSIA, Econlit, ERIC, IBSS, Medline, PsycINFO, and SSCI). Searches within each database combined controlled vocabulary and natural language terms, with appropriate wildcards, for three concepts: TVET, employment, and study design. Second, specialist databases and library catalogues were searched, including grey literature, regional and topic-specific sources. Third, the websites of government agencies, research centres, foundations, professional associations, and other relevant organisations were searched. Fourth, the reference lists of previous reviews and included studies were checked. Fifth, citation searches of included studies were conducted. Sixth, information about additional relevant studies was requested from authors of included studies and other key experts in the field.

Data extraction and critical appraisal

Data were independently extracted from all included studies by pairs of reviewers using a coding tool specifically designed for this review. Studies were quality assessed using a tool developed specifically for assessing risk of bias in experimental and quasi-experimental designs based on statistical methods (Hombrados and Waddington 2012 ).

Where data allowed, effect sizes were computed for each study and random-effects inverse-variance weighted meta-analytic methods were used to synthesise the evidence. We followed Petrosino et al. ( 2012 ) in the use of standardised mean differences (SMDs) for synthesising both continuous and dichotomous outcomes. The different formulas used in calculating effect sizes are detailed in the full technical report. The included studies were analysed by outcome and Hedges’ g effect sizes were reported. To visibly examine between-study variability in the effect size estimates, forest plots were used to display the estimated effect sizes from each study along with their 95 per cent confidence intervals (CIs). Heterogeneity was explored using both the Q test and the I 2 index (Deeks et al. 2001 ; Higgins and Green 2011 ). Moderator analyses were performed to examine possible reasons for variability in effects. Due to the small number of available studies, an analogue to the ANOVA analysis (univariate) approach was used (Lipsey and Wilson 2001 ). The categorical variables which identified the sub-groups used in this exercise were specified in advance.

Size of literature reviewed

Of the 8072 articles identified using the electronic search, 145 were deemed potentially relevant on the basis of title and abstract and the full papers were retrieved for a full text review. The full papers of a further 76 items identified through the non-database search were also manually examined. This process resulted in a total of 30 different studies, described in 55 separate publications, being identified as meeting the eligibility criteria. Of these, 26 studies were included in the review. a The study search and selection process is detailed in Figure  1 .

Study search and selection process flowchart.

Descriptive analysis

Publication dates of the 26 included studies ranged between 2001 and 2011, and over three-quarters were published as technical reports. Three studies were randomised experiments and 23 studies used a quasi-experimental design (QED). The QED evaluations used different econometric techniques to address selection bias and net out the impacts of other factors, most commonly propensity score matching (Rosenbaum and Rubin 1983 ). The majority of study samples included both male and female participants, and a single study focused exclusively on young women. The risk of bias assessments resulted in five studies graded medium quality (Attanasio et al. 2011 ; Card et al. 2011 ; Chong and Galdo 2006 ; Hicks et al. 2011 ; Espinoza 2010 ), 10 graded low/medium (Acero et al. 2011 ; Aedo and Nuñez 2004 ; Aedo and Pizarro 2004 ; Analítica Consultores 2006 ; Chong et al. 2008 ; Díaz and Jaramillo 2006 ; Elías et al. 2004 ; Ibarraran and Rosas-Shady 2006 ; Jaramillo et al. 2007 ; Ñopo et al. 2007 ) and 11 graded low (Alzuá and Brassiolo 2006 ; Alzúa et al. 2007 ; Benus et al. 2001 ; Bidani et al. 2002 ; Chun and Watanabe 2011 ; Delajara et al. 2006 ; Dmitrijeva 2009 ; López-Acevedo 2003 ; Medina and Nuñez 2005 ; Mensch et al. 2004 ; van Gameren 2010 ). d

Twenty different TVET programme interventions were evaluated. Settings include 10 upper-middle income countries (Argentina; Bosnia and Herzegovina; Brazil; Chile; China; Colombia; Dominican Republic; Latvia; Mexico; Panama and Peru); two lower-middle income countries (India and Bhutan); and one low-income country (Kenya). e Twelve programmes were specifically targeted at youth. The majority were targeted at disadvantaged populations, based on criteria such as household income, education level, and employment experience. A few were occupation- rather than age-focused (targeting, for example, demobilised soldiers or unemployed former employees of state-owned enterprises). A small number of interventions were directed exclusively at either urban or rural residents.

The 20 TVET interventions consisted of one or more of the different models or approaches to vocational skills development (see Table  1 ). The different types of TVET are not evenly represented, however; for instance, no apprenticeship training programmes were evaluated and a very small number of studies focused on technical education or vocational education. Many of the interventions in the included studies were complex, multi-component programmes. Alongside TVET-related training, some programmes incorporated labour mediation services (such as job readiness skills), some included training specifically for self-employment and some included other types of training (such as basic skills). However, the largest category of multi-component programme involved components consisting of different types of TVET. There were nine two-phase TVET interventions, most of which combined theoretical and practical training, usually in the format of classroom-based vocational training followed by a period of on-the-job training to provide beneficiaries with work experience. These interventions typically focused on short-term semi-skilled training in specific occupations demanded in the private sector.

Primary study authors often did not provide all the necessary data for computing effect sizes. It was possible to calculate at least one effect size for 10 of the studies included in the review (Acero et al. 2011 ; Aedo and Nuñez 2004 ; Aedo and Pizarro 2004 ; Attanasio et al. 2011 ; Card et al. 2011 ; Elías et al. 2004 ; Espinoza 2010 ; Hicks et al. 2011 ; Ibarraran and Rosas-Shady 2006 ; Medina and Nuñez 2005 ). The findings from these studies were statistically combined using meta-analytic techniques.

Overall mean effects of interventions

The overall mean effect sizes for five outcomes are presented in Table  2 . For three outcomes, an overall positive and statistically significant effect on TVET on youth was found: overall paid employment ( g  = 0.134; 95% CI [0.024, 0.243]); formal employment ( g  = 0.199; 95% CI [0.055, 0.344]); and earnings ( g  = 0.127; 95% CI [0.043, 0.21]). For working hours worked, the overall effect was also positive, but non-significant ( g  = 0.043; 95% CI [-0.017, 0.104]). In contrast, the overall effect on self-employment earnings was negative, and non-significant ( g  = -0.025; 95% CI [-0.11, 0.061]). b

Analysis of homogeneity

For three outcomes (overall paid employment, formal employment, and earnings) the grand mean provides some evidence that the TVET interventions were, on average, effective; although, effects are generally small and difficult to detect. Furthermore, in each case the result of the statistical test for homogeneity was statistically significant (Q = 23.8, p < 0.05; Q = 11.1, p < 0.05; Q = 25.5, p < 0.05), indicating that studies disagreed on the magnitude of effect. Our next step was to explore the reasons for this variability between studies.

Analysis of moderator effects

Moderator analyses were performed to examine independent variables that may be contributing to the heterogeneity. The results are presented in Table  3 .

Relationship of study characteristics to effect size

Two factors potentially associated with treatment effect were examined: study quality and length of follow up (i.e., timing of outcome measurement). As there were no high quality studies included in the review, studies rated medium quality were compared with those rated low or low-medium. c Short-term treatment effects (measured at approximately six months) were compared with medium-term effects (measured at approximately 12 months). The low quality studies were found to have consistently larger mean effects than the medium quality studies; and, for two outcomes, differences in mean effects observed between studies according to study quality were statistically significant: paid employment (Q b  = 6.49, p = 0.0108); formal employment (Q b  = 10.6, p = 0.0012). Although studies that measured short-term treatment effects consistently had slightly larger mean effects than studies in the medium-term follow up group, no statistically significant relationship between length of follow up and treatment effect was observed for any of the outcomes for which we had data.

Relationship of participant characteristics (gender) to effect size

For one of the five outcome categories (weekly hours), statistically significant differences in mean treatment effects were observed between studies according to gender (Qb = 10.1; p = 0.00151). Treatment effects for female youth were positive ( g  = 0.16; 95% CI [0.04, 0.28]), while those for male youth were negative ( g  = -0.09; 95% CI [-0.2, 0.01]). However, male youth participation in TVET demonstrated effects that were not statistically different from zero, lessening our confidence in this result.

Relationship of intervention characteristics (type) to effect size

Slight differences in mean effects between two-phase models of TVET and other models were observed, with two-phases usually, but not always, producing the larger effects. However, these differences were not statistically significant.

Conclusions

This review improves upon prior work by statistically synthesising TVET intervention research, strengthening the evidence base on which current policies and practices can draw. However, the nature of the available evidence, and limitations in the review itself, are such that drawing strong inferences from the results of the analyses is not recommended and considerable caution should be used when interpreting and applying the findings of the review.

Despite renewed investment in TVET and increased pressure for evidence-based decision-making, there remains an overall scarcity of research in this area and specific knowledge gaps. The interventions included in this systematic review are not representative of all TVET programmes in existence. Studies were undertaken in a relatively small number of counties, predominantly in Latin America and the Caribbean. Some types of TVET are poorly represented in the review; for example, no studies examined the effectiveness of apprenticeship training. The quality of the existing evidence base is also a concern. Only three RCTs were identified, and methodological and reporting shortcomings of the included studies were such that none provided high quality evidence to help answer the review questions.

Overall, interventions included in the meta-analyses were found to demonstrate a significant, though small, positive effect on a limited range of labour market outcomes: overall paid employment ( g  = 0.134; 95% CI [0.024, 0.243]); formal employment ( g  = 0.199; 95% CI [0.055, 0.344]); and monthly earnings ( g  = 0.127; 95% CI [0.043, 0.21]). However, the observed heterogeneity was also significant, indicating that different studies point to somewhat different conclusions. Attempts to explain the heterogeneity in overall mean effects suggest that methods matter. The low quality studies have consistently larger mean effects than the medium quality studies. For two outcomes, evidence of a statistically significant relationship between study quality and effect size magnitude suggest that the overall mean effect may be inflated and that our conclusions about treatment effect should be based only on those studies rated medium quality: paid employment ( g  = 0.06; 95% CI [-0.01, 0.12]); formal employment ( g  = 0.12; 95% CI [0.05, 0.19]). Treatment effects for both self-employment earnings and weekly hours worked were non-significant. On the whole, the other independent variables that were tested were unable to explain variability in effects. Length of follow up did not demonstrate a significant relationship with treatment effect. Two-phase TVET interventions (classroom-based vocational training followed by on-the-job training) produced statistically similar effects to other TVET models. Gender may have some influence, as an increase in the number of weekly hours worked by youth following participating in TVET appears to be driven by the effect on young women. It is important to note, however, that due to an insufficient number of studies reporting relevant data, only some of the moderator analyses that had been planned a priori could be performed. Furthermore, due to the very small number of studies for most of the variables tested, and thus low statistical power, we may not have been able to detect moderator effects that may indeed be present. In addition, there may be other moderating variables that either were not tested in this study or measured in the primary reports, such as implementation fidelity, whether the intervention was theoretically informed, and so on, which could account for the differences in effects between studies.

While a key strength of this study is its application of systematic review principles to improve upon prior work. There are several important limitations to this review that should be acknowledged. First, not all eligible studies are included in the meta-analysis. Second, the methods for calculating comparable effect sizes from studies using complex econometrics methods, as used in this review, are under-developed and require further research (for a complete discussion, see Becker and Wu 2007 ; Duvendack et al. 2012 ; Lipsey and Wilson 2001 ). Third, no high quality studies were identified and some of the methodological concerns associated with those that were included may mean that the studies have yielded biased estimates of treatment effect. All conclusions from the current review are therefore sensitive to the possibility that the results from the meta-analyses may be over- or under-estimating the effects of TVET on the outcome variables of interest.

Limitations notwithstanding, this systematic review provides some evidence to support the claim that participation in TVET improves the labour market situation of youth in LMICs, on average, when compared to youth who do not participate, with the strength of the evidence strongest for formal employment and monthly earnings. Thus, it is reasonable to conclude that it is worthwhile to continue investment in TVET provision. Although, statistically, the effects of TVET on youth may be small, even a slight increase in the employment rate, for example, could potentially translate into fairly large numbers of young people entering the labour market, especially where programmes are delivered nationally. Through its use of statistical meta-analysis, this review has improved upon previous work; nevertheless, there are two main related areas of likely disappointment for policymakers and programme implementers. First, not only were the observed effects generally small and difficult to detect, the review does not provide conclusive evidence of the effectiveness of TVET on youth employment outcomes. This suggests an urgent need to improve the rigour of the evidence base, and will require policymakers, practitioners and other relevant stakeholders engaging critically with the barriers to effective research production and dissemination in this area. The commissioning of RCTs and robust QEDs is crucial for generating evidence capable of supporting causal claims. To improve potential generalisability, it is also important that appropriate support, financial and otherwise, is given to rigorous outcome research on all types of TVET that have been implemented internationally for young people, and in a broader range of geographical settings than covered by the current evidence base. A cost-benefit analysis was outside the scope of the current review; however, attention should be given to the collection of data regarding the costs of TVET interventions, and examinations of cost-effectiveness should be a priority in future impact evaluations and systematic reviews. Second, the available evidence does not allow recommendations to be made for, or against, investments in a particular model of TVET. Although claims have been made in the literature and media for the success or effectiveness of a particular format of TVET widely implemented in Latin America and the Caribbean, the evidence from this systematic review suggests that such claims are premature. In the absence of evidence in support of any one model or approach to vocational skills development, it seems reasonable in the meantime for policymakers and other stakeholders to select the least costly, most culturally acceptable, and/or most easily implemented programmes, while at the same time becoming good consumers of evidence and working towards the strengthening of the evidence base on TVET. This review has demonstrated that not only is there is a clear need for additional primary research in this area, but that more of the same will not suffice. The methodological shortcomings of the current evidence base, and specific knowledge gaps, suggest a number of future research priorities. These are summarised in Table  4 .

a Four eligible Spanish-language studies could not be included in the review. Of these, one study was judged as meeting the review selection criteria on the basis of detailed descriptions in previous literature reviews, but efforts to obtain a copy of the publication were unsuccessful. The three remaining eligible non-English language studies were not included in the review due to exhaustion of financial resources. All four studies evaluated ProJoven, a Peruvian programme that was evaluated by a number of studies that were included in the review.

b A preliminary assessment of the ‘direction of effects’ of the 16 studies not included in the meta-analyses suggests that these studies generally support the meta-analytic findings.

c For this exercise, all of these studies were grouped together into a single category labelled low.

d One study (Espinoza 2010 ) was graded medium quality overall; however, individual assessments for the different outcome categories varied. For employment outcomes, this study was graded as proving medium quality evidence; for monthly earnings, the grading was low/medium.

e World Bank country classifications have been revised twice since the conduct of this review, and Chile and Latvia are currently ranked as high income economies.

Authors’ information

JT is a lecturer in Social Policy, based at the EPPI-Centre (Evidence for Policy and Practice Co-ordinating Centre), Social science Research Unit, Institute of Education, University of London. JH is a research analyst at J-PAL (Abdul Latif Jameel Poverty Action Lab), based at its regional office in Latin America at the Pontificia Universidad Católica de Chile.

Abbreviations

Low- and middle-income country

Quasi-experimental design

Randomised controlled trial

Standardised mean difference

Technical and vocational education and training.

Acero C, Alvarado M, Bravo D, Contreras D, Ruiz-Tagle J: Evaluación de impacto del Programa Jóvenes al Bicentenario para la cohorte de participantes en el Año 2008 . 2011. . Accessed 25 Oct 2013 http://www.pnud.cl/proyectos/Documentos%20de%20proyectos/2011/Informe%20Evaluaci%C3%B3n%20Impacto%20JB_FINAL.pdf . Accessed 25 Oct 2013

Google Scholar  

Aedo C, Nuñez S: The impact of training policies in Latin America and the Caribbean: The case of Programa Joven (Research Network Working Paper #R-483) . Washington: Inter-American Development Bank; 2004.

Aedo C, Pizarro M: Rentabilidad económica del programa de capacitación laboral de jóvenes 'Chile Joven' . Santiago de Chile: INACAP and Mideplan; 2004.

African Union: Strategy to revitalize technical and vocational education and training (TVET) in Africa. Paper presented at the Meeting of the Bureau of the Conference of Ministers of Education of the African Union (COMEDAF II+), Addis Ababa, May 2007 . 2007. . Accessed 25 Oct 2013 http://info.worldbank.org/etools/docs/library/243614/TVET%20Strategy%20in%20Africa.pdf . Accessed 25 Oct 2013

Alzuá ML, Brassiolo P: The impact of training policies in Argentina: An evaluation of Proyecto Joven (Working Paper: OVE/WP-15/06) . Washington: Inter-American Development Bank; 2006.

Alzúa ML, Nahirñak P, Alvarez De Toledo B: Evaluation of Entra 21 using quantitative and qualitative data. Q-squared Working Paper No. 41 . University of Toronto Centre for International Studies; 2007. . Accessed 25 Oct 2013 http://www.trentu.ca/ids/documents/Q2_WP41_Alzua_etal.pdf . Accessed 25 Oct 2013

Analítica Consultores: Estudio de evaluacion de la estrategia Bécate . Ciudad de México: Secretaría del Trabajo y Previsión Social; 2006.

Angel-Urdinola DF, Semlali A, Brodmann S: Non-public provision of active labor market programs in Arab-Mediterranean countries: An inventory of youth programs (SP Discussion Paper No. 1005) . Washington: World Bank; 2010.

Attanasio O, Kugler A, Meghir C: Subsidizing vocational training for disadvantaged youth in Colombia: Evidence from a randomized trial. Am Econ J Appl Econ 2011, 3 (3):188–220. 10.1257/app.3.3.188

Article   Google Scholar  

Becker B, Wu MJ: The synthesis of regression slopes in meta-analysis. Stat Sci 2007, 22: 414–429. 10.1214/07-STS243

Benus J, Rude J, Patrabansh S: Impact of the Emergency Demobilization and Reintegration Project (Prepared for: U.S. Department of Labor Bureau of International Affairs Office of Foreign Relations) . 2001. . Accessed 25 Oct 2013 http://pdf.usaid.gov/pdf_docs/Pcaab161.pdf . Accessed 25 Oct 2013

Betcherman G, Olivas K, Dar A: Impacts of active labour market programs: New evidence from evaluations with particular attention to developing and transition countries (Social Protection Discussion Paper Series No. 0402) . Washington: World Bank; 2004.

Betcherman G, Godfrey M, Puerto S, Rother F, Stavreska A: A review of interventions to support young workers: Findings of the youth employment inventory (Social Protection Discussion Paper No. 0715) . Washington: World Bank; 2007.

Bidani B, Goh C, O'Leary CJ: Has training helped employ Xiagang in China? A tale from two cities (Report for the World Bank's Poverty Reduction and Economic Unit East Asia and Pacific Region) . Washington: World Bank; 2002.

Card D, Kluve J, Weber A: Active labour market policy evaluations: A meta-analysis. Econ J 2010, 120: F452-F477. 10.1111/j.1468-0297.2010.02387.x

Card D, Ibarraran P, Regalia F, Rosas-Shady D, Soares Y: The labor market impacts of youth training in the Dominican Republic. J Labor Econ 2011, 29 (2):267–300. 10.1086/658090

Chong A, Galdo J: Does the quality of training programs matter? Evidence from bidding processes data (IZA Discussion Paper No. 2202) . Bonn: Institute for the Study of Labor; 2006.

Chong A, Galdo J, Saavedra J: Informality and productivity in the labor market in Peru. J Econ Policy Reform 2008, 11 (4):229–245. 10.1080/17487870802543480

Chun N, Watanabe M: Can skill diversification improve welfare in rural areas? Evidence from the Rural Skills Development Project in Bhutan (ADB Economics Working Paper Series No. 260) . Manila: Asian Development Bank; 2011.

Deeks JJ, Altman DA, Bradburn MJ: Statistical methods for examining heterogeneity and combining results from several studies in meta-analysis. In Systematic reviews in health care: Meta-analysis in context . Edited by: Egger M, Davey Smith G, Altman DG. London: BMJ Publishing Group; 2001:313–335.

Chapter   Google Scholar  

Delajara M, Freije S, Soloaga I: An evaluation of training for the unemployed in Mexico (Working Paper: OVE/WP-09/06) . Washington: Inter-American Development Bank; 2006.

Díaz JJ, Jaramillo M: An evaluation of the Peruvian ‘youth labor training program’ – Projoven (Working Paper: OVE/WP-10/06) . Washington: Inter-American Development Bank; 2006.

Dmitrijeva J: The employment effects of public training programme: Evidence from Latvian micro and macro data . 2009. . Accessed 25 Oct 2013 http://www.univ-orleans.fr/leo/images/espace_commun/seminaires/semmar2009/WP_57.pdf . Accessed 25 Oct 2013

Duvendack M, Hombrados JG, Palmer-Jones R, Waddington H: Assessing ‘what works’ in international development: meta-analysis for sophisticated dummies. J Dev Eff 2012, 4: 456–471. 10.1080/19439342.2012.710642

Elías VJ, Ruiz Núñez FR, Cossa R, Bravo D: An econometric cost-benefit analysis of Argentina’s youth training program (Research Network Working Paper #R-482) . Washington: Inter-American Development Bank; 2004.

Espinoza H: Impact evaluation of a job-training programme for disadvantaged youths: The case of Projoven . Maastricht: Boekenplan; 2010.

Fares J, Puerto OS: Towards comprehensive training (Social Protection Discussion Paper No. 0924) . Washington: World Bank; 2009.

Greenberg DH, Michalopoulos C, Robins PK: A meta-analysis of government-sponsored training programs. Ind Labor Relat Rev 2003, 57 (1):31–53. 10.2307/3590980

Hicks JH, Kremer M, Mbiti I, Miguel E: Vocational education voucher delivery and labor market returns: A randomized evaluation among Kenyan youth (Report for Spanish Impact Evaluation Fund (SIEF) Phase II) . Washington: World Bank; 2011.

Higgins JPT, Green S (Eds): Cochrane handbook for systematic reviews of interventions . Chichester: Wiley-Blackwell; 2011.

Hombrados JG, Waddington H: Internal validity in social experiments and quasi-experiments: An assessment tool for reviewers . London: Mimeo; 2012.

Ibarraran P, Rosas-Shady D: Impact evaluation of the job training component (PROCAJOVEN) of the assistance program for the building of a training and employment system in Panama (PN0125) (Ex-Post Project Evaluation Report: OVE/EPPER-02/06) . Washington: Inter-American Development Bank; 2006.

International Labour Organization: Global employment trends for youth: 2011 update . Geneva: International Labour Organization; 2011.

International Labour Organization: Global employment trends 2012: Preventing a deeper jobs crisis . Geneva: International Labour Organization; 2012.

Jaramillo M, Galdo J, Montalva V: Do the poorest among the poor benefit from active labor market programs? Evidence from Peru’s Projoven . 2007. . Accessed 25 Oct 2013 http://www.depeco.econo.unlp.edu.ar/cedlas/ien/pdfs/meeting2007/papers/jaramillo.pdf . Accessed 25 Oct 2013

Katz E: Programs promoting young women’s employment: What works? Background paper for Adolescent Girls Initiative . Washington: World Bank; 2008.

King K, Palmer R: Planning for technical and vocational skills development . Paris: UNESCO International Institute for Educational Planning; 2010.

Kluve J: The effectiveness of European active labor market policy (IZA Discussion Paper 2018) . Bonn: Institute for the Study of Labor; 2006.

Kluve J, Schmidt C: Can training and employment subsidies combat European unemployment? Econ Policy 2002, 35: 409–448.

Lipsey MW, Wilson D: Practical meta-analysis . London: Sage; 2001.

López-Acevedo G: A reassessment of technical education in Mexico. J Career Tech Educ 2003, 19 (2):59–81.

Medina C, Nuñez J: The impact of private and public job training in Colombia (Research Network Working Paper #R-484) . Washington: Inter-American Development Bank; 2005.

Mensch BS, Grant MJ, Sebastian MP, Hewett PC, Huntington D: The effect of a livelihoods intervention in an urban slum in India: Do vocational counseling and training alter the attitudes and behavior of adolescent girls? (Working Paper No. 194) . New York: Population Council; 2004.

Ñopo H, Robles M, Saavedra J: Occupational training to reduce gender segregation: The impacts of ProJoven (Research Department Working Paper #623) . Washington: Inter-American Development Bank; 2007.

Papaioannou D, Sutton A, Carroll C, Booth A, Wong R: Literature searching for social science reviews: consideration of a range of search techniques. Health Info Libr J 2009, 27: 114–122. 10.1111/j.1471-1842.2009.00863.x

Péano S, Vergel de Dios B, Atchoaréna D, Mendoza U: Investment in technical vocational education and training (TVET) in the Philippines . Paris: International Institute for Educational Planning; 2008.

Petrosino A, Morgan C, Fronius TA, Tanner-Smith EE, Boruch RF: Interventions in developing nations for improving primary and secondary school enrollment of children: A systematic review. Campbell Syst Rev 2012, 2012: 19. DOI: 10.4073/csr.2012.19 DOI: 10.4073/csr.2012.19

Puerto OS: Interventions to support young workers in Latin America and the Caribbean: Regional report for the Youth Employment Inventory . Washington: World Bank; 2007.

Rosenbaum PR, Rubin DB: The central role of the propensity score in observational studies for causal effects. Biometrika 1983, 70 (1):41–55. 10.1093/biomet/70.1.41

Schucan Bird K, Tripney J: Systematic literature searching in policy relevant, inter-disciplinary reviews: An example from culture and sport. Res Synth Methods 2011, 2 (3):163–173. 10.1002/jrsm.48

Stavreska A: Interventions to support young workers in South, East Asia and the Pacific: Regional report for the Youth Employment Inventory . Washington: World Bank; 2006.

Tripney J, Newman M, Hovish K, Brown C: Protocol: Post-basic technical and vocational education and training (TVET) interventions to improve employability and employment of TVET graduates in low- and middle-income countries. 2012. http://campbellcollaboration.org/lib/project/227/ . Accessed 25 Oct 2013

Tripney J, Hombrados J, Newman M, Hovish K, Brown C, Steinka-Fry K, Wilkey E: Technical and vocational education and training (TVET) interventions to improve the employability and employment of young people in low- and middle-income countries: A systematic review. Campbell Syst Rev 2013, 2013: 9. DOI: 10.4073/csr.2013.9 DOI: 10.4073/csr.2013.9

United Nations: World youth report . Geneva: United Nations; 2012.

United Nations Educational, Scientific and Cultural Organization: Reaching the marginalised. Education for All Global Monitoring Report 2010 . Paris: UNESCO; 2010.

United Nations Educational, Scientific and Cultural Organization: Youth and skills: Putting education to work. Education for All Global Monitoring Report 2012 . Paris: UNESCO; 2012.

Van Adams A: The role of youth skills development in the transition to work: A global review (HDNCY No. 5) . Washington: World Bank; 2007.

van Gameren E: Evaluación de impacto del programa de apoyo al empleo . Ciudad de México: Secretaría de Trabajo y Previsión Social (STPS); 2010.

Download references

Acknowledgements

This manuscript arose from a systematic review commissioned in 2011 by the Australian Agency for International Development (AusAID), in response to the International Initiative for Impact Evaluation (3ie) systematic reviews call 3. JT was the grant holder and principal investigator. JH participated in the full review on an un-funded basis. JT and JH wish to acknowledge the contributions of those who helped in the creation of the full review: Mark Newman was involved in data collection and advised on the statistical analysis; Kimberley Hovish, Chris Brown, Katarzyna Steinka-Fry and Eric Wilkey participated in the acquisition of data. Special thanks are due to Sandra Jo Wilson, editor of the Education Coordination Group (ECG), for her guidance throughout the initial project. The funding agency informed the scope and development of the full review. The preparation of this manuscript was unfunded, and the funding agency had no role in the writing of this manuscript, or the decision to submit the manuscript for publication. The opinions expressed in this manuscript are those of the authors and do not necessarily reflect those of the funding agency. The full version of the review is published in the Campbell library ( http://www.campbellcollaboration.org/lib/project/227/ ), and the 3ie database of systematic reviews ( http://www.3ieimpact.org/en/evidence/systematic-reviews/details/152/ ).

Author information

Authors and affiliations.

EPPI-Centre, Institute of Education, University of London, 18 Woburn Square, London, WC1H 0NR, UK

Janice S Tripney

Abdul Latif Jameel Poverty Action Lab (J-PAL), Pontificia Universidad Católica de Chile, Av. Vicuna MacKenna, Santiago, 4860, Chile

Jorge G Hombrados

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Janice S Tripney .

Additional information

Competing interests.

The authors declare that they have no competing interests.

Authors’ contributions

JT conceived and designed the study; collected, analysed and interpreted the data; and drafted the manuscript. JH participated in the coordination of the study and data acquisition; advised on the statistical analysis and interpretation of data; and revised the manuscript. Both authors read and approved the final manuscript.

Authors’ original submitted files for images

Below are the links to the authors’ original submitted files for images.

Authors’ original file for figure 1

Rights and permissions.

Open Access This article is distributed under the terms of the Creative Commons Attribution 2.0 International License ( https://creativecommons.org/licenses/by/2.0 ), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and permissions

About this article

Cite this article.

Tripney, J.S., Hombrados, J.G. Technical and vocational education and training (TVET) for young people in low- and middle-income countries: a systematic review and meta-analysis. Empirical Res Voc Ed Train 5 , 3 (2013). https://doi.org/10.1186/1877-6345-5-3

Download citation

Received : 11 November 2013

Accepted : 12 November 2013

Published : 20 December 2013

DOI : https://doi.org/10.1186/1877-6345-5-3

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• International Labour Organization
• Formal Employment
• Monthly Earning
• Campbell Collaboration
• Apprenticeship Training

Advertisement

Technology professional development for teacher educators: A literature review and proposal for further research

• Review Paper
• Published: 06 July 2021
• Volume 1 , article number  200 , ( 2021 )

Cite this article

• Foster Gondwe   ORCID: orcid.org/0000-0001-5716-3538 1  

1173 Accesses

2 Citations

Explore all metrics

Teacher educators still face challenges to model effective technology integration into their work partly because they lack necessary technology competencies. With teacher educator technology competencies and technology professional development gaining prominence internationally recently, there is a need to continue making sense of the existing literature on the subject. This article is a literature review and includes proposals for new areas of research that can extend our knowledge of technology professional development (TPD) for teacher educators. Thirty papers published between 2000 and 2019 were included in the review. The findings include knowns and unknowns on the evaluation components of TPD teacher educators. The article has suggested further research on understanding teacher educators’ technology professional development from student teachers’ perspectives; the link between teacher educators’ technology competencies and their professional roles; and balancing between formal and informal TPD. The unique contribution of the paper is a proposed framework for conceptualizing teacher educators’ technology professionalism.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Similar content being viewed by others

Technology Teacher Educators

Where is Technology in the ‘Golden Thread’ of Teacher Professional Development?

Data availability.

Although data sharing is not applicable to this article as no new data were analysed in this study, the data that support the findings of this study are available from the corresponding author upon request.

Archambault L, Wetzel K, Foulger TS, Kim Williams M (2010) Professional development 2.0: transforming teacher education pedagogy with 21st century tools. J Digit Learn Teach Educ. https://doi.org/10.1080/21532974.2010.10784651

Article   Google Scholar  

Borg S, Clifford I, Htut KP (2018) Having an EfECT: professional development for teacher educators in Myanmar. Teach Teach Educ. https://doi.org/10.1016/j.tate.2018.02.010

Capobianco B, Lehman J (2006) Integrating technology to foster inquiry in an elementary science methods course: an action research study of one teacher educator’s initiatives in a PT3 Project. J Comput Math Sci Teach 25(2):123–146

Google Scholar  

Drent M, Meelissen M (2008) Which factors obstruct or stimulate teacher educators to use ICT innovatively? Comput Educ. https://doi.org/10.1016/J.COMPEDU.2007.05.001

Duran M, Brunvand S, Fossum P (2009) Preparing Science teachers to teach with technology: Exploring a K-16 Networked Learning Community Approach. T he Turkish Online Journal of Educational Technology 8(4): 21–42. Available via http://www.tojet.net/articles/v8i4/843.pdf

Elamin C, Fordham N, Hammond DR, O’bannon B, Vannetta R, Gruber K (2008) Infusing technology into the elementary classroom: a school/university partnership model. Comput Schools 19(1–2):149–162. https://doi.org/10.1300/J025v19n01_11

European Commission (2013) Supporting teacher educators. Educ Train. https://doi.org/10.1007/978-3-540-49527-7_2

Evans L (2008) Professionalism, professionality and the development of education professionals. Br J Educ Stud. https://doi.org/10.1111/j.1467-8527.2007.00392.x

Evans L (2014) Leadership for professional development and learning: enhancing our understanding of how teachers develop. Camb J Educ. https://doi.org/10.1080/0305764X.2013.860083

Evans L (2015) Professionalism and professional development: what these research fields look like today—and what tomorrow should bring. Hillary Place Papers. Available via https://pdfs.semanticscholar.org/e356/d2d12d67039503d358ba2bb7106f52f5bca1.pdf

Evans L (2019) Implicit and informal professional development: what it ‘looks like’, how it occurs, and why we need to research it. Prof Dev Educ. https://doi.org/10.1080/19415257.2018.1441172

Fereday J, Muir-Cochrane E (2006) Demonstrating rigor using thematic analysis: a hybrid approach of inductive and deductive coding and theme development. Int J Qual Methods. https://doi.org/10.1177/160940690600500107

Fletcher T, Bullock SM (2015) Reframing pedagogy while teaching about teaching online: a collaborative self-study. Prof Dev Educ. https://doi.org/10.1080/19415257.2014.938357

Foulger TS, Buss RR, Wetzel K, Lindsey LA (2015) Instructors’ growth in TPACK: teaching technology-infused methods courses to preservice teachers. J Digit Learn Teach Educ. https://doi.org/10.1080/21532974.2015.1055010

Foulger TS, Graziano KJ, Schmidt-crawford DA, Slykhuis DA (2017) Teacher educator technology competencies. J Technol Teach Educ 25:413–448

Fowler Z, Stanley G, Murray J, Jones M, McNamara O (2013) Research capacity-building with new technologies within new communities of practice: reflections on the first year of the Teacher Education Research Network. Prof Dev Educ. https://doi.org/10.1080/19415257.2012.744769

Franklin CA, Sessoms DB (2005) A Situative perspective on a collaborative model for integrating technology into teaching. J Educ Comput Res. https://doi.org/10.2190/BQCM-R7AW-PH1B-KRRE

Grant MM (2004) Learning to teach with the web: factors influencing teacher education faculty. Internet High Educ. https://doi.org/10.1016/j.iheduc.2004.09.005

Heck D, Sweeney TA (2013) Using most significant change stories to document the impact of the teaching teachers for the future project: an australian teacher education story. Austral Educ Comput 27(3):36–47

Hong CE, Van Riper I (2016) Enhancing teacher learning from guided video analysis of literacy instruction: an interdisciplinary and collaborative approach. J Inquiry Action Educ 7(2):94–110

Jaipal-Jamani K, Figg C, Gallagher T, Scott R, Ciampa K (2015) Collaborative professional development in higher education: developing knowledge of technology enhanced teaching. J Effect Teach 17(2):90–111

Jaramillo-Baquerizo C, Valcke M, Vanderlinde R (2019) Professional development initiatives for university teachers: variables that influence the transfer of learning to the workplace. Innov Educ Teach Int. https://doi.org/10.1080/14703297.2018.1479283

Johnson H, Ehrlich S, Watts-Taffe S, Williams C (2014) Who am I here? Disrupted identities and gentle shifts when teaching in cyberspace. J Instruct Res 3:43–54

Kastberg S, Lynch-Davis K, D’Ambrosio B (2014) Examining mathematics teacher educators’ emerging practices in online environments. Contemp Issues Technol Teach Educ 14(4):384–400

Kennedy A (2014) Understanding CPD: the need for theory to impact on policy and practice. Prof Dev Educ 40(5):688–697

Kennedy M (2016) How does professional development improve teaching? Rev Educ Res. https://doi.org/10.3102/0034654315626800

King F (2014) Evaluating the impact of teacher professional development: an evidence-based framework. Prof Dev Educ. https://doi.org/10.1080/19415257.2013.823099

Kosnik C, Menna L, Dharamshi P, Miyata C, Cleovoulou Y, Beck C (2015) Four spheres of knowledge required: an international study of the professional development of literacy/English teacher educators. J Educ Teach. https://doi.org/10.1080/02607476.2014.992634

Lavonen J, Lattu M, Juuti K, Meisalo V (2006) Strategy based development of teacher educators’ ICT competence through a cooperative staff development project. Eur J Teach Educ. https://doi.org/10.1080/02619760600617433

Lawless KA, Pellegrino JW (2007) Professional development in integrating technology into teaching and learning: knowns, unknowns, and ways to pursue better questions and answers. Rev Educ Res. https://doi.org/10.3102/0034654307309921

Lidolf S, Pasco D (2020) Educational technology professional development in higher education: a systematic literature review of empirical research. Front Educ. https://doi.org/10.3389/feduc.2020.00035

Lunenberg M, Dengerink J, Korthagen F (2014) The Professional teacher educator: roles, behaviour, and professional development of teacher educators. Springer, New York

Book   Google Scholar  

Murphy KL, Richards J, Lewis C (2005) Strengthening Educational Technology in K-8 Urban schools and in preservice teacher education: a practitioner-faculty collaborative process. J Technol Teach Educ 13(1):125–139

Murray J (2014) Teacher educators’ constructions of professionalism: a case study. Asia Pac J Teach Educ. https://doi.org/10.1080/1359866X.2013.870971

Parrish AH, Sadera WA (2019) A review of faculty development models that build teacher educators’ technology competencies. J Technol Teach Educ 27(4):437–464

Peeraer J, Van Petegem P (2012) The limits of programmed professional development on integration of information and communication technology in education. Austral J Educ Technol. https://doi.org/10.14742/ajet.809

Ping C, Schellings G, Beijaard D (2018) Teacher educators’ professional learning: a literature review. Teach Teach Educ. https://doi.org/10.1016/j.tate.2018.06.003

Reading C, Doyle H (2013) Teacher educators as learners: enabling learning while developing innovative practice in ict-rich education. Aust Educ Comput 27(3):109–116

Reese J (2016) Virtual mentoring of preservice teachers: mentors’ perceptions. J Music Teach Educ. https://doi.org/10.1177/1057083715577793

Rienties B, Brouwer N, Bohle Carbonell K, Townsend D, Rozendal AP, van der Loo J, Dekker P, Lygo-Baker S (2013a) Online training of TPACK skills of higher education scholars: a cross-institutional impact study. Eur J Teach Educ. https://doi.org/10.1080/02619768.2013.801073

Rienties B, Brouwer N, Lygo-Baker S (2013b) The effects of online professional development on higher education teachers’ beliefs and intentions towards learning facilitation and technology. Teach Teach Educ. https://doi.org/10.1016/j.tate.2012.09.002

Rosaen CL, Hobson S, Khan G (2003) Making connections: collaborative approaches to preparing today’s and tomorrow’s teachers to use technology. J Technol Teach Educ 12(2):281–306

Ross V, Chan E (2016) Personal practical knowledge of teacher educators. In: Loughran J, Hamilton M L (eds) International handbook of teacher education, vol 2. https://doi.org/10.1007/978-981-10-0369-1_1

Sahlberg P, Hasak J, Rodriguez V (2017) Hard questions on global educational change: policies, practices, and the future of education . Teachers College Press.

Schols M (2015) Continuing technology professional development: a technology learning preferences instrument to support teacher educators ’ workplace learning [University of Roehampton]. Available via file:///C:/Users/BP0002115378-0/OneDrive/Research Programm/Articles/Maurice_Schols_Thesis.pdf

Schrum L, Skeele R, Grant M (2002) One college of education’s effort to infuse technology: a systemic approach to revisioning teaching and learning. J Res Technol Educ. https://doi.org/10.1080/15391523.2002.10782385

Soto M, Gupta D, Dick L, Appelgate M (2019) Bridging distances: professional development for higher education faculty through technology-facilitated lesson study. J Univ Teach Learn Pract

Teclehaimanot B, Lamb A (2005) Technology-rich faculty development for teacher educators: the evolution of a program. Contemp Issues Technol Teach Educ 5(3):330–344

Triggs P, John P (2004) From transaction to transformation: information and communication technology, professional development and the formation of communities of practice. J Comput Assist Learn. https://doi.org/10.1111/j.1365-2729.2004.00101.x

Uerz D, Volman M, Kral M (2018) Teacher educators’ competencies in fostering student teachers’ proficiency in teaching and learning with technology: an overview of relevant research literature. Teach Teach Educ. https://doi.org/10.1016/j.tate.2017.11.005

Vanassche E, Kelchtermans G (2014) Teacher educators’ professionalism in practice: positioning theory and personal interpretative framework. Teach Teach Educ. https://doi.org/10.1016/j.tate.2014.08.006

Vanassche E, Kidd W, Murray J (2019) Articulating, reclaiming and celebrating the professionalism of teacher educators in England. Eur J Teach Educ. https://doi.org/10.1080/02619768.2019.1628211

Vasinda S, Ryter DA, Hathcock S, Wang Q (2017) Access is not enough: a collaborative autoethnographic study of affordances and challenges of teacher educators’ iPad integration in elementary education methods courses. Contemp Issues Technol Teach Educ CITE J 17(3):411–431

Vaughan M, Beers C, Burnaford G (2015) The impact of iPads on teacher educator practice: a collaborative professional development initiative. Int J Technol Teach Learn 11(1):21–34

Wambugu PW (2018) Massive open online courses (MOOCs) for professional teacher and teacher educator development: a case of TESSA MOOC in Kenya. Universal Journal of Educational Research. https://doi.org/10.13189/ujer.2018.060604

Webster-Wright A (2009) Reframing professional development through understanding authentic professional learning. Rev Educ Res. https://doi.org/10.3102/0034654308330970

Wepner SB, Bowes KA, Serotkin R (2005) Lessons learned from implementing technology in an undergraduate elementary education program. J Comput Teach Educ. https://doi.org/10.1080/10402454.2005.10784527

Wright VH, Wilson EK (2007) A partnership of educators to promote technology integration: designing a master technology teacher program. Education 128:80–86

Willis J, Thompson A, Sadera W (1999) Research on technology and teacher education: Current status and future directions. Educ Tech Res Dev. https://doi.org/10.1007/BF02299596

Download references

This study did not receive any funding.

Author information

Authors and affiliations.

Hiroshima University, Japan/University of Malawi, Zomba, Malawi

Foster Gondwe

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Foster Gondwe .

Ethics declarations

Conflict of interest.

The author does not have any conflict of interest.

See Table 2 .

Rights and permissions

Reprints and permissions

About this article

Gondwe, F. Technology professional development for teacher educators: A literature review and proposal for further research. SN Soc Sci 1 , 200 (2021). https://doi.org/10.1007/s43545-021-00184-9

Download citation

Received : 13 November 2020

Accepted : 09 June 2021

Published : 06 July 2021

DOI : https://doi.org/10.1007/s43545-021-00184-9

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Teacher educator
• Technology professional development
• Technology professionalism
• Teacher educator technology competence
• Find a journal
• Publish with us
• Track your research

• Reference Manager
• Simple TEXT file

People also looked at

Review article, a review on virtual reality skill training applications.

• 1 Design Computing and Extended Reality Group, Volgenau School of Engineering, Computer Science, George Mason University, Fairfax, VA, United States
• 2 Virtual Reality Lab, Department of Computer Graphics Technology, Purdue University, West Lafayette, IN, United States
• 3 Purdue Human Resource Development Virtual Lab, Department of Technology Leadership and Innovation, Purdue University, West Lafayette, IN, United States

This study aimed to discuss the research efforts in developing virtual reality (VR) technology for different training applications. To begin with, we describe how VR training experiences are typically created and delivered using the current software and hardware. We then discuss the challenges and solutions of applying VR training to different application domains, such as first responder training, medical training, military training, workforce training, and education. Furthermore, we discuss the common assessment tests and evaluation methods used to validate VR training effectiveness. We conclude the article by discussing possible future directions to leverage VR technology advances for developing novel training experiences.

1 Introduction

Thanks to the recent growth of consumer-grade virtual reality (VR) devices, VR has become much more affordable and available. Recent advances in VR technology also support the creation, application, evaluation, and delivery of interactive VR applications at a lower cost. The VR research community is more active than ever. The IEEE Virtual Reality conference has received a record number of submissions in 2020, an approximate 10% growth compared to the previous year (2019.) The latest commercial standalone VR headset Oculus Quest 2, which was released in 2020 has become the fastest selling VR headset Lang (2021) . Such trends contribute to the increasing popularity and success of VR training across different domains. For example, VR training is the most common use of VR within enterprise—62% according to a recent survey Ostrowski (2018) .

According to Merriam Webster 1 , training is defined either as 1) the act, process, or method of one that trains and 2) the skill, knowledge, or experience acquired by one that trains; or the state of being trained. Conventionally, training happens with physical setups such as classrooms and laboratory spaces through presentations and hands-on practice. However, there are cases in which trainees must travel to specific facilities to receive proper training. To date, VR technology makes it possible to provide real-world training through virtual environments while providing an effective and immersive training experience.

As virtual environments are used to train users to perform real-world tasks and procedures, it is important to compare real-world training with VR-based training. In general, real-world training has the following limitations: 1) it could be time-consuming due to the efforts and time needed to set up the real-world training site and to travel to the site; 2) it could be expensive due to the cost of preparing real-world training materials and hiring human coaches; 3) it could be unappealing and unintuitive due to the lack of visual hints such as 3D animations for illustrating skills and processes; and 4) it could not be possible to train some skills in the real world such as emergency procedures that can be only safely trained in simulators.

Depending on the domain, VR could drastically reduce the cost of training while increasing the number of training scenarios. As VR training scenarios mainly involve computer-generated 3D graphics, VR developers can easily create a variety of scenarios from existing 3D assets, which can be repeatedly applied for training different people. Delivered through the internet, the scenarios are also convenient and inexpensive to access. Although VR training does not guarantee a lower cost, the advantages that those systems bring to the training audience could justify the investment costs.

VR technologies allow trainees to learn in the comfort of their personal space. For example, a worker might learn to use a piece of equipment or follow safety procedures using VR at home. Depending on the domain, VR training could also provide privacy for a trainee, and this is especially important for cases in which a trainee might feel uncomfortable about his/her actions in a real-world training process due to the presence of observers. However, there are training situations that need to happen in the presence of instructors as they need to provide early feedback on the trainee’s performance and alert the trainee to issues that they may be experiencing, including negative trends in their performance.

Moreover, VR training provides a safe environment with minimal exposure to dangerous situations [e.g., fires Backlund et al., 2007 ; Conges et al., 2020 ; explosions, and natural disasters Li et al., 2017 ]. For example, an instructor can control a virtual fire (position, intensity, and damage caused) in which a firefighter is being trained to evacuate a building. Such scenarios could allow people to easily train in a safe environment that mimics real-world challenges.

VR technologies alter users’ perceptions such that they are replaced by or blended with the output of a computer program. There are several elements of VR experiences distinguished roughly by the degree to which the user’s sensory perception is co-opted. Head-mounted VR displays with headphones, wearable haptic clothing, and equipment that allows the user to walk freely exemplify the most common consumer-grade VR setup. VR is not limited to visual information, as the auditory, haptic, kinesthetic, and olfactory senses can be technologically simulated. Additionally, high-quality virtual content designs greatly improve VR training experiences. We discuss how different types of VR setups and simulations are used to create effective VR experiences for different training purposes. Figure 1 shows some VR training applications.

FIGURE 1 . VR training has been widely adopted by different domains. Image courtesy of the paper authors.

Motivated by recent advances in VR training, this study presents previous works to inform the community about the current state and future directions of VR training research. Many works are published on this topic every year and a workshop [“TrainingXR” at IEEE VR TRAININGXR@IEEEVR 2020, 2020 ] was recently established to fill the gap in this subdomain of VR research. We explore the methods, techniques, and prior developments in VR training that have been discussed in previous works. Most of the works that we selected belong to simulation-based training via VR technologies. These works often use a simulator that is either controlled by a real instructor or driven by a set of training tasks to improve the trainee’s performance on certain tasks or skills. Furthermore, the work has been validated through experiments and has some preliminary results. However, as there is a substantial amount of work that satisfies such criteria, it is not possible to cover and discuss of all them in this study. Our goal is to show how VR training can be applied to several major domains through some representative examples.

We limit the scope of this study to VR training. We did not consider training involving other extended reality domains (e.g., augmented and mixed reality). There are many conceivable perspectives through which to review VR training, from software choices to software development, from graphics fidelity to run-time efficiency, and from interaction techniques to interaction realism. To provide a succinct overview, in this article, we discuss VR training from the following perspectives:

Introduce the main technology for creating and delivering VR training experiences (Section 2)

Give an overview of VR training in different domains (Section 3)

Discuss the evaluation methods for assessing VR training effectiveness (Section 4)

Identify open issues and promising research directions for VR training (Section 5)

2 Creation Pipeline of VR Training Experiences

VR training applications have been developed for a variety of domains, including construction safety training ( Filigenzi et al., 2000 ; Nasios, 2002 ; Li et al., 2018 ), natural disaster escape training ( Smith and Ericson, 2009 ; Li et al., 2017 ), education ( Le et al., 2015 ; Wang et al., 2018 ), etc. VR training applications are typically created according to specific tasks characterized by a fixed procedure and a practical purpose for a specific training target. Thus their implementations are generally miscellaneous and different from each other ( Lang et al., 2018 ; Si et al., 2018 ; Taupiac et al., 2019 ).

Xie et al. (2006) explored generalized descriptions of different training scenarios. They integrated the creation process of the virtual reality safety training system with a scenario-based design method to construct a general framework for creating VR training experiences. Martin et al. (2009) , Martin and Hughes (2010) developed software for automatic scenario generation, aiming at exploring the conceptual design of scenario generation approaches. Lin et al. (2002) presented an architecture of VR-based training systems, as well as an introduction to virtual training task planning and some task-oriented training scenario models. Overall, the creation pipeline of VR training can be roughly divided into three stages: 1) task analysis, 2) training scenario sketching, and 3) implementation.

2.1 Task Analysis

One of the first steps that should be considered before training is the task analysis, which is the foundation that lies under the development process of any training objectives. The aim of task analysis was to set up the information needs for the specific training task and to summarize all the factors that should be involved in the training system during the design process. Early attempts for task analysis can be dated back to the 1940s. One famous example is the therbligs that was proposed by Gilbreth and Carey (2013) . Although the therbligs has improved a lot in describing and representing some cognitive tasks such as search and find, Schraagen et al. (2000) compared to original scientific management methods most of them still focused on the description of physical behavior and relatively simple decisions ( Clark and Estes, 1996 ) and usually the task analysis would emphasize the detailed procedures of the task. However, low-level task analysis is gradually replaced by more advanced problem-solving methods that allow not only overt physical behaviors but also complex cognitive tasks. Hierarchical task analysis (HTA) and cognitive task analysis (CTA) are the most famous and widely used ones among those new methods. Generally speaking, instead of merely concentrating on simple tasks, HTA describes “goals,” which are composed of different units, while CTA focuses more on dealing with unexpected situations. Salmon et al. (2010) provided a conclusive comparison of the two methods.

2.2 Training Scenario Sketching

The purpose of scenario sketching, also called scenario design, was to obtain detailed descriptions of how people accomplish a task. For example, Carroll (2000) provides a general definition and introduction for scenario-based design. Farmer et al. (2017) explained the detailed methodology for specifying and analyzing training tasks. Recently, event templates have also been integrated into the scenario description for virtual training environments Papelis and Watson (2018) . These works in devising efficient and comprehensive representations of training scenarios cannot only provide a general framework and inspiring new ideas for designing new VR training applications but also offer a theoretical basis for developing procedural approaches to automatically generate virtual training environments.

2.3 Implementation

2.3.1 software.

While modeling and simulations are the essential components of VR training software nowadays, traditional methods for training are generally based on technical manuals or multimedia resources Chao et al. (2017) . Although such resources are simple to use and easy to produce, trainees lack the immersive experience, and therefore, the training results are not comparable to VR training ( Hamilton et al., 2002 ; Lovreglio et al., 2020 ). In this article, we summarize VR training software in two aspects listed below.

2.3.1.1 Development Tools

A number of tools were developed in the past. Among them, the Virtual Reality Modeling Language (VRML) that was first introduced in 1994, was intended for the development of “virtual worlds” without dependency on headsets. After VRML faded out, a wide array of other programming languages and tools picked up where it left off. Among them we found 3DMLW, which discontinued in 2016, COLLADA 2 , O3D 3 , and X3D 4 , which is the official successor of VRML. Nowadays, game engines are one of the most popular development tools for virtual reality applications, with multiple packages and software development kits that support mainstream VR devices. For example, Unity 3D ( Linowes, 2015 ) and Unreal Engine ( Whiting and Donaldson, 2014 ) are popular choices. Both of them have integrated APIs which VR developers can easily use after installing the corresponding SDKs.

3D modeling is also important in creating virtual training environments. There are plenty of resources such as 3D models, 3D scenes, and sample projects on the Unity’s and Unreal’s asset stores, making the content creation process easier for people who may not have any knowledge or skills in 3D modeling. In addition, if developers want to customize their 3D content, they can use advanced 3D modeling software such as 3ds Max, Maya, and Blender to build and modify 3D models. On the other hand, Virtual Reality Peripheral Network (VRPN) ( Taylor et al., 2001 ) provides local or networked access to various tracking, button, joystick, sound, and other devices used in VR systems.

2.3.1.2 Procedural Generation

Procedural generation techniques are also widely used in creating virtual content such as virtual environments, 3D models, and game level designs. These contents are typically created by designers manually from scratch. The content creation process is usually labor-intensive and time-consuming as it involves a lot of trials and errors. Procedural generation of 3D virtual content is a common approach for creating certain types of 3D content [e.g., terrains Smelik et al., 2009 , cities Parish and Müller, 2001 , and room layouts Yu et al., 2011 ]. As described above, exploring scenario-based descriptions for VR training can help devise procedural generation methods for training scenarios. Related works include Smelik et al. (2011) , Smelik et al. (2014) , Freiknecht and Effelsberg (2017) , Lang et al. (2018) , and Li et al. (2020) . Basically, the general process starts by specifying the training scenario, including training tasks, environments, baselines, and randomness level. Then, based on the parameterized and pre-described scenario, a procedural generation framework is constructed, which applies an automatic process (e.g., a stochastic optimization process) to generate the whole training scenario.

2.3.2 Hardware

The development of VR training is closely related to the advances of VR devices. Before the advances of consumer-grade VR devices and due to the lack of computation power Biocca and Delaney (1995) , many training tasks were conducted using 1) window (computer) systems that provide a portal into an interactive 3D virtual world, 2) mirror systems in which users can observe themselves moving in a virtual world through a projection screen, 3) vehicle systems in which the user enters what appears to be a vehicle and operates controls that simulate movement in the virtual world, and 4) Cave Automatic Virtual Environment (CAVE) systems Cruz-Neira et al. (1992) in which users enter a room where they are surrounded by large screens that project a nearly continuous virtual space.

The ease of access of the current VR devices provides many benefits in VR training. Anthes et al. (2016) summarized the most commonly used VR hardware and software. Although many problems, such as motion sickness, still exist, there have been efforts on identifying suitable situations where available VR devices such as head-mounted displays (HMDs) can help in training skills. For example, Jensen and Konradsen (2018) found that cognitive skills, psychomotor skills, and affective skills can be better enhanced using HMDs, compared to other methods with a lower immersion level.

Besides HMDs, tracking and interaction devices, such as Leap Motion and Kinect sensors, have also been used in previously conducted VR training research projects. For example, Iosa et al. (2015) demonstrated the feasibility of using a Leap Motion sensor for rehabilitation of subacute stroke by conducting therapy experiments among elderly people. Wang et al. (2017) developed a VR training application based on a Leap Motion sensor to facilitate upper limb functional recovery process based on the conclusion drawn in the study by Iosa et al. (2015) . On the other hand, Kinect sensors are frequently used in motor-function recovery training of limbs ( Bao et al., 2013 ; Sin and Lee, 2013 ; Park et al., 2017 ). Bogatinov et al. (2017) explored the possibility for integrating a Kinect sensor to provide firearms shooting training scenarios with a low cost. What is more, sports training applications such as dance ( Kyan et al., 2015 ), motion game ( Zhang et al., 2019 ), badminton ( He et al., 2014 ), table tennis ( Liu et al., 2020 ), and golf ( Lin et al., 2013 ) commonly use a Kinect sensor.

Depth-based sensors like Kinect and Leap Motion provide a direct and convenient solution for basic motion tracking of the human body at an affordable cost. Some companies use motion capture suit to increase the tracking precision. Commercial products like PrioVR, TESLASUIT, and HoloSuit all use embedded sensors and haptic devices to provide full-body tracking and haptic feedback of the user. Besides body tracking, there are devices for improving VR experiences through locomotion. Treadmill-based platforms (e.g., Cyberith Virtualizer, KAT VR, and Virtuix Omni), which physically simulate the feeling of walking and running in VR, are popular devices for VR locomotion. They allow the user to walk in a virtual environment.

Researchers have also investigated the relationship between training outcomes and different VR devices Howard (2019) . Research in comparing the pedagogical uses of different VR headsets has been conducted as VR applications in education and training become more and more popular Papachristos et al. (2017) , Moro et al. (2017) . Although there are some disputes about whether mobile VR headsets tend to induce nausea and blurred vision Moro et al. (2017) , previous results indicate little differences in immersion levels between high-end HMDs like Oculus Rift and low-end HMDs such as mobile VR headsets in aspects such as spatial presence Amin et al. (2016) , usability, and training outcomes Papachristos et al. (2017) .

3 Virtual Reality Training Domains

In this section, we comprehensively review the challenges and approaches of employing VR training in different domains. We reviewed a total of 48 VR training applications and works. Approximately 54% of the works used head-mount displays (HMDs) in their framework, 6% used mobile VR, 10% used CAVE-like VR monitors, and 27% did not specify which VR devices they used. In addition, 12% of the work used additional trackers or sensors, for example, depth sensors to capture hand gesture information. 10% used advanced devices like treadmills or motion platforms to provide immersive navigation experiences. 21% of the works used custom hardware like Arduino, or other 3D-printed parts to satisfy their tasks’ needs. Most of the works (83%) that we reviewed are simulation-based applications, which simulate a real-world scenario for performing certain tasks in virtual reality. Finally, 71% of the works involved user evaluation, where the average number of the participants is 34 ± 29.

3.1 First Responder Training

First responders (e.g., police officers, firefighters, and emergency medical services) frequently face dangerous situations that are uncommon in other work fields ( Reichard and Jackson, 2010 ). First responders’ abilities to deal with critical events increases with the extent of training received. Successful mission completion requires intensive training. In contrast, inadequate training may cause the first responders to fail to complete missions and may even cause personnel injury. Setting up such a training environment in the real world can be expensive and time-consuming ( Karabiyik et al., 2019 ). Training first responders in virtual reality can significantly reduce training costs and improve performances ( Koutitas et al., 2020 ). The training setup can be highly customized to train first responders to deal with critical events effectively. Some government departments have begun to use virtual training as part of combat training. For example, the New York Police Department (NYPD) uses VR for active shooter training ( Melnick, 2019 ).

Wilkerson et al. (2008) proposed a method of using an immersive simulation, delivered through a cave automatic virtual environment (CAVE, see Figure 2 ), for training first responders for mass casualty incidents. A CAVE projected stereo images on the walls and floor to provide a fully immersive experience that included unrestricted navigation, interaction with virtual objects, and physical objects placed in the environment. The training experience was also enhanced by directional sounds. Similarly, a CAVE can be used to design and deliver other extreme, realistic, rare, but significant training scenarios. It allows trainees to train in a safe and immersive environment while reducing training costs.

FIGURE 2 . An example of the cave automatic virtual environment (CAVE). Image courtesy of Wilkerson et al. (2008) .

Conges et al. (2020) proposed an EGCERSIS project which provided a framework for training a crisis management team on critical sites. The team consisted of three actors: a policeman for traffic control, for example, pulling up a cordon to block an area; a fireman for dealing with danger, for example, using a fire hose to extinguish fires and clear obstacles, and a medical doctor for rescuing the wounded, for example, measuring vital signs and bandaging the wounded. Each actor used a unique set of tools to complete his/her mission. Using the framework, Conges et al. (2020) also demonstrated a scenario where a first responder team performed a mission to rescue an injured man lying on the floor of a subway station platform that was on fire. Mossel et al. (2017) proposed a VR platform that enables immersive first responder training on-site with squad leaders. Unlike the training of team members’ professional and practical ability, the training of team leader focuses more on leadership ability. Mossel et al. (2017) designed four leadership indicators in his method: 1) PLAN, 2) DO, 3) CHECK, and 4) ACT. It is referred to as the PDCA cycle. Specifically, the training involves exploring the site, making decisions, giving commands to request supply personnel, evaluating the results of commands, and identifying deviations from the plan, and adapting actions. In the early presentation, Mossel et al. (2017) demonstrated a scenario prototype which simulated a small fire and a traffic accident with injured people and animated bystanders.

Another important VR training project in the first responder domain is firefighting. Backlund et al. (2007) presented Sidh , a gamified firefighter training simulator. The simulator is run on a CAVE. Players interacted with the virtual environment through a set of sensors to complete 16 firefighter training levels. In addition to training professionals, some fire simulators aimed to train the general public about fire safety as well ( Cha et al., 2012 ).

The use of extended reality techniques to train naval firefighters was proposed by Heirman et al. (2020) . Since the Navy has strict firefighter training requirements, they devised a VR controller based on the firehouse used in daily training. Therefore, the trainee can simulate the use of machinery in the real world to the maximum in VR. Vehicles that use hydrogen as fuel may become a reality in the future. However, there are no suitable training tools for dealing with hydrogen vehicle accidents. Pinto et al. (2019) proposed the use of VR on rescue training for hydrogen vehicles accidents. Due to the difference between hydrogen and general gasoline characteristics, they used simulation technology to simulate the scene of a hydrogen vehicle after an accident in VR, providing a suitable platform for the training of rescuers. They demonstrated a solution composed of three components: tutorial mode, training mode, and certification mode.

Although the training of rare and high-risk scenarios are worthy of attention, the common and low-risk scenarios should not be ignored. Along this direction, Haskins et al. (2020) raised a set of challenges in real-world training: communication, situational awareness, high-frequency low-risk incidents, and assessment training. They explored a set of immersive training scenarios for the first responders for such scenarios. The training scenarios included the demolition of a bomb, using a fire truck panel (see Figure 3 ), handling hazardous materials, and monitoring a police checkpoint. In the bomb demolition scenario, a trainee wearing an HMD device played as a first responder, while another trainee played as a supporter giving complex bomb demolition instructions. The supporter gave instructions to the first responder about how to demolish the bomb, while the first responder described what he saw. Similarly, the other training scenarios showed simulated virtual environments that trained first responders to practice specialized missions in a safe and low-cost manner.

FIGURE 3 . A fire truck panel in VR. Image courtesy of Haskins et al. (2020) .

Koutitas et al. (2019) proposed to use the VR platform to train the crews of Ambulance Bus to improve their professional capabilities. Different from general ambulance training, the internal facilities of Ambulance Buses are more complicated and require higher professionalism in work tasks. They demonstrated the training environment inside the ambulance bus in VR and AR, respectively. The user study divided the subjects into three controlled groups, each in AR, VR, and PowerPoint presentation training modes. The results showed that after a week of training, subjects’ error rate in the AR and VR groups was significantly less than that in the PowerPoint presentation group.

Due to the limitations of VR setups, such as the limited range of motion, the common range of motion is within 3 × 2 m 2 , which cannot well simulate a training that requires an extensive range of movement. For example, long-distance chasing training cannot be conducted. Besides, there is also the lack of operation feedback, so it does not function well in training scenarios that require a fine operation. For example, if a nurse does not bandage the wounds of a patient properly, it may increase the suffering of the patient. It could be challenging to simulate the responses of a patient realistically in VR-based nurse training.

VR training will likely supplement rather than replace the traditional training model ( Heirman et al., 2020 ). Compared with the traditional model, VR training’s low-cost and rapid deployment characteristics are suitable for training novices who are just getting started. The traditional training mode can be used as an assessment method after the training is completed. In the future, the alternate uses of traditional training mode and mature VR training mode will reduce the cost and improve the quality of training.

3.2 Medical Training

Researchers have thoroughly explored using virtual reality for medical training. Virtual reality has been used for treatment, therapy, surgery and creating experiences by simulating impairments. Previous work suggested that using a properly designed simulation-based training can significantly reduce errors of health-care workers and improve patient safety ( Salas et al., 2005 ). Furthermore, researchers tackled the challenges of creating realistic training experiences for medical professionals by enhancing virtual patient systems.

Some VR programs were created to treat visual impairments, while others aimed to further the understanding of visual impairments through simulations. Hurd et al. (2019) explored the usage of VR video games for Amblyopia treatment. Amblyopia, commonly referred to as lazy eye, is a vision disorder occurring due to faulty connections between the eyes and brain. Typical treatments of the disorder involve occluding the patient’s dominant eye via wearing an eye-patch to strengthen the neurological connection between the weak eye and the brain. Hurd et al. (2019) designed engaging video games that were played using a VR headset with blurred vision. These therapy sessions were found to improve the visual acuity of participants in a user study.

ICthroughVR ( Krösl et al., 2019 ) was created to simulate the effects of cataracts using VR as shown in Figure 4 . This simulation is useful for evaluating architectural designs. More specifically, it evaluates lighting conditions along with the placement of emergency exit signs. The researchers adaptively reduced the HMD’s contrast to simulate these effects. They similarly blurred, clouded, and tinted the user’s vision. The researchers also utilized eye-tracking to simulate light sensitivity by monitoring changes in the user’s pupil size.

FIGURE 4 . ICthroughVR simulates the effects of cataracts in VR. Image courtesy of Krösl et al. (2019) .

Medical professionals also investigated using VR for surgical training. Gallagher et al. (1999) compared traditional training with VR training in the “fulcrum effect” of the body wall in surgical instructions. The results indicated that the VR group performed significantly better than the traditional group. Grantcharov et al. (2004) also validated that VR training outperformed traditional training using a randomized clinical trial of the performance of laparoscopic cholecystectomy. Similar results have been found in the studies by Westwood et al. (1998) , Seymour et al. (2002) , Gurusamy et al. (2009) .

Researchers ventured to use VR to mitigate phantom limb pain for amputees ( Fukumori et al., 2014 , Ambron et al., 2018 ; Willis et al., 2019 ). In mirror therapy, a mirror is placed in front of the amputee’s affected limb to block it and to reflect the patient’s unaffected limb. This simulates motion in the affected limb and stimulates the corresponding areas of the brain, thus reducing pain. Virtual mirror therapy was conducted on amputees by placing them in virtual environments and using a data glove or visual tracking to simulate the affected limb’s movement ( Fukumori et al., 2014 ; Ambron et al., 2018 ). Researchers also incorporated tactile sensation into the therapy session ( Willis et al., 2019 ).

Several works explored the use of virtual agents to facilitate health-care training. Hochreiter et al. (2015) demonstrated the use of a projector and infrared (IR) camera setup to render an animated physical–virtual patient. The IR cameras enable the patient to sense touch events and react to them with an animation event. Daher et al. (2018) studied the efficacy of this physical–virtual patient in a medical training scenario, which was designed to train health-care professionals to assess patients who show neurological symptoms of a stroke. The researchers compared the physical–virtual patient with a high-fidelity touch-aware mannequin and found that the physical–virtual patient improved the health-care training experience for subjects.

VR also shows significant use for post-war recovery. Rizzo et al. (2014) used a VR system as an exposure therapy (VRET) tool to help soldiers suffering from posttraumatic stress disorder (PTSD). A Middle East–themed virtual city showing deserts and roads, called Virtual Iraq, was created to mimic the experience of soldiers deployed in Iraq. A clinical trial protocol was conducted with 20 active-duty service members who were recently redeployed from Iraq. These soldiers experienced no improvement under traditional PTSD treatment, including group counseling and medication. After 5 weeks of VRET treatment (each soldier received less than 11 sessions of treatment on average), 16 out of the 20 students no longer showed significant signs of PTSD. They showed a significant reduction in anxiety and depression. The improvement in their condition still persisted at a 3-month follow-up posttreatment.

Researchers also studied the usability of virtual patients in clinical psychology. Kenny et al. (2009) created a virtual patient simulating PTSD in the form of a 3D character that is responsive to clinicians’ speech and gestures. The researchers also conducted a user study to evaluate psychological variables like immersiveness and presence. They also assessed the clinician’s openness to absorbing experiences ( Huguet et al., 2016 ), unlike other works that create virtual experiences mimicking traditional training, exploring the use of perturbed communication for medical training. Huguet et al. (2016) created a multiagent virtual experience, simulating a hospital. The agents within the hospital are equipped with an erroneous communication model to induce stress for the trainee.

Due to the importance of tactile feedback in medical professions, most training tools require tailor-made equipment. For example, Gallagher et al. (1999) and Grantcharov et al. (2004) used a VR setup outfitted with two laparoscopic instruments that the surgeon manipulated to simulate a surgical procedure. Moreover, Hochreiter et al. (2015) and Daher et al. (2018) built a specialty rig to simulate a physical–virtual patient. In this subfield of training, researchers must not only provide a faithful visual VR experience but also develop realistic and responsive physical devices. The creation of such training devices and experiences requires expertize in rendering, simulation, haptics, psychology, and physiology.

3.3 Military Training

Military training could be dangerous and risky. Many soldiers die due to noncombat causes or accidents every year ( Mann and Fischer, 2019 ). Technologies have allowed us to simulate different conditions of environments, such as day and night, different weather types, and other scenarios ( Rushmeier et al., 2019 ; Shirk, 2019 ). Militaries began to use simulation software or serious games as training tools. Hussain et al. (2009) developed the Flooding Control Trainer (FTC) to train new recruits at the U.S. Navy in various skills. U.S. Aviation began to experiment with VR training to supplement the traditional hands-on training through the Aviator Training Next (ATN) program. Their preliminary results suggested that the VR training produced pilots with comparable quality and competence as pilots trained in a live aircraft ( Dalladaku et al., 2020 ). Traditional training methods are often constrained by the real training environment and some specific devices or equipment. VR training provides a safe, controlled virtual environment at a relatively low cost for militaries to practice their technical skills and to improve their cognitive functions ( Zyda, 2005 ).

Many traditional training methods require specific locations or sets of equipment. For example, the Landing Signal Officer Trainer, Device 2H111, is located in Oceana, Virginia. It is used to prepare all Landing Signal Officers (LSOs) for past decades. In order to create an immersive environment, the room consists of several large screen displays and physical renditions of the actual instructions. Due to these constraints, each LSO needs to share the room which houses the equipment, leaving gaps between training periods. Greunke and Sadagic (2016) created a lightweight VR LSO training system, as shown in Figure 5 , that uses commercial off-the-shelf devices to support the capabilities of Device 2H111. Besides the basic capabilities of 2H111, the VR LSO training system included additional features that the 2H111 did not provide, such as speech recognition and visual cues to help trainees in the virtual environments. This approach was presented to real world LSOs at an LSO school, where a side-by-side comparison was made. The feedback was positive. The VR training system supports all elements needed for 2H111 with additional features. Similarly, Doneda and de Oliveira (2020) created the Helicopter Visual Signal Simulator (HVSS) to help LS trainees better improve their skills after theoretical studies. A user study was conducted with 15 LSOs, and the results demonstrated that the HVSS is comparable with the actual training, while incurring a much lower cost.

FIGURE 5 . A user demonstrates using the Landing Signal Officer Display System for training. Image courtesy of Greunke and Sadagic (2016) .

Furthermore, Girardi and de Oliveira (2019) observed that the training of army artillery is often constrained by the high costs and the limited instruction area. They presented a Brazilian Army VR simulator where the instructor can train observers at a minimum cost and without space limitation. The simulator was validated with 13 experienced artillery officers. The results indicated that the simulator attained the expectations for both presence and effectiveness. Siu et al. (2016) created VR surgical training system specified for health-care personal transitioning between operation and civilian. The training system incorporates the needs and skills that are essential to the militaries and adaptively generates optimal training tasks for the health-care workers to improve their efficiency in reskilling.

Another similar scenario where the VR greatly improved training quality and condition compared to the traditional training method was presented by Taupiac et al. (2019) . The authors created virtual reality training software that helps the French Army infantrymen to learn the calibration procedure of the IR sights on a combat system called FELIN. In order to practice on the actual FELIN system, the soldiers need to repeatedly practice on the traditional software until they make no mistake. Traditional system only provides a 2D program for soldiers to practice calibration procedure, whereas the new VR method allows them to practice in a virtual environment with 3D-printed rifle model, as shown in Figure 6 , which provides a similar control and feeling of use as its real-world counterpart. An ad hoc study was conducted on a group of French soldiers to compare these two methods. The result indicated that the VR method greatly improved soldiers’ learning efficiency and their intrinsic motivation to perform training tasks.

FIGURE 6 . A 3D-printed rifle rack to simulate the actual rifle feeling for soldiers to practice the FELIN in VR. Image courtesy of Taupiac et al. (2019) .

Overall, VR technologies show significant effects in technical training. These technologies greatly reduce military training expenses by replacing some part of the expensive equipment with affordable VR devices. VR devices will not fully replace traditional equipment, but can serve as a more affordable alternative for training. Researchers Greunke and Sadagic (2016) , and Taupiac et al. (2019) have validated that the effect of VR training is comparable to the actual training in some cases, for example, for improving the proficiency of some military equipment usage.

3.4 Transportation

Vehicle simulators provide a safe, controlled training environment which allows people to practice before operating the actual vehicle. Companies use professional simulators such as Iowa driving simulators ( Freeman et al., 1995 ) to test their products or simulate the driving experiences. These platform-based simulators are usually bulky and very expensive for general public use. On the other hand, traditional flat-screen–based training program could be ineffective due to the lack of immersiveness. In current practice, people go to a driving school or hire a human coach to learn driving, which is labor-intensive and could be costly. Moreover, behind-the-wheel training could lead to a high anxiety of the trainees, resulting in a steeper learning curve and a longer training time. As VR devices are getting more affordable, researchers start to explore the possibilities of simulated driving training in virtual environments. VR filled the gap between platform-based and flat-screen–based simulators. A major advantage of VR simulation training is that it allows people to practice under immersive, simulated dangerous scenarios.

To improve driving habits, Lang et al. (2018) created a system to automatically synthesize a training program that improves bad driving habits of drivers ( Figure 7 ). The system uses an eye-tracking VR headset to detect common driving errors such as not checking blind spots or not paying attention to pedestrians on the street. Based on the collected eye movement data, the system synthesizes a personalized training route through which the trainee will encounter events and practice their skills, hence improving their driving habits. The results indicated that the personalized VR driving system improved drivers’ bad driving habits. The training effect is more significant and long-lasting compared to that of traditional training methods (e.g., booklets, videos).

FIGURE 7 . A user improves her driving skills through a personalized VR driving training program. Image courtesy of Lang et al. (2018) .

Besides vehicle training simulators, VR is also employed for assistive transportation training, such as wheelchair training, for people with physical or cognitive disabilities. The history of VR wheelchair training can be traced back to the late 1990s, when Inman et al. (1997) first applied VR technology for simulated wheelchair training. They found that children were more motivated in VR training compared to traditional training. Even for some children with significant learning disabilities, they were still able to concentrate on the assigned tasks and follow instructions to gain operating skills. However, there were some children who favored flat-screen over HMDs for training, mainly due to the low-computing power and the technological limitations of HMDs at that time. Furthermore, Harrison et al. (2002) conducted an assessment which suggests that virtual environments are potentially useful for training inexperienced wheelchair users.

In view of the recent advancements of VR technology, John et al. (2018) conducted a validation study to compare VR training with traditional wheelchair training simulators. They created several tasks in the VR environment, including navigating through a maze, a room with obstacles, and other designed paths. The experiments showed a significant improvement in navigation performance and driving skills resulting from the use of the VR training system. Niniss and Nadif (2000) first explored simulating the kinematics behavior of powered wheelchairs using VR. Herrlich et al. (2010) adopted driving characteristics that are commonly used in electric wheelchairs to create virtual simulated wheelchairs. The system was validated through a physics simulator engine. Rodriguez (2015) later developed a wheelchair simulator that can help disabled children familiarize themselves with a wheelchair. In addition, Li et al. (2020) developed an approach to automatically synthesizing wheelchair training scenes for VR training ( Figure 8 ). The approach was able to generate different scenarios with training paths of different moving distance, numbers of rotations, and narrowness, for practicing different wheelchair control skills. Participants showed improvement in wheelchair control skills in terms of proficiency and precision after training with the generated scenes in VR.

FIGURE 8 . A user performs a wheelchair training in VR. Courtesy of Li et al. (2020) .

The vehicle and wheelchair VR training examples demonstrated that VR training is suitable for improving driving skills. Furthermore, research indicated that the participants are more concentrated and involved in VR than traditional driving training through a monitor ( Lhemedu-Steinke et al., 2018 ). As mentioned previously, VR reduces the risks of training; beginner drivers can improve their driving skills in the virtual environment without risks of accidents.

VR also benefits assistive training. For example, those who suffer from lower body disability due to accidents or sickness may need to practice the use of a wheelchair. VR provides a possible solution for wheelchair training in virtual environments. Overall, VR training in the transportation field is not limited to car driving or wheelchair control training. Researchers also explore the possibilities of other vehicle control training via VR, such as aircraft control ( Wang et al., 2019 ), excavator operation ( Sekizuka et al., 2019 ), and forklift operations ( Lustosa et al., 2018 ), among others.

3.5 Workforce Training

Many industrial projects and research have been devoted to creating or adopting virtual reality training for their needs as VR could reduce the loss of productivity during training and resolve the lack of qualified training instructors. For example, Manufacturing Extension Partnerships (MEPs) established by the U.S. government offers virtual training and resources to manufacturers for maintaining productivity on top of improving employee stability ( Brahim, 2020 ). Retail businesses such as Walmart and Lowe’s use VR to deliver common and uncommon scenarios and train their employees to achieve a faster assimilation process ( VIAR, 2019 ). Besides, VR vocational training could mitigate unemployment by offering a variety of hands-on experiences and workplace exploration to reskill the workforce. For example, Prasolova-Førland et al. (2019) presented a virtual internship project that introduced a variety of occupations as well as workforce training for young job seekers, while Baur et al. (2013) created a realistic job interview simulation that used social cues to simulate mock interviews. Both virtual simulators help fill the communication gap between the employers and job seekers in the labor market.

It is critical for a training session to faithfully mimic a real working environment and possible challenges that a worker may face in the workplace. Compared to a traditional training environment, virtual reality provides a low-cost and replicable training to satisfy such needs while offering a mistake-tolerant environment to relieve stress that a new worker may have Mechlih, 2016 ). For example, Di Loreto et al. (2018) created a virtual reality work-at-height simulator to reproduce real situations with high fidelity for training. Such a simulator not only allows companies to train their employees for performing work-at-height operations but also detects if new workers have acrophobia and help them assimilate to a work-at-height environment. Similarly, training for power line operation and maintenance are necessary to prevent injuries or fatalities. Borba et al. (2016) have worked on a procedure-training simulator that focused on workplace safety and risk assessment of common power line maintenance and operation. This simulator shown in Figure 9 included a real basket and a stick to simulate daily challenges imposed on the operators in an urban setting. It also enabled student workers to learn from the mistakes and successes of their peer’s performances. Moreover, a virtual application for safety training in the construction site was developed and assessed ( Hafsia et al., 2018 ). Many researchers agreed that VR training could add value by reducing safety and health issues for high-risk occupations. For example, immersive virtual construction safety training experiences were developed ( Li et al., 2012 , Bhoir and Esmaeili, 2015 ; Le et al., 2015 ; Froehlich and Azhar, 2016 ). Sacks et al. (2013) indicated that virtual construction safety training is feasible and effective, especially in terms of teaching workers to identify and inspect construction safety risks. Aati et al. (2020) developed an immersive virtual platform for enhancing the training for work zone inspectors. Through the simulation of VR-based safety training programs developed by Zhao and Lucas (2015) , users can effectively enhance their hazard inspection skills.

FIGURE 9 . A user learning power line operation and maintenance skills in VR. Image courtesy of Borba et al. (2016) .

Many scholars and industries have extensively researched integrating VR with virtual and physical factories by providing high-fidelity simulation tools to handle the concurrent evolution of products, processes and production systems ( Tolio et al., 2010 ). There is a growing trend for designing VR training in assembly and manufacturing contexts. A study evaluated VR assembly training in terms of the effectiveness and transfer of training and found that VR training enhances assembly time and reduces error rate compared to traditional or baseline training groups Abidi et al. (2019) . Besides, VR training can save costs and reduce risks for normal factory operation practice such as the risk of machinery and equipment operation, the risk associated with the movement of workers during work, and the risk of emergencies ( Lacko, 2020 ). However, VR training does not always enable collaboration with others in real time. Yildiz et al. (2019) proposed a collaborative and coordinated VR training design model for the wind turbine assembly scenario. This simulation model overcomes the challenge of multiuser VR data integration with virtual and physical factories.

Last but not least, both employees and employers could benefit from workplace VR training. On one hand, new workers can familiarize themselves with the workplace environment and operation through replicable, mistake-tolerant VR training. Employers can use the training results to evaluate whether their employees are ready to work in the field. With a headset, as well as a few motion capture sensors and motion trackers, workplace VR training research has mainly focused on two aspects: 1) improving workers’ skill in performing machinery and equipment operations (e.g., work-at-height training), 2) raising workers’ awareness at work by providing safety and risk assessment training for high-risk occupation (e.g., hazard inspection at a construction site).

However, group training for emergency response (e.g., coping with fire and earthquake emergencies) still has much room for development. Multiuser VR training in real time is still an open research and engineering challenge. Moreover, we observe that most VR training setups use handheld controllers to capture the hand motion of users. For a complex task (e.g., wind turbine assembly), hand tracker devices like VR gloves are needed for capturing a trainee’s detailed hand motion to evaluate his work performance. We expect that the advancement and popularity of VR gloves will lead to more realistic and practical VR training applications.

3.6 Interpersonal Skills Training

Aside from training for highly specialized technical skills, VR has recently shown promise for interpersonal skills training. As VR hardware and software improve, the potential for rich interactions among people—a crucial factor for interpersonal skills development—becomes more likely, especially due to the psychological experience of “being there” ( Colbert et al., 2016 ). For instance, VR has recently shown promise for investigating communication in health-care teams, although little work has examined VR in training settings ( Cordar et al., 2017 ). VR has also shown promise for helping people collaboratively build plans for startup companies while having to negotiate team roles as they progress ( Kiss et al., 2015 ). Similarly, VR has also demonstrated effectiveness in intercultural competence development ( Akdere et al., 2021a ).

A recent meta-analysis suggests that ongoing technical advances position VR to deliver effective training for human resource management, including issues related to recruiting and selecting new employees, and increasing knowledge and skills of current employees ( Stone et al., 2015 ). Furthermore, VR is being combined with other emerging technologies, such as haptic interfaces, to support workforce training ( Grajewski et al., 2015 ). The most critical interpersonal skills include openness, empathy, and verbal and nonverbal communication.

3.6.1 Openness

Scholars have long recognized the importance of the affective and behavioral learning dimensions to honing communication skills ( Greene, 2003 ); in other words, cognitive development alone is insufficient. Good communicators, those who are best suited to maximize organizational tacit knowledge both as senders/speakers and receivers/listeners, need positive attitudes. Intercultural openness is widely recognized as an important component of intercultural competency. For example, in the AAC&U’s VALUE rubric for intercultural knowledge and competence, the attitude of openness involves both willingness to interact with culturally different others and the capacity to suspend judgment and consider alternate interpretations during those interactions.

Many other models of intercultural competency, including Deardorff’s Pyramid model ( Deardorff, 2006 ), highlights the importance of openness. In fact, Deardorff’s model places openness at the base of the pyramid, implying that development of intercultural competency cannot continue without this foundational attitude. As such a well-recognized and foundational aspect of intercultural communication, openness is therefore a suitable focus for the VR-based training simulations. Given the nature of openness, a VR environment presents unique training opportunities allowing learners to fully experience controversial issues and topics and therefore enabling them to reflect on their individual convictions and opinions, thus developing the competence for being open. Contributing to the formation of cyberculture VR training “is shown to bring the functional conditionality to the forefront” ( Gilyazova, 2019 ). Openness is also critical in team settings. In a study among surgeons, personal traits such as openness demonstrated “an important role in team performance, which in turn is highly relevant for optimal surgical performance” ( Rosenthal et al., 2013 ). VR is proven to be “particularly useful for promoting change within students’ conceptualizations of representation, narrativity and affect” ( Dittmer, 2010 ), thus promoting the notion of openness.

One way to evaluate openness development in the VR environment by using a pre- and posttest design, as well as a valid and reliable scale such as the Attitudinal and Behavioral Openness Scale ( Caligiuri et al., 2000 ). This demonstrably valid instrument is a self-report Likert-scale survey of 16 items that is easily combined with other measures for online administration of data collection in pre- and posttraining assessments. We also recommend a direct measure for openness. Both at the beginning and the end of the VR simulation, one could collect data on a performance task incorporated seamlessly into the storyline. This task can involve choosing team members from a number of culturally diverse groups; the homogeneity or heterogeneity of teams that participants construct will function in this case as a proxy for their willingness to engage across cultural differences.

3.6.2 Empathy

Empathy is a complex skill that involves understanding the experiences, perspectives, and emotions of others. Intercultural empathy is especially challenging because the cultural differences often function as barriers to such understanding of others. Deardorff places it toward the top of the pyramid in her model because the performance of this skill often requires positive attitudes such as openness as well as a certain threshold of cultural knowledge. Without these foundational components, empathetic responses are difficult, if not impossible. A recent study on the efficacy of VR environments in developing empathy in psychology education reported that “VR-based simulations of psychopathology may offer a promising platform for delivering a constructionist approach” ( Formosa et al., 2018 ). Similarly, in the field of history, researchers argue that a VR environment is effective in helping learners develop historical empathy, explain why it is important, and discuss how it is taught ( Sweeney et al., 2018 ). van Loon et al. (2018) explored whether VR is driven by increased empathy and whether the effect extends to ostensibly real-stakes behavioral games, and they reported that VR can increase prosocial behavior toward others.

In the context of intercultural competence, empathy is important because members of different cultures often struggle not only to understand the perspectives of other cultures but even to recognize those perspectives as valid ( Johnson and Johnson, 2010 ). A VR training environment presents a new opportunity to help individuals develop empathy. An instrument that was developed to measure empathetic responses across ethnic and racial differences is the Scale of Ethnocultural Empathy Wang et al. (2003) . This self-report instrument encompasses 4 factors: empathetic feeling and expression, empathetic perspective-taking, acceptance of cultural differences, and empathetic awareness. The instrument includes a Likert scale of 30 items and may be included with the other measures used for pre- and posttest design.

3.6.3 Verbal and Nonverbal Communication

Regardless of what other components a scholar may or may not include in various definitions of intercultural competence, verbal and nonverbal communication skills are a constant across all models ( Spitzberg and Changnon, 2009 ). Roberts et al. (2016) demonstrated “an approach in which a wide range of nonverbal communication between client and therapist can be contextualized within a shared simulation, even when the therapist is in the clinic and the client at home.” They ensured that VR is reported to allow participants to use nonverbal communication grounded in both the experience of the exposure and the current surroundings. Allmendinger (2010) studied “whether the actual use of nonverbal signals can affect the sense of social presence and thus help to establish and maintain the learner’s motivation and provide support for structuring social interaction in learning situations,” and reported that nonverbal communication can “support the conversational flow,” “provide cognitive support,” and “express emotions and attitudes.”

Verbal communication, on the other hand, is also critical for intercultural competence. In a study exploring enhancing verbal communication through VR environments in medical education, Chellali et al. (2011) reported that VR training “improves user’s collaborative performance.” Houtkamp et al. (2012) reported that “users may experience higher levels of presence, engagement, and arousal when sound is included in the simulations.”

Undoubtedly, the capacity to express oneself clearly and appropriately and the reciprocal ability to accurately interpret the verbal and nonverbal messages of others are vital to interacting successfully across cultural differences. The use of effective communication strategies can be measured with the Communication Scale ( Barkman and Machtmes, 2001 ). This 23-item Likert-scale survey targets communication habits such as clearly articulating ideas, paying attention to the ways that others feel, using one’s own body language to emphasize spoken messages, using nonverbal signals to interpret their speech, and listening actively. This scale can also be included in the pre- and posttest design.

4 Assessment and Evaluation Methods

Training evaluation has traditionally focused on identifying immediate participant reactions to the overall training and on measuring the participant’s level of learning upon completion of training. Organizational calls for more accountability during the last decade shifted the focus to return on investment ( Akdere and Egan, 2020 ). Current technological advancements have impacted both the delivery of training (virtual and simulated) and the evaluation of training (biometrics and analytics). VR simulation training can be assessed and evaluated through a number of modes of inquiry such as quantitative and qualitative.

4.1 Quantitative Measure

The most commonly used method is the quantitative method in general, and the pre- and posttest experiment design in particular. Measurements (such as the participant’s performance) are taken prior to and after a treatment. The measurements typically take the form of a set of questionnaires before the treatment and the same set of questionnaires administered after the intervention. This enables the evaluation of any impact of training on the participant. Using various statistical analysis techniques (e.g., t-tests), the quantitative approach allows us to determine if the training induces any statistically significant improvement in performance.

As a newly established form of research, biometrics is defined as the use of distinctive, measurable, and physical characteristics to describe individuals, which can generally be divided into two groups: those that serve as identifiers of the body (such as fingerprints and retina scanning) and those that pertain to psychophysiological measures of human behavior and psychological state ( Jain et al., 2007 ). These include the following:

Eye-tracking, which measures the distance between the pupil center and the reflection of the cornea, revealing the fixation and duration of an individual’s eye movement behavior ( Bergstrom and Schall, 2014 ).

Facial expression analysis (fEMG), which records facial muscle activities via electrodes placed on two major facial skin regions in response to different emotion expressions ( Yoo et al., 2017 ; McGhee et al., 2018 ).

Galvanic skin response (GSR), which reflects the variation of skin conductance caused by changes of sweat glands ( Tarnowski et al., 2018 ).

Heart rate variability (HRV), which indicates an individual’s variance of heart rate speed ( Williams et al., 2015 ).

GSR, HRV, and eye-tracking are the most widely used biometric indices because of their ability to predict the affective state of the subject ( Cacioppo et al., 2007 ). Inclusion of biometric measurements in VR training helps researchers collect and analyze the emotional feedback of the trainees, broadening researchers’ understanding of VR experiences in addition to traditional evaluation approaches ( Hickman and Akdere, 2018 ). Biometrics is best used in conjunction with conventional research methods, as physiological responses to stimuli are involuntary and beyond conscious control. To sum, biometrics can provide valuable information as to the emotional and attentive state of the trainee in real time, without the limitations of language or bias. However, concerns for biometric data safety and ethics should also be considered as part of the biometric-based assessment and evaluation approaches ( Akdere et al., 2021b ).

4.2 Qualitative Evaluation

The next approach to training evaluation is qualitative design, which has interpretivism, epistemological orientation, and constructionism as its main ontological orientation ( Bryman, 2006 ). It is typically used to describe complex phenomena ( Zikmund et al., 2010) . For VR training, qualitative research helps us understand an individual’s experiences related to a phenomenon. Such understanding is critical to enable and better support the “transfer of training,” which plays an important role as organizations equip their workforce with the relevant knowledge, skills, and abilities required to perform various job activities ( Burke and Hutchins, 2007 ). However, according to Grossman and Salas (2011) , the concept of transfer of either learning or training is a source of continuous study and debate, making it difficult for organizations to determine exactly what critical factors should be addressed when dealing with this topic. Using the Baldwin and Ford (1988) model of transfer and identify, Grossman and Salas (2011) suggest that “the factors relating to trainee characteristics (cognitive ability, self-efficacy, motivation, and perceived utility of training), training design (behavioral modeling, error management, and realistic training environments), and work environment (transfer climate, support, opportunity to perform, and follow-up)” are the ones “that have exhibited the strongest, most consistent relationships with the transfer of training.” Bossard et al. (2008) also raise important concerns about the transfer of learning and training in virtual environments. The authors point out the importance of focusing on the content in any training effort, referring to Detterman (1993) ’s view on training transfer, which states that it is important to train people on the exact areas/topics needed for people to learn.

Another potential challenge of the transfer of training in virtual environments is situated learning. Lave argued that knowledge and skills are context-bound and consequently pose the challenge of conceptually learning in a simulated virtual environment and practically applying it to a real-life environment (transfer of training) ( Lave, 1988) . The critical role of the design of virtual environments and its impact on the transfer of training should be taken into account, starting with the design stage rather than posttraining, while embedding the critical learning elements that will be assessed as part of the evaluation of training ( Bossard et al., 2008 ). This view is in line with Grossman and Salas’ identification of “training design” and “realistic training environments” as critical factors to the improvement of transfer success 2011.

Compared to situations with positive or no training transfer, negative training transfer raises more concern because it not only creates no value but further detriments the organization ( Wagner and Campbell, 1994 ). Negative transfer occurs when the training outcome hinders real-world performance or even confuses the employee about the work instead of helping the employee improve, further risking the effective utilization of employee’s knowledge, skills, and abilities at work, and complicating organizational efforts on training. Such problems can definitely happen in VR-based training when the virtual environment is ineffective at simulating and reflecting real-world tasks ( Rose et al., 2000 ; Michalski et al., 2019 ).

Some virtual environments fail to provide haptic feedback to trainees when performing tasks. Specifically, handlers and 3D mice are common input devices in a VR training platform. By clicking and holding the triggers/button, trainees manipulate objects through grasping or pulling them in the virtual environment ( Hamblin, 2005 ). Such activities can be inaccurate compared to performing tasks in the real-world that require high-level motor skills, such as surgery and sports ( Issurin, 2013 ; Våpenstad et al., 2017 ). When accuracy and precision are the priority of tasks, the VR training platform should combine visual, auditory, and haptic sensors together to imitate the real-world environment. Therefore, it is vital for the VR-based training design to incorporate the training transfer phase into the process to achieve higher training transfer rates among VR training participants.

For VR training, self-reported participant reflection is a commonly used qualitative approach which can be conducted verbally or by text. This method provides the participants with an opportunity to reflect more deeply about their training experiences through probing. The reflection inquired about how the training experience affected the participant, what new insights it generated, how the participant is going to feel/behave differently after the training, and so forth. Overall, the self-reported reflection method not only serves as a training evaluation method for VR-based simulations but also provides a further learning opportunity to participants.

5 Summary and Future Directions

We introduced the VR training creation pipeline, including training scenario sketching and implementation of VR training applications. We reviewed some of the latest works from many important VR training domains. We also discussed how to assess and evaluate VR training. We discuss some future directions of VR training in the following.

5.1 Future Devices

To improve VR experiences, companies are working on improving the resolution and field of view of VR headsets. There are efforts on improving VR experiences from other perspectives too, for example, using haptics to increase the sense of presence (SoP). In current practice, the haptics of VR experiences are often quite limited. Most VR setups use fixed-shape controllers with a joystick. The haptic feedback is provided by some simple vibrations generated by a motor. The latest controller from Valve called Index provides hand recognition which detects finger movements. Fang et al. (2020) developed a device called Wireality which enables a user to feel by hands the shape of virtual objects in the physical world.

Besides haptics, other devices try to improve the immersiveness of VR from other perspectives. For example, FEELREAL Sensory Mask can simulate more than 200 different scents for a user to smell in VR ( FEELREAL, 2020) . Cai et al. (2020) developed pneumatic gloves called ThermAirGlove , which provide thermal feedback when a user is interacting with virtual objects. Ranasinghe et al. (2017) developed a device called Ambiotherm which uses a fan and other hardware to simulate temperature and wind condition in VR. Future VR training applications can incorporate these devices to enhance the realism and immersiveness of training.

5.2 Networked VR

Networked VR enables multiple users to perform training simultaneously in a shared virtual space. Currently, applications such as VRChat and Rec Room allow multiple users to explore and chat in a virtual area for entertaining purposes. While the Internet enables people to “chat” with a stranger across the sea, networked VR enables people to “meet” at the same place. With virtual avatars, hand gesture recognition, facial emotion mapping, and other upcoming technologies, networked VR experiences will facilitate people around the world to socialize and collaborate. On the other hand, improvements in network quality and speed will make networked VR even more accessible. The fifth-generation (5G) network and cloud computing make it possible for people to use VR without an expensive local setup. To sum, networked VR will likely make VR training more available, accessible, and scalable.

5.3 Multiuser Training

Multiuser VR training is very useful in some domains. For example, first responders such as firefighters usually work in a squad. Each person in the squad has specific duties. To maximize the performance of a squad, seamless cooperation is needed. Multiuser VR training provides an affordable solution to individuals such that they can practice their specialized skills and communication skills whenever and wherever they want. For example, a group of medical professionals could practice surgery in VR even if some of the members are located remotely. In fact, NYPD has started to use collaborative VR training software for active shooter training ( Melnick, 2019) . Apart from emergency training, multiuser VR training could motivate and engage groups of learners. Multiuser training could induce positive peer pressure, where individuals support each other during the training process.

5.4 AR-Based Training

Augmented reality (AR) overlays real-world objects with a layer of virtual objects or information. Common AR devices include AR headsets (e.g., Microsoft Hololens and Magic Leap headsets) and AR-supported smartphones. Compared to navigating fully immersed VR scenes, users have more control over the physical world when using AR devices. AR has been applied for training in domains such as manufacturing, construction and medical ( Rushmeier et al., 2019) . For example, AR reduced the difficulty and workload for mechanics by adding extra labels and indicators to the machinery during training and maintenance ( Ramakrishna et al., 2016 ). Medical professionals adopt AR for teaching neurosurgical procedures ( de Ribaupierre et al., 2015 ; Si et al., 2018 ). AR training applications are also developed for improving work efficiency on construction sites ( Kirchbach and Runde, 2011 ; Zollmann et al., 2014 ). Continuous advancements in AR devices and computer vision techniques (e.g., object recognition and tracking) promise to enhance the applicability of AR for in situ training in many real-world scenarios.

Author Contributions

BX: manuscript preparation, figure design, and figure captions. HL, RA, YZ, FL, YJ, CL, and WL: manuscript preparation. HH, MA, CM, and L-FY: survey design, overall editing, and project oversight. All authors contributed to the article and approved the submitted version.

Publication of this article was funded by Purdue University Libraries Open Access Publishing Fund. This study was supported by two grants (award numbers: 1942531 & 2000779) funded by the National Science Foundation.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

The authors would like to thank their colleagues from different disciplines on providing insights about the application of VR to different domains.

1 https://www.merriam-webster.com/dictionary/training

https://www.khronos.org/collada/

3 https://code.google.com/archive/p/o3d/

4 https://www.web3d.org/x3d/what-x3d

Aati, K., Chang, D., Edara, P., and Sun, C. (2020). Immersive Work Zone Inspection Training Using Virtual Reality. Transp. Res. Rec. 2674 (12), 224–232. doi:10.1177/0361198120953146

CrossRef Full Text | Google Scholar

Abidi, M. H., Al-Ahmari, A., Ahmad, A., Ameen, W., and Alkhalefah, H. (2019). Assessment of Virtual Reality-Based Manufacturing Assembly Training System. Int. J. Adv. Manuf. Technol. 105, 3743–3759. doi:10.1007/s00170-019-03801-3

Akdere, M., and Egan, T. (2020). Transformational Leadership and Human Resource Development: Linking Employee Learning, Job Satisfaction, and Organizational Performance. Hum. Resour. Dev. Q. 31, 393–421. doi:10.1002/hrdq.21404

Akdere, M., Acheson, K., and Jiang, Y. (2021a). An Examination of the Effectiveness of Virtual Reality Technology for Intercultural Competence Development. Int. J. Intercult. Relat. 82, 109–120. doi:10.1016/j.ijintrel.2021.03.009

Akdere, M., Jiang, Y., and Lobo, F., (2021b). Evaluation and Assessment of Virtual Reality-Based Simulated Training: Exploring the Human-Technology Frontier. Eur. J. Train. Dev . doi:10.15405/epsbs.2021.02.17

Allmendinger, K. (2010). Social Presence in Synchronous Virtual Learning Situations: The Role of Nonverbal Signals Displayed by Avatars. Educ. Psychol. Rev. 22, 41–56. doi:10.1007/s10648-010-9117-8

Ambron, E., Miller, A., Kuchenbecker, K. J., Buxbaum, L. J., and Coslett, H. B. (2018). Immersive Low-Cost Virtual Reality Treatment for Phantom Limb Pain: Evidence from Two Cases. Front. Neurol. 9, 67. doi:10.3389/fneur.2018.00067

PubMed Abstract | CrossRef Full Text | Google Scholar

Amin, A., Gromala, D., Tong, X., and Shaw, C. (2016). “Immersion in Cardboard Vr Compared to a Traditional Head-Mounted Display,” in Virtual, Augmented and Mixed Reality . Editors S. Lackey, and R. Shumaker (Cham: Springer International Publishing ), 269–276. doi:10.1007/978-3-319-39907-2_25

Anthes, C., García-Hernández, R. J., Wiedemann, M., and Kranzlmüller, D. (2016). “State of the Art of Virtual Reality Technology”, in IEEE Aerospace Conf. , Big Sky, MT , 1–19.

Google Scholar

Backlund, P., Engstrom, H., Hammar, C., Johannesson, M., and Lebram, M. (2007). “Sidh – a Game Based Firefighter Training Simulation”, in 11th International Conference Information Visualization , Zurich, Switzerland , July 4–6, 2007 (New York, NY: IEEE ), 899–907. doi:10.1109/iv.2007.100

Baldwin, T. T., and Ford, J. K. (1988). Transfer of Training: A Review and Directions for Future Research. Pers. Psychol. 41, 63–105. doi:10.1111/j.1744-6570.1988.tb00632.x

Bao, X., Mao, Y., Lin, Q., Qiu, Y., Chen, S., Li, L., et al. (2013). Mechanism of Kinect-Based Virtual Reality Training for Motor Functional Recovery of Upper Limbs after Subacute Stroke. Neural Regen. Res. 8, 2904–2913. doi:10.3969/j.issn.1673-5374.2013.31.003

Barkman, S., and Machtmes, K. (2001). Four-fold: A Research-Based Model Designing and Evaluting the Impact of Youth Development Programs. News & Views 54, 3–8.

Baur, T., Damian, I., Gebhard, P., Porayska-Pomsta, K., and André, E. (2013). “A Job Interview Simulation: Social Cue-Based Interaction with a Virtual Character,” in 2013 International Conference on Social Computing (IEEE) , Alexandria, VA , Sep 8–14, 2013 (New York, NY: IEEE ), 220–227.

Bergstrom, J. R., and Schall, A. (2014). Eye Tracking in User Experience Design . Amsterdam, Netherlands: Elsevier .

Bhoir, S., and Esmaeili, B. (2015). “State-of-the-art Review of Virtual Reality Environment Applications in Construction Safety”, in. In AEI 2015 , Milwaukee, WI , March 24–27, 2015 (Reston, VA: American Society of Civil Engineers ), 457–468.

Biocca, F., and Delaney, B. (1995). Immersive Virtual Reality Technology. Commun. Age Virtual Real. 15, 10–5555.

Bogatinov, D., Lameski, P., Trajkovik, V., and Trendova, K. M. (2017). Firearms Training Simulator Based on Low Cost Motion Tracking Sensor. Multimed. Tools Appl. 76, 1403–1418. doi:10.1007/s11042-015-3118-z

Borba, E. Z., Cabral, M., Montes, A., Belloc, O., and Zuffo, M. (2016). “Immersive and Interactive Procedure Training Simulator for High Risk Power Line Maintenance”, in ACM SIGGRAPH 2016 VR Village , Anaheim, California , July, 2016 (New York, NY: Association for Computing Machinery ). doi:10.1145/2929490.2929497

Bossard, C., Kermarrec, G., Buche, C., and Tisseau, J. (2008). Transfer of Learning in Virtual Environments: a New Challenge?. Virtual Real. 12, 151–161. doi:10.1007/s10055-008-0093-y

Brahim, Z. (2020). Nhmep Offers Virtual Training and Resources to Assist Manufacturers. New Hampshire Manufacturing Extension Partnership — NH MEP. Available at: https://www.nhmep.org/nhmep-offers-virtual-training-and-resources-to-assist-manufacturers/ (Accessed April 23, 2020).

Bryman, A. (2006). Integrating Quantitative and Qualitative Research: How Is it Done? Qual. Res. 6, 97–113. doi:10.1177/1468794106058877

Burke, L. A., and Hutchins, H. M. (2007). Training Transfer: An Integrative Literature Review. Hum. Res. Dev. Rev. 6, 263–296. doi:10.1177/1534484307303035

Cacioppo, J. T., Tassinary, L. G., and Berntson, G. (2007). Handbook of Psychophysiology . Cambridge, United Kingdom: Cambridge University Press .

Cai, S., Ke, P., Narumi, T., and Zhu, K. (2020). “Thermairglove: A Pneumatic Glove for Thermal Perception and Material Identification in Virtual Reality,” in IEEE Conference on Virtual Reality and 3D User Interfaces , Atlanta, GA , March 22–26, 2020 (New York, NY: IEEE ), 248–257.

Caligiuri, P. M., Jacobs, R. R., and Farr, J. L. (2000). The Attitudinal and Behavioral Openness Scale: Scale Development and Construct Validation. Int. J. Intercult. Relat. 24, 27–46. doi:10.1016/s0147-1767(99)00021-8

Carroll, J. M. (2000). Making Use: Scenario-Based Design of Human-Computer Interactions . Cambridge, MA: MIT Press . doi:10.1145/347642.347652

CrossRef Full Text

Cha, M., Han, S., Lee, J., and Choi, B. (2012). A Virtual Reality Based Fire Training Simulator Integrated with Fire Dynamics Data. Fire Saf. J. 50, 12–24. doi:10.1016/j.firesaf.2012.01.004

Chao, C.-J., Wu, S.-Y., Yau, Y.-J., Feng, W.-Y., and Tseng, F.-Y. (2017). Effects of Three-Dimensional Virtual Reality and Traditional Training Methods on Mental Workload and Training Performance. Hum. Fact. Man. 27, 187–196. doi:10.1002/hfm.20702

Chellali, A., Dumas, C., and Milleville-Pennel, I. (2011). Influences of Haptic Communication on a Shared Manual Task. Interact. Comput. 23, 317–328. doi:10.1016/j.intcom.2011.05.002

Clark, R. E., and Estes, F. (1996). Cognitive Task Analysis for Training. Int. J. Educ. Res. 25, 403–417. doi:10.1016/s0883-0355(97)81235-9

Colbert, A., Yee, N., and George, G. (2016). The Digital Workforce and the Workplace of the Future. Acad. Manage. J. 59 (3), 731–739. doi:10.5465/amj.2016.4003

Conges, A., Evain, A., Benaben, F., Chabiron, O., and Rebiere, S. (2020). “Crisis Management Exercises in Virtual Reality”, in IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops , Atlanta, GA , March 22–26, 2020 (New York, NY: IEEE ), 87–92.

Cordar, A., Wendling, A., White, C., Lampotang, S., and Lok, B. (2017). “Repeat After me: Using Mixed Reality Humans to Influence Best Communication Practices”, in 2017 IEEE Virtual Reality , Los Angeles, CA , March 18–22, 2017 (New York, NY: IEEE ), 148–156.

Cruz-Neira, C., Sandin, D. J., DeFanti, T. A., Kenyon, R. V., and Hart, J. C. (1992). The Cave: Audio Visual Experience Automatic Virtual Environment. Commun. ACM 35, 64–72. doi:10.1145/129888.129892

Daher, S., Hochreiter, J., Norouzi, N., Gonzalez, L., Bruder, G., and Welch, G. (2018). “Physical-virtual Agents for Healthcare Simulation”, in Proceedings of the 18th International Conference on Intelligent Virtual Agents IVA ’18 , Sydney, NSW , Nov, 2018 (New York, NY: Association for Computing Machinery ), 99–106.

Dalladaku, Y., Kelley, J., Lacey, B., Mitchiner, J., Welsh, B., and Beigh, M. (2020). “Assessing the Effectiveness of Virtual Reality in the Training of Army Aviators”, in Proceedings of the 2020 Annual General Donald R. Keith Memorial Capstone Conference , New York, NY , April 30, 2020 .

de Ribaupierre, S., Armstrong, R., Noltie, D., Kramers, M., and Eagleson, R. (2015). “Vr and Ar Simulator for Neurosurgical Training,” in 2015 IEEE Virtual Reality (VR) , Arles, France , March 23–27, 2015 (New York, NY: IEEE ), 147–148.

Deardorff, D. K. (2006). Identification and Assessment of Intercultural Competence as a Student Outcome of Internationalization. J. Stud. Int. Educ. 10, 241–266. doi:10.1177/1028315306287002

Detterman, D. K. (1993). The Case for the Prosecution: Transfer as an Epiphenomenon . (New York, NY: Ablex Publishing ), 1–24. doi:10.21236/ada264836

Di Loreto, C., Chardonnet, J.-R., Ryard, J., and Rousseau, A. (2018). “Woah: A Virtual Reality Work-At-Height Simulator”, in IEEE Conference on Virtual Reality and 3D User Interfaces , Reutlingen, Germany , March 18–22, 2018 (New York, NY: IEEE ), 281–288.

Dittmer, J. (2010). Immersive Virtual Worlds in University-Level Human Geography Courses. Int. Res. Geogr. Environ. Educ. 19, 139–154. doi:10.1080/10382046.2010.482222

Doneda, A. L. C., and de Oliveira, J. C. (2020). “Helicopter Visual Signaling Simulation: Integrating Vr and Ml into a Low-Cost Solution to Optimize Brazilian Navy Training”, in 2020 22nd Symposium on Virtual and Augmented Reality , Porto de Galinhas, Brazil , Nov 7, 2020 (New York, NY: IEEE ), 434–442. doi:10.1109/SVR51698.2020.00071

Fang, C., Zhang, Y., Dworman, M., and Harrison, C. (2020). “Wireality: Enabling Complex Tangible Geometries in Virtual Reality with Worn Multi-String Haptics”, in Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems , Honolulu HI , April, 2020 (New York, NY: Association for Computing Machinery ), 1–10.

Farmer, E., Van Rooij, J., Riemersma, J., and Jorna, P. (2017). Handbook of Simulator-Based Training . United Kingdom: Routledge . doi:10.4324/9781315253671

Feelreal, (2020). Feelreal Multisensory Vr Mask. Available at: https://feelreal.com/ (Accessed May 10, 2020).

Filigenzi, M. T., Orr, T. J., and Ruff, T. M. (2000). Virtual Reality for Mine Safety Training. Appl. Occup. Environ. Hyg. 15, 465–469. doi:10.1080/104732200301232

Formosa, N. J., Morrison, B. W., Hill, G., and Stone, D. (2018). Testing the Efficacy of a Virtual Reality‐based Simulation in Enhancing Users' Knowledge, Attitudes, and Empathy Relating to Psychosis. Aust. J. Psychol. 70, 57–65. doi:10.1111/ajpy.12167

Freeman, J., Watson, G., Papelis, Y., Lin, T., Tayyab, A., Romano, R., et al. (1995). The Iowa Driving Simulator: An Implementation and Application Overview. SAE Technical Paper , Tech. Rep. doi:10.4271/950174

Freiknecht, J., and Effelsberg, W. (2017). A Survey on the Procedural Generation of Virtual Worlds. Multimodal. Technol. Interact. 1, 27. doi:10.3390/mti1040027

Froehlich, M. A., and Azhar, S. (2016). “Investigating Virtual Reality Headset Applications in Construction”, in Proceedings of the 52nd Associated Schools of Construction Annual International Conference , 13–16.

Fukumori, S., Gofuku, A., Isatake, K., and Sato, K. (2014). “Mirror Thrapy System Based Virtual Reality for Chronic Pain in Home Use”, in IECON 2014 - 40th Annual Conference of the IEEE Industrial Electronics Society , Dallas, TX , Oct 29, 2014 (New York, NY: IEEE ), 4034–4039.

Gallagher, A., McClure, N., McGuigan, J., Crothers, I., and Browning, J. (1999). Virtual Reality Training in Laparoscopic Surgery: a Preliminary Assessment of Minimally Invasive Surgical Trainer Virtual Reality (Mist Vr). Endoscopy 31, 310–313. doi:10.1055/s-1999-15

Gilbreth, F. B., and Carey, E. G. (2013). Cheaper by the Dozen . New York, NY: Open Road Media .

Gilyazova, O. S. (2019). The Relationship between Virtual and Actual Reality: Phenomenological-Ontological Approach. Taksad 8, 196–204. doi:10.7596/taksad.v8i1.1921

Girardi, R., and de Oliveira, J. C. (2019). “Virtual Reality in Army Artillery Observer Training”, in 2019 21st Symposium on Virtual and Augmented Reality , Rio de Janeiro, Brazil , Oct 28, 2019 (New York, NY: IEEE ), 25–33. doi:10.1109/SVR.2019.00021

Grajewski, D., Górski, F., Hamrol, A., and Zawadzki, P. (2015). Immersive and Haptic Educational Simulations of Assembly Workplace Conditions. Proced. Comput. Sci. 75, 359–368. doi:10.1016/j.procs.2015.12.258

Grantcharov, T. P., Kristiansen, V. B., Bendix, J., Bardram, L., Rosenberg, J., and Funch-Jensen, P. (2004). Randomized Clinical Trial of Virtual Reality Simulation for Laparoscopic Skills Training. Br. J. Surg. 91, 146–150. doi:10.1002/bjs.4407

Greene, J. C. (2003). Handbook of Communication and Social Interaction Skills . London, United Kingdom: Psychology Press .

Greunke, L., and Sadagic, A. (2016). Taking Immersive Vr Leap in Training of Landing Signal Officers. IEEE Trans. Vis. Comput. Graphics 22, 1482–1491. doi:10.1109/tvcg.2016.2518098

Grossman, R., and Salas, E. (2011). The Transfer of Training: what Really Matters. Int. J. Train. Develop. 15, 103–120. doi:10.1111/j.1468-2419.2011.00373.x

Gurusamy, K. S., Aggarwal, R., Palanivelu, L., and Davidson, B. R. (2009). Virtual Reality Training for Surgical Trainees in Laparoscopic Surgery. Cochrane Database Syst. Rev. 1, CD006575. doi:10.1002/14651858.cd006575.pub2

Hafsia, M., Monacelli, E., and Martin, H. (2018). “Virtual Reality Simulator for Construction Workers”, in Proceedings of the Virtual Reality International Conference-Laval Virtual , Laval, France , April, 2018 (New York, NY: Association for Computing Machinery ), 1–7.

Hamblin, C. J. (2005). Transfer of Training from Virtual Reality Environments. PhD thesis. Wichita (Kansas): Wichita State University .

Hamilton, E. C., Scott, D. J., Fleming, J. B., Rege, R. V., Laycock, R., Bergen, P. C., et al. (2002). Comparison of Video Trainer and Virtual Reality Training Systems on Acquisition of Laparoscopic Skills. Surg. Endosc. 16, 406–411. doi:10.1007/s00464-001-8149-z

Harrison, A., Derwent, G., Enticknap, A., Rose, F. D., and Attree, E. A. (2002). The Role of Virtual Reality Technology in the Assessment and Training of Inexperienced Powered Wheelchair Users. Disabil. Rehabil. 24, 599–606. doi:10.1080/09638280110111360

Haskins, J., Zhu, B., Gainer, S., Huse, W., Eadara, S., Boyd, B., et al. (2020). “Exploring Vr Training for First Responders”, in IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops , Atlanta, GA , March 22–26, 2020 (New York, NY: IEEE ), 57–62.

He, Z. D., Hu, R. M., and Xu, J. C. (2014). The Development of Badminton Auxiliary Training System Based on Kinect Motion Capture. Adv. Mat. Res. 926–930, 2735–2738. doi:10.4028/www.scientific.net/amr.926-930.2735

Heirman, J., Selleri, S., De Vleeschauwer, T., Hamesse, C., Bellemans, M., Schoofs, E., et al. (2020). “Exploring the Possibilities of Extended Reality in the World of Firefighting”, in 2020 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR) . Los Alamitos, CA , Dec 14, 2020 (Washington: IEEE Computer Society) , 266–273. doi:10.1109/AIVR50618.2020.00055

Herrlich, M., Meyer, R., Malaka, R., and Heck, H. (2010). “Development of a Virtual Electric Wheelchair - Simulation and Assessment of Physical Fidelity Using the Unreal Engine 3”, in International Conference on Entertainment Computing . Springer , 286–293. doi:10.1007/978-3-642-15399-0_29

Hickman, L., and Akdere, M. (2018). “Developing Intercultural Competencies through Virtual Reality: Internet of Things Applications in Education and Learning”, in 2018 15th Learning and Technology Conference (L&T) , Jeddah, Saudi Arabia (New York, NY: IEEE ), 24–28.

Hochreiter, J., Daher, S., Nagendran, A., Gonzalez, L., and Welch, G. (2015). Touch Sensing on Non-parametric Rear-Projection Surfaces: A Physical-Virtual Head for Hands-On Healthcare Training. IEEE Virtual Real. , 69–74. doi:10.1109/VR.2015.7223326

Houtkamp, J. M., Toet, A., and Bos, F. A. (2012). Task-relevant Sound and User Experience in Computer-Mediated Firefighter Training. Simul. Gaming 43, 778–802. doi:10.1177/1046878112444564

Howard, M. C. (2019). Virtual Reality Interventions for Personal Development: A Meta-Analysis of Hardware and Software. Hum. Comput. Interact. 34, 205–239. doi:10.1080/07370024.2018.1469408

Huguet, L., Lourdeaux, D., Sabouret, N., and Ferrer, M.-H. (2016). Perturbed Communication in a Virtual Environment to Train Medical Team Leaders. Stud. Health Technol. Inform. 220, 146–149.

PubMed Abstract Google Scholar

Hurd, O., Kurniawan, S., and Teodorescu, M. (2019). “Virtual Reality Video Game Paired with Physical Monocular Blurring as Accessible Therapy for Amblyopia”, in IEEE Conference on Virtual Reality and 3D User Interfaces , Osaka, Japan , March 23, 2019 (New York, NY: IEEE ), 492–499.

Hussain, T. S., Roberts, B., Menaker, E. S., Coleman, S. L., Centreville, V., Pounds, K., et al. (2009). “Designing and Developing Effective Training Games for the Us Navy”, in The Interservice/Industry Training, Simulation & Education Conference , 1.

Inman, D. P., Loge, K., and Leavens, J. (1997). VR Education and Rehabilitation. Commun. ACM 40, 53–58. doi:10.1145/257874.257886

Iosa, M., Morone, G., Fusco, A., Castagnoli, M., Fusco, F. R., Pratesi, L., et al. (2015). Leap Motion Controlled Videogame-Based Therapy for Rehabilitation of Elderly Patients with Subacute Stroke: a Feasibility Pilot Study. Top. Stroke Rehabil. 22, 306–316. doi:10.1179/1074935714z.0000000036

Issurin, V. B. (2013). Training Transfer: Scientific Background and Insights for Practical Application. Sports Med. 43, 675–694. doi:10.1007/s40279-013-0049-6

Jain, A. K., Flynn, P., and Ross, A. A. (2007). Handbook of Biometrics . Berlin, Germany: Springer Science & Business Media .

Jensen, L., and Konradsen, F. (2018). A Review of the use of Virtual Reality Head-Mounted Displays in Education and Training. Educ. Inf. Technol. 23, 1515–1529. doi:10.1007/s10639-017-9676-0

John, N. W., Pop, S. R., Day, T. W., Ritsos, P. D., and Headleand, C. J. (2018). The Implementation and Validation of a Virtual Environment for Training Powered Wheelchair Manoeuvres. IEEE Trans. Vis. Comput. Graph. 24, 1867–1878. doi:10.1109/tvcg.2017.2700273

Johnson, M., and Johnson, L. (2010). Generations, Inc.: From Boomers to Linksters–Managing The Friction between Generations at Work , New York, NY: Amacom .

Karabiyik, U., Mousas, C., Sirota, D., Iwai, T., and Akdere, M. (2019). “A Virtual Reality Framework for Training Incident First Responders and Digital Forensic Investigators” in International Symposium on Visual Computing , Springer , 469–480. doi:10.1007/978-3-030-33723-0_38

Kenny, P. G., Parsons, T. D., and Rizzo, A. A. (2009). “Human Computer Interaction in Virtual Standardized Patient Systems,” in Human-Computer Interaction. Interacting in Various Application Domains . Editor J. A. Jacko (Berlin, Heidelberg: Springer Berlin Heidelberg) ), 514–523. doi:10.1007/978-3-642-02583-9_56

Kirchbach, K., and Runde, C. (2011). “Optimized Work Flow through Vr and Ar Technology on Construction Sites”, in 2011 15th International Conference on Information Visualisation , (Washington, DC: IEEE Computer Society ), 549–551.

Kiss, L., Hámornik, B. P., Köles, M., Baranyi, P., Galambos, P., and Persa, G. (2015). “Training of Business Skills in Virtual Reality”, in 6th IEEE International Conference on Cognitive Infocommunications (CogInfoCom) , Debrecen, Hungary , Sep 11–14, 2017 (New York, NY: IEEE ), 215–216.

Koutitas, G., Smith, K. S., Lawrence, G., Metsis, V., Stamper, C., Trahan, M., et al. (2019). “A Virtual and Augmented Reality Platform for the Training of First Responders of the Ambulance Bus”, in Proceedings of the 12th ACM International Conference on PErvasive Technologies Related to Assistive Environments , Rhodes, Greece , June, 2019 (New York, NY: Association for Computing Machinery ), 299–302. doi:10.1145/3316782.3321542

Koutitas, G., Smith, S., and Lawrence, G. (2020). Performance Evaluation of AR/VR Training Technologies for Ems First Responders. Virtual Real. 25, 1–12. doi:10.1007/s10055-020-00436-8

Krösl, K., Elvezio, C., Hürbe, M., Karst, S., Wimmer, M., and Feiner, S. (2019). “Icthroughvr: Illuminating Cataracts through Virtual Reality,” in IEEE Conference on Virtual Reality and 3D User Interfaces , Osaka, Japan , March 23, 2019 (New York, NY: IEEE ), 655–663.

Kyan, M., Sun, G., Li, H., Zhong, L., Muneesawang, P., Dong, N., et al. (2015). An Approach to Ballet Dance Training through Ms Kinect and Visualization in a Cave Virtual Reality Environment. ACM Trans. Intell. Syst. Technol. 6, 1–37. doi:10.1145/2735951

Lacko, J. (2020). “Health Safety Training for Industry in Virtual Reality”, in 2020 Cybernetics & Informatics (K&I) , Velke Karlovice, Czech Republic , Jan 29, 2020 (New York, NY: IEEE ), 1–5. doi:10.1109/ki48306.2020.9039854

Lang, Y., Wei, L., Xu, F., Zhao, Y., and Yu, L. (2018). “Synthesizing Personalized Training Programs for Improving Driving Habits via Virtual Reality”, in IEEE Conference on Virtual Reality and 3D User Interfaces , 297–304.

Lang, B. (2021). Facebook Ceo: Quest 2 ’on Track to Be First Mainstream Vr Headset’, Next Headset Confirmed. Road to VR. Available at: https://www.roadtovr.com/zuckerberg-quest-2-mainstream-vr-headset-facebook-q4-2020-earnings/ .

Lave, J. (1988). Cognition in Practice: Mind, Mathematics and Culture in Everyday Life . Cambridge, United Kingdom: Cambridge University Press . doi:10.1017/cbo9780511609268

Le, Q. T., Pedro, A., and Park, C. S. (2015). A Social Virtual Reality Based Construction Safety Education System for Experiential Learning. J. Intell. Robot Syst. 79, 487–506. doi:10.1007/s10846-014-0112-z

Lhemedu-Steinke, Q., Meixner, G., and Weber, M. (2018). “Comparing Vr Display with Conventional Displays for User Evaluation Experiences”, in IEEE Conference on Virtual Reality and 3D User Interfaces , Tuebingen, Germany , March 18, 2018 (New York, NY: IEEE ), 583–584. doi:10.1109/VR.2018.8446076

Li, H., Chan, G., and Skitmore, M. (2012). Multiuser Virtual Safety Training System for Tower Crane Dismantlement. J. Comput. Civ. Eng. 26, 638–647. doi:10.1061/(asce)cp.1943-5487.0000170

Li, C., Liang, W., Quigley, C., Zhao, Y., and Yu, L.-F. (2017). Earthquake Safety Training through Virtual Drills. IEEE Trans. Vis. Comput. Graph. 23, 1275–1284. doi:10.1109/tvcg.2017.2656958

Li, X., Yi, W., Chi, H.-L., Wang, X., and Chan, A. P. C. (2018). A Critical Review of Virtual and Augmented Reality (Vr/ar) Applications in Construction Safety. Autom. Constr. 86, 150–162. doi:10.1016/j.autcon.2017.11.003

Li, W., Talavera, J., Samayoa, A. G., Lien, J.-M., and Yu, L.-F. (2020). “Automatic Synthesis of Virtual Wheelchair Training Scenarios”, in IEEE Conference on Virtual Reality and 3D User Interfaces . 539–547.

Lin, F., Ye, L., Duffy, V. G., and Su, C.-J. (2002). Developing Virtual Environments for Industrial Training. Inf. Sci. 140, 153–170. doi:10.1016/s0020-0255(01)00185-2

Lin, Y.-H., Huang, S.-Y., Hsiao, K.-F., Kuo, K.-P., and Wan, L.-T. (2013). “A Kinect-Based System for Golf Beginners' Training”, in Information Technology Convergence . ( Springer ), 121–129. doi:10.1007/978-94-007-6996-0_13

Linowes, J. (2015). Unity Virtual Reality Projects . 2nd Edn. Birmingham, United Kingdom: Packt Publishing Ltd .

Liu, H., Wang, Z., Mousas, C., and Kao, D. (2020). “Virtual Reality Racket Sports: Virtual Drills for Exercise and Training”, in IEEE International Symposium on Mixed and Augmented Reality (ISMAR) , Porto de Galinhas, Brazil , Nov 9, 2020 (New York, NY: IEEE ), 566–576.

Lovreglio, R., Duan, X., Rahouti, A., Phipps, R., and Nilsson, D. (2020). Comparing the Effectiveness of Fire Extinguisher Virtual Reality and Video Training. Virtual Real. 1–13. doi:10.1007/s10055-020-00447-5

Lustosa, E. B. S., de Macedo, D. V., and Formico Rodrigues, M. A. (2018). “Virtual Simulator for Forklift Training”, in 2018 20th Symposium on Virtual and Augmented Reality , Foz do Iguacu, Brazil , Oct 28. 2018 (New York, NY: IEEE ), 18–26. doi:10.1109/SVR.2018.00016

Mann, C. T., and Fischer, H. (2019). Recent Trends in Active-Duty Military Deaths .

Martin, G. A., and Hughes, C. E. (2010). “A Scenario Generation Framework for Automating Instructional Support in Scenario-Based Training”, in Proceedings of the 2010 Spring Simulation Multiconference , Orlando, FL , April, 2010 (San Diego, CA: Society for Computer Simulation International ), 1–6.

Martin, G., Schatz, S., Bowers, C., Hughes, C. E., Fowlkes, J., and Nicholson, D. (2009). “Automatic Scenario Generation Through Procedural Modeling for Scenario-Based Training”, in Proceedings of the Human Factors and Ergonomics Society Annual Meeting (Los Angeles, CA: SAGE Publications Sage ), 1949–1953. doi:10.1177/154193120905302615

McGhee, J. T., Mohsen Fatoorechi, B., Cleal, A., and Charles Nduka, M. (2018). Towards a Novel Biometric Facial Input for Emotion Recognition and Assistive Technology for Virtual Reality. Int. J. Child Health Hum. Develop. 11, 243–247.

Mechlih, H. (2016). “New Employee Education Using 3d Virtual Manufacturing”, in 13th Learning and Technology Conference (L&T) (New York, NY: IEEE ), 1–3.

Melnick, K. (2019). Nypd Uses Location-Based Vr for Active Shooter Training. VR Scout , May 14, 2019.

Michalski, S. C., Szpak, A., Saredakis, D., Ross, T. J., Billinghurst, M., and Loetscher, T. (2019). Getting Your Game on: Using Virtual Reality to Improve Real Table Tennis Skills. PloS One 14, e0222351. doi:10.1371/journal.pone.0222351

Moro, C., Štromberga, Z., and Stirling, A. (2017). Virtualisation Devices for Student Learning: Comparison between Desktop-Based (Oculus Rift) and Mobile-Based (Gear Vr) Virtual Reality in Medical and Health Science Education. Australas. J. Educ. Tech. 33. doi:10.14742/ajet.3840

Mossel, A., Froeschl, M., Schoenauer, C., Peer, A., Goellner, J., and Kaufmann, H. (2017). “Vronsite: Towards Immersive Training of First Responder Squad Leaders in Untethered Virtual Reality”, in IEEE Virtual Reality , Los Angeles, CA , March 18, 2017 (New York, NY: IEEE ), 357–358. doi:10.1109/VR.2017.7892324

Nasios, K. (2002). Improving Chemical Plant Safety Training Using Virtual Reality. PhD thesis. Nottingham (United Kingdom): University of Nottingham .

Niniss, H., and Nadif, A. (2000). “Simulation of the Behaviour of a Powered Wheelchair Using Virtual Reality”, in 3rd International Conference on Disability, Virtual Reality and Associated Technologies , 9–14.

Ostrowski, S. (2018). Growing Impact of Emerging Technologies Examined in Comptia Research Briefs. PR Newswire. doi:10.5592/co/cetra.2018.889

Papachristos, N. M., Vrellis, I., and Mikropoulos, T. A. (2017). “A Comparison Between Oculus Rift and a Low-Cost Smartphone Vr Headset: Immersive User Experience and Learning”, in IEEE 17th International Conference on Advanced Learning Technologies (ICALT) , Timisoara, Romania , July 3, 2017 (New York, NY: IEEE ), 477–481.

Papelis, Y., and Watson, G. S. (2018). “Using Event Templates to Accellerate Scenario Development in Virtual Training Environments”, in Proceedings of the 50th Computer Simulation Conference , Bordeaux, France , July, 2018 (San Diego, CA: Society for Computer Simulation International ), 16.

Parish, Y. I., and Müller, P. (2001). “Procedural Modeling of Cities,” in Proceedings of the 28th annual conference on Computer graphics and interactive techniques (New York, NY: Association for Computing Machinery ), 301–308. doi:10.1145/383259.383292

Park, D.-S., Lee, D.-G., Lee, K., and Lee, G. (2017). Effects of Virtual Reality Training Using Xbox Kinect on Motor Function in Stroke Survivors: a Preliminary Study. J. Stroke Cerebrovasc. Dis. 26, 2313–2319. doi:10.1016/j.jstrokecerebrovasdis.2017.05.019

Pinto, D., Peixoto, B., Gonçalves, G., Melo, M., Amorim, V., and Bessa, M. (2019). “Developing Training Applications for Hydrogen Emergency Response Training”, in 2019 International Conference on Graphics and Interaction (ICGI) , Faro, Portugal , Nov 21, 2019 (New York, NY: IEEE ), 130–136. doi:10.1109/ICGI47575.2019.8955091

Prasolova-Førland, E., Forninykh, M., and Ekelund, O. I. (2019). “Empowering Young Job Seekers with Virtual Reality”, in IEEE Conference on Virtual Reality and 3D User Interfaces , Osaka, Japan , March 23, 2019 (New York, NY: IEEE ), 295–302.

Ramakrishna, P., Hassan, E., Hebbalaguppe, R., Sharma, M., Gupta, G., Vig, L., et al. (2016). “An Ar Inspection Framework: Feasibility Study with Multiple Ar Devices,” in IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct) , Merida, Mexico , Sep 19, 2016 (New York, NY: IEEE ), 221–226.

Ranasinghe, N., Jain, P., Karwita, S., Tolley, D., and Do, E. Y.-L. (2017). “Ambiotherm: Enhancing Sense of Presence in Virtual Reality by Simulating Real-World Environmental Conditions”, in Proceedings Of the 2017 CHI Conference On Human Factors In Computing Systems , Denver, CO , May, 2017 (New York, NY: Association for Computing Machinery ), 1731–1742.

Reichard, A. A., and Jackson, L. L. (2010). Occupational Injuries Among Emergency Responders. Am. J. Ind. Med. 53, 1–11. doi:10.1002/ajim.20772

Rizzo, A., Hartholt, A., Grimani, M., Leeds, A., and Liewer, M. (2014). Virtual Reality Exposure Therapy for Combat-Related Posttraumatic Stress Disorder. Computer 47, 31–37. doi:10.1109/mc.2014.199

Roberts, D., Fairchild, A., Campion, S., Garcia Jimenez, A., Wolff, R., et al. (2016). “Bringing the Client and Therapist Together in Virtual Reality Telepresence Exposure Therapy”, in Proceedings of the 11th International Conference on Disability, Virtual Reality and Associated Technologies ( University of Reading ), 157–163.

Rodriguez, N. (2015). “Development of a Wheelchair Simulator for Children with Multiple Disabilities”, in 2015 3rd IEEE VR International Workshop on Virtual and Augmented Assistive Technology , Arles, France , March 23, 2015 (New York, NY: IEEE ), 19–21.

Rose, F. D., Attree, E. A., Brooks, B. M., Parslow, D. M., and Penn, P. R. (2000). Training in Virtual Environments: Transfer to Real World Tasks and Equivalence to Real Task Training. Ergonomics 43, 494–511. doi:10.1080/001401300184378

Rosenthal, R., Schäfer, J., Hoffmann, H., Vitz, M., Oertli, D., and Hahnloser, D. (2013). Personality Traits and Virtual Reality Performance. Surg. Endosc. 27, 222–230. doi:10.1007/s00464-012-2424-z

Rushmeier, H., Madathil, K. C., Hodgins, J., Mynatt, B., Derose, T., Macintyre, B., et al. (2019). Content Generation for Workforce Training. Available at: https://cra.org/ccc/events/content-generation-for-workforce-training/ .

Sacks, R., Perlman, A., and Barak, R. (2013). Construction Safety Training Using Immersive Virtual Reality. Constr. Man.. Econ. 31, 1005–1017. doi:10.1080/01446193.2013.828844

Salas, E., Wilson, K. A., Burke, C. S., and Priest, H. A. (2005). Using Simulation-Based Training to Improve Patient Safety: What Does it Take? Jt. Comm. J. Qual. Patient Saf. 31, 363–371. doi:10.1016/s1553-7250(05)31049-x

Salmon, P., Jenkins, D., Stanton, N., and Walker, G. (2010). Hierarchical Task Analysis vs. Cognitive Work Analysis: Comparison of Theory, Methodology and Contribution to System Design. Theor. Issues Ergon. Sci. 11, 504–531. doi:10.1080/14639220903165169

Schraagen, J. M., Chipman, S. F., and Shalin, V. L. (2000). Cognitive Task Analysis . London, United Kingdom: Psychology Press . doi:10.1037/e446712006-001

Sekizuka, R., Ito, M., Saiki, S., Yamazaki, Y., and Kurita, Y. (2019). “Evaluation System for Hydraulic Excavator Operation Skill Using Remote Controlled Excavator and Virtual Reality”, in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , Macau, China , Nov 8, 2019 (New York, NY: IEEE ), 3229–3234. doi:10.1109/IROS40897.2019.8968213

Seymour, N. E., Gallagher, A. G., Roman, S. A., O’Brien, M. K., Bansal, V. K., Andersen, D. K., et al. (2002). Virtual Reality Training Improves Operating Room Performance. Ann. Surg. 236, 458–464. doi:10.1097/00000658-200210000-00008

Shirk, P.. (2019). 1st Wxg’s Virtual Training Brings Real Benefits. Available at: https://www.designworldonline.com/ .

Si, W., Liao, X., Wang, Q., and Heng, P. (2018). “Augmented Reality-Based Personalized Virtual Operative Anatomy for Neurosurgical Guidance and Training,” in IEEE Conference on Virtual Reality and 3D User Interfaces (VR) , Tuebingen, Germany , March 18, 2018 (New York, NY: IEEE ), 683–684. doi:10.1109/VR.2018.8446450

Sin, H., and Lee, G. (2013). Additional Virtual Reality Training Using Xbox Kinect in Stroke Survivors with Hemiplegia. Am. J. Phys. Med. Rehabil. 92, 871–880. doi:10.1097/phm.0b013e3182a38e40

Siu, K.-C., Best, B. J., Kim, J. W., Oleynikov, D., and Ritter, F. E. (2016). Adaptive Virtual Reality Training to Optimize Military Medical Skills Acquisition and Retention. Mil. Med. 181, 214–220. doi:10.7205/milmed-d-15-00164

Smelik, R. M., De Kraker, K. J., Tutenel, T., Bidarra, R., and Groenewegen, S. A. (2009). “A Survey of Procedural Methods for Terrain Modelling”, in Proceedings of the CASA Workshop on 3D Advanced Media in Gaming and Simulation , 25–34.

Smelik, R. M., Tutenel, T., de Kraker, K. J., and Bidarra, R. (2011). A Declarative Approach to Procedural Modeling of Virtual Worlds. Comput. Graph. 35 (2), 352–363. doi:10.1016/j.cag.2010.11.011

Smelik, R. M., Tutenel, T., Bidarra, R., and Benes, B. (2014). A Survey on Procedural Modelling for Virtual Worlds. Comput. Graph. Forum 33, 31–50. doi:10.1111/cgf.12276

Smith, S., and Ericson, E. (2009). Using Immersive Game-Based Virtual Reality to Teach Fire-Safety Skills to Children. Virtual Real. 13, 87–99. doi:10.1007/s10055-009-0113-6

Spitzberg, B. H., and Changnon, G. (2009). Conceptualizing Intercultural Competence . Editor D. Deardorf (Thousand Oaks, CA: Sage ).

Stone, D. L., Deadrick, D. L., Lukaszewski, K. M., and Johnson, R. (2015). The Influence of Technology on the Future of Human Resource Management. Hum. Res. Man. Rev. 25, 216–231. doi:10.1016/j.hrmr.2015.01.002

Sweeney, S. K., Newbill, P., Ogle, T., and Terry, K. (2018). Using Augmented Reality and Virtual Environments in Historic Places to Scaffold Historical Empathy. TechTrends 62, 114–118. doi:10.1007/s11528-017-0234-9

Tarnowski, P., Kołodziej, M., Majkowski, A., and Rak, R. J. (2018). “Combined Analysis of Gsr and Eeg Signals for Emotion Recognition”, in International Interdisciplinary PhD Workshop , Poland , May 9, 2018 (New York, NY: IEEE ), 137–141.

Taupiac, J., Rodriguez, N., Strauss, O., and Rabier, M. (2019). “Ad-hoc Study on Soldiers Calibration Procedure in Virtual Reality”, in IEEE Conference on Virtual Reality and 3D User Interfaces , Osaka, Japan , March 23, 2019 (New York, NY: IEEE ), 190–199.

Taylor, R. M., Hudson, T. C., Seeger, A., Weber, H., Juliano, J., and Helser, A. T. (2001). “Vrpn: a Device-independent, Network-Transparent Vr Peripheral System”, in Proceedings of the ACM Symposium on Virtual Reality Software and Technology , Baniff, AB , November, 2001 (New York, NY: Association for Computing Machinery ), 55–61.

Tolio, T., Ceglarek, D., ElMaraghy, H. A., Fischer, A., Hu, S. J., Laperrière, L., et al. (2010). SPECIES-Co-evolution of Products, Processes and Production Systems. CIRP Ann. 59, 672–693. doi:10.1016/j.cirp.2010.05.008

Trainingxr@Ieeevr2020 (2020). Workshop on 3d Content Creation for Simulated Training in Extended Reality(trainingxr). Available at: https://sites.google.com/view/trainingxrieeevr2020 .

Våpenstad, C., Hofstad, E. F., Bø, L. E., Kuhry, E., Johnsen, G., Mårvik, R., et al. (2017). Lack of Transfer of Skills after Virtual Reality Simulator Training with Haptic Feedback. Minimally Invasive Ther. Allied Tech. 26, 346–354. doi:10.1080/13645706.2017.1319866

van Loon, A., Bailenson, J., Zaki, J., Bostick, J., and Willer, R. (2018). Virtual Reality Perspective-Taking Increases Cognitive Empathy for Specific Others. PloS One 13, e0202442. doi:10.1371/journal.pone.0202442

Viar (2019). Retail Training: The Role of Virtual Reality in the Retail L&d Process. Viar360. Available at: https://www.viar360.com/the-role-of-virtual-reality-in-retail-training/ .

Wagner, R. J., and Campbell, J. (1994). Outdoor-Based Experiential Training: Improving Transfer of Training Using Virtual Reality. J. Manage. Develop. 13, 4–11. doi:10.1108/02621719410063350

Wang, Y.-W., Davidson, M. M., Yakushko, O. F., Savoy, H. B., Tan, J. A., and Bleier, J. K. (2003). The Scale of Ethnocultural Empathy: Development, Validation, and Reliability. J. Couns. Psychol. 50, 221–234. doi:10.1037/0022-0167.50.2.221

Wang, Z.-R., Wang, P., Xing, L., Mei, L.-P., Zhao, J., and Zhang, T. (2017). Leap Motion-Based Virtual Reality Training for Improving Motor Functional Recovery of Upper Limbs and Neural Reorganization in Subacute Stroke Patients. Neural Regen. Res. 12, 1823–1831. doi:10.4103/1673-5374.219043

Wang, P., Wu, P., Wang, J., Chi, H.-L., and Wang, X. (2018). A Critical Review of the Use of Virtual Reality in Construction Engineering Education and Training. Int. J. Environ. Res. Public. Health. 15, 1204. doi:10.3390/ijerph15061204

Wang, Y., Zhang, J., Wang, P., and Zhang, L. (2019). “Dynamic Flight Simulator for Low-Altitude Planes”, in International Conference on Virtual Reality and Visualization (ICVRV) , Hong Kong, China , Nov 18, 2019 , 277–278. doi:10.1109/ICVRV47840.2019.00068

Westwood, J., Hoffman, H., Stredney, D., and Weghorst, S. (1998). Validation of Virtual Reality to Teach and Assess Psychomotor Skills in Laparoscopic Surgery: Results from Randomised Controlled Studies Using the Mist Vr Laparoscopic Simulator. Stud. Health Technol. Inform. 50, 124.

Whiting, N., and Donaldson, N. (2014). “Lessons from Integrating the Oculus Rift into Unreal Engine 4”, in Oculus Connect Developers Conference .

Wilkerson, W., Avstreih, D., Gruppen, L., Beier, K.-P., and Woolliscroft, J. (2008). Using Immersive Simulation for Training First Responders for Mass Casualty Incidents. Acad. Emerg. Med. 15, 1152–1159. doi:10.1111/j.1553-2712.2008.00223.x

Williams, D. P., Cash, C., Rankin, C., Bernardi, A., Koenig, J., and Thayer, J. F. (2015). Resting Heart Rate Variability Predicts Self-Reported Difficulties in Emotion Regulation: a Focus on Different Facets of Emotion Regulation. Front. Psychol. 6, 261. doi:10.3389/fpsyg.2015.00261

Willis, D., Powell, W., Powell, V., and Stevens, B. (2019). “Visual Stimulus Disrupts the Spatial Localization of a Tactile Sensation in Virtual Reality”, in IEEE Conference on Virtual Reality and 3D User Interfaces , Osaka, Japan , March 23–27, 2019 (New York, NY: IEEE ), 484–491.

Xie, H., Tudoreanu, E., and Shi, W. (2006). Development of a Virtual Reality Safety-Training System for Construction Workers. Digital Library of Construction Informatics and Information Technology in Civil Engineering and Construction . Available at: https://www.semanticscholar.org/paper/Development-of-a-Virtual-Reality-Safety-Training-Shi-Tudoreanu/421177aa83476bb021c638626f058545dc6d89d1 .

Yildiz, E., Møller, C., Melo, M., and Bessa, M. (2019). “Designing Collaborative and Coordinated Virtual Reality Training Integrated with Virtual and Physical Factories”, in International Conference on Graphics and Interaction 2019 , Faro, Portugal , Nov 21, 2019 (New York, NY: IEEE ). doi:10.1109/icgi47575.2019.8955033

Yoo, J. W., Lee, D. R., Cha, Y. J., and You, S. H. (2017). Augmented Effects of Emg Biofeedback Interfaced with Virtual Reality on Neuromuscular Control and Movement Coordination during Reaching in Children with Cerebral Palsy. Neurorehabilitation 40, 175–185. doi:10.3233/nre-161402

Yu, L. F., Yeung, S. K., Tang, C. K., Terzopoulos, D., Chan, T. F., and Osher, S. J. (2011). Make it Home: Automatic Optimization of Furniture Arrangement. ACM Trans. Graphics (Tog) 30. doi:10.1145/2010324.1964981

Zhang, Y., Xie, B., Huang, H., Ogawa, E., You, T., and Yu, L.-F. (2019). “Pose-guided Level Design”, in Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems , Glasgow, United Kingdom , May, 2019 (New York, NY: Association for Computing Machinery ), 1–12.

Zhao, D., and Lucas, J. (2015). Virtual Reality Simulation for Construction Safety Promotion. Int. J. Inj. Control Saf. Promot. 22, 57–67. doi:10.1080/17457300.2013.861853

Zikmund, W. G., Babin, B. J., Carr, J. C., and Griffin, M. (2010). Business Research Methods . Mason, OH: South Western Cengage Learning .

Zollmann, S., Hoppe, C., Kluckner, S., Poglitsch, C., Bischof, H., and Reitmayr, G. (2014). Augmented Reality for Construction Site Monitoring and Documentation. Proc. IEEE 102, 137–154. doi:10.1109/jproc.2013.2294314

Zyda, M. (2005). From Visual Simulation to Virtual Reality to Games. Computer 38, 25–32. doi:10.1109/MC.2005.297

Keywords: virtual reality, training, simulation, content creation, personalization

Citation: Xie B, Liu H, Alghofaili R, Zhang Y, Jiang Y, Lobo FD, Li C, Li W, Huang H, Akdere M, Mousas C and Yu L-F (2021) A Review on Virtual Reality Skill Training Applications. Front. Virtual Real. 2:645153. doi: 10.3389/frvir.2021.645153

Received: 22 December 2020; Accepted: 19 April 2021; Published: 30 April 2021.

Reviewed by:

Copyright © 2021 Xie, Liu, Alghofaili, Zhang, Jiang, Lobo, Li, Li, Huang, Akdere, Mousas and Yu. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Lap-Fai Yu, [email protected]

This article is part of the Research Topic

Professional Training in Extended Reality: Challenges and Solutions

An official website of the United States government.

Here’s how you know

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• American Job Centers
• Apprenticeship
• Demonstration Grants
• Farmworkers
• Federal Bonding Program
• Foreign Labor Certification
• Indians and Native Americans
• Job Seekers
• Layoffs and Rapid Response
• National Dislocated Worker Grants
• Older Workers
• Skills Training Grants
• Trade Adjustment Assistance
• Unemployment Insurance
• Workforce Innovation and Opportunity Act (WIOA)
• WIOA Adult Program
• Advisories and Directives
• Regulations
• Labor Surplus Area
• Performance
• Recovery-Ready Workplace Resource Hub
• Research and Evaluation
• ETA News Releases
• Updates for Workforce Professionals
• Regional Offices
• Freedom of Information Act
• Office of Apprenticeship
• Office of Foreign Labor Certification
• Office of Grants Management
• Office of Job Corps
• Office of Unemployment Insurance (1-877-S-2JOBS)

Considerations Regarding Future Research on Use of Fees in Employment and Training Administration (ETA) Programs: Discussion Paper

Publication info, research methodology, country, state or territory, description.

This paper explores ideas for future research on application or user fees in programs administered by the Employment and Training Administration (in the U.S. Department of Labor). The paper briefly reviews federal law and regulations, related research studies, and key factors that can be used to guide possible research. The paper focuses on research related to use of fees with employers who, under title I of the Workforce Innovation and Opportunity Act, can be charged fees at the local level for certain customized services (such as for recruitment events and human resource consultation services). Possible research studies identified in the paper include: 1) a descriptive study of current use of fees by local programs (as well as of comparable commercial services to which employers have access); 2) a pilot or demonstration to test use of fees for employers in different circumstances and different types of services, and 3) a feasibility study on use of more rigorous methods (such as a randomized controlled trial or a quasi-experimental design) to test use of fees with employers.

Final Report: Considerations Regarding Future Research on Use of Fees in Employment and Training Administration (ETA) Programs:  Discussion Paper

NTRS - NASA Technical Reports Server

Available downloads, related records.

Naval Research Program

Research proposal guide - naval research program, research proposal guide.

Once an Initial Research Estimate Form (IREF)  is validated and selected for funding, the next step is to complete a Research Proposal. Proposals must be completed and approved before funding can be authorized and released.

Faculty that have an IREF validated and selected for funding are required to complete the following steps before funding will be authorized and established. More details for each step are provided below; this list can be used as a checklist if desired.

Please notify NRP at [email protected]  if you have any reservations about accepting your funded project (e.g. impending retirement, emergent obligations, etc.).

Jump to section

Eligibility, annual pi training, acknowledgment of terms.

• Technical Proposal/Narrative

Proposal tab

Nps personnel tab, proposal questions tab, proposal data tab, abstracts and attachments tab, proposal budget tab, review process, proposal amendments, troubleshooting.

More events

Events & Deadlines

Quick links, principal investigators.

The Principal Investigator (PI) is the researcher who has primary responsibility for the design, execution, and management of a sponsored research project and is named on the proposal to the sponsoring agency. The PI has the primary responsibility for the fulfillment of the Statement of Work. Even when collaborating with one or more Co-PIs, the PI has the ultimate responsibility for the project and remains the sole individual responsible for managing expenditures in support of the project. 

Only eligible Naval Postgraduate School faculty participating in the mission of NPS may submit proposals and act as a PI/PD/Co-I/Co-PD for sponsored projects. Individuals in a faculty or staff category other than those listed in the SPPGM-22 require a waiver to be eligible. Sponsored Program Policy/Guidance Memo 22 (SPPGM-22): Who can be a PI/PD/Co-I/Co-PD?  (PDF, 4 January 2023) PI/PD/Co-I/Co-PD Justification Memorandum (Waiver Form)  (PDF)

^ Back to top

Training is required for anyone who functions as a PI/PD or has direct access to funds on sponsored projects. Annual PI Training is completed within Sakai, and includes four modules:

• Accountability/Fiscal Law: 13/14 required to pass
• Protection of Human Subjects: 7/8 required to pass
• OPSEC for the PI/PD: pass/fail
• Stewardship: 18/20 required to pass

PIs must pass all modules for training to be complete. Please visit Sponsored Programs Related Annual Online Trainings  to begin.

Please send a screenshot or export of your passing scores to NRP upon successful completion of your Annual PI Training.

All PIs and Co-PIs must digitally sign the Acknowledgment of Terms form . This form must then be uploaded as an attachment to your Research Proposal.

To digitally sign the form, you may need to download and save it to your computer, and open in Adobe Reader. Opening it in your web browser doesn't reliably activate the signature fields.

When uploading as an attachment, go to the Abstracts and Attachments tab in Coeus, and upload the form using the "Supplementary Documentation" category.

Please do not email any documents directly to NRP unless requested to do so.

PDF, 142KB. Last updated November 2023

All PIs must complete a Technical Proposal/Narrative, using the official NRP template.  These documents must be submitted as attachments to your Research Proposal.

• Read all template guidance prior to starting your proposal
• Complete all required template elements
• Supporting TASKS section must include direct correlation/justification for all budget expenses
• Coeus is approved for CUI, but please mark all documents accordingly if applicable
• Remove the first page containing template guidance before submitting to Coeus

Topic Advocate signatures are no longer required. Research Proposals will be routed through the chain for approval by NPS personnel and teams using Coeus. For efficiency purposes and to prevent delays, please  do not bypass  this process.

Word, 57KB. Last updated October 2023

Coeus Proposal Package

Koali Coeus is the system NRP uses to manage and review Research Proposals. To submit a Research Proposal, you will use a Proposal Development Document (PDD).

NRP creates empty PDD shells within Coeus, to pre-populate required information and simplify setup. Each PI will receive a Coeus-generated, automated email prompting you to begin your proposal.

Please email the NRP office at [email protected] with any questions, or if you don't receive your Coeus access email.

• Period of performance and milestone dates, in all proposal documents, must match.
• Proposals are only accepted through Coeus. Incomplete proposal packages will be rejected.
• Spend plans must be realistic as you will be required to execute as scheduled (burn rate).
• Expenses must include direct correlation/justification to the TASKS in your technical proposal.
• If someone is assisting you in preparing your proposal package, give them a link to this page.

This tab will be completed by NRP. Please do not make any changes to the fields present on this tab.

At minimum, you must include the PI and any Co-PIs. You can be a PI on no more than two concurrent NRP projects across all Cycle Years, and no more than one NRP project per Cycle Year.

• Enter the last name, email address, etc. of the person you wish to add. You only need to fill out one field, and the search is not case-sensitive. Click search .
• Click return value  on the far left of the row containing the correct personnel record.
• The top section of the page will now have the person's name, as well as a  Proposal Role  section. Select the appropriate role, then click  add person  below that.

If additional staff/faculty are expected to participate but have yet to be identified, input their positions in your project using To be named  instead of Employee Search .

Please answer all Proposal Questions, including any sub-questions that may appear. If you have any questions about this tab, please contact NRP .

Please fill out all Proposal Data. If you have any questions about this tab, please contact NRP .

To select multiple items in a "select all that apply" list, use  Ctrl  on Windows, or  Cmd  on Mac.

Upload your required attachments including your  Acknowledgment of Terms form  and your Technical Proposal/Narrative  under the categories below:

• Acknowledgment of Terms:  "Supplementary Documentation"
• Technical Proposal/Narrative:  "Technical Proposal/Narrative"
• Budget Justification:  "Budget Justification"
• Equipment Justification:  "Equipment"
• You must upload a waiver for  each  person requiring one

Copy-paste the abstract from your Technical Proposal/Narrative into the  Publicly Releasable Abstract  section, under Enter Abstract . Make sure to click  add  or it will not save to your PDD correctly.

When entering keywords, please enter one keyword per line,  not  a comma-separated list. Keywords are used by other integrated systems and comma-separating them can cause errors in the data integrations.

The quarterly spend plan you enter into Coeus generates your  required  burn rate schedule. This data is reported to the Budget Submission Office (BSO) and the burn rate must be executed as input into Coeus. PIs are expected to plan for salary adjustments within their budget. The authorized project amount is not increased due to promotions or Cost of Living Allowance increases.

Ensure your quarterly spend plan is realistic by accounting for:

• Cost-of-living adjustments
• Raises and promotions
• Reasonable flexibility

Official research proposal budgets must be submitted using the Coeus budget proposal tool. The  FY24 NRP Budget Spreadsheet  is used for budget/spend plan updates during the PoP. The spreadsheet can be used for general offline proposal budget planning purposes but it is highly recommended that you use the  Coeus training instance  to draft your proposal budget.

Your proposal budget is an embedded document within your PDD, and has tabs of its own. NRP has created a quarterly budget shell for you to fill out; open this budget by using the  open  button on the far right of the page.

Using your IREF budget allocations, enter quarterly expenses in the  NPS Labor  and Other Direct Costs  (Non-Labor) tabs. Your PDD's budget must be equal to or less than the budget proposed in your IREF.

If financial expenditure questions arise during budget development or execution, PIs should consult with the NRP SPFA .

How can the money be spent?

• NRP funds are RDT&E BA 6.6 and are appropriated solely for specific selected NRP projects. There must be a direct relationship between funds spent and the selected NRP research project.
• This money cannot be used for academic/curriculum support.
• This money cannot be used for office supplies, printers or cell phones.
• This money cannot be used to hire administrative personnel.

Financial Terms

• Indirect Costs:  The NRP uses NR&DE funds, and indirect cost are not collected. The indirect rate is 0%.
• Fringe  (aka Acceleration or Fully Burdened Rate): Fringe is always included in the cost of payroll regardless of how your payroll is being charged.  Fringe addresses the actual cost of benefits paid by the government for each employee (TSP, FERS, Medicare, FEGLI, TSP Matching, Annual Leave, Sick Leave, Vacation).  NPS recommends using 52.5% for projection purposes.
• Overhead:  The NRP takes a small percentage off the top of the annual budget to run the program. Therefore, no overhead cost needs to be factored into each individual project budget.

Spend Plan Justification

The Description & Purpose field(s) in Coeus for all expenses (Other Direct Costs/Non-Labor) must include a direct correlation/justification to the TASKS in your Technical Proposal/Narrative. Do not use blanket terms.

If your budget includes costs that exceed the NRP allowances, attach a Budget Justification document. Alternatively, you can include line-item Budget Justification notes in your PDD's budget.

• Ensure that quarters with proposed travel include appropriate corresponding labor hours
• For projects with budgets exceeding $175,000, total travel will be capped at $26,250. This cap is for the entire project team: PI, Co-PI, researchers and students who are listed in the approved project proposal.
• People who are not listed in the specific NRP project are not allowed to utilize these funds. Travel percentage and total cap may change in FY25.
• Unique business such as project kick-off meetings, mid-year progress review meetings, final projects delivery meetings, data collection, and/or one conference attendance that are directly tied to the project can be conducted via travel, but all “regular business” should be conducted on Teams. Virtual/online conferences are preferred and encouraged. Labor must be charged to the project at the same time you are on travel.
• Ensure travel and labor expenses align with research completion timelines, to include any travel for the purpose of final debriefing/delivering the final product(s)
• Students:  Student participation is allowed; however, travel requests must state how the travel applies to the associated NRP research.
• DTS form justification box must include: NRP project number, PI Name, detailed purpose of the travel i.e., how the travel is directly related to the NRP project, travel is/is not included in the original proposal.
• All travelers are required to submit a  Travel Report  after each trip.
• Academic purposes
• Thesis development
• Student graduations
• Other research projects
• List all known project personnel. List additional planned but unidentified individuals under "To be named." Notify the NRP SPFA  of project personnel changes immediately to avoid a payroll approval delay.
• Labor is charged using actual benefits and varies per individual. Rates are updated/calculated in Coeus routinely. Consult your SPFA for individual rates. NRP is exempt from other Indirect Costs.
• NRP funds are not appropriate for employee cash awards. All awards using NRP funds will be reversed upon detection.
• The fully burdened amount is listed in Coeus. PIs are expected to plan for salary adjustments within their budget. The authorized project amount is not increased due to promotions or Cost of Living Allowance increases .

External Support

If you are intending/planning to outsource labor/skills that cannot be performed by NPS personnel you must obtain approval from your department chair.

NRP funds cannot be used to hire administrative personnel.

Acquisitions

All purchases must align with the tasks and deliverables cited in your proposal and require a detailed justification to be submitted in ERP. As per the Annual PI Training, purchasing for "the greater good" is not allowed.

Provide an explanation, in plain language, detailing how each purchase contributes to the tasks and deliverables of the project. Blanket terms such as "Mission Essential / Critical" are not a valid justification.

Total purchasing exceeding 25% of your project's total budget will require additional justification.

• Orders should be submitted early in order to contribute to the project deliverable(s), and must be acquired within the project's Period of Performance.
• Acquisitions for computers, equipment, contracts, and MIPRS are  only  approved for the benefit of the selected NRP project. Therefore, each item must be ordered soon enough that it arrives early enough to contribute to the project deliverables.
• PIs may be allowed to purchase equipment with justification (e.g., computers) once every three years, however purchasing peripheral equipment that is considered office equipment is not allowed.
• All equipment must be shipped directly to the warehouse and registered in the NPS property accounting system prior to receipt under the PI’s name. The PI is accountable to produce records during an audit.
• Cell phones & cell phone services
• Printers, ink, and toner
• Office supplies
• Publications (Please contact NRP for NRP-related publication expenses)
• Items considered to be for general purposes
• Checklist of OSHE Related Hazards to Consider
• Include labor, time, and costs for safety controls.
• Include safety and environmental planning and training hours.
• Include funds for safety assessments if needed, protective equipment and physical controls.
• Critical to plan ahead for off-campus activities, UUVs, lithium batteries, RF emitter, hazmat, lasers, etc.

Review  Safety Information Needed from PIs for Project Descriptions

For safety questions or concerns email  [email protected]  or visit the  Safety Review and Planning page on the  Safety website.

Finalizing your budget

Once your budget is complete, save it with the button at the bottom, and use the blue  Return to Proposal  button at the top right. Check the  Final  box for your budget in the  Proposal Budget  tab.

• Coeus User Guide
• Contact NRP
• Contact Coeus Ombudsman
• Contact NRP SPFA for financial expenditure questions
• Budget Spreadsheet

Submit Your Proposal

Once your PDD is completed, run Data Validation .

• Go to the Proposal Actions tab
• On the  Data Validation  section, click the  show  button to expand it
• Click  turn on validation
• Go through each of the  Validation Errors  and  Warnings , and fix them
• Reach out to NRP at  [email protected]  if you need any help

Once your PDD passes validation, please use the  Submit  button at the bottom of the  Proposal Actions  tab to submit it for approval. NRP will receive an automated email from Coeus letting us know when your PDD is ready to review.

See the Troubleshooting guide below for help resolving errors. NRP is also happy to assist; contact us at [email protected] .

Once your PDD is successfully submitted, it goes through several stages of review:

• Your department chair reviews and approves your PDD. If you have Co-PIs listed, their department chairs must review and approve your PDD as well.
• NRP reviews your PDD for completeness, accuracy, and adherence to requirements and guidance.
• NRP's Financial Manager reviews your budget and spending allocations.
• NRP's Program Manager reviews and approves your PDD.
• The Vice Provost for Research and Innovation reviews and approves your PDD.

Once the VPR has approved your PDD, NRP double-checks a few more requirements:

• Completion of your annual PI training
• Completion and submission of all previous Cycle Years' required research deliverables
• Successful approval from the  Human Research Protection Program Office & Institutional Review Board (IRB) , if required

Once these are complete, NRP then issues funding for you to begin your research.

To check your proposal's current status in the review process:

• Log in to Coeus
• In the  Proposals  section of the Coeus homepage, click "Search Proposals"
• In the  Document ID  field, enter the five-digit Document ID of your proposal. If you don't know the Document ID, please reach out to NRP and we can provide that to you.
• Click the  Search  button
• Click "view" on the left of the row your proposal shows up in, in the search results
• Go to the  Proposal Actions  tab
• Open the  Route Log  section by clicking the "show" button on it
• In Action List to Complete:  Approval is required
• In Action List to FYI:  Approval is not required; the person was notified for their situational awareness
• If multiple people are listed within the same approval line, only one of them is required to approve

To make amendments to an approved and finalized PDD, email the Coeus Ombudsman  with your amendment request.

The types of changes that require an amendment through Coeus are:

• Changes to the Period of Performance
• Budget increases
• Changes to the PI or Co-PI(s)
• Topic Advocate changes
• Statement of Work changes

Here are solutions to common problems encountered through the Research Proposal process.

If you're still having trouble, please reach out to NRP . We're happy to help!

PDDs are "checked out" when someone opens them to edit them, and locked until the person editing clicks the  close  button to "check in" the proposal.  Closing the browser tab will not check in the PDD.

If your proposal was locked by someone else, Coeus will display their name at the top of the PDD. Simply email them and request that they open the proposal and close it using the  close  button at the bottom of the page rather than just closing the browser tab.

If you're not able to get a hold of the person your PDD is checked out to, email the Coeus Ombudsman to request an administrator to unlock your PDD.

Reach out to NRP and let us know; we can add them as an authorized user to your PDD.

Opening the PDF in your browser doesn't always activate the signature fields. Download it to your computer and open it in Adobe Reader or Adobe Acrobat. The fields should show up.

• SUBSCRIPTION SERVICES
• EDITORIAL CALENDAR
• EVENTS / WEBINARS
• INDUSTRY LINKS
• BUYERS DIRECTORY

Irving Pulp & Paper Launches New Technical Training Program at Saint John Pulp Mill

May 23, 2024 - Irving Pulp & Paper recently announced that its pulp mill in Saint John, New Brunswick (Canada) has launched an internal technical training program to improve collaboration and knowledge sharing across Irving Pulp & Paper called the University of Reversing Falls.

Over the past six months, the training program gave team members comprehensive knowledge of the mill's operational processes to help them excel in their careers. The training program was instructed by 13 highly skilled and experienced Irving Pulp & Paper employees from a variety of disciplines and areas of the mill.

According to the company, after 24 hours of in-person training, 24 Irving Pulp & Paper employees successfully completed the University of Reversing Falls.

"Being able to continuously grow and develop our people is key to our business. Here at Irving Pulp & Paper we are committed to developing our team members to allow them the opportunity to continue growing in their careers," said Dennis Mott, Mill Manager at Irving Pulp & Paper.

The company added, "We are very proud of our employees and are happy to share their success."

Irving Pulp & Paper produces softwood and hardwood grades of kraft pulp — Northern Bleached Softwood Kraft (NBSK) and Northern Bleached Hardwood Kraft (NBHK). The mill specializes in fiber for premium tissue products in global markets, producing exceptional tissue quality NBSK pulp.

SOURCE: Irving Pulp & Paper

Paper Industry Newsletter Stay on top of paper industry news from around the world with PaperAge's free weekly newsletter. Delivered every Thursday. Sign up today!

• Subscription Services
• Editorial Calendar
• Media Guide
• Events / Webinars
• Industry Links
• Buyers Directory
• About PaperAge

PaperAge PO Box 904, Cohasset, MA 02025 (USA) Phone: 781-378-1253

Editorial Contact – Print, Website, Newsletter John O'Brien, Managing Editor Phone: 781-536-4925 email: [email protected]

Advertising Sales – Print and Digital Michael O'Brien, Publisher Phone: 781-378-1253 email: [email protected]

PaperAge. Copyright © O'Brien Publications, Inc. All rights reserved.

Help | Advanced Search

Computer Science > Computation and Language

Title: chameleon: mixed-modal early-fusion foundation models.

Abstract: We present Chameleon, a family of early-fusion token-based mixed-modal models capable of understanding and generating images and text in any arbitrary sequence. We outline a stable training approach from inception, an alignment recipe, and an architectural parameterization tailored for the early-fusion, token-based, mixed-modal setting. The models are evaluated on a comprehensive range of tasks, including visual question answering, image captioning, text generation, image generation, and long-form mixed modal generation. Chameleon demonstrates broad and general capabilities, including state-of-the-art performance in image captioning tasks, outperforms Llama-2 in text-only tasks while being competitive with models such as Mixtral 8x7B and Gemini-Pro, and performs non-trivial image generation, all in a single model. It also matches or exceeds the performance of much larger models, including Gemini Pro and GPT-4V, according to human judgments on a new long-form mixed-modal generation evaluation, where either the prompt or outputs contain mixed sequences of both images and text. Chameleon marks a significant step forward in a unified modeling of full multimodal documents.

Submission history

Access paper:.

• HTML (experimental)
• Other Formats

References & Citations

• Google Scholar
• Semantic Scholar

BibTeX formatted citation

Bibliographic and Citation Tools

Code, data and media associated with this article, recommenders and search tools.

• Institution

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs .