risk assessment in research paper

Home / Resources / ISACA Journal / Issues / 2021 / Volume 2 / Risk Assessment and Analysis Methods

Risk assessment and analysis methods: qualitative and quantitative.

A risk assessment determines the likelihood, consequences and tolerances of possible incidents. “Risk assessment is an inherent part of a broader risk management strategy to introduce control measures to eliminate or reduce any potential risk- related consequences.” 1 The main purpose of risk assessment is to avoid negative consequences related to risk or to evaluate possible opportunities.

It is the combined effort of:

• “…[I]dentifying and analyzing possible future events that could adversely affect individuals, assets, processes and/or the environment (i.e.,risk analysis)”
• “…[M]aking judgments about managing and tolerating risk on the basis of a risk analysis while considering influencing factors (i.e., risk evaluation)” 2

Relationships between assets, processes, threats, vulnerabilities and other factors are analyzed in the risk assessment approach. There are many methods available, but quantitative and qualitative analysis are the most widely known and used classifications. In general, the methodology chosen at the beginning of the decision-making process should be able to produce a quantitative explanation about the impact of the risk and security issues along with the identification of risk and formation of a risk register. There should also be qualitative statements that explain the importance and suitability of controls and security measures to minimize these risk areas. 3

In general, the risk management life cycle includes seven main processes that support and complement each other ( figure 1 ):

• Determine the risk context and scope, then design the risk management strategy.
• Choose the responsible and related partners, identify the risk and prepare the risk registers.
• Perform qualitative risk analysis and select the risk that needs detailed analysis.
• Perform quantitative risk analysis on the selected risk.
• Plan the responses and determine controls for the risk that falls outside the risk appetite.
• Implement risk responses and chosen controls.
• Monitor risk improvements and residual risk.

Qualitative and Quantitative Risk Analysis Techniques

Different techniques can be used to evaluate and prioritize risk. Depending on how well the risk is known, and if it can be evaluated and prioritized in a timely manner, it may be possible to reduce the possible negative effects or increase the possible positive effects and take advantage of the opportunities. 4 “Quantitative risk analysis tries to assign objective numerical or measurable values” regardless of the components of the risk assessment and to the assessment of potential loss. Conversely, “a qualitative risk analysis is scenario-based.” 5

Qualitative Risk The purpose of qualitative risk analysis is to identify the risk that needs detail analysis and the necessary controls and actions based on the risk’s effect and impact on objectives. 6 In qualitative risk analysis, two simple methods are well known and easily applied to risk: 7

• Keep It Super Simple (KISS) —This method can be used on narrow-framed or small projects where unnecessary complexity should be avoided and the assessment can be made easily by teams that lack maturity in assessing risk. This one-dimensional technique involves rating risk on a basic scale, such as very high/high/medium/low/very.
• Probability/Impact —This method can be used on larger, more complex issues with multilateral teams that have experience with risk assessments. This two-dimensional technique is used to rate probability and impact. Probability is the likelihood that a risk will occur. The impact is the consequence or effect of the risk, normally associated with impact to schedule, cost, scope and quality. Rate probability and impact using a scale such as 1 to 10 or 1 to 5, where the risk score equals the probability multiplied by the impact.

Qualitative risk analysis can generally be performed on all business risk. The qualitative approach is used to quickly identify risk areas related to normal business functions. The evaluation can assess whether peoples’ concerns about their jobs are related to these risk areas. Then, the quantitative approach assists on relevant risk scenarios, to offer more detailed information for decision-making. 8 Before making critical decisions or completing complex tasks, quantitative risk analysis provides more objective information and accurate data than qualitative analysis. Although quantitative analysis is more objective, it should be noted that there is still an estimate or inference. Wise risk managers consider other factors in the decision-making process. 9

Although a qualitative risk analysis is the first choice in terms of ease of application, a quantitative risk analysis may be necessary. After qualitative analysis, quantitative analysis can also be applied. However, if qualitative analysis results are sufficient, there is no need to do a quantitative analysis of each risk.

Quantitative Risk A quantitative risk analysis is another analysis of high-priority and/or high-impact risk, where a numerical or quantitative rating is given to develop a probabilistic assessment of business-related issues. In addition, quantitative risk analysis for all projects or issues/processes operated with a project management approach has a more limited use, depending on the type of project, project risk and the availability of data to be used for quantitative analysis. 10

The purpose of a quantitative risk analysis is to translate the probability and impact of a risk into a measurable quantity. 11 A quantitative analysis: 12

• “Quantifies the possible outcomes for the business issues and assesses the probability of achieving specific business objectives”
• “Provides a quantitative approach to making decisions when there is uncertainty”
• “Creates realistic and achievable cost, schedule or scope targets”

Consider using quantitative risk analysis for: 13

• “Business situations that require schedule and budget control planning”
• “Large, complex issues/projects that require go/no go decisions”
• “Business processes or issues where upper management wants more detail about the probability of completing on schedule and within budget”

The advantages of using quantitative risk analysis include: 14

• Objectivity in the assessment
• Powerful selling tool to management
• Direct projection of cost/benefit
• Flexibility to meet the needs of specific situations
• Flexibility to fit the needs of specific industries
• Much less prone to arouse disagreements during management review
• Analysis is often derived from some irrefutable facts

THE MOST COMMON PROBLEM IN QUANTITATIVE ASSESSMENT IS THAT THERE IS NOT ENOUGH DATA TO BE ANALYZED.

To conduct a quantitative risk analysis on a business process or project, high-quality data, a definite business plan, a well-developed project model and a prioritized list of business/project risk are necessary. Quantitative risk assessment is based on realistic and measurable data to calculate the impact values that the risk will create with the probability of occurrence. This assessment focuses on mathematical and statistical bases and can “express the risk values in monetary terms, which makes its results useful outside the context of the assessment (loss of money is understandable for any business unit). 15  The most common problem in quantitative assessment is that there is not enough data to be analyzed. There also can be challenges in revealing the subject of the evaluation with numerical values or the number of relevant variables is too high. This makes risk analysis technically difficult.

There are several tools and techniques that can be used in quantitative risk analysis. Those tools and techniques include: 16

• Heuristic methods —Experience-based or expert- based techniques to estimate contingency
• Three-point estimate —A technique that uses the optimistic, most likely and pessimistic values to determine the best estimate
• Decision tree analysis —A diagram that shows the implications of choosing various alternatives
• Expected monetary value (EMV) —A method used to establish the contingency reserves for a project or business process budget and schedule
• Monte Carlo analysis —A technique that uses optimistic, most likely and pessimistic estimates to determine the business cost and project completion dates
• Sensitivity analysis —A technique used to determine the risk that has the greatest impact on a project or business process
• Fault tree analysis (FTA) and failure modes and effects analysis (FMEA) —The analysis of a structured diagram that identifies elements that can cause system failure

There are also some basic (target, estimated or calculated) values used in quantitative risk assessment. Single loss expectancy (SLE) represents the money or value expected to be lost if the incident occurs one time, and an annual rate of occurrence (ARO) is how many times in a one-year interval the incident is expected to occur. The annual loss expectancy (ALE) can be used to justify the cost of applying countermeasures to protect an asset or a process. That money/value is expected to be lost in one year considering SLE and ARO. This value can be calculated by multiplying the SLE with the ARO. 17 For quantitative risk assessment, this is the risk value. 18

USING BOTH APPROACHES CAN IMPROVE PROCESS EFFICIENCY AND HELP ACHIEVE DESIRED SECURITY LEVELS.

By relying on factual and measurable data, the main benefits of quantitative risk assessment are the presentation of very precise results about risk value and the maximum investment that would make risk treatment worthwhile and profitable for the organization. For quantitative cost-benefit analysis, ALE is a calculation that helps an organization to determine the expected monetary loss for an asset or investment due to the related risk over a single year.

For example, calculating the ALE for a virtualization system investment includes the following:

• Virtualization system hardware value: US$1 million (SLE for HW)
• Virtualization system management software value: US$250,000 (SLE for SW)
• Vendor statistics inform that a system catastrophic failure (due to software or hardware) occurs one time every 10 years (ARO = 1/10 = 0.1)
• ALE for HW = 1M * 1 = US$100,000
• ALE for SW = 250K * 0.1 = US$25,000

In this case, the organization has an annual risk of suffering a loss of US$100,000 for hardware or US$25,000 for software individually in the event of the loss of its virtualization system. Any implemented control (e.g., backup, disaster recovery, fault tolerance system) that costs less than these values would be profitable.

Some risk assessment requires complicated parameters. More examples can be derived according to the following “step-by-step breakdown of the quantitative risk analysis”: 19

• Conduct a risk assessment and vulnerability study to determine the risk factors.
• Determine the exposure factor (EF), which is the percentage of asset loss caused by the identified threat.
• Based on the risk factors determined in the value of tangible or intangible assets under risk, determine the SLE, which equals the asset value multiplied by the exposure factor.
• Evaluate the historical background and business culture of the institution in terms of reporting security incidents and losses (adjustment factor).
• Estimate the ARO for each risk factor.
• Determine the countermeasures required to overcome each risk factor.
• Add a ranking number from one to 10 for quantifying severity (with 10 being the most severe) as a size correction factor for the risk estimate obtained from company risk profile.
• Determine the ALE for each risk factor. Note that the ARO for the ALE after countermeasure implementation may not always be equal to zero. ALE (corrected) equals ALE (table) times adjustment factor times size correction.
• Calculate an appropriate cost/benefit analysis by finding the differences before and after the implementation of countermeasures for ALE.
• Determine the return on investment (ROI) based on the cost/benefit analysis using internal rate of return (IRR).
• Present a summary of the results to management for review.

Using both approaches can improve process efficiency and help achieve desired security levels. In the risk assessment process, it is relatively easy to determine whether to use a quantitative or a qualitative approach. Qualitative risk assessment is quick to implement due to the lack of mathematical dependence and measurements and can be performed easily. Organizations also benefit from the employees who are experienced in asset/processes; however, they may also bring biases in determining probability and impact. Overall, combining qualitative and quantitative approaches with good assessment planning and appropriate modeling may be the best alternative for a risk assessment process ( figure 2 ). 20

Qualitative risk analysis is quick but subjective. On the other hand, quantitative risk analysis is optional and objective and has more detail, contingency reserves and go/no-go decisions, but it takes more time and is more complex. Quantitative data are difficult to collect, and quality data are prohibitively expensive. Although the effect of mathematical operations on quantitative data are reliable, the accuracy of the data is not guaranteed as a result of being numerical only. Data that are difficult to collect or whose accuracy is suspect can lead to inaccurate results in terms of value. In that case, business units cannot provide successful protection or may make false-risk treatment decisions and waste resources without specifying actions to reduce or eliminate risk. In the qualitative approach, subjectivity is considered part of the process and can provide more flexibility in interpretation than an assessment based on quantitative data. 21 For a quick and easy risk assessment, qualitative assessment is what 99 percent of organizations use. However, for critical security issues, it makes sense to invest time and money into quantitative risk assessment. 22 By adopting a combined approach, considering the information and time response needed, with data and knowledge available, it is possible to enhance the effectiveness and efficiency of the risk assessment process and conform to the organization’s requirements.

1 ISACA ® , CRISC Review Manual, 6 th Edition , USA, 2015, https://store.isaca.org/s/store#/store/browse/detail/a2S4w000004Ko8ZEAS 2 Ibid. 3 Schmittling, R.; A. Munns; “Performing a Security Risk Assessment,” ISACA ® Journal , vol. 1, 2010, https://www.isaca.org/resources/isaca-journal/issues 4 Bansal,; "Differentiating Quantitative Risk and Qualitative Risk Analysis,” iZenBridge,12 February 2019, https://www.izenbridge.com/blog/differentiating-quantitative-risk-analysis-and-qualitative-risk-analysis/ 5 Tan, D.; Quantitative Risk Analysis Step-By-Step , SANS Institute Information Security Reading Room, December 2020, https://www.sans.org/reading-room/whitepapers/auditing/quantitative-risk-analysis-step-by-step-849 6 Op cit Bansal 7 Hall, H.; “Evaluating Risks Using Qualitative Risk Analysis,” Project Risk Coach, https://projectriskcoach.com/evaluating-risks-using-qualitative-risk-analysis/ 8 Leal, R.; “Qualitative vs. Quantitative Risk Assessments in Information Security: Differences and Similarities,” 27001 Academy, 6 March 2017, https://advisera.com/27001academy/blog/2017/03/06/qualitative-vs-quantitative-risk-assessments-in-information-security/ 9 Op cit Hall 10 Goodrich, B.; “Qualitative Risk Analysis vs. Quantitative Risk Analysis,” PM Learning Solutions, https://www.pmlearningsolutions.com/blog/qualitative-risk-analysis-vs-quantitative-risk-analysis-pmp-concept-1 11 Meyer, W. ; “Quantifying Risk: Measuring the Invisible,” PMI Global Congress 2015—EMEA, London, England, 10 October 2015, https://www.pmi.org/learning/library/quantitative-risk-assessment-methods-9929 12 Op cit Goodrich 13 Op cit Hall 14 Op cit Tan 15 Op cit Leal 16 Op cit Hall 17 Tierney, M.; “Quantitative Risk Analysis: Annual Loss Expectancy," Netwrix Blog, 24 July 2020, https://blog.netwrix.com/2020/07/24/annual-loss-expectancy-and-quantitative-risk-analysis 18 Op cit Leal 19 Op cit Tan 20 Op cit Leal 21 ISACA ® , Conductin g a n IT Security Risk Assessment, USA, 2020, https://store.isaca.org/s/store#/store/browse/detail/a2S4w000004KoZeEAK 22 Op cit Leal

Volkan Evrin, CISA, CRISC, COBIT 2019 Foundation, CDPSE, CEHv9, ISO 27001-22301-20000 LA

Has more than 20 years of professional experience in information and technology (I&T) focus areas including information systems and security, governance, risk, privacy, compliance, and audit. He has held executive roles on the management of teams and the implementation of projects such as information systems, enterprise applications, free software, in-house software development, network architectures, vulnerability analysis and penetration testing, informatics law, Internet services, and web technologies. He is also a part-time instructor at Bilkent University in Turkey; an APMG Accredited Trainer for CISA, CRISC and COBIT 2019 Foundation; and a trainer for other I&T-related subjects. He can be reached at [email protected] .

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 08 April 2024

A case study on the relationship between risk assessment of scientific research projects and related factors under the Naive Bayesian algorithm

• Xuying Dong 1 &
• Wanlin Qiu 1  

Scientific Reports volume  14 , Article number:  8244 ( 2024 ) Cite this article

347 Accesses

Metrics details

• Computer science
• Mathematics and computing

This paper delves into the nuanced dynamics influencing the outcomes of risk assessment (RA) in scientific research projects (SRPs), employing the Naive Bayes algorithm. The methodology involves the selection of diverse SRPs cases, gathering data encompassing project scale, budget investment, team experience, and other pertinent factors. The paper advances the application of the Naive Bayes algorithm by introducing enhancements, specifically integrating the Tree-augmented Naive Bayes (TANB) model. This augmentation serves to estimate risk probabilities for different research projects, shedding light on the intricate interplay and contributions of various factors to the RA process. The findings underscore the efficacy of the TANB algorithm, demonstrating commendable accuracy (average accuracy 89.2%) in RA for SRPs. Notably, budget investment (regression coefficient: 0.68, P < 0.05) and team experience (regression coefficient: 0.51, P < 0.05) emerge as significant determinants obviously influencing RA outcomes. Conversely, the impact of project size (regression coefficient: 0.31, P < 0.05) is relatively modest. This paper furnishes a concrete reference framework for project managers, facilitating informed decision-making in SRPs. By comprehensively analyzing the influence of various factors on RA, the paper not only contributes empirical insights to project decision-making but also elucidates the intricate relationships between different factors. The research advocates for heightened attention to budget investment and team experience when formulating risk management strategies. This strategic focus is posited to enhance the precision of RAs and the scientific foundation of decision-making processes.

Similar content being viewed by others

Beyond probability-impact matrices in project risk management: A quantitative methodology for risk prioritisation

A method for managing scientific research project resource conflicts and predicting risks using BP neural networks

Prediction of SMEs’ R&D performances by machine learning for project selection

Introduction.

Scientific research projects (SRPs) stand as pivotal drivers of technological advancement and societal progress in the contemporary landscape 1 , 2 , 3 . The dynamism of SRP success hinges on a multitude of internal and external factors 4 . Central to effective project management, Risk assessment (RA) in SRPs plays a critical role in identifying and quantifying potential risks. This process not only aids project managers in formulating strategic decision-making approaches but also enhances the overall success rate and benefits of projects. In a recent contribution, Salahuddin 5 provides essential numerical techniques indispensable for conducting RAs in SRPs. Building on this foundation, Awais and Salahuddin 6 delve into the assessment of risk factors within SRPs, notably introducing the consideration of activation energy through an exploration of the radioactive magnetohydrodynamic model. Further expanding the scope, Awais and Salahuddin 7 undertake a study on the natural convection of coupled stress fluids. However, RA of SRPs confronts a myriad of challenges, underscoring the critical need for novel methodologies 8 . Primarily, the intricate nature of SRPs renders precise RA exceptionally complex and challenging. The project’s multifaceted dimensions, encompassing technology, resources, and personnel, are intricately interwoven, posing a formidable challenge for traditional assessment methods to comprehensively capture all potential risks 9 . Furthermore, the intricate and diverse interdependencies among various project factors contribute to the complexity of these relationships, thereby limiting the efficacy of conventional methods 10 , 11 , 12 . Traditional approaches often focus solely on the individual impact of diverse factors, overlooking the nuanced relationships that exist between them—an inherent limitation in the realm of RA for SRPs 13 , 14 , 15 .

The pursuit of a methodology capable of effectively assessing project risks while elucidating the intricate interplay of different factors has emerged as a focal point in SRPs management 16 , 17 , 18 . This approach necessitates a holistic consideration of multiple factors, their quantification in contributing to project risks, and the revelation of their correlations. Such an approach enables project managers to more precisely predict and respond to risks. Marx-Stoelting et al. 19 , current approaches for the assessment of environmental and human health risks due to exposure to chemical substances have served their purpose reasonably well. Additionally, Awais et al. 20 highlights the significance of enthalpy changes in SRPs risk considerations, while Awais et al. 21 delve into the comprehensive exploration of risk factors in Eyring-Powell fluid flow in magnetohydrodynamics, particularly addressing viscous dissipation and activation energy effects. The Naive Bayesian algorithm, recognized for its prowess in probability and statistics, has yielded substantial results in information retrieval and data mining in recent years 22 . Leveraging its advantages in classification and probability estimation, the algorithm presents a novel approach for RA of SRPs 23 . Integrating probability analysis into RA enables a more precise estimation of project risks by utilizing existing project data and harnessing the capabilities of the Naive Bayesian algorithms. This method facilitates a quantitative, statistical analysis of various factors, effectively navigating the intricate relationships between them, thereby enhancing the comprehensiveness and accuracy of RA for SRPs.

This paper seeks to employ the Naive Bayesian algorithm to estimate the probability of risks by carefully selecting distinct research project cases and analyzing multidimensional data, encompassing project scale, budget investment, and team experience. Concurrently, Multiple Linear Regression (MLR) analysis is applied to quantify the influence of these factors on the assessment results. The paper places particular emphasis on exploring the intricate interrelationships between different factors, aiming to provide a more specific and accurate reference framework for decision-making in SRPs management.

This paper introduces several innovations and contributions to the field of RA for SRPs:

Comprehensive Consideration of Key Factors: Unlike traditional research that focuses on a single factor, this paper comprehensively considers multiple key factors, such as project size, budget investment, and team experience. This holistic analysis enhances the realism and thoroughness of RA for SRPs.

Introduction of Tree-Enhanced Naive Bayes Model: The naive Bayes algorithm is introduced and further improved through the proposal of a tree-enhanced naive Bayes model. This algorithm exhibits unique advantages in handling uncertainty and complexity, thereby enhancing its applicability and accuracy in the RA of scientific and technological projects.

Empirical Validation: The effectiveness of the proposed method is not only discussed theoretically but also validated through empirical cases. The analysis of actual cases provides practical support and verification, enhancing the credibility of the research results.

Application of MLR Analysis: The paper employs MLR analysis to delve into the impact of various factors on RA. This quantitative analysis method adds specificity and operability to the research, offering a practical decision-making basis for scientific and technological project management.

Discovery of New Connections and Interactions: The paper uncovers novel connections and interactions, such as the compensatory role of team experience for budget-related risks and the impact of the interaction between project size and budget investment on RA results. These insights provide new perspectives for decision-making in technology projects, contributing significantly to the field of RA for SRPs in terms of both importance and practical value.

The paper is structured as follows: “ Introduction ” briefly outlines the significance of RA for SRPs. Existing challenges within current research are addressed, and the paper’s core objectives are elucidated. A distinct emphasis is placed on the innovative aspects of this research compared to similar studies. The organizational structure of the paper is succinctly introduced, providing a brief overview of each section’s content. “ Literature review ” provides a comprehensive review of relevant theories and methodologies in the domain of RA for SRPs. The current research landscape is systematically examined, highlighting the existing status and potential gaps. Shortcomings in previous research are analyzed, laying the groundwork for the paper’s motivation and unique contributions. “ Research methodology ” delves into the detailed methodologies employed in the paper, encompassing data collection, screening criteria, preprocessing steps, and more. The tree-enhanced naive Bayes model is introduced, elucidating specific steps and the purpose behind MLR analysis. “ Results and discussion ” unfolds the results and discussions based on selected empirical cases. The representativeness and diversity of these cases are expounded upon. An in-depth analysis of each factor’s impact and interaction in the context of RA is presented, offering valuable insights. “ Discussion ” succinctly summarizes the entire research endeavor. Potential directions for further research and suggestions for improvement are proposed, providing a thoughtful conclusion to the paper.

Literature review

A review of ra for srps.

In recent years, the advancement of SRPs management has led to the evolution of various RA methods tailored for SRPs. The escalating complexity of these projects poses a challenge for traditional methods, often falling short in comprehensively considering the intricate interplay among multiple factors and yielding incomplete assessment outcomes. Scholars, recognizing the pivotal role of factors such as project scale, budget investment, and team experience in influencing project risks, have endeavored to explore these dynamics from diverse perspectives. Siyal et al. 24 pioneered the development and testing of a model geared towards detecting SRPs risks. Chen et al. 25 underscored the significance of visual management in SRPs risk management, emphasizing its importance in understanding and mitigating project risks. Zhao et al. 26 introduced a classic approach based on cumulative prospect theory, offering an optional method to elucidate researchers’ psychological behaviors. Their study demonstrated the enhanced rationality achieved by utilizing the entropy weight method to derive attribute weight information under Pythagorean fuzzy sets. This approach was then applied to RA for SRPs, showcasing a model grounded in the proposed methodology. Suresh and Dillibabu 27 proposed an innovative hybrid fuzzy-based machine learning mechanism tailored for RA in software projects. This hybrid scheme facilitated the identification and ranking of major software project risks, thereby supporting decision-making throughout the software project lifecycle. Akhavan et al. 28 introduced a Bayesian network modeling framework adept at capturing project risks by calculating the uncertainty of project net present value. This model provided an effective means for analyzing risk scenarios and their impact on project success, particularly applicable in evaluating risks for innovative projects that had undergone feasibility studies.

A review of factors affecting SRPs

Within the realm of SRPs management, the assessment and proficient management of project risks stand as imperative components. Consequently, a range of studies has been conducted to explore diverse methods and models aimed at enhancing the comprehension and decision support associated with project risks. Guan et al. 29 introduced a new risk interdependence network model based on Monte Carlo simulation to support decision-makers in more effectively assessing project risks and planning risk management actions. They integrated interpretive structural modeling methods into the model to develop a hierarchical project risk interdependence network based on identified risks and their causal relationships. Vujović et al. 30 provided a new method for research in project management through careful analysis of risk management in SRPs. To confirm the hypothesis, the study focused on educational organizations and outlined specific project management solutions in business systems, thereby improving the business and achieving positive business outcomes. Muñoz-La Rivera et al. 31 described and classified the 100 identified factors based on the dimensions and aspects of the project, assessed their impact, and determined whether they were shaping or directly affecting the occurrence of research project accidents. These factors and their descriptions and classifications made significant contributions to improving the security creation of the system and generating training and awareness materials, fostering the development of a robust security culture within organizations. Nguyen et al. concentrated on the pivotal risk factors inherent in design-build projects within the construction industry. Effective identification and management of these factors enhanced project success and foster confidence among owners and contractors in adopting the design-build approach 32 . Their study offers valuable insights into RA in project management and the adoption of new contract forms. Nguyen and Le delineated risk factors influencing the quality of 20 civil engineering projects during the construction phase 33 . The top five risks identified encompass poor raw material quality, insufficient worker skills, deficient design documents and drawings, geographical challenges at construction sites, and inadequate capabilities of main contractors and subcontractors. Meanwhile, Nguyen and Phu Pham concentrated on office building projects in Ho Chi Minh City, Vietnam, to pinpoint key risk factors during the construction phase 34 . These factors were classified into five groups based on their likelihood and impact: financial, management, schedule, construction, and environmental. Findings revealed that critical factors affecting office building projects encompassed both natural elements (e.g., prolonged rainfall, storms, and climate impacts) and human factors (e.g., unstable soil, safety behavior, owner-initiated design changes), with schedule-related risks exerting the most significant influence during the construction phase of Ho Chi Minh City’s office building projects. This provides construction and project management practitioners with fresh insights into risk management, aiding in the comprehensive identification, mitigation, and management of risk factors in office building projects.

While existing research has made notable strides in RA for SRPs, certain limitations persist. These studies limitations in quantifying the degree of influence of various factors and analyzing their interrelationships, thereby falling short of offering specific and actionable recommendations. Traditional methods, due to their inherent limitations, struggle to precisely quantify risk degrees and often overlook the intricate interplay among multiple factors. Consequently, there is an urgent need for a comprehensive method capable of quantifying the impact of diverse factors and revealing their correlations. In response to this exigency, this paper introduces the TANB model. The unique advantages of this algorithm in the RA of scientific and technological projects have been fully realized. Tailored to address the characteristics of uncertainty and complexity, the model represents a significant leap forward in enhancing applicability and accuracy. In comparison with traditional methods, the TANB model exhibits greater flexibility and a heightened ability to capture dependencies between features, thereby elevating the overall performance of RA. This innovative method emerges as a more potent and reliable tool in the realm of scientific and technological project management, furnishing decision-makers with more comprehensive and accurate support for RA.

Research methodology

This paper centers on the latest iteration of ISO 31000, delving into the project risk management process and scrutinizing the RA for SRPs and their intricate interplay with associated factors. ISO 31000, an international risk management standard, endeavors to furnish businesses, organizations, and individuals with a standardized set of risk management principles and guidelines, defining best practices and establishing a common framework. The paper unfolds in distinct phases aligned with ISO 31000:

Risk Identification: Employing data collection and preparation, a spectrum of factors related to project size, budget investment, team member experience, project duration, and technical difficulty were identified.

RA: Utilizing the Naive Bayes algorithm, the paper conducts RA for SRPs, estimating the probability distribution of various factors influencing RA results.

Risk Response: The application of the Naive Bayes model is positioned as a means to respond to risks, facilitating the formulation of apt risk response strategies based on calculated probabilities.

Monitoring and Control: Through meticulous data collection, model training, and verification, the paper illustrates the steps involved in monitoring and controlling both data and models. Regular monitoring of identified risks and responses allows for adjustments when necessary.

Communication and Reporting: Maintaining effective communication throughout the project lifecycle ensures that stakeholders comprehend the status of project risks. Transparent reporting on discussions and outcomes contributes to an informed project environment.

Data collection and preparation

In this paper, a meticulous approach is undertaken to select representative research project cases, adhering to stringent screening criteria. Additionally, a thorough review of existing literature is conducted and tailored to the practical requirements of SRPs management. According to Nguyen et al., these factors play a pivotal role in influencing the RA outcomes of SRPs 35 . Furthermore, research by He et al. underscored the significant impact of team members’ experience on project success 36 . Therefore, in alignment with our research objectives and supported by the literature, this paper identifies variables such as project scale, budget investment, team member experience, project duration, and technical difficulty as the focal themes. To ensure the universality and scientific rigor of our findings, the paper adheres to stringent selection criteria during the project case selection process. After preliminary screening of SRPs completed in the past 5 years, considering factors such as project diversity, implementation scales, and achieved outcomes, five representative projects spanning diverse fields, including engineering, medicine, and information technology, are ultimately selected. These project cases are chosen based on their capacity to represent various scales and types of SRPs, each possessing a typical risk management process, thereby offering robust and comprehensive data support for our study. The subsequent phase involves detailed data collection on each chosen project, encompassing diverse dimensions such as project scale, budget investment, team member experience, project cycle, and technical difficulty. The collected data undergo meticulous preprocessing to ensure data quality and reliability. The preprocessing steps comprised data cleaning, addressing missing values, handling outliers, culminating in the creation of a self-constructed dataset. The dataset encompasses over 500 SRPs across diverse disciplines and fields, ensuring statistically significant and universal outcomes. Particular emphasis is placed on ensuring dataset diversity, incorporating projects of varying scales, budgets, and team experience levels. This comprehensive coverage ensures the representativeness and credibility of the study on RA in SRPs. New influencing factors are introduced to expand the research scope, including project management quality (such as time management and communication efficiency), historical success rate, industry dynamics, and market demand. Detailed definitions and quantifications are provided for each new variable to facilitate comprehensive data processing and analysis. For project management quality, consideration is given to time management accuracy and communication frequency and quality among team members. Historical success rate is determined by reviewing past project records and outcomes. Industry dynamics are assessed by consulting the latest scientific literature and patent information. Market demand is gauged through market research and user demand surveys. The introduction of these variables enriches the understanding of RA in SRPs and opens up avenues for further research exploration.

At the same time, the collected data are integrated and coded in order to apply Naive Bayes algorithm and MLR analysis. For cases involving qualitative data, this paper uses appropriate coding methods to convert it into quantitative data for processing in the model. For example, for the qualitative feature of team member experience, numerical values are used to represent different experience levels, such as 0 representing beginners, 0 representing intermediate, and 2 representing advanced. The following is a specific sample data set example (Table 1 ). It shows the processed structured data, and the values in the table represent the specific characteristics of each project.

Establishment of naive Bayesian model

The Naive Bayesian algorithm, a probabilistic and statistical classification method renowned for its effectiveness in analyzing and predicting multi-dimensional data, is employed in this paper to conduct the RA for SRPs. The application of the Naive Bayesian algorithm to RA for SRPs aims to discern the influence of various factors on the outcomes of RA. The Naive Bayesian algorithm, depicted in Fig.  1 , operates on the principles of Bayesian theorem, utilizing posterior probability calculations for classification tasks. The fundamental concept of this algorithm hinges on the assumption of independence among different features, embodying the “naivety” hypothesis. In the context of RA for SRPs, the Naive Bayesian algorithm is instrumental in estimating the probability distribution of diverse factors affecting the RA results, thereby enhancing the precision of risk estimates. In the Naive Bayesian model, the initial step involves the computation of posterior probabilities for each factor, considering the given RA result conditions. Subsequently, the category with the highest posterior probability is selected as the predictive outcome.

Naive Bayesian algorithm process.

In Fig.  1 , the data collection process encompasses vital project details such as project scale, budget investment, team member experience, project cycle, technical difficulty, and RA results. This meticulous collection ensures the integrity and precision of the dataset. Subsequently, the gathered data undergoes integration and encoding to convert qualitative data into quantitative form, facilitating model processing and analysis. Tailored to specific requirements, relevant features are chosen for model construction, accompanied by essential preprocessing steps like standardization and normalization. The dataset is then partitioned into training and testing sets, with the model trained on the former and its performance verified on the latter. Leveraging the training data, a Naive Bayesian model is developed to estimate probability distribution parameters for various features across distinct categories. Ultimately, the trained model is employed to predict new project features, yielding RA results.

Naive Bayesian models, in this context, are deployed to forecast diverse project risk levels. Let X symbolize the feature vector, encompassing project scale, budget investment, team member experience, project cycle, and technical difficulty. The objective is to predict the project’s risk level, denoted as Y. Y assumes discrete values representing distinct risk levels. Applying the Bayesian theorem, the posterior probability P(Y|X) is computed, signifying the probability distribution of projects falling into different risk levels given the feature vector X. The fundamental equation governing the Naive Bayesian model is expressed as:

In Eq. ( 1 ), P(Y|X) represents the posterior probability, denoting the likelihood of the project belonging to a specific risk level. P(X|Y) signifies the class conditional probability, portraying the likelihood of the feature vector X occurring under known risk level conditions. P(Y) is the prior probability, reflecting the antecedent likelihood of the project pertaining to a particular risk level. P(X) acts as the evidence factor, encapsulating the likelihood of the feature vector X occurring.

The Naive Bayes, serving as the most elementary Bayesian network classifier, operates under the assumption of attribute independence given the class label c , as expressed in Eq. ( 2 ):

The classification decision formula for Naive Bayes is articulated in Eq. ( 3 ):

The Naive Bayes model, rooted in the assumption of conditional independence among attributes, often encounters deviations from reality. To address this limitation, the Tree-Augmented Naive Bayes (TANB) model extends the independence assumption by incorporating a first-order dependency maximum-weight spanning tree. TANB introduces a tree structure that more comprehensively models relationships between features, easing the constraints of the independence assumption and concurrently mitigating issues associated with multicollinearity. This extension bolsters its efficacy in handling intricate real-world data scenarios. TANB employs conditional mutual information \(I(X_{i} ;X_{j} |C)\) to gauge the dependency between attributes \(X_{j}\) and \(X_{i}\) , thereby constructing the maximum weighted spanning tree. In TANB, any attribute variable \(X_{i}\) is permitted to have at most one other attribute variable as its parent node, expressed as \(Pa\left( {X_{i} } \right) \le 2\) . The joint probability \(P_{con} \left( {x,c} \right)\) undergoes transformation using Eq. ( 4 ):

In Eq. ( 4 ), \(x_{r}\) refers to the root node, which can be expressed as Eq. ( 5 ):

TANB classification decision equation is presented below:

In the RA of SRPs, normal distribution parameters, such as mean (μ) and standard deviation (σ), are estimated for each characteristic dimension (project scale, budget investment, team member experience, project cycle, and technical difficulty). This estimation allows the calculation of posterior probabilities for projects belonging to different risk levels under given feature vector conditions. For each feature dimension \({X}_{i}\) , the mean \({mu}_{i,j}\) and standard deviation \({{\text{sigma}}}_{i,j}\) under each risk level are computed, where i represents the feature dimension, and j denotes the risk level. Parameter estimation employs the maximum likelihood method, and the specific calculations are as follows.

In Eqs. ( 7 ) and ( 8 ), \({N}_{j}\) represents the number of projects belonging to risk level j . \({x}_{i,k}\) denotes the value of the k -th item in the feature dimension i . Finally, under a given feature vector, the posterior probability of a project with risk level j is calculated as Eq. ( 9 ).

In Eq. ( 9 ), d represents the number of feature dimensions, and Z is the normalization factor. \(P(Y=j)\) represents the prior probability of category j . \(P({X}_{i}\mid Y=j)\) represents the normal distribution probability density function of feature dimension i under category j . The risk level of a project can be predicted by calculating the posterior probabilities of different risk levels to achieve RA for SRPs.

This paper integrates the probability estimation of the Naive Bayes model with actual project risk response strategies, enabling a more flexible and targeted response to various risk scenarios. Such integration offers decision support to project managers, enhancing their ability to address potential challenges effectively and ultimately improving the overall success rate of the project. This underscores the notion that risk management is not solely about problem prevention but stands as a pivotal factor contributing to project success.

MLR analysis

MLR analysis is used to validate the hypothesis to deeply explore the impact of various factors on RA of SRPs. Based on the previous research status, the following research hypotheses are proposed.

Hypothesis 1: There is a positive relationship among project scale, budget investment, and team member experience and RA results. As the project scale, budget investment, and team member experience increase, the RA results also increase.

Hypothesis 2: There is a negative relationship between the project cycle and the RA results. Projects with shorter cycles may have higher RA results.

Hypothesis 3: There is a complex relationship between technical difficulty and RA results, which may be positive, negative, or bidirectional in some cases. Based on these hypotheses, an MLR model is established to analyze the impact of factors, such as project scale, budget investment, team member experience, project cycle, and technical difficulty, on RA results. The form of an MLR model is as follows.

In Eq. ( 10 ), Y represents the RA result (dependent variable). \({X}_{1}\) to \({X}_{5}\) represent factors, such as project scale, budget investment, team member experience, project cycle, and technical difficulty (independent variables). \({\beta }_{0}\) to \({\beta }_{5}\) are the regression coefficients, which represent the impact of various factors on the RA results. \(\epsilon\) represents a random error term. The model structure is shown in Fig.  2 .

Schematic diagram of an MLR model.

In Fig.  2 , the MLR model is employed to scrutinize the influence of various independent variables on the outcomes of RA. In this specific context, the independent variables encompass project size, budget investment, team member experience, project cycle, and technical difficulty, all presumed to impact the project’s RA results. Each independent variable is denoted as a node in the model, with arrows depicting the relationships between these factors. In an MLR model, the arrow direction signifies causality, illustrating the influence of an independent variable on the dependent variable.

When conducting MLR analysis, it is necessary to estimate the parameter \(\upbeta\) in the regression model. These parameters determine the relationship between the independent and dependent variables. Here, the Ordinary Least Squares (OLS) method is applied to estimate these parameters. The OLS method is a commonly used parameter estimation method aimed at finding parameter values that minimize the sum of squared residuals between model predictions and actual observations. The steps are as follows. Firstly, based on the general form of an MLR model, it is assumed that there is a linear relationship between the independent and dependent variables. It can be represented by a linear equation, which includes regression coefficients β and the independent variable X. For each observation value, the difference between its predicted and actual values is calculated, which is called the residual. Residual \({e}_{i}\) can be expressed as:

In Eq. ( 11 ), \({Y}_{i}\) is the actual observation value, and \({\widehat{Y}}_{i}\) is the value predicted by the model. The goal of the OLS method is to adjust the regression coefficients \(\upbeta\) to minimize the sum of squared residuals of all observations. This can be achieved by solving an optimization problem, and the objective function is the sum of squared residuals.

Then, the estimated value of the regression coefficient \(\upbeta\) that minimizes the sum of squared residuals can be obtained by taking the derivative of the objective function and making the derivative zero. The estimated values of the parameters can be obtained by solving this system of equations. The final estimated regression coefficient can be expressed as:

In Eq. ( 13 ), X represents the independent variable matrix. Y represents the dependent variable vector. \(({X}^{T}X{)}^{-1}\) is the inverse of a matrix, and \(\widehat{\beta }\) is a parameter estimation vector.

Specifically, solving for the estimated value of regression coefficient \(\upbeta\) requires matrix operation and statistical analysis. Based on the collected project data, substitute it into the model and calculate the residual. Then, the steps of the OLS method are used to obtain parameter estimates. These parameter estimates are used to establish an MLR model to predict RA results and further analyze the influence of different factors.

The degree of influence of different factors on the RA results can be determined by analyzing the value of the regression coefficient β. A positive \(\upbeta\) value indicates that the factor has a positive impact on the RA results, while a negative \(\upbeta\) value indicates that the factor has a negative impact on the RA results. Additionally, hypothesis testing can determine whether each factor is significant in the RA results.

The TANB model proposed in this paper extends the traditional naive Bayes model by incorporating conditional dependencies between attributes to enhance the representation of feature interactions. While the traditional naive Bayes model assumes feature independence, real-world scenarios often involve interdependencies among features. To address this, the TANB model is introduced. The TANB model introduces a tree structure atop the naive Bayes model to more accurately model feature relationships, overcoming the limitation of assuming feature independence. Specifically, the TANB model constructs a maximum weight spanning tree to uncover conditional dependencies between features, thereby enabling the model to better capture feature interactions.

Assessment indicators

To comprehensively assess the efficacy of the proposed TANB model in the RA for SRPs, a self-constructed dataset serves as the data source for this experimental evaluation, as outlined in Table 1 . The dataset is segregated into training (80%) and test sets (20%). These indicators cover the accuracy, precision, recall rate, F1 value, and Area Under the Curve (AUC) Receiver Operating Characteristic (ROC) of the model. The following are the definitions and equations for each assessment indicator. Accuracy is the proportion of correctly predicted samples to the total number of samples. Precision is the proportion of Predicted Positive (PP) samples to actual positive samples. The recall rate is the proportion of correctly PP samples among the actual positive samples. The F1 value is the harmonic average of precision and recall, considering the precision and comprehensiveness of the model. The area under the ROC curve measures the classification performance of the model, and a larger AUC value indicates better model performance. The ROC curve suggests the relationship between True Positive Rate and False Positive Rate under different thresholds. The AUC value can be obtained by accumulating the area of each small rectangle under the ROC curve. The confusion matrix is used to display the prediction property of the model in different categories, including True Positive (TP), True Negative (TN), False Positive (FP), and False Negative (FN).

The performance of TANB in RA for SRPs can be comprehensively assessed to understand the advantages, disadvantages, and applicability of the model more comprehensively by calculating the above assessment indicators.

Results and discussion

Accuracy analysis of naive bayesian algorithm.

On the dataset of this paper, Fig.  3 reveals the performance of TANB algorithm under different assessment indicators.

Performance assessment of TANB algorithm on different projects.

From Fig.  3 , the TANB algorithm performs well in various projects, ranging from 0.87 to 0.911 in accuracy. This means that the overall accuracy of the model in predicting project risks is quite high. The precision also maintains a high level in various projects, ranging from 0.881 to 0.923, indicating that the model performs well in classifying high-risk categories. The recall rate ranges from 0.872 to 0.908, indicating that the model can effectively capture high-risk samples. Meanwhile, the AUC values in each project are relatively high, ranging from 0.905 to 0.931, which once again emphasizes the effectiveness of the model in risk prediction. From multiple assessment indicators, such as accuracy, precision, recall, F1 value, and AUC, the TANB algorithm has shown good risk prediction performance in representative projects. The performance assessment results of the TANB algorithm under different feature dimensions are plotted in Figs.  4 , 5 , 6 and 7 .

Prediction accuracy of TANB algorithm on different budget investments.

Prediction accuracy of TANB algorithm on different team experiences.

Prediction accuracy of TANB algorithm at different risk levels.

Prediction accuracy of TANB algorithm on different project scales.

From Figs.  4 , 5 , 6 and 7 , as the level of budget investment increases, the accuracy of most projects also shows an increasing trend. Especially in cases of high budget investment, the accuracy of the project is generally high. This may mean that a higher budget investment helps to reduce project risks, thereby improving the prediction accuracy of the TANB algorithm. It can be observed that team experience also affects the accuracy of the model. Projects with high team experience exhibit higher accuracy in TANB algorithms. This may indicate that experienced teams can better cope with project risks to improve the performance of the model. When budget investment and team experience are low, accuracy is relatively low. This may imply that budget investment and team experience can complement each other to affect the model performance.

There are certain differences in the accuracy of projects under different risk levels. Generally speaking, the accuracy of high-risk and medium-risk projects is relatively high, while the accuracy of low-risk projects is relatively low. This may be because high-risk and medium-risk projects require more accurate predictions, resulting in higher accuracy. Similarly, project scale also affects the performance of the model. Large-scale and medium-scale projects exhibit high accuracy in TANB algorithms, while small-scale projects have relatively low accuracy. This may be because the risks of large-scale and medium-scale projects are easier to identify and predict to promote the performance of the model. In high-risk and large-scale projects, accuracy is relatively high. This may indicate that the impact of project scale is more significant in specific risk scenarios.

Figure  8 further compares the performance of the TANB algorithm proposed here with other similar algorithms.

Performance comparison of different algorithms in RA of SRPs.

As depicted in Fig.  8 , the TANB algorithm attains an accuracy and precision of 0.912 and 0.920, respectively, surpassing other algorithms. It excels in recall rate and F1 value, registering 0.905 and 0.915, respectively, outperforming alternative algorithms. These findings underscore the proficiency of the TANB algorithm in comprehensively identifying high-risk projects while sustaining high classification accuracy. Moreover, the algorithm achieves an AUC of 0.930, indicative of its exceptional predictive prowess in sample classification. Thus, the TANB algorithm exhibits notable potential for application, particularly in scenarios demanding the recognition and comprehensiveness requisite for high-risk project identification. The evaluation results of the TANB model in predicting project risk levels are presented in Table 2 :

Table 2 demonstrates that the TANB model surpasses the traditional Naive Bayes model across multiple evaluation metrics, including accuracy, precision, and recall. This signifies that, by accounting for feature interdependence, the TANB model can more precisely forecast project risk levels. Furthermore, leveraging the model’s predictive outcomes, project managers can devise tailored risk mitigation strategies corresponding to various risk scenarios. For example, in high-risk projects, more assertive measures can be implemented to address risks, while in low-risk projects, risks can be managed more cautiously. This targeted risk management approach contributes to enhancing project success rates, thereby ensuring the seamless advancement of SRPs.

The exceptional performance of the TANB model in specific scenarios derives from its distinctive characteristics and capabilities. Firstly, compared to traditional Naive Bayes models, the TANB model can better capture the dependencies between attributes. In project RA, project features often exhibit complex interactions. The TANB model introduces first-order dependencies between attributes, allowing features to influence each other, thereby more accurately reflecting real-world situations and improving risk prediction precision. Secondly, the TANB model demonstrates strong adaptability and generalization ability in handling multidimensional data. SRPs typically involve data from multiple dimensions, such as project scale, budget investment, and team experience. The TANB model effectively processes these multidimensional data, extracts key information, and achieves accurate RA for projects. Furthermore, the paper explores the potential of using hybrid models or ensemble learning methods to further enhance model performance. By combining other machine learning algorithms, such as random forests and support vector regressors with sigmoid kernel, through ensemble learning, the shortcomings of individual models in specific scenarios can be overcome, thus improving the accuracy and robustness of RA. For example, in the study, we compared the performance of the TANB model with other algorithms in RA, as shown in Table 3 .

Table 3 illustrates that the TANB model surpasses other models in terms of accuracy, precision, recall, F1 value, and AUC value, further confirming its superiority and practicality in RA. Therefore, the TANB model holds significant application potential in SRPs, offering effective decision support for project managers to better evaluate and manage project risks, thereby enhancing the likelihood of project success.

Analysis of the degree of influence of different factors

Table 4 analyzes the degree of influence and interaction of different factors.

In Table 4 , the regression analysis results reveal that budget investment and team experience exert a significantly positive impact on RA outcomes. This suggests that increasing budget allocation and assembling a team with extensive experience can enhance project RA outcomes. Specifically, the regression coefficient for budget investment is 0.68, and for team experience, it is 0.51, both demonstrating significant positive effects (P < 0.05). The P-values are all significantly less than 0.05, indicating a significant impact. The impact of project scale is relatively small, at 0.31, and its P-value is also much less than 0.05. The degree of interaction influence is as follows. The impact of interaction terms is also significant, especially the interaction between budget investment and team experience and the interaction between budget investment and project scale. The P value of the interaction between budget investment and project scale is 0.002, and the P value of the interaction between team experience and project scale is 0.003. The P value of the interaction among budget investment, team experience, and project scale is 0.005. So, there are complex relationships and interactions among different factors, and budget investment and team experience significantly affect the RA results. However, the budget investment and project scale slightly affect the RA results. Project managers should comprehensively consider the interactive effects of different factors when making decisions to more accurately assess the risks of SRPs.

The interaction between team experience and budget investment

The results of the interaction between team experience and budget investment are demonstrated in Table 5 .

From Table 5 , the degree of interaction impact can be obtained. Budget investment and team experience, along with the interaction between project scale and technical difficulty, are critical factors in risk mitigation. Particularly in scenarios characterized by large project scales and high technical difficulties, adequate budget allocation and a skilled team can substantially reduce project risks. As depicted in Table 5 , under conditions of high team experience and sufficient budget investment, the average RA outcome is 0.895 with a standard deviation of 0.012, significantly lower than assessment outcomes under other conditions. This highlights the synergistic effects of budget investment and team experience in effectively mitigating risks in complex project scenarios. The interaction between team experience and budget investment has a significant impact on RA results. Under high team experience, the impact of different budget investment levels on RA results is not significant, but under medium and low team experience, the impact of different budget investment levels on RA results is significantly different. The joint impact of team experience and budget investment is as follows. Under high team experience, the impact of budget investment is relatively small, possibly because high-level team experience can compensate for the risks brought by insufficient budget to some extent. Under medium and low team experience, the impact of budget investment is more significant, possibly because the lack of team experience makes budget investment play a more important role in RA. Therefore, team experience and budget investment interact in RA of SRPs. They need to be comprehensively considered in project decision-making. High team experience can compensate for the risks brought by insufficient budget to some extent, but in the case of low team experience, the impact of budget investment on RA is more significant. An exhaustive consideration of these factors and their interplay is imperative for effectively assessing the risks inherent in SRPs. Merely focusing on budget allocation or team expertise may not yield a thorough risk evaluation. Project managers must scrutinize the project’s scale, technical complexity, and team proficiency, integrating these aspects with budget allocation and team experience. This holistic approach fosters a more precise RA and facilitates the development of tailored risk management strategies, thereby augmenting the project’s likelihood of success. In conclusion, acknowledging the synergy between budget allocation and team expertise, in conjunction with other pertinent factors, is pivotal in the RA of SRPs. Project managers should adopt a comprehensive outlook to ensure sound decision-making and successful project execution.

Risk mitigation strategies

To enhance the discourse on project risk management in this paper, a dedicated section on risk mitigation strategies has been included. Leveraging the insights gleaned from the predictive model regarding identified risk factors and their corresponding risk levels, targeted risk mitigation measures are proposed.

Primarily, given the significant influence of budget investment and team experience on project RA outcomes, project managers are advised to prioritize these factors and devise pertinent risk management strategies.

For risks stemming from budget constraints, the adoption of flexible budget allocation strategies is advocated. This may involve optimizing project expenditures, establishing financial reserves, or seeking additional funding avenues.

In addressing risks attributed to inadequate team experience, measures such as enhanced training initiatives, engagement of seasoned project advisors, or collaboration with experienced teams can be employed to mitigate the shortfall in expertise.

Furthermore, recognizing the impact of project scale, duration, and technical complexity on RA outcomes, project managers are advised to holistically consider these factors during project planning. This entails adjusting project scale as necessary, establishing realistic project timelines, and conducting thorough assessments of technical challenges prior to project commencement.

These risk mitigation strategies aim to equip project managers with a comprehensive toolkit for effectively identifying, assessing, and mitigating risks inherent in SRPs.

This paper delves into the efficacy of the TANB algorithm in project risk prediction. The findings indicate that the algorithm demonstrates commendable performance across diverse projects, boasting high precision, recall rates, and AUC values, thereby outperforming analogous algorithms. This aligns with the perspectives espoused by Asadullah et al. 37 . Particular emphasis was placed on assessing the impact of variables such as budget investment levels, team experience, and project size on algorithmic performance. Notably, heightened budget investment and extensive team experience positively influenced the results, with project size exerting a comparatively minor impact. Regression analysis elucidates the magnitude and interplay of these factors, underscoring the predominant influence of budget investment and team experience on RA outcomes, whereas project size assumes a relatively marginal role. This underscores the imperative for decision-makers in projects to meticulously consider the interrelationships between these factors for a more precise assessment of project risks, echoing the sentiments expressed by Testorelli et al. 38 .

In sum, this paper furnishes a holistic comprehension of the Naive Bayes algorithm’s application in project risk prediction, offering robust guidance for practical project management. The paper’s tangible applications are chiefly concentrated in the realm of RA and management for SRPs. Such insights empower managers in SRPs to navigate risks with scientific acumen, thereby enhancing project success rates and performance. The paper advocates several strategic measures for SRPs management: prioritizing resource adjustments and team training to elevate the professional skill set of team members in coping with the impact of team experience on risks; implementing project scale management strategies to mitigate potential risks by detailed project stage division and stringent project planning; addressing technical difficulty as a pivotal risk factor through assessment and solution development strategies; incorporating project cycle adjustment and flexibility management to accommodate fluctuations and mitigate associated risks; and ensuring the integration of data quality management strategies to bolster data reliability and enhance model accuracy. These targeted risk responses aim to improve the likelihood of project success and ensure the seamless realization of project objectives.

Achievements

In this paper, the application of Naive Bayesian algorithm in RA of SRPs is deeply explored, and the influence of various factors on RA results and their relationship is comprehensively investigated. The research results fully prove the good accuracy and applicability of Naive Bayesian algorithm in RA of science and technology projects. Through probability estimation, the risk level of the project can be estimated more accurately, which provides a new decision support tool for the project manager. It is found that budget input and team experience are the most significant factors affecting the RA results, and their regression coefficients are 0.68 and 0.51 respectively. However, the influence of project scale on the RA results is relatively small, and its regression coefficient is 0.31. Especially in the case of low team experience, the budget input has a more significant impact on the RA results. However, it should also be admitted that there are some limitations in the paper. First, the case data used is limited and the sample size is relatively small, which may affect the generalization ability of the research results. Second, the factors concerned may not be comprehensive, and other factors that may affect RA, such as market changes and policies and regulations, are not considered.

The paper makes several key contributions. Firstly, it applies the Naive Bayes algorithm to assess the risks associated with SRPs, proposing the TANB and validating its effectiveness empirically. The introduction of the TANB model broadens the application scope of the Naive Bayes algorithm in scientific research risk management, offering novel methodologies for project RA. Secondly, the study delves into the impact of various factors on RA for SRPs through MLR analysis, highlighting the significance of budget investment and team experience. The results underscore the positive influence of budget investment and team experience on RA outcomes, offering valuable insights for project decision-making. Additionally, the paper examines the interaction between team experience and budget investment, revealing a nuanced relationship between the two in RA. This finding underscores the importance of comprehensively considering factors such as team experience and budget investment in project decision-making to achieve more accurate RA. In summary, the paper provides crucial theoretical foundations and empirical analyses for SRPs risk management by investigating RA and its influencing factors in depth. The research findings offer valuable guidance for project decision-making and risk management, bolstering efforts to enhance the success rate and efficiency of SRPs.

This paper distinguishes itself from existing research by conducting an in-depth analysis of the intricate interactions among various factors, offering more nuanced and specific RA outcomes. The primary objective extends beyond problem exploration, aiming to broaden the scope of scientific evaluation and research practice through the application of statistical language. This research goal endows the paper with considerable significance in the realm of science and technology project management. In comparison to traditional methods, this paper scrutinizes project risk with greater granularity, furnishing project managers with more actionable suggestions. The empirical analysis validates the effectiveness of the proposed method, introducing a fresh perspective for decision-making in science and technology projects. Future research endeavors will involve expanding the sample size and accumulating a more extensive dataset of SRPs to enhance the stability and generalizability of results. Furthermore, additional factors such as market demand and technological changes will be incorporated to comprehensively analyze elements influencing the risks of SRPs. Through these endeavors, the aim is to provide more precise and comprehensive decision support to the field of science and technology project management, propelling both research and practice in this domain to new heights.

Limitations and prospects

This paper, while employing advanced methodologies like TANB models, acknowledges inherent limitations that warrant consideration. Firstly, like any model, TANB has its constraints, and predictions in specific scenarios may be subject to these limitations. Subsequent research endeavors should explore alternative advanced machine learning and statistical models to enhance the precision and applicability of RA. Secondly, the focus of this paper predominantly centers on the RA for SRPs. Given the unique characteristics and risk factors prevalent in projects across diverse fields and industries, the generalizability of the paper results may be limited. Future research can broaden the scope of applicability by validating the model across various fields and industries. The robustness and generalizability of the model can be further ascertained through the incorporation of extensive real project data in subsequent research. Furthermore, future studies can delve into additional data preprocessing and feature engineering methods to optimize model performance. In practical applications, the integration of research outcomes into SRPs management systems could provide more intuitive and practical support for project decision-making. These avenues represent valuable directions for refining and expanding the contributions of this research in subsequent studies.

Data availability

All data generated or analysed during this study are included in this published article [and its Supplementary Information files].

Moshtaghian, F., Golabchi, M. & Noorzai, E. A framework to dynamic identification of project risks. Smart and sustain. Built. Environ. 9 (4), 375–393 (2020).

Google Scholar  

Nunes, M. & Abreu, A. Managing open innovation project risks based on a social network analysis perspective. Sustainability 12 (8), 3132 (2020).

Article   Google Scholar  

Elkhatib, M. et al. Agile project management and project risks improvements: Pros and cons. Mod. Econ. 13 (9), 1157–1176 (2022).

Fridgeirsson, T. V. et al. The VUCAlity of projects: A new approach to assess a project risk in a complex world. Sustainability 13 (7), 3808 (2021).

Salahuddin, T. Numerical Techniques in MATLAB: Fundamental to Advanced Concepts (CRC Press, 2023).

Book   Google Scholar  

Awais, M. & Salahuddin, T. Radiative magnetohydrodynamic cross fluid thermophysical model passing on parabola surface with activation energy. Ain Shams Eng. J. 15 (1), 102282 (2024).

Awais, M. & Salahuddin, T. Natural convection with variable fluid properties of couple stress fluid with Cattaneo-Christov model and enthalpy process. Heliyon 9 (8), e18546 (2023).

Article   CAS   PubMed   PubMed Central   Google Scholar  

Guan, L., Abbasi, A. & Ryan, M. J. Analyzing green building project risk interdependencies using Interpretive Structural Modeling. J. Clean. Prod. 256 , 120372 (2020).

Gaudenzi, B. & Qazi, A. Assessing project risks from a supply chain quality management (SCQM) perspective. Int. J. Qual. Reliab. Manag. 38 (4), 908–931 (2021).

Lee, K. T., Park, S. J. & Kim, J. H. Comparative analysis of managers’ perception in overseas construction project risks and cost overrun in actual cases: A perspective of the Republic of Korea. J. Asian Archit. Build. Eng. 22 (4), 2291–2308 (2023).

Garai-Fodor, M., Szemere, T. P. & Csiszárik-Kocsir, Á. Investor segments by perceived project risk and their characteristics based on primary research results. Risks 10 (8), 159 (2022).

Senova, A., Tobisova, A. & Rozenberg, R. New approaches to project risk assessment utilizing the Monte Carlo method. Sustainability 15 (2), 1006 (2023).

Tiwari, P. & Suresha, B. Moderating role of project innovativeness on project flexibility, project risk, project performance, and business success in financial services. Glob. J. Flex. Syst. Manag. 22 (3), 179–196 (2021).

de Araújo, F., Lima, P., Marcelino-Sadaba, S. & Verbano, C. Successful implementation of project risk management in small and medium enterprises: A cross-case analysis. Int. J. Manag. Proj. Bus. 14 (4), 1023–1045 (2021).

Obondi, K. The utilization of project risk monitoring and control practices and their relationship with project success in construction projects. J. Proj. Manag. 7 (1), 35–52 (2022).

Atasoy, G. et al. Empowering risk communication: Use of visualizations to describe project risks. J. Constr. Eng. Manage. 148 (5), 04022015 (2022).

Dandage, R. V., Rane, S. B. & Mantha, S. S. Modelling human resource dimension of international project risk management. J. Global Oper. Strateg. Sourcing 14 (2), 261–290 (2021).

Wang, L. et al. Applying social network analysis to genetic algorithm in optimizing project risk response decisions. Inf. Sci. 512 , 1024–1042 (2020).

Marx-Stoelting, P. et al. A walk in the PARC: developing and implementing 21st century chemical risk assessment in Europe. Arch. Toxicol. 97 (3), 893–908 (2023).

Awais, M., Salahuddin, T. & Muhammad, S. Evaluating the thermo-physical characteristics of non-Newtonian Casson fluid with enthalpy change. Thermal Sci. Eng. Prog. 42 , 101948 (2023).

Article   CAS   Google Scholar  

Awais, M., Salahuddin, T. & Muhammad, S. Effects of viscous dissipation and activation energy for the MHD Eyring-Powell fluid flow with Darcy-Forchheimer and variable fluid properties. Ain Shams Eng. J. 15 (2), 102422 (2024).

Yang, L., Lou, J. & Zhao, X. Risk response of complex projects: Risk association network method. J. Manage. Eng. 37 (4), 05021004 (2021).

Acebes, F. et al. Project risk management from the bottom-up: Activity Risk Index. Cent. Eur. J. Oper. Res. 29 (4), 1375–1396 (2021).

Siyal, S. et al. They can’t treat you well under abusive supervision: Investigating the impact of job satisfaction and extrinsic motivation on healthcare employees. Rationality Society 33 (4), 401–423 (2021).

Chen, D., Wawrzynski, P. & Lv, Z. Cyber security in smart cities: A review of deep learning-based applications and case studies. Sustain. Cities Soc. 66 , 102655 (2021).

Zhao, M. et al. Pythagorean fuzzy TODIM method based on the cumulative prospect theory for MAGDM and its application on risk assessment of science and technology projects. Int. J. Fuzzy Syst. 23 , 1027–1041 (2021).

Suresh, K. & Dillibabu, R. A novel fuzzy mechanism for risk assessment in software projects. Soft Comput. 24 , 1683–1705 (2020).

Akhavan, M., Sebt, M. V. & Ameli, M. Risk assessment modeling for knowledge based and startup projects based on feasibility studies: A Bayesian network approach. Knowl.-Based Syst. 222 , 106992 (2021).

Guan, L., Abbasi, A. & Ryan, M. J. A simulation-based risk interdependency network model for project risk assessment. Decis. Support Syst. 148 , 113602 (2021).

Vujović, V. et al. Project planning and risk management as a success factor for IT projects in agricultural schools in Serbia. Technol. Soc. 63 , 101371 (2020).

Muñoz-La Rivera, F., Mora-Serrano, J. & Oñate, E. Factors influencing safety on construction projects (FSCPs): Types and categories. Int. J. Environ. Res. Public Health 18 (20), 10884 (2021).

Article   PubMed   PubMed Central   Google Scholar  

Nguyen, P. T. & Nguyen, P. C. Risk management in engineering and construction: A case study in design-build projects in Vietnam. Eng. Technol. Appl. Sci. Res 10 , 5237–5241 (2020).

Nguyen PT, Le TT. Risks on quality of civil engineering projects-an additive probability formula approach//AIP Conference Proceedings. AIP Publishing, 2798(1) (2023).

Nguyen, P.T., Phu, P.C., Thanh, P.P., et al . Exploring critical risk factors of office building projects. 8 (2), 0309–0315 (2020).

Nguyen, H. D. & Macchion, L. Risk management in green building: A review of the current state of research and future directions. Environ. Develop. Sustain. 25 (3), 2136–2172 (2023).

He, S. et al. Risk assessment of oil and gas pipelines hot work based on AHP-FCE. Petroleum 9 (1), 94–100 (2023).

Asadullah, M. et al. Evaluation of machine learning techniques for hypertension risk prediction based on medical data in Bangladesh. Indones. J. Electr. Eng. Comput. Sci. 31 (3), 1794–1802 (2023).

Testorelli, R., de Araujo, F., Lima, P. & Verbano, C. Fostering project risk management in SMEs: An emergent framework from a literature review. Prod. Plan. Control 33 (13), 1304–1318 (2022).

Download references

Author information

Authors and affiliations.

Institute of Policy Studies, Lingnan University, Tuen Mun, 999077, Hong Kong, China

Xuying Dong & Wanlin Qiu

You can also search for this author in PubMed   Google Scholar

Contributions

Xuying Dong and Wanlin Qiu played a key role in the writing of Risk Assessment of Scientific Research Projects and the Relationship between Related Factors Based on Naive Bayes Algorithm. First, they jointly developed clearly defined research questions and methods for risk assessment using the naive Bayes algorithm at the beginning of the research project. Secondly, Xuying Dong and Wanlin Qiu were responsible for data collection and preparation, respectively, to ensure the quality and accuracy of the data used in the research. They worked together to develop a naive Bayes algorithm model, gain a deep understanding of the algorithm, ensure the effectiveness and performance of the model, and successfully apply the model in practical research. In the experimental and data analysis phase, the collaborative work of Xuying Dong and Wanlin Qiu played a key role in verifying the validity of the model and accurately assessing the risks of the research project. They also collaborated on research papers, including detailed descriptions of methods, experiments and results, and actively participated in the review and revision process, ensuring the accuracy and completeness of the findings. In general, the joint contribution of Xuying Dong and Wanlin Qiu has provided a solid foundation for the success of this research and the publication of high-quality papers, promoted the research on the risk assessment of scientific research projects and the relationship between related factors, and made a positive contribution to the progress of the field.

Corresponding author

Correspondence to Wanlin Qiu .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary information., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Dong, X., Qiu, W. A case study on the relationship between risk assessment of scientific research projects and related factors under the Naive Bayesian algorithm. Sci Rep 14 , 8244 (2024). https://doi.org/10.1038/s41598-024-58341-y

Download citation

Received : 30 October 2023

Accepted : 27 March 2024

Published : 08 April 2024

DOI : https://doi.org/10.1038/s41598-024-58341-y

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Naive Bayesian algorithm
• Scientific research projects
• Risk assessment
• Factor analysis
• Probability estimation
• Decision support
• Data-driven decision-making

By submitting a comment you agree to abide by our Terms and Community Guidelines . If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing: AI and Robotics newsletter — what matters in AI and robotics research, free to your inbox weekly.

• Open access
• Published: 09 December 2020

Data of risk analysis management in university campuses

• Alireza Dehdashti 1 ,
• Farin Fatemi 1 ,
• Muhammadreza Janati 2 ,
• Fatemeh Asadi 2 &
• Marzieh Belji Kangarloo 2  

BMC Research Notes volume  13 , Article number:  554 ( 2020 ) Cite this article

3632 Accesses

2 Citations

Metrics details

This data paper aims to provide the data set of a practical method to health, safety, and environmental risk assessment to assess and rank potential threats/hazards and to prevent and decrease the accidents and harmful consequences at an academic setting. Descriptive data on type of hazards, places, and persons at risk were collected. Quantitative data on risk probability and severity of identified hazards were determined. Additionally, the descriptive statistics and analytical tests were applied to create a concise perspective on health, safety and environmental hazards/threats situation in research location under study. The dataset further provides information on the prioritization of determined risks according to the relevant scores and levels for doing the relevant control measures to remove and mitigate the related risks.

Data description

This paper provides data of comprehensive risk assessment of health, safety and, environmental hazards of academic setting. For each identified hazard, the descriptive and numeric data are available. The information about the risk level and prevention or mitigation measure related to each hazard is provided. Additionally, the statistical tests are applied for determining the relations among the variables under study. The data and methodology on risk assessment in this article may be used to manage variety of risks in higher education institutions.

Risk assessment is the process of evaluating risks to persons’ safety and health from workplace hazards [ 1 ]. This data note aims to provide comprehensive information about the Health, Safety and Environment (HSE) risk assessment. Such these data sets are necessary for accident prevention and decrease the harmful impacts such as death, injuries and damages to structures and equipment at academic settings [ 2 , 3 ]. Also, such these risk assessment results promote information sharing across the university systems about best practices to mitigate the vulnerability of hazards [ 4 ]. Corresponding mitigation measures for each campus’s highest risks is beneficial to create a University-wide relative risk ranking of all threat events and to summarize the status of campus mitigation measures [ 5 ]. Taking steps to either eliminate or to reduce risks (as far as reasonably practicable) by introducing control measures should be done in the risk management phase [ 6 ]. To achieve these goals, we developed two checklists to collect descriptive and quantitative data based on the evidence review [ 7 , 8 ]. The laboratories, public offices, and peripheral areas of the academic setting were included in this study. Besides, we planned to determine the hazard mitigation strategies for each identified hazard in the assessed areas of the academic setting in the applied checklist [ 9 ]. In the current paper, we described the collected data during the HSE risk assessment in this study. A research paper was written up and published based on our data and methodology to further describe and prioritize risks in terms of varying risk impacts in the university environment [ 10 ]. These data may help researchers to assess probability, impact, and mitigation risks in detail and find a proactive approach toward risks in higher education institutions.

This cross-sectional study was conducted from June to July 2018 at the School of Public Health, Semnan, University of Medical Sciences, Semnan, Iran. For implementing a successful HSE risk assessment, we needed to identify the potential hazards into three individual domains of health, safety, and environment in different areas of the academic setting. First, a list of hazards in three sections of health, safety, and environment was developed in a checklist. The reviewed literature on concerning recent accidents or incidents within universities was much helpful in defining the list of HSE hazards. The hazards checklist was completed via on-site walking and interviews by three trained students in the 39 locations of the academic setting. Additionally, the persons at risk including students, staff, or faculties were determined for each hazard in the assessed location. The descriptive analysis indicated that the most frequency of hazards was related to health hazards (50.3%). The safety and environmental hazards were 44%, and 5.7%, respectively.

Second, the research team analyzed the risks based on the risk matrix in ISO 31000 [ 8 ] (see Table  1 ). Risk analysis was done by estimating the probability and severity of risks for identified hazards. The probability of occurrence metrics was measured on a five-point scale from “not applicable” to “inevitable”. The interpretation of each scale has been mentioned in data file 1 [ 11 ]. The novelty of this study is determining the severity rate in terms of human, equipment, and institution. To estimate the effect of each identified hazard on three mentioned terms, we applied two items with a five-point score response [ 12 ]. Responses were scored and averaged to obtain an overall severity score. The severity metrics were measured on a five-point scale from “very low” to “very high”. The final risk scores were calculated from multiplying probability by severity and categorized into 3 risk levels, risks rated in the top category (red) are larger than those rated in moderate category (yellow) and low category (green).

Based on the results, hazards with high risk level were belonged to safety category that they were included 5.7%, required immediate mitigation measures. From all identified health, safety and environmental hazards, 6.5%, 36% and 43% were categorized in moderate risk level, respectively. These hazards need to be corrected in the near future. The rest of identified HSE hazards had acceptable risk score and categorized in low risk hazards. Therefore, the mitigation measures were recommended in appropriate to determined risk levels for identified hazards.

In data file 2, in order to create a more detailed view on health, safety and environmental hazards/threats situation, the analytical tests were applied to examine associations among the main variables under study [ 13 ]. Type of analytical tests (Chi square or Kruskal–Wallis)was used based on type of variable that is qualitative or quantitative. The most important finding is related to significant relation between the calculated risk scores and the type of hazards. The safety-related hazards indicated a statistically higher contribution to the total risk score when compared to health and environmental hazards. The descriptive analysis and providing analytical information helps to withstand and cope with the adverse effects of accidents and emergencies [ 14 , 15 ].

Limitations

Although risk matrices can indeed be very useful in risk analysis, but can mistakenly assign higher qualitative ratings to smaller risks that leading to wrong risk management decision. It means that we might have quantitatively overestimated or underestimated the risk scores for identified hazards. Probably, applying new approaches to risk matrix such as new scaling and scoring methods to risk matrix extensions depends on the location of risk assessment in future studies would minimize the mentioned concern.

The data of this study were collected in a typical higher education institution with its exclusive hazards, which may limit its interpretation of safety, health, and environmental risks. However, the authors believe that an integrated approach for obtaining data will be beneficial to compare various domains of hazards in a workplace.

Availability of data and materials

The data described in this Data note can be freely and openly accessed via http://figshare.com . Please see Table  1 and Refs. [ 11 , 13 ] for details and links to the data.

Abbreviations

Health, safety, environment

Sarker MN, Yang B, Lv Y, Huq ME, Kamruzzaman MM. Climate change adaptation and resilience through big data. Int J Adv Comput Sci Appl. 2020;11:533–9. https://doi.org/10.14569/IJACSA.2020.0110368 .

Article   Google Scholar  

Farrokhi M, Khankeh H, Poursadeqiyan M. The necessity to establish health, safety and environment management major at the University of Social Welfare and Rehabilitation Sciences. Health Emerg Disasters Quarterly. 2019;4(3):119–26. http://hdq.uswr.ac.ir/article-1-205-en.html .

Salazar-Escoboza MA, Laborin-Alvarez JF, Alvarez-Chavez CR, Noriega-Orozco L, Borbon-Morales C. Safety climate perceived by users of academic laboratories in higher education institutes. Saf Sci. 2020;121:93–9. http://www.sciencedirect.com/science/article/pii/S0925753519306198 .

Skurka C, Quick BL, Reynolds-Tylus T, Short T, Bryan AL. An evaluation of a college campus emergency preparedness intervention. J Saf Res. 2018;65(67–72):000434745100008.

Google Scholar  

Yang N, Wu YM. University public emergency plan formulation: Current situation and countermeasures. Jun H, Hong C, Jianbin J, editors 2010. 40–3 p.000289654800010.

Marincioni F, Fraboni R. A baseline assessment of emergency planning and preparedness in Italian universities. Disasters. 2012;36(2):291–315. https://doi.org/10.1111/j.1467-7717.2011.01259.x .

Article   PubMed   Google Scholar  

Example risk assessments United Kingdom: Aberystwyth University; 2018. https://www.aber.ac.uk/en/hse/proc-prac/risk-assessment/#template-and-guidance .

Risk management framework. Second ed: Goverment of South Australia; 2012. https://dhs.sa.gov.au/data/assets/pdf_file/0008/9782/risk-management-framework.pdf .

Ilo PI, Izuagbe R, Mole AJC, Ekwueme L. Measuring disaster preparedness and response practices in university libraries in Nigeria: the role of disaster equipment. Int J Disaster Risk Reduct. 2018;31:85–91.

Dehdashti A, Fatemi F, Jannati M, Asadi F, Kangarloo MB. Applying health, safety, and environmental risk assessment at academic settings. BMC Public Health; 2020. https://bmcpublichealth.biomedcentral.com/articles/10.1186/s12889-020-09419-5 .

Fatemi F. Risk assessment data file. figshare. Dataset. 2020. https://doi.org/10.6084/m9figshare.13106825.v1 .

Nouri J, Mansouri N, Abbaspour M, Karbasi AR, Omidvari M. Designing a developed model for assessing the disaster induced vulnerability value in educational centers. Saf Sci. 2011;49(5):679–85. https://doi.org/10.1016/j.ssci.2011.01.002 .

Fatemi F. Analytical data. figshare. Dataset. 2020. https://doi.org/10.6084/m9.figshare.13106843.v1 .

Sarker MNI, Peng Y, Yiran C, Shouse RC. Disaster resilience through big data: Way to environmental sustainability. Int J Disaster Risk Reduct. 2020;51 March:101769. https://doi.org/10.1016/j.ijdrr.2020.101769 .

Tajdinan S, Afshari D. Checking of human errors in Ancoiler Device Control Room of Ahvaz Pipe Mill using SHERPA and HET methods at year 1390. Iran Occup Health. 2013;10(3). http://ioh.iums.ac.ir/article-1-771-en.html .

Download references

Acknowledgements

We thank all colleagues and staff of the School of Public Health who supported and assisted for conducting this research.

There is no funding source for this research.

Author information

Authors and affiliations.

Research Center for Health Sciences and Technologies, Semnan University of Medical Sciences, Semnan, Iran

Alireza Dehdashti & Farin Fatemi

Student Research Committee, Semnan University of Medical Siences, Semnan, Iran

Muhammadreza Janati, Fatemeh Asadi & Marzieh Belji Kangarloo

You can also search for this author in PubMed   Google Scholar

Contributions

MJ, FA and MB collected data and contributed to entering data into dataset. FF and AD designed study, analyzed data, and prepared the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Farin Fatemi .

Ethics declarations

Ethics approval and consent to participate.

This study was approved by the Ethics Committee Review Board at Semnan university of Medical Sciences (IR.SEMUMS.REC.1398.131). All the participants signed a consent form and were informed on the purpose of the study prior to interview as per local protocol on research ethics.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Dehdashti, A., Fatemi, F., Janati, M. et al. Data of risk analysis management in university campuses. BMC Res Notes 13 , 554 (2020). https://doi.org/10.1186/s13104-020-05397-4

Download citation

Received : 17 October 2020

Accepted : 24 November 2020

Published : 09 December 2020

DOI : https://doi.org/10.1186/s13104-020-05397-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Risk assessment
• Academic settings

BMC Research Notes

ISSN: 1756-0500

• Submission enquiries: [email protected]
• General enquiries: [email protected]

Information

• Author Services

Initiatives

You are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.

All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess .

Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.

Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.

Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.

Original Submission Date Received: .

• Active Journals
• Find a Journal
• Proceedings Series
• For Authors
• For Reviewers
• For Editors
• For Librarians
• For Publishers
• For Societies
• For Conference Organizers
• Open Access Policy
• Institutional Open Access Program
• Special Issues Guidelines
• Editorial Process
• Research and Publication Ethics
• Article Processing Charges
• Testimonials
• Preprints.org
• SciProfiles
• Encyclopedia

Article Menu

• Subscribe SciFeed
• Recommended Articles
• Google Scholar
• on Google Scholar
• Table of Contents

Find support for a specific problem in the support section of our website.

Please let us know what you think of our products and services.

Visit our dedicated information section to learn more about MDPI.

JSmol Viewer

Future horizons: the potential role of artificial intelligence in cardiology.

1. Introduction

1.1. terminology, 1.1.1. machine learning (ml), 1.1.2. deep learning (dl), 1.1.3. artificial neural networks (anns), 1.1.4. convolutional neural networks (cnns), 2. materials and methods, 3.1. electrocardiography (ecg), 3.1.1. echocardiography.

• Assessing and monitoring the left ventricular systolic and diastolic function;
• Evaluation of the right ventricular function;
• Evaluation and quantification of the cardiac chamber size;
• Assessing the functional significance of a valvular lesion and the evaluation of the prosthetic valve structure and function;
• Identification of the cardiac source of embolism and the evaluation of the cardiac masses;
• Evaluation of pericardial diseases [ 58 ].

3.1.2. Coronary Angiography

3.1.3. cardiac computed angiography, 3.1.4. computed tomography, 3.1.5. cardiac mri, 4. discussions, 4.1. challenges in ai implementation, 4.2. limitations, 5. conclusions, author contributions, institutional review board statement, informed consent statement, data availability statement, conflicts of interest.

• Flora, G.D.; Nayak, M.K. A Brief Review of Cardiovascular Diseases, Associated Risk Factors and Current Treatment Regimes. Curr. Pharm. Des. 2019 , 25 , 4063–4084. [ Google Scholar ] [ CrossRef ]
• Roth, G.A.; Mensah, G.A.; Johnson, C.O.; Addolorato, G.; Ammirati, E.; Baddour, L.M.; Barengo, N.C.; Beaton, A.Z.; Benjamin, E.J.; Benziger, C.P.; et al. Global Burden of Cardiovascular Diseases and Risk Factors, 1990–2019: Update from the GBD 2019 Study. J. Am. Coll. Cardiol. 2020 , 76 , 2982–3021. [ Google Scholar ] [ CrossRef ]
• Haq, I.U.; Chhatwal, K.; Sanaka, K.; Xu, B. Artificial Intelligence in Cardiovascular Medicine: Current Insights and Future Prospects. Vasc. Health Risk Manag. 2022 , 18 , 517. [ Google Scholar ] [ CrossRef ]
• Amisha; Malik, P.; Pathania, M.; Rathaur, V.K. Overview of artificial intelligence in medicine. J. Fam. Med. Prim. Care 2019 , 8 , 2328. [ Google Scholar ] [ CrossRef ]
• Hamet, P.; Tremblay, J. Artificial intelligence in medicine. Metabolism 2017 , 69S , S36–S40. [ Google Scholar ] [ CrossRef ]
• Poalelungi, D.G.; Musat, C.L.; Fulga, A.; Neagu, M.; Neagu, A.I.; Piraianu, A.I.; Fulga, I. Advancing Patient Care: How Artificial Intelligence Is Transforming Healthcare. J. Pers. Med. 2023 , 13 , 1214. [ Google Scholar ] [ CrossRef ]
• Xu, N.; Yang, D.; Arikawa, K.; Bai, C. Application of artificial intelligence in modern medicine. Clin. eHealth 2023 , 6 , 130–137. [ Google Scholar ] [ CrossRef ]
• Fogel, A.L.; Kvedar, J.C. Artificial intelligence powers digital medicine. NPJ Digit. Med. 2018 , 1 , 5. [ Google Scholar ] [ CrossRef ]
• Zhang, J.; Zhang, J.; Jin, J.; Jiang, X.; Yang, L.; Fan, S.; Zhang, Q.; Chi, M. Artificial intelligence applied in cardiovascular disease: A bibliometric and visual analysis. Front. Cardiovasc. Med. 2024 , 11 , 1323918. [ Google Scholar ] [ CrossRef ]
• Noorbakhsh-Sabet, N.; Zand, R.; Zhang, Y.; Abedi, V. Artificial Intelligence Transforms the Future of Healthcare. Am. J. Med. 2019 , 132 , 795. [ Google Scholar ] [ CrossRef ]
• Bostrom, N. How Long Before Superintelligence? Int. J. Future Stud. 1998 , 2 , 11–30. [ Google Scholar ]
• Sun, K.; Roy, A.; Tobin, J.M. Artificial intelligence and machine learning: Definition of terms and current concepts in critical care research. J. Crit. Care 2024 , 82 , 154792. [ Google Scholar ] [ CrossRef ]
• Chakraborty, C.; Bhattacharya, M.; Pal, S.; Lee, S.-S. From machine learning to deep learning: Advances of the recent data-driven paradigm shift in medicine and healthcare. Curr. Res. Biotechnol. 2024 , 7 , 100164. [ Google Scholar ] [ CrossRef ]
• Botvinick, M.; Ritter, S.; Wang, J.X.; Kurth-Nelson, Z.; Blundell, C.; Hassabis, D. Reinforcement Learning, Fast and Slow. Trends Cogn. Sci. 2019 , 23 , 408–422. [ Google Scholar ] [ CrossRef ]
• Jakhar, D.; Kaur, I. Artificial intelligence, machine learning and deep learning: Definitions and differences. Clin. Exp. Dermatol. 2020 , 45 , 131–132. [ Google Scholar ] [ CrossRef ]
• Piccialli, F.; Di Somma, V.; Giampaolo, F.; Cuomo, S.; Fortino, G. A survey on deep learning in medicine: Why, how and when? Inf. Fusion. 2021 , 66 , 111–137. [ Google Scholar ] [ CrossRef ]
• Dongare, A.D.; Kharde, R.R.; Kachare, A.D. Introduction to Artificial Neural Network. Certif. Int. J. Eng. Innov. Technol. (IJEIT) 2008 , 9001 , 2277–3754. [ Google Scholar ]
• Micheli-Tzanakou, E. Artificial Neural Networks: Definitions, Methods, Applications. In Supervised and Unsupervised Pattern Recognition ; CRC Press: Boca Raton, FL, USA, 2017; pp. 61–78. [ Google Scholar ]
• Zhou, S.K.; Rueckert, D.; Fichtinger, G. (Eds.) Handbook of Medical Image Computing and Computer Assisted Intervention ; Academic Press: Cambridge, MA, USA, 2020. [ Google Scholar ]
• Berezsky, O.; Liashchynskyi, P.; Pitsun, O.; Izonin, I. Synthesis of Convolutional Neural Network architectures for biomedical image classification. Biomed. Signal Process Control 2024 , 95 , 106325. [ Google Scholar ] [ CrossRef ]
• Piraianu, A.I.; Fulga, A.; Musat, C.L.; Ciobotaru, O.R.; Poalelungi, D.G.; Stamate, E.; Ciobotaru, O.; Fulga, I. Enhancing the Evidence with Algorithms: How Artificial Intelligence Is Transforming Forensic Medicine. Diagnostics 2023 , 13 , 2992. [ Google Scholar ] [ CrossRef ]
• Herman, R.; Meyers, H.P.; Smith, S.W.; Bertolone, D.T.; Leone, A.; Bermpeis, K.; Viscusi, M.M.; Belmonte, M.; Demolder, A.; Boza, V.; et al. International evaluation of an artificial intelligence-powered electrocardiogram model detecting acute coronary occlusion myocardial infarction. Eur. Heart J. Digit. Health 2023 , 5 , 123–133. [ Google Scholar ] [ CrossRef ]
• Nogimori, Y.; Sato, K.; Takamizawa, K.; Ogawa, Y.; Tanaka, Y.; Shiraga, K.; Masuda, H.; Matsui, H.; Kato, M.; Daimon, M.; et al. Prediction of adverse cardiovascular events in children using artificial intelligence-based electrocardiogram. Int. J. Cardiol. 2024 , 406 , 132019. [ Google Scholar ] [ CrossRef ]
• Hillis, J.; Bizzo, B.C.; Mercaldo, S.; Ghatak, A.; Macdonald, A.; Halle, M.; Schultz, A.; L’Italien, E.; Tam, V.; Awad, A.; et al. Detection of Hypertrophic Cardiomyopathy on Electrocardiogram Using Artificial Intelligence. J. Am. Coll. Cardiol. 2024 , 83 , 2609. [ Google Scholar ] [ CrossRef ]
• Haimovich, J.S.; Diamant, N.; Khurshid, S.; Di Achille, P.; Reeder, C.; Friedman, S.; Singh, P.; Spurlock, W.; Ellinor, P.T.; Philippakis, A.; et al. Artificial intelligence–enabled classification of hypertrophic heart diseases using electrocardiograms. Cardiovasc. Digit. Health J. 2023 , 4 , 48–59. [ Google Scholar ] [ CrossRef ]
• Harmon, D.M.; Mangold, K.; Baez Suarez, A.; Scott, C.G.; Murphree, D.H.; Malik, A.; Attia, Z.I.; Lopez-Jimenez, F.; Friedman, P.A.; Dispenzieri, A.; et al. Postdevelopment Performance and Validation of the Artificial Intelligence-Enhanced Electrocardiogram for Detection of Cardiac Amyloidosis. JACC Adv. 2023 , 2 , 100612. [ Google Scholar ] [ CrossRef ]
• Butler, L.; Karabayir, I.; Kitzman, D.W.; Alonso, A.; Tison, G.H.; Chen, L.Y.; Chang, P.P.; Clifford, G.; Soliman, E.Z.; Akbilgic, O. A generalizable electrocardiogram-based artificial intelligence model for 10-year heart failure risk prediction. Cardiovasc. Digit. Health J. 2023 , 4 , 183–190. [ Google Scholar ] [ CrossRef ]
• Awasthi, S.; Sachdeva, N.; Gupta, Y.; Anto, A.G.; Asfahan, S.; Abbou, R.; Bade, S.; Sood, S.; Hegstrom, L.; Vellanki, N.; et al. Identification and risk stratification of coronary disease by artificial intelligence-enabled ECG. eClinicalMedicine 2023 , 65 , 102259. [ Google Scholar ] [ CrossRef ]
• Lee, Y.H.; Hsieh, M.T.; Chang, C.C.; Tsai, Y.L.; Chou, R.H.; Lu, H.H.S.; Huang, P.-H. Improving detection of obstructive coronary artery disease with an artificial intelligence-enabled electrocardiogram algorithm. Atherosclerosis 2023 , 381 , 117238. [ Google Scholar ] [ CrossRef ]
• Valente Silva, B.; Marques, J.; Nobre Menezes, M.; Oliveira, A.L.; Pinto, F.J. Artificial intelligence-based diagnosis of acute pulmonary embolism: Development of a machine learning model using 12-lead electrocardiogram. Rev. Port. Cardiol. 2023 , 42 , 643–651. [ Google Scholar ] [ CrossRef ]
• Sau, A.; Ibrahim, S.; Kramer, D.B.; Waks, J.W.; Qureshi, N.; Koa-Wing, M.; Keene, D.; Malcolme-Lawes, L.; Lefroy, D.C.; Linton, N.W.; et al. Artificial intelligence–enabled electrocardiogram to distinguish atrioventricular re-entrant tachycardia from atrioventricular nodal re-entrant tachycardia. Cardiovasc. Digit. Health J. 2023 , 4 , 60–67. [ Google Scholar ] [ CrossRef ]
• Shimojo, M.; Inden, Y.; Yanagisawa, S.; Suzuki, N.; Tsurumi, N.; Watanabe, R.; Nakagomi, T.; Okajima, T.; Suga, K.; Tsuji, Y.; et al. A novel practical algorithm using machine learning to differentiate outflow tract ventricular arrhythmia origins. J. Cardiovasc. Electrophysiol. 2023 , 34 , 627–637. [ Google Scholar ] [ CrossRef ]
• Shiokawa, N.; Izumo, M.; Shimamura, T.; Kurosaka, Y.; Sato, Y.; Okamura, T.; Akashi, Y.J. Accuracy and Efficacy of Artificial Intelligence-Derived Automatic Measurements of Transthoracic Echocardiography in Routine Clinical Practice. J. Clin. Med. 2024 , 13 , 1861. [ Google Scholar ] [ CrossRef ]
• Sveric, K.M.; Ulbrich, S.; Dindane, Z.; Winkler, A.; Botan, R.; Mierke, J.; Trausch, A.; Heidrich, F.; Linke, A. Improved assessment of left ventricular ejection fraction using artificial intelligence in echocardiography: A comparative analysis with cardiac magnetic resonance imaging. Int. J. Cardiol. 2024 , 394 , 131383. [ Google Scholar ] [ CrossRef ]
• Slivnick, J.A.; Gessert, N.T.; Cotella, J.I.; Oliveira, L.; Pezzotti, N.; Eslami, P.; Sadeghi, A.; Wehle, S.; Prabhu, D.; Waechter-Stehle, I.; et al. Echocardiographic Detection of Regional Wall Motion Abnormalities Using Artificial Intelligence Compared to Human Readers. J. Am. Soc. Echocardiogr. 2024. [ CrossRef ]
• Kampaktsis, P.N.; Bohoran, T.A.; Lebehn, M.; McLaughlin, L.; Leb, J.; Liu, Z.; Moustakidis, S.; Siouras, A.; Singh, A.; Hahn, R.T.; et al. An attention-based deep learning method for right ventricular quantification using 2D echocardiography: Feasibility and accuracy. Echocardiography 2024 , 41 , e15719. [ Google Scholar ] [ CrossRef ]
• Murayama, M.; Sugimori, H.; Yoshimura, T.; Kaga, S.; Shima, H.; Tsuneta, S.; Mukai, A.; Nagai, Y.; Yokoyama, S.; Nishino, H.; et al. Deep learning to assess right ventricular ejection fraction from two-dimensional echocardiograms in precapillary pulmonary hypertension. Echocardiography 2024 , 41 , e15812. [ Google Scholar ] [ CrossRef ]
• Hsia, B.C.; Lai, A.; Singh, S.; Samtani, R.; Bienstock, S.; Liao, S.; Stern, E.; LaRocca, G.; Sanz, J.; Lerakis, S.; et al. Validation of American Society of Echocardiography Guideline-Recommended Parameters of Right Ventricular Dysfunction Using Artificial Intelligence Compared with Cardiac Magnetic Resonance Imaging. J. Am. Soc. Echocardiogr. 2023 , 36 , 967–977. [ Google Scholar ] [ CrossRef ]
• Anand, V.; Weston, A.D.; Scott, C.G.; Kane, G.C.; Pellikka, P.A.; Carter, R.E. Machine Learning for Diagnosis of Pulmonary Hypertension by Echocardiography. Mayo Clin. Proc. 2024 , 99 , 260–270. [ Google Scholar ] [ CrossRef ]
• Oikonomou, E.K.; Holste, G.; Yuan, N.; Coppi, A.; McNamara, R.L.; Haynes, N.A.; Vora, A.N.; Velazquez, E.J.; Li, F.; Menon, V.; et al. A Multimodal Video-Based AI Biomarker for Aortic Stenosis Development and Progression. JAMA Cardiol. 2024 , 9 , 534–544. [ Google Scholar ] [ CrossRef ]
• Krishna, H.; Desai, K.; Slostad, B.; Bhayani, S.; Arnold, J.H.; Ouwerkerk, W.; Hummel, Y.; Lam, C.S.; Ezekowitz, J.; Frost, M.; et al. Fully Automated Artificial Intelligence Assessment of Aortic Stenosis by Echocardiography. J. Am. Soc. Echocardiogr. 2023 , 36 , 769–777. [ Google Scholar ] [ CrossRef ]
• Guo, Y.; Xia, C.; Zhong, Y.; Wei, Y.; Zhu, H.; Ma, J.; Li, G.; Meng, X.; Yang, C.; Wang, X.; et al. Machine learning-enhanced echocardiography for screening coronary artery disease. Biomed. Eng. Online 2023 , 22 , 44. [ Google Scholar ] [ CrossRef ]
• Molenaar, M.A.; Bouma, B.J.; Asselbergs, F.W.; Verouden, N.J.; Selder, J.L.; Chamuleau, S.A.J.; Schuuring, M.J. Explainable machine learning using echocardiography to improve risk prediction in patients with chronic coronary syndrome. Eur. Heart J. Digit. Health 2024 , 5 , 170–182. [ Google Scholar ] [ CrossRef ]
• Lu, N.; Vaseli, H.; Mahdavi, M.; Taheri Dezaki, F.; Luong, C.; Yeung, D.; Gin, K.; Tsang, M.; Nair, P.; Jue, J.; et al. Automated Atrial Fibrillation Diagnosis by Echocardiography without ECG: Accuracy and Applications of a New Deep Learning Approach. Diseases 2024 , 12 , 35. [ Google Scholar ] [ CrossRef ]
• Brown, K.; Roshanitabrizi, P.; Rwebembera, J.; Okello, E.; Beaton, A.; Linguraru, M.G.; Sable, C.A. Using Artificial Intelligence for Rheumatic Heart Disease Detection by Echocardiography: Focus on Mitral Regurgitation. J. Am. Heart Assoc. 2024 , 13 , e031257. [ Google Scholar ] [ CrossRef ]
• Steffner, K.R.; Christensen, M.; Gill, G.; Bowdish, M.; Rhee, J.; Kumaresan, A.; He, B.; Zou, J.; Ouyang, D. Deep learning for transesophageal echocardiography view classification. Sci. Rep. 2024 , 14 , 11. [ Google Scholar ] [ CrossRef ]
• In Kim, Y.; Roh, J.H.; Kweon, J.; Kwon, H.; Chae, J.; Park, K.; Lee, J.-H.; Jeong, J.-O.; Kang, D.-Y.; Lee, P.H.; et al. Artificial intelligence-based quantitative coronary angiography of major vessels using deep-learning. Int. J. Cardiol. 2024 , 405 , 131945. [ Google Scholar ] [ CrossRef ]
• Rinehart, S.; Raible, S.J.; Ng, N.; Mullen, S.; Huey, W.; Rogers, C.; Pursnani, A. Utility of Artificial Intelligence Plaque Quantification: Results of the DECODE Study. J. Soc. Cardiovasc. Angiogr. Interv. 2024 , 3 , 101296. [ Google Scholar ] [ CrossRef ]
• Omori, H.; Matsuo, H.; Fujimoto, S.; Sobue, Y.; Nozaki, Y.; Nakazawa, G.; Takahashi, K.; Osawa, K.; Okubo, R.; Kaneko, U.; et al. Determination of lipid-rich plaques by artificial intelligence-enabled quantitative computed tomography using near-infrared spectroscopy as reference. Atherosclerosis 2023 , 386 , 117363. [ Google Scholar ] [ CrossRef ]
• Toggweiler, S.; Wyler von Ballmoos, M.C.; Moccetti, F.; Douverny, A.; Wolfrum, M.; Imamoglu, Z.; Mohler, A.; Gülan, U.; Kim, W.-K. A fully automated artificial intelligence-driven software for planning of transcatheter aortic valve replacement. Cardiovasc. Revascularization Med. 2024. [ CrossRef ]
• Salehi, M.; Maiter, A.; Strickland, S.; Aldabbagh, Z.; Karunasaagarar, K.; Thomas, R.; Lopez-Dee, T.; Capener, D.; Dwivedi, K.; Sharkey, M.; et al. Clinical assessment of an AI tool for measuring biventricular parameters on cardiac MR. Front. Cardiovasc. Med. 2024 , 11 , 1279298. [ Google Scholar ] [ CrossRef ]
• Ghanbari, F.; Joyce, T.; Lorenzoni, V.; Guaricci, A.I.; Pavon, A.G.; Fusini, L.; Andreini, D.; Rabbat, M.G.; Aquaro, G.D.; Abete, R.; et al. AI Cardiac MRI Scar Analysis Aids Prediction of Major Arrhythmic Events in the Multicenter DERIVATE Registry. Radiology 2023 , 307 , e222239. [ Google Scholar ] [ CrossRef ]
• Kaplan Berkaya, S.; Uysal, A.K.; Sora Gunal, E.; Ergin, S.; Gunal, S.; Gulmezoglu, M.B. A survey on ECG analysis. Biomed. Signal Process Control 2018 , 43 , 216–235. [ Google Scholar ] [ CrossRef ]
• Ogah, O.S.; Oladapo, O.O.; Adebiyi, A.A.; Adebayo, A.K.; Aje, A.; Ojji, D.B. Electrocardiographic left ventricular hypertrophy with strain pattern: Prevalence, mechanisms and prognostic implications. Cardiovasc. J. Afr. 2008 , 19 , 39. [ Google Scholar ]
• Bornstein, A.B.; Rao, S.S.; Marwaha, K. Left Ventricular Hypertrophy. [Updated 8 August 2023]. In StatPearls [Internet] ; StatPearls Publishing: Treasure Island, FL, USA, 2024. Available online: https://www.ncbi.nlm.nih.gov/books/NBK557534/ (accessed on 11 June 2024).
• Savarese, G.; Becher, P.M.; Lund, L.H.; Seferovic, P.; Rosano, G.M.C.; Coats, A.J.S. Global burden of heart failure: A comprehensive and updated review of epidemiology. Cardiovasc. Res. 2023 , 118 , 3272–3287. [ Google Scholar ] [ CrossRef ]
• Ojha, M.K.; Wadhwani, S.; Wadhwani, A.K.; Shukla, A. Automatic detection of arrhythmias from an ECG signal using an auto-encoder and SVM classifier. Phys. Eng. Sci. Med. 2022 , 45 , 665–674. [ Google Scholar ] [ CrossRef ]
• Ahmed, I.; Sasikumar, N. Echocardiography Imaging Techniques. [Updated 30 July 2023]. In StatPearls [Internet] ; StatPearls Publishing: Treasure Island, FL, USA, 2024. Available online: https://www.ncbi.nlm.nih.gov/books/NBK572130/ (accessed on 11 June 2024).
• Vidal-Perez, R.; Grapsa, J.; Bouzas-Mosquera, A.; Fontes-Carvalho, R.; Vazquez-Rodriguez, J.M. Current role and future perspectives of artificial intelligence in echocardiography. World J. Cardiol. 2023 , 15 , 284. [ Google Scholar ] [ CrossRef ]
• Carabello, B.A.; Paulus, W.J. Aortic stenosis. Lancet 2009 , 373 , 956–966. [ Google Scholar ] [ CrossRef ]
• Ring, L.; Shah, B.N.; Bhattacharyya, S.; Harkness, A.; Belham, M.; Oxborough, D.; Pearce, K.; Rana, B.S.; Augustine, D.X.; Robinson, S.; et al. Echocardiographic assessment of aortic stenosis: A practical guideline from the British Society of Echocardiography. Echo Res. Pract. 2021 , 8 , G19. [ Google Scholar ] [ CrossRef ]
• Stamate, E.; Piraianu, A.-I.; Ciobotaru, O.R.; Crassas, R.; Duca, O.; Fulga, A.; Grigore, I.; Vintila, V.; Fulga, I.; Ciobotaru, O.C. Revolutionizing Cardiology through Artificial Intelligence—Big Data from Proactive Prevention to Precise Diagnostics and Cutting-Edge Treatment—A Comprehensive Review of the Past 5 Years. Diagnostics 2024 , 14 , 1103. [ Google Scholar ] [ CrossRef ]
• Suzuki, N.; Asano, T.; Nakazawa, G.; Aoki, J.; Tanabe, K.; Hibi, K.; Ikari, Y.; Kozuma, K. Clinical expert consensus document on quantitative coronary angiography from the Japanese Association of Cardiovascular Intervention and Therapeutics. Cardiovasc. Interv. Ther. 2020 , 35 , 105. [ Google Scholar ] [ CrossRef ]
• Ramjattan, N.A.; Lala, V.; Kousa, O.; Shams, P.; Makaryus, A.N. Coronary CT Angiography. [Updated 19 January 2024]. In StatPearls [Internet] ; StatPearls Publishing: Treasure Island, FL, USA, 2024. Available online: https://www.ncbi.nlm.nih.gov/books/NBK470279/ (accessed on 11 June 2024).
• Blanke, P.; Schoepf, U.J.; Leipsic, J.A. CT Transcatheter Aortic Valve Replacement. Radiology 2013 , 269 , 650–669. [ Google Scholar ] [ CrossRef ]
• Tseng, W.Y.I.; Su, M.Y.M.; Tseng, Y.H.E. Introduction to Cardiovascular Magnetic Resonance: Technical Principles and Clinical Applications. Acta Cardiol. Sin. 2016 , 32 , 129. [ Google Scholar ] [ CrossRef ]
• He, J.; Baxter, S.L.; Xu, J.; Xu, J.; Zhou, X.; Zhang, K. The practical implementation of artificial intelligence technologies in medicine. Nat. Med. 2019 , 25 , 30. [ Google Scholar ] [ CrossRef ]
• Kolla, L.; Parikh, R.B. Uses and limitations of artificial intelligence for oncology. Cancer 2024 , 130 , 2101–2107. [ Google Scholar ] [ CrossRef ]
• Oh, S.; Kim, J.H.; Choi, S.W.; Lee, H.J.; Hong, J.; Kwon, S.H. Physician Confidence in Artificial Intelligence: An Online Mobile Survey. J. Med. Internet Res. 2019 , 21 , e12422. [ Google Scholar ] [ CrossRef ]
• Patel, V.; Shah, M. Artificial intelligence and machine learning in drug discovery and development. Intell. Med. 2022 , 2 , 134–140. [ Google Scholar ] [ CrossRef ]
• Busnatu, Ș.; Niculescu, A.G.; Bolocan, A.; Petrescu, G.E.D.; Păduraru, D.N.; Năstasă, I.; Lupușoru, M.; Geantă, M.; Andronic, O.; Grumezescu, A.M.; et al. Clinical Applications of Artificial Intelligence—An Updated Overview. J. Clin. Med. 2022 , 11 , 2265. [ Google Scholar ] [ CrossRef ]
• Chan, B. Black-box assisted medical decisions: AI power vs. ethical physician care. Med. Health Care Philos. 2023 , 26 , 285. [ Google Scholar ] [ CrossRef ]
• Poon, A.I.F.; Sung, J.J.Y. Opening the black box of AI-Medicine. J. Gastroenterol. Hepatol. 2021 , 36 , 581–584. [ Google Scholar ] [ CrossRef ]
• Durán, J.M.; Jongsma, K.R. Who is afraid of black box algorithms? On the epistemological and ethical basis of trust in medical AI. J. Med. Ethics 2021 , 47 , 329–335. [ Google Scholar ] [ CrossRef ]
• Gerke, S.; Minssen, T.; Cohen, G. Ethical and legal challenges of artificial intelligence-driven healthcare. Artif. Intell. Healthc. 2020 , 295–336. [ Google Scholar ] [ CrossRef ]
• Tang, L.; Li, J.; Fantus, S. Medical artificial intelligence ethics: A systematic review of empirical studies. Digit. Health 2023 , 9 . [ Google Scholar ] [ CrossRef ]
• Sana, M.K.; Hussain, Z.M.; Maqsood, M.H.; Shah, P.A. Artificial intelligence in celiac disease. Comput. Biol. Med. 2020 , 125 , 103996. [ Google Scholar ] [ CrossRef ]
• Vinny, P.W.; Vishnu, V.Y.; Padma Srivastava, M.V. Artificial Intelligence shaping the future of neurology practice. Med. J. Armed Forces India 2021 , 77 , 276–282. [ Google Scholar ] [ CrossRef ]

Click here to enlarge figure

Share and Cite

Patrascanu, O.S.; Tutunaru, D.; Musat, C.L.; Dragostin, O.M.; Fulga, A.; Nechita, L.; Ciubara, A.B.; Piraianu, A.I.; Stamate, E.; Poalelungi, D.G.; et al. Future Horizons: The Potential Role of Artificial Intelligence in Cardiology. J. Pers. Med. 2024 , 14 , 656. https://doi.org/10.3390/jpm14060656

Patrascanu OS, Tutunaru D, Musat CL, Dragostin OM, Fulga A, Nechita L, Ciubara AB, Piraianu AI, Stamate E, Poalelungi DG, et al. Future Horizons: The Potential Role of Artificial Intelligence in Cardiology. Journal of Personalized Medicine . 2024; 14(6):656. https://doi.org/10.3390/jpm14060656

Patrascanu, Octavian Stefan, Dana Tutunaru, Carmina Liana Musat, Oana Maria Dragostin, Ana Fulga, Luiza Nechita, Alexandru Bogdan Ciubara, Alin Ionut Piraianu, Elena Stamate, Diana Gina Poalelungi, and et al. 2024. "Future Horizons: The Potential Role of Artificial Intelligence in Cardiology" Journal of Personalized Medicine 14, no. 6: 656. https://doi.org/10.3390/jpm14060656

Article Metrics

Article access statistics, further information, mdpi initiatives, follow mdpi.

Subscribe to receive issue release notifications and newsletters from MDPI journals

Academia.edu no longer supports Internet Explorer.

To browse Academia.edu and the wider internet faster and more securely, please take a few seconds to  upgrade your browser .

•  We're Hiring!
•  Help Center

Risk Assessment & Risk Management

• Most Cited Papers
• Most Downloaded Papers
• Newest Papers
• Save to Library
• Last »
• Financial Risk Management Follow Following
• Risk Management Follow Following
• Risk assessment Follow Following
• Konsumsi Protein 2012 Follow Following
• Risk Follow Following
• Risk Analysis Follow Following
• Risk and Vulnerability Follow Following
• Certified Financial Analyst Follow Following
• Disaster risk management Follow Following
• Disaster risk reduction Follow Following

Enter the email address you signed up with and we'll email you a reset link.

• Academia.edu Publishing
•   We're Hiring!
•   Help Center
• Find new research papers in:
• Health Sciences
• Earth Sciences
• Cognitive Science
• Mathematics
• Computer Science
• Academia ©2024

• Open access
• Published: 17 June 2024

Patient outcome quality indicators for older persons in acute care: original development data using interRAI AC-CGA

• Melinda G. Martin-Khan 1 , 2 ,
• Leonard C. Gray 1 ,
• Caroline Brand 3 , 4 , 5 ,
• Olivia Wright 6 ,
• Nancy A. Pachana 7 ,
• Gerard J. Byrne 8 , 9 ,
• Mark D. Chatfield 1 ,
• Richard Jones 10 , 11 ,
• John Morris 11 ,
• Catherine Travers 12 ,
• Joanne Tropea 4 , 13 ,
• Beibei Xiong 1 ,
• The Research Collaborative for Quality Care: Acute Care Panel &

The Research Collaborative for Quality Care: Dementia Care Panel

BMC Geriatrics volume  24 , Article number:  527 ( 2024 ) Cite this article

Metrics details

A range of strategies are available that can improve the outcomes of older persons particularly in relation to basic activities of daily living during and after an acute care (AC) episode. This paper outlines the original development of outcome-oriented quality indicators (QIs) in relation to common geriatric syndromes and function for the care of the frail aged hospitalized in acute general medical wards.

Design QIs were developed using evidence from literature, expert opinion, field study data and a formal voting process. A systematic literature review of literature identified existing QIs (there were no outcome QIs) and evidence of interventions that improve older persons’ outcomes in AC. Preliminary indicators were developed by two expert panels following consideration of the evidence. After analysis of the data from field testing (indicator prevalence, variability across sites), panel meetings refined the QIs prior to a formal voting process.

Data was collected in nine Australian general medical wards.

Participants

Patients aged 70 years and over, consented within 24 h of admission to the AC ward.

Measurements

The interRAI Acute Care – Comprehensive Geriatric Assessment (interRAI AC-CGA) was administered at admission and discharge; a daily risk assessment in hospital; 28-day phone follow-up and chart audit.

Ten outcome QIs were established which focused on common geriatric syndromes and function for the care of the frail aged hospitalized in acute general medical wards.

Ten outcome QIs were developed. These QIs can be used to identify areas where specific action will lead to improvements in the quality of care delivered to older persons in hospital.

Peer Review reports

Older persons who are hospitalized for acute illnesses are at risk of both losing independence and institutionalization [ 1 , 2 ]. A range of strategies are available that can improve their outcomes at discharge, particularly in relation to the ability to perform basic activities of daily living and discharge to long term residential care [ 2 ]. The challenge is in identifying areas where limited time and resources should be focused to make appropriate changes.

Quality assurance activities can be complex and multifactorial. Prior literature tempers expectations between process and outcomes [ 3 ]. It can be challenging to record a link between changed processes and improved outcomes. Despite this, measuring patient outcomes and improving clinical practice to optimize patient outcomes in the long term are important. Quality indicators (QIs) (structure, process and outcome) are one way to measure quality of care [ 4 ]. QIs are commonly used in hospitals, but there are no published sets of outcome QIs for geriatric syndromes and function for the care of older persons in acute care (AC) prior to the interRAI AC QIs [ 5 , 6 , 7 ].

QI development is most effective when it includes expert opinion, a review of the scientific literature and field study data [ 8 , 9 ]. Measuring quality of care based on patient outcomes has inherent appeal, but there are major concerns. There are attribution problems (difficulties measuring the aspect of quality precisely or exclusively) [ 1 ]; and problems with inaccurate trigger rates (triggering based on other random factors and resulting in inaccurate results). Depending on the location of the hospital, socio-economic and health demographic factors will also impact the bed-use and presentation profile. Any development of outcome QIs must take these factors into account [ 10 ] because a QI that measures a hospital’s performance is only meaningful when it measures factors that hospital can control, and includes mechanisms to adjust for factors that hospital cannot control.

The aim of this study was to develop outcome oriented QIs in relation to common geriatric syndromes and function for the care of the frail aged hospitalized in acute general medical wards. These QIs have since been adopted into an international assessment system for AC and a supplementary set created for use in a surgical setting. This paper outlines the original development project.

This project was carried out in three phases: Development of a preliminary QI set; field study; analysis and compilation of a definitive QI set. The research methodology for this project has been described in detail elsewhere [ 10 ]. A truncated description is provided here.

Phase 1: Development of a preliminary QI set

Literature summary.

A scoping review of the scientific literature pertaining to adverse geriatric outcomes in acute hospitals was compiled. Publications with the highest levels of evidence in each domain of interest were identified together with existing guidelines and published QIs relevant to hospitalized older persons (with an interpretation of methodological quality and applicability).

Expert panels

Given the scope of the project, two expert panels were required: panel one focused on generic outcomes for older persons (e.g., continence) [Acute Care Panel, Supplementary Box 1 ]; and panel two focused on cognitive health (e.g., the recognition of cognitive impairment) [Dementia Care Panel, Supplementary Box 2 ]. Expertise criteria for panel participation was established a priori. Experience included: allied health, consumer advocacy, dementia, general medicine, geriatric medicine, gerontic nursing, quality research methodology, and AC public hospital health care. Participating sites for data collection were recruited from two Australian states (Queensland, Victoria), and coordinators at each site participated in the expert panels. Based on matching site coordinator's experience with the a priori list, any remaining gaps were filled by invitation from the research team.

Expert Panel Meetings. The literature summaries were divided, based on relevance, between the two expert panels. Initially, each panel met for two days to review their respective summaries and conceptualise potential QIs. Panel members recommended QIs according to predefined criteria: being amenable to change, within the control of the hospital, had evidence to support a link between the actions of a hospital and the outcome, and which were clinically significant for an older person and therefore warranted attention. Each QI was defined (i.e. identifying the variables needed for calculating the numerator, denominator, inclusion, and exclusion criteria). The QIs were linked with variables from the interRAI AC – Comprehensive Geriatric Assessment (AC- CGA) tool [ 11 ] for data collection in the first instance, to score the QIs prior to evaluation in phase 3.

Phase 2: Field Study

Patients aged 70 years and older admitted to nine Australian AC general medical hospitals, who were likely to stay in hospital for at least 48 h, were invited to participate. The data collection took an average of 7 months (range 4–10 months). Data collection period by hospital type (tertiary or regional) is shown on a radar chart in the Supplementary Fig.  1 . Hospitals included five metropolitan tertiary teaching hospitals and four regional hospitals. Ethics approval was obtained from the relevant organisations.

Data collection

Each hospital nominated one or two registered nurses with gerontic experience to work full-time on the project as data collectors during the study period. One full-time registered nurse with gerontic experience, trained in project data collection, was rostered each day per site for 7.5 h of data collection a day across five days each week (Monday to Friday) during the data collection period. All nurse assessors attended a three-day training program at the Centre for Health Services Research (CHSR) prior to commencing data collection. Training involved understanding the project protocol and minimising potential research bias, administration and scoring of the formal assessment tools, and the completion of the study recruitment database. A regular teleconference schedule was established for interaction between sites during the data collection period and protocols for data storage and transfer were put in place. All site research nurses were trained to administer the interRAI AC-CGA by the same experienced interRAI assessment trainer. Previous studies have shown very high levels of reliability on observed agreement for the interRAI AC-CGA [ 12 , 13 ].

Demographic data was collected on patients who consented to participate in the study. The interRAI AC-CGA was administered within 24 h of admission to the ward for consenting patients, with daily follow-up and review at discharge. At 28 days, a follow-up telephone call to the patient’s or carer’s home was made to collect data on readmission and post-discharge change of living arrangement, this included the Mini-Mental State Examination 22-item telephone version (ALFI-MMSE 22-items) [ 14 , 15 ]. The data collected at 28 days was part of demographic data and was not used in the calculation of any QIs. Verification of re-admission, including diagnostic codes, was obtained by State data custodians or the relevant hospital data custodian.

Phase 3: Analysis and final consultation (with expert panels)

Statistical analysis.

Data was entered into IBM SPSS Statistics v26 2019. General descriptive statistics (frequencies, standard deviations and means) were used to describe characteristics of enrolled participants and to compare between hospitals, using demographic data, and cognitive and functional scales derived from the interRAI AC-CGA.

Each QI was scored for the total sample, and for each individual hospital. A trigger rate (percentage) was calculated for each QI (the proportion of patients who triggered the indicator event (the numerator) divided by the total number of eligible patients (the denominator)).

The expert-panel-identified covariates were analysed for statistical relevance to the sample by presenting the QI trigger rates when the covariate was present and absent. This was followed by descriptive statistics (odds ratios with 95% confidence intervals, and p-values) to identify statistically significant factors for each QI. All covariate information was provided, noting that multiple risk adjustments cannot be applied simultaneously to this sample given the small sample size.

Using the RAND-UCLA consensus method [ 16 ] two rounds of voting for each panel were completed. Analysis of voting round data was completed after each round. For each QI the following were calculated using the panel members’ individual votes of 1–9: the 30th and 70th percentile, the Interpercentile Range Adjusted for Symmetry, the Interpercentile Range, Interpercentile Range Central Point, the Asymmetry Index, median and mean deviation from the median. This data were used to apply decision rules, defined a priori, to identify valid or invalid QIs [ 17 ]. Only votes from round 2 were utilised to determine the final QI set.

Expert Panel

A report was prepared for each panel (there was no overlap of indicators between panels) with a description of the preliminary indicators suggested from Phases 1 and 2, feedback provided by the interRAI AC Network and the interRAI Instrument Committee, and any recommendations (Supplementary Fig.  2 ). Any new research published in the interim was also provided. Each panel held a second two-day workshop to scrutinise the preliminary QIs which had now been tested in the field. QIs were modified based on recommendations from the expert panel following interpretation of the field data, evidence from the literature, and expert opinion.

Risk Adjustment: Risk adjustment was considered for each QI separately at the final expert panel meetings. Risk adjustment was approved if there was agreement amongst the panel members that factors, outside the control of the hospital, would impact the QI’s outcome, and those factors couldn’t be dealt with by excluding patients from the denominator. Risk adjustment was not approved in situations where it was recognised that certain patient characteristics did pose extra challenges but this didn’t obviate the extra care that was required to ensure patients’ safety and well-being. For example, the Dementia Care panel recognized that patients with dementia were at higher risk of delirium in hospital, but this did not justify risk adjustment for the delirium QI. It was important that staff identify these patients and provide care to minimise the risk of delirium rather than accept that it was inevitable. For each QI, the prevalence of the outcome across all patients in hospital datasets, after exclusions, is the facility-level observed QI score. Risk adjustment was calculated, when the final set of outcome QIs was identified, using logistic regression models [ 10 , 18 ].

Each panel voted on an independent set of QIs which were combined to establish the final outcome QI set. Each member voted on a scale of 1–9 with reference to specific criteria. A second voting sheet was prepared individually for each panel member which showed the spread of panel votes for each QI and the personal vote of that panel member, along with the QI data analysis (Valid or not, etc.). A teleconference was held to discuss the round 1 results. Where the votes for a QI was wide-spread, it generally indicated a misunderstanding, a poorly constructed QI, or diverse opinion amongst the panel members. The teleconference enabled clarification and rectified any misunderstandings. Each panel member voted a second time to conclude the final outcome AC set.

Data from 643 patients at nine hospitals were analysed and used to calculate each individual QI. No patients withdrew during the in-hospital data collection. Characteristics of the included sample (Table 1 ) is provided.

The Research Collaboration for Quality Care (RCQC) Outcome QIs for Older Persons in AC

Ten outcome indicators were developed for the AC QIs for Older Persons. Table 2 describes each QI. Each QI has been classified according to the Institute of Medicine (IOM) framework, which is a useful way of considering the domains of care, and how this set of outcome QIs sit within those domains [ 19 , 20 ]. The domains of care, as defined in the IOM framework are: safety, effectiveness, patient-centeredness, timeliness, efficiency, and equity. Figure  1 presents the QIs and displays the observed between site variation of scores within each QI. Table 3 provides raw summary data for each QI. In all cases, the QIs are interpreted by understanding that a lower score reflects a better outcome for the patient and the hospital.

Field study data of outcome indicators for older adults, ordered by lowest to highest performing in terms of overall trigger rate . Note: The size of the marker is scaled according to the denominator

RCQC Outcome QIs for older persons in AC:

The proportion of female patients with a new urinary catheter on admission. Functional decline guidelines recommend careful assessment of the need for an indwelling urinary catheter prior to use given the evidence of a causal link between indwelling catheter use and urinary tract infections [ 21 , 22 ]. The focus of this QI is on female patients as it was felt that clinically there was less justification for using catheters in women than men and that this better represented quality of care (face validity) than a QI for all patients.

The proportion of patients with delirium-indicating behaviors present at discharge. There is evidence for delirium reducing interventions in AC. Australian functional decline guidelines (expert opinion levels of evidence) recommend consideration of transitional care needs and community-based strategies for people discharged from hospital and residential care to take into account delirium issues [ 21 , 22 ]. This QI considers the potential risk of discharging older people from hospital if they have delirium, without firstly being aware that delirium is present, and second using clinical judgement to decide if discharge is the best course of action. The panel recognises that in some cases, with careful discharge planning, the best choice for a patient with delirium may be discharge.

The proportion of patients discharged with worse levels of cognitive function compared with premorbid levels. Declining Mini-Mental State Examination scores have been shown to predict mortality [ 23 ]. Interventions can change the outcome and assist in maintaining cognitive functions [ 24 ]. There is variation in recoverable cognitive dysfunction amongst older patients in AC [ 25 ]. This QI was included by the panel because of the impact of cognitive decline, over and above that of delirium, for older patients.

The proportion of patients discharged with worse mobility levels compared with pre-morbid levels. Levels of mobility, self-care needs and cognitive and behavioral status are factors often associated with needing additional ‘non-acute’ care while in hospital. This extra care is more commonly needed by older people who have issues with mobility when in hospital. Addressing support for mobility (and other self-care needs) in hospital impacts future care needs for older patients (i.e. after the hospital episode) [ 26 ]. The panel acknowledge the high level evidence in support of this QI, which indicates the use of multidisciplinary interventions (which include a component of exercise) to reduce hospital length of stay, increase proportion of patients discharged home and reduce costs of stay for older acute hospitalized inpatients [ 21 , 22 ].

The proportion of patients with pre-hospital decline who failed to return to preadmission function (or better) by discharge. There is a significant body of evidence that shows older people decline in function by the time of discharge from AC, and that this negatively impacts quality of life of hospitalized older people [ 27 ]. Recommendations of the Australian Functional Decline Guidelines, based on expert opinion, include individualized care plans which encourage appropriate incidental activity throughout the day and minimize bed rest, and recommendations to assess and modify the environment to encourage independence and mobility [ 21 , 22 ].

The proportion of patients with no pre-morbid pain who reported both pain at admission and unimproved pain at discharge. There is evidence that improved strategies to manage pain (i.e. quality improvement collaborative program) improve processes of care (such as improved use of pain assessment tools), but only limited evidence of links to patient outcomes (i.e. sufficient pain treatment for patients) [ 28 , 29 ].

The proportion of patients with a new or worsening pressure ulcer at discharge when compared with admission. A review of studies reporting the implementation of pressure ulcer reduction strategies ( n  = 16) reported a positive reduction in the outcome pressure ulcer incidence [ 30 ]. There is strong evidence to link specific practices in AC to reducing the risk of pressure ulcers (in patients with high risk) [ 21 , 22 ]. The panel recognized that the incidence of pressure ulcers was small in this study but that the clinical impact was significant and therefore it was included as an indicator of quality of care.

The proportion of patients who fell (at least once) during the hospital episode. Some falls are preventable while some are not preventable [ 31 ]. There is sound evidence to support the use of multifaceted approaches to reduce the falls rate in hospital settings. There is some evidence to support the use of specialist geriatric wards to reduce falls incidence and fall-related injuries among post-operative hip fracture patients [ 32 , 33 ]. The panel recognized that the incidence of falls was small in this study but that the clinical impact for older persons was significant and therefore it was included as an indicator of quality of care.

The proportion of patients with prolonged length of stay. Generally, injured older adults (> = 65 years) have an average length of hospital stay, including admissions to ICU, exceeding 10 days. A retrospective study of the characteristics and outcomes of injured older adults after hospital admission (N = 6,069) from the Queensland Trauma Registry identified that the average length of stay (including patients in ICU) was eight days (interquartile range 5–15), with 33.8% of cases with a major injury developing a complication [ 34 ]. There is evidence to indicate that the risk of iatrogenic complications is higher if an older person is hospitalized for longer than one week. These complications are frequently preventable and are often associated with a rapid decline in muscle strength, aerobic capacity and pulmonary ventilation that older patients experience [ 35 ]. For the majority of older patients, ‘waiting for placement’ is not a contributor to their length of stay. Most older people are discharged to their usual place of residence [ 36 ].

The proportion of community-dwelling patients discharged to long-term care. There is good evidence to support the use of multidisciplinary interventions that include a component of exercise to reduce in-hospital length of stay, increase the proportion of patients discharge directly home and reduce costs of stay for older acute hospitalised inpatients [ 21 , 22 ]. A review focused on patients regaining functional independence in the acute setting following hip fracture concluded that patients who stayed longer in the acute setting, received physical therapy more than once a day had improved odds of regaining independence in bed mobility, transfers and ambulation. Patients who regained independence along with use of physical therapy services had improved odds of being discharged directly to the home from the acute setting [ 37 ].

No covariates for risk adjustment were recommended for three QIs: Delirium, Self Care, and Pain. The remaining QIs each had covariates recommended for risk adjustment (Table  4 ). The initial step for industry uptake has been swift. The final set of QIs were presented to the interRAI Instrument Committee. The RCQC Patient Outcome QIs for Older Persons in AC were formally endorsed by interRAI and adopted (except for the cognition indicator) for use with the interRAI AC suite [ 7 , 11 ] with modification recommended for the surgical context [ 38 ]. Software vendors who provide electronic versions of the interRAI AC now also include algorithms to run the AC outcome indictors derived from the data collected by staff when completing care assessments.

The aim of this project was to develop outcome-oriented QIs in relation to common geriatric syndromes and function for the care of the frail aged hospitalized in acute general medical wards derived from the interRAI AC-CGA assessment tool clinical data items. The interRAI assessments were selected because they are implemented as care focused sources of data collection. From that care assessment data, a sub-set of items are used to score the QIs, but it is assumed that the assessment is collected in full initially by care staff for the primary purpose of delivering care to a patient in a health service. There is no additional burden of collecting data for administrative or quality care purposes. Many of the QIs can be scored without the interRAI AC-CGA, or they can be scored using a measure the facility currently uses if only internal benchmarking is occurring. QIs have been clearly marked in Tables 2 and Supplementary Table  1 to indicate where this applies.

Ten outcome QIs have been developed, with nine formally endorsed by interRAI for use in the interRAI AC suite (except for the cognition indicator as it was decided delirium was the primary quality issue relevant in AC at this time). The interRAI AC suite is a set of tools, primarily used by nurses at a ward level for risk assessment, to guide clinical care planning and preparation for specialist consultation. The QIs were developed to support improvements in clinical care at the ward level. They can be used to benchmark existing levels of care, and measure change after the implementation of interventions. Future work in quality improvement and outcome QIs will focus on understanding the extent to which staff self-direct the modification of their clinical activities when they are aware of the QI scoring.

When the study was carried out originally the primary data collection tool was the interRAI AC-CGA (a longer assessment). There is now a shorter nursing assessment, a 15-min interRAI AC, and supplementary surgical QIs have been developed. The Supplementary Table 1 illustrates how the QIs can be scored using administrative or clinical data, or interRAI data (i.e. Some QIs can be scored without the interRAI AC as they are straightforward prevalence or incidence indicators). These are listed in the Supplementary Table  1 and indicated with an * including bladder catheter, pain, skin integrity, falls, prolonged stay, institutional placement. At present, some QIs are scored using algorithms calculated from assessment data in the interRAI AC-CGA and the interRAI AC (delirium, cognition, self-care, mobility). Detailed information regarding the scoring is available in the interRAI clinical and applications manual for the interRAI AC [ 39 ] which is sourced at the interRAI website ( www.interRAI.com ). For example, QIs scored using scales inherent in the interRAI AC, and if the QI was to be implemented and scored using a different system (such as a cognitive tool, rather than the interRAI cognitive CPS scale), the benchmarks in this paper would not be comparable. The QIs scored using interRAI scales or specific items are marked as such in Table  2 and the Supplementary Table  1 : these are the Delirium, Cognition, Mobility and Self Care QIs. For organisations not using the interRAI assessment to score these QIs, it is possible to use validated assessments as measures for internal purposes (for example, when scoring the delirium QI) but this will not enable immediate comparison for external benchmarking purposes as different measures have been used, nor can the data in this paper be used as a baseline.

We acknowledge the limitation of this project, that it was carried out in general medical wards at nine Australian hospitals in two States. We recognize that geriatric specific wards would have the potential to score differently as it is possible that the specific training and expertise that staff have in these units would change the outcomes for older persons (possibly improved outcomes; lower scores). The application of these QIs in general medical wards internationally may also generate different results, despite this, generalizability is not impacted as the criteria for quality of care is the same around the world. We were also limited in the information we could collect regarding participating patients and did not have consistency across all sites. This information would have increased our understanding of the generalisability of the findings and will be a strong focus for future work. Second, this paper describes the development of a set of outcome QIs. Further research to show the extent to which these indicators respond to interventions designed to improve patient outcomes is needed to validate their use in benchmarking quality improvement activities. The current set of indicators have been selected based on face validity in relation to responsiveness (i.e. there is published evidence that interventions can change the outcome in the area of interest). The interRAI AC-CGA has been used as the minimum data set for the development of the QIs (and they can also be scored on the companion interRAI AC). A strength in this approach is that interRAI assessments are an accepted international standard for minimum datasets and in many countries the assessments are mandated as part of health care data collection in a variety of health care settings. This creates an opportunity for these QIs to be utilised for international benchmarking in acute care in the future. Conversely, uptake maybe hindered by organisations which choose not to access licensed standardised assessment tools for their data collection and patient assessment. For those organisations, sufficient information is in the public domain for internal benchmarking activities utilising these QIs without the interRAI AC assessment but this limits the potential for international benchmarking.

Prior to this study, there were no published outcome indicators for older persons in AC. This work was a first step towards supporting improvement in the care of older persons in an acute setting. In their current format, these QIs can be applied in an acute hospital ward and the results can be compared with the field study data listed here as a benchmark. Poor scores on particular QIs can help to focus quality assurance time and resources to improving care where it is needed most.

Conclusions

Two expert panels were formed to review the scientific literature and interpret field study data to develop a set of outcome QIs. Ten outcome QIs were developed. These QIs can be used to identify areas where specific action will lead to improvements in the quality of care delivered to older persons in hospital.

Availability of data and materials

The datasets generated and/or analysed during the current study are not publicly available due to privacy or ethical restrictions but are available from the corresponding author on reasonable request.

Abbreviations

Quality Indicator

interRAI Acute Care - Comprehensive Geriatric Assessment

Research Collaboration for Quality Care

Anell A, Glenngård AH. The use of outcome and process indicators to incentivize integrated care for frail older people: a case study of primary care services in Sweden. Int J Integr Care. 2014;14:e038–e038.

Article   PubMed   PubMed Central   Google Scholar  

Landefeld CS, Palmer RM, Kresevic DM, Fortinsky RH, Kowal J. A randomized trial of care in a hospital medical unit especially designed to improve the functional outcomes of acutely ill older patients. N Engl J Med. 1995;332(20):1338–44.

Article   CAS   PubMed   Google Scholar  

Schneider EC. Hospital quality management: a shape-shifting cornerstone in the foundation for high-quality health care. International Journal For Quality In Health Care: Journal Of The International Society For Quality In Health Care / Isqua. 2014;26(Suppl 1):1–1.

Article   Google Scholar  

Donabedian A. The quality of care: How can it be assessed? JAMA. 1988;260(12):1743–8.

Ahmed N, Taylor K, McDaniel Y, Dyer CB. The role of an Acute Care for the Elderly unit in achieving hospital quality indicators while caring for frail hospitalized elders. Popul Health Manag. 2012;15(4):236–40.

Article   PubMed   Google Scholar  

Katterl R, Anikeeva O, Butler C, Brown L, Smith B, Bywood P. Potentially avoidable hospitalisations in Australia: Causes for hospitalisations and primary health care interventions. PHC RIS Policy Issue Review. Adelaide: Primary Health Care Research & Information Service; 2012. https://core.ac.uk/reader/14947500 .

Gray LC, Beattie E, Boscart VM, Henderson A, Hornby-Turner YC, Hubbard RE, Wood S, Peel NM. Development and testing of the interRAI acute care: a standardized assessment administered by nurses for patients admitted to acute care. Health Services Insights. 2018;11:1178632918818836.

De Koning J, Burgers JS, Klazinga N. Appraisal of Indicators through Research and Evaluation (English version based on original Dutch version 2.0). Amsterdam: University of Amsterdam; 2007.

Burkett E, Martin-Khan MG, Gray LC. Quality indicators in the care of older persons in the emergency department: A systematic review of the literature. Australas J Ageing. 2017;36(4):286–98.

Brand C, Martin-Khan M, Wright O, Jones R, Morris J, Travers C, Tropea J, Gray L. Development of quality indicators for monitoring outcomes of frail elderly hospitalised in acute care health settings: Study Protocol. BMC Health Services Research. 2011;11:281. https://doi.org/10.1186/1472-6963-11-281

Gray L, Arino-Blasco S, Berg K, Fries B, Heckman G, Jonsson P, Kergoat M-J, Morris J, Peel N, Sinha S, et al. interRAI Acute Care for Comprehensive Geriatric Assessment (AC-CGA) Form and User’s Manual Version 9.3. Washington, DC: interRAI; 2017.

Google Scholar  

Wellens NI, Van Lancker A, Flamaing J, Gray L, Moons P, Verbeke G, Boonen S, Milisen K. Interrater reliability of the interRAI Acute Care (interRAI AC). Arch Gerontol Geriatr. 2012;55(1):165–72.

Wellens NI, Deschodt M, Boonen S, Flamaing J, Gray L, Moons P, Milisen K. Validity of the interRAI Acute Care based on test content: a multi-center study. Aging Clin Exp Res. 2011;23(5–6):476–86.

Roccaforte WH, Burke WJ, Bayer BL, Wengel SP. Validation of a Telephone Version of the Mini-Mental State Examination. J Am Geriatr Soc. 1992;40(7):697–702.

Kennedy RE, Williams CP, Sawyer P, Allman RM, Crowe M. Comparison of in-person and telephone administration of the Mini-Mental State Examination in the University of Alabama at Birmingham Study of Aging. J Am Geriatr Soc. 2014;62(10):1928–32.

Fitch K, Bernstein S, Aguilar M, Burnand B, LaCalle J, Lazaro P, van het Loo M, McDonnell J, Vader J, Kahan J. The RAND/UCLA appropriateness method user’s manual. Santa Monica: RAND; 2001.

Basger BJ, Chen TF, Moles RJ. Validation of prescribing appropriateness criteria for older Australians using the RAND/UCLA appropriateness method. BMJ Open. 2012;2(5): e001431.

Abt Associates Inc. National nursing home quality measures: User’s manual. Cambridge, MA: Abt Associates Inc.; 2004. https://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/NursingHomeQualityInits/downloads/nhqiqmusersmanual.pdf .

Institute of Medicine (US) Committee on Quality of Health Care in America. Crossing the Quality Chasm: a New Health System for the 21st Century. Washington, DC: National Academies Press (US); 2001. https://doi.org/10.17226/10027 .

Romano P, Hussey P, Ritley D. Selecting Quality and Resource Use Measures: A Decision Guide for Community Quality Collaboratives. Rockville, MD, USA: US Department of Health and Human Services, Agency for Healthcare Research and Quality; 2010. https://www.ahrq.gov/sites/default/files/publications/files/perfmeas.pdf .

Clinical Epidemiology and Health Service Evaluation Unit, Melbourne Health. Best practice approaches to minimise functional decline in the older person across the acute, sub-acute and residential aged care sectors. Melbourne, VIC: Victorian Government Department of Human Services; 2004. https://www.vgls.vic.gov.au/client/en_AU/search/asset/1160801/0 .

Clinical Epidemiology and Health Service Evaluation Unit, Melbourne Health. Best practice approaches to minimise functional decline in the older person across the acute, sub-acute and residential aged care sectors: Update. Melbourne, VIC: Victorian Government Department of Human Services; 2007. https://www.vgls.vic.gov.au/client/en_AU/search/asset/1161133/0 .

Lavery LL, Dodge HH, Snitz B, Ganguli M. Cognitive decline and mortality in community-based cohort: the Monongahela valley independent elders survey. J Am Geriatr Soc. 2009;57(1):94–100.

Inouye SK, Bogardus ST, Baker DI, Leo-Summers L, Cooney LM. The hospital elder life program: a model of care to prevent cognitive and functional decline in older hospitalized patients. J Am Geriatr Soc. 2000;48(12):1697–706.

Chen CC-H, Chiu M-J, Chen S-P, Cheng C-M, Huang G-H. Patterns of cognitive change in elderly patients during and 6 months after hospitalisation: A prospective cohort study. Int J Nurs Stud. 2011;48(3):338–46.

To T, Davies O, Sincock J, Whitehead C. Multidisciplinary care needs in an Australian tertiary teaching hospital. Aust Health Rev. 2010;34(2):234–8.

Mudge A, O’Rouke P, Denaro C. Timing and Risk Factors for Functional Changes Associated with Medical Hospitalisation in Older Patients. J Gerontol A Biol Sci Med Sci. 2010;65(8):866–72.

Morrison R, Meier D, Fischberg D, Moore C, Degenholtz H, Litke A, Maroney-Galin C, Siu A. Improving the management of pain in hospitalised adults. Arch Intern Med. 2006;166:1033–9.

Haller G, Agoritsas T, Luthy C, Piguet V, Griesser A, Perneger T. Collaborative Quality Improvement to Manage Pain in Acute Care Hospitals. Pain Med. 2011;12:138–47.

Soban L, Hempel S, Munjas B, Miles J, Rubenstein L. Preventing pressure ulcers in hospitals: a systematic review of nurse-focused quality improvement interventions. The Joint Commission Journal on Quality and Patient Safety. 2011;37(6):245–52.

Australian Commission on Safety and Quality in Health Care (ACSQHC). Preventing falls and harm from falls in older people: Best practice guidelines for Australian hospitals. Sydney, NSW: Commonwealth of Australia ACSQHC. 2009. https://www.safetyandquality.gov.au/publications-and-resources/resource-library/preventing-falls-and-harm-falls-older-people-best-practice-guidelines-australian-hospitals .

Australian Health Ministers’ Advisory Council (AHMAC). Best Practice Approaches to Minimise Functional Decline in the Older Person across the Acute, Sub-Acute and Residential Aged Care Settings. Melboure: Victorian Government Department of Human Services; 2004.

Cameron I, Murray G, Gillespie L, Robertson M, Hill K, Cumming R, Kerse N. Interventions for preventing falls in older people in nursing care facilities and hospitals. Cochrane Database Syst Rev. 2010;1:CD005465.

Aitken L, Burmeister E, Lang J, Chaboyer W, Richmond T. Characteristics and outcomes of injured older adults after hospital admission. J Am Geriatr Soc. 2010;58(3):442–9.

Creditor M. Hazards of hospitalization of the elderly. Ann Intern Med. 1993;118(3):219–23.

Gray L, Dakin L. Australian and New Zealand Society for Geriatric Medicine position statement - older persons in acute hospitals awaiting transfer to a residential aged care facility. Australas J Ageing. 2011;30(1):43–6.

Guccione A, Fagerson T, Anderson J. Regaining functional independence in the acute care setting following hip fracture. Phys Ther. 1996;76(8):818–26.

Wood T, Chatfield M, Gray L, Peel N, Freeman S, Martin-Khan M. Examining the adaptability and validity of interRAI acute care quality indicators in a surgical context. SAGE Open Medicine. 2022;10:20503121221103221.

Gray L, Arino-Blasco S, Berg K, Bula C, Gambassi G, Heckman G, Jonsson P, Kergoat M, Leff B, Martin-Khan M, et al. interRAI clinical and management applications manual: for use with the interRAI Acute Care Assessment Instrument Version 9.1. Washington, DC: interRAI; 2013.

Download references

Acknowledgements

We thank the staff and patients at each participating hospital for their involvement in this study and the site coordinators (implementation of the protocol and oversight of data collection at each site) for their implementation support: Austin Hospital, Victoria; Cairns Base Hospital, Queensland; Princess Alexandra Hospital, Queensland; Redcliffe Hospital, Queensland; Royal Brisbane and Women’s Hospital, Queensland; The Northern Hospital, Victoria; The Royal Melbourne Hospital, Victoria; The Prince Charles Hospital, Queensland; Toowoomba Hospital, Queensland.

The Research Collaborative for Quality Care: Acute Care Panel 

Caroline Brand 3,4,5 , Leonard C. Gray 1 , Nancye Peel 1 , Olivia Wright 6 , Alison Mudge 1,14 , Jeffrey Rowland 15 , & Kwang Lim 16

14 Royal Brisbane Women’s Hospital, Brisbane, QLD, Australia.

15 Prince Charles Hospital, Brisbane, QLD, Australia.

16 Northern Hospital, Northern Health, Melbourne, VIC, Australia.

Elizabeth Beattie 17 , Caroline Brand 3,4,5 , Gerard Byrne 8,9 , Leonard C. Gray 1 , Nancy A. Pachana 7 , Eddy Strivens 18 , Paul Varghese 19 , & Catherine Yelland 15

17 School of Nursing, Queensland University of Technology, Brisbane, QLD, Australia.

18 James Cook University, Cairns Hospital, Queensland Health, Brisbane, QLD, Australia.

19 Princess Alexandra Hospital, Queensland Health, Brisbane, QLD, Australia.

Funding for this project was provided by the National Health and Medical Research Council (NHMRC) Project grants scheme (569682) which funded the work of developing QIs for older adults and the primary project funding. The Wicking Trust funded the work specific to developing QIs for people with dementia including the dementia expert panels. Alzheimer’s Australia Viertel Foundation post-doctoral fellowship funded the post-doctoral researcher, Dr. Melinda Martin-Khan. The research was designed, implemented, and analysed independently, with no involvement from the funding organisation.

Author information

Authors and affiliations.

Centre for Health Services Research, The University of Queensland, Building 33, Princess Alexandra Hospital, 34 Cornwall St, Brisbane, QLD, 4102, Australia

Melinda G. Martin-Khan, Leonard C. Gray, Mark D. Chatfield & Beibei Xiong

Department of Health and Life Sciences, University of Exeter, Exeter, UK

Melinda G. Martin-Khan

Department of Clinical Epidemiology, Biostatistics and Health Services Research, Melbourne Health, The University of Melbourne and The Royal Melbourne Hospital, Melbourne, VIC, Australia

Caroline Brand

Department of Medicine, University of Melbourne, Parkville, VIC, Australia

Caroline Brand & Joanne Tropea

Centre for Research Excellence in Patient Safety (CREPS), Monash University, Melbourne, VIC, Australia

Nutrition and Dietetics, School of Human Movement and Nutrition Sciences, The University of Queensland, Brisbane, QLD, Australia

Olivia Wright

School of Psychology, The University of Queensland, Brisbane, QLD, Australia

Nancy A. Pachana

Mayne Academy of Psychiatry, Faculty of Medicine, The University of Queensland, Brisbane, QLD, Australia

• Gerard J. Byrne

Mental Health Service, Royal Brisbane and Women’s Hospital, Brisbane, QLD, Australia

The Warren Alpert Medical School, Brown University, Boston, MA, USA

Richard Jones

Institute of Ageing Research, Boston, MA, USA

Richard Jones & John Morris

School of Health and Rehabilitation Sciences, The University of Queensland, Brisbane, QLD, Australia

Catherine Travers

Department of Aged Care, Royal Melbourne Hospital, Melbourne, VIC, Australia

Joanne Tropea

You can also search for this author in PubMed   Google Scholar

The Research Collaborative for Quality Care: Acute Care Panel

• Leonard C. Gray
• , Olivia Wright
• , Alison Mudge
• , Jeffrey Rowland
•  & Kwang Lim
• , Leonard C. Gray
• , Nancy A. Pachana
• , Catherine Travers
• , Elizabeth Beattie
• , Eddy Strivens
•  & Paul Varghese

Contributions

MMK and LG contributed to the study concept and design. MMK, LG, CB, OW, NP, GB, CT, and JT contributed to the acquisition, analysis, and interpretation of data. MMK drafted the manuscript. MMK, MC, RJ, JM, and BX contributed to statistical analysis. MMK and LG obtained funding and provided study supervision. All authors contributed to critical revisions of the manuscript for important intellectual content. All authors have read and approved the submission of this manuscript.

Corresponding author

Correspondence to Leonard C. Gray .

Ethics declarations

Ethics approval and consent to participate.

Ethics approval was obtained from: The University of Queensland Medical Research Ethics Committee [2008001564]; Princess Alexandra Hospital Human Research Ethics Committee [2008/130]; Redcliffe-Caboolture Human Research Ethics Committee [HREC/09/QNRC/6]; Royal Brisbane & Women’s Hospital Human Research Ethics Committee [HREC/09/QRBW/75]; The Prince Charles Committee Human Research Ethics Committee [HREC/09/QPCH/50]; The Northern Health Human Research Ethics Committee Continuing Care [CC17/09]; Melbourne Health Mental Health Research and Ethics Committee [2009.613]; The Cairns & Hinterland Health Service District Human Research and Ethics Committee [HREC/09/QCH/33–576]; and Austin Health Human Research Ethics Committee [H2009/03513]. Informed consent was obtained from all study participants. The study was performed in accordance with the Declaration of Helsinki guidelines and regulations.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests. LG, JM, RJ and MMK are fellows of the interRAI research consortium, which is a not-for-profit organization registered in the United States. Fellows contribute to the interRAI effort purely on a voluntary basis.

Additional information

Publisher ’ s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary material 1., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Martin-Khan, M.G., Gray, L.C., Brand, C. et al. Patient outcome quality indicators for older persons in acute care: original development data using interRAI AC-CGA. BMC Geriatr 24 , 527 (2024). https://doi.org/10.1186/s12877-024-04980-9

Download citation

Received : 30 October 2023

Accepted : 16 April 2024

Published : 17 June 2024

DOI : https://doi.org/10.1186/s12877-024-04980-9

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Quality indicators
• Quality of care

BMC Geriatrics

ISSN: 1471-2318

• Submission enquiries: [email protected]
• General enquiries: [email protected]

Application of Fuzzy Decision Support Systems in Risk Assessment of Southeast Asian Labor Market

• Research Article
• Open access
• Published: 19 June 2024
• Volume 17 , article number  153 , ( 2024 )

Cite this article

You have full access to this open access article

• Zujie Zhang 1  

The labor market in Southeast Asia has several complicated issues, including the quick changes in the economy, the various behaviors of employees, and the shifting regulatory settings. To foster sustainable regional development and make decisions based on accurate information, a comprehensive risk assessment is important. Conventional methodologies, on the other hand, often fail to reflect the multifaceted personality of issues about the labor market. In light of recent economic changes and the complexity of applicable policies, this paper recommends a fuzzy decision support system to enhance risk assessment (FDSS-ERA) that could enhance risk assessment in Southeast Asian labor markets. The model methodically assesses hazards, using a mixture of fuzzy logic sense and decision assistance, incorporating changes in employment trends and demographics. The results highlight fuzzy logic’s role in decision-making for effective risk management and policy interventions, showing improved risk comprehension. While educated policy choice making can lead to equitable development in Southeast Asia’s labor work marketplace, FDSS is a potential method. In conclusion, to overcome the many obstacles that stand in the way of chance assessment in the labor market in Southeast Asia, FDSS provides a powerful strategy. Because it offers more advanced analytical tools, this research assists decision-makers in the region in developing more effective guidelines, proactively reducing risks, and achieving projects that promote sustainable development.

Avoid common mistakes on your manuscript.

1 Introduction

Rapid economic developments, shifting regulatory landscapes, and unique worker traits are some of the features that define the labor market in Southeast Asia [ 1 ]. Consequently, the atmosphere to which the stakeholders were exposed was difficult. In this quickly developing industry, it is very necessary to carry out realistic risk assessments to ease the making of informed decisions and maintain sustainable growth [ 2 ]. The conventional methods of risk assessment, on the other hand, have been unable to adequately address the multifaceted and varied constraints that exist within the labor market of Southeast Asia [ 3 ]. Several regional economic conditions are subject to sudden shifts, which constitute a considerable barrier [ 4 ]. Several factors contribute to variations in the labor market [ 5 ]. These factors include global economic trends, technical breakthroughs, and geopolitical concerns that influence agriculture, production, and services. Regulatory frameworks are still in the process of being constructed, which further complicates the predicament. The principles, laws, and rules that regulate labor law are difficult for governments to adjust to accommodate the constantly shifting social, economic, and environmental situations. The strategies for managing labor and risk exposure are significantly impacted as a result of these alterations [ 6 ].

Additionally, the labor market in Southeast Asia is different due to the various workers present there, and these workers cover specific knowledge ranges, educational qualifications, and cultural contexts [ 7 ]. Using state-of-the-art methodologies that go beyond conventional risk assessment methods is necessary to address the risks connected with this diversity effectively. These risks include variations in competencies, language barriers, and cultural differences. To tackle these troubles, it is essential to employ inventive techniques that may alter the ever-changing and complicated characteristics of the labor market in Southeast Asia. Hence, investigating and using advanced methodologies, including FDSS, provide promising possibilities for augmenting risk evaluation methodologies within the area.

The shifting dynamics of the job market and gender balance are using pregnant women to look for riskier work situations. There are gaps in research regarding pregnancy-related exposures, together with physical variables and work absence, and their effect on the health of each mother and child. The existing studies on the correlation between physically demanding jobs and negative pregnancy consequences are scarce, and there is a lack of expertise concerning the impact of work absence at some stage in pregnancy [ 8 ]. This research investigates the economic ramifications of disasters in the labor market, emphasizing the results from medium- to lengthy-term periods. A simulation model primarily based on agent-primarily based methodology explores two conditions following an earthquake in Jerusalem. The findings indicate that labor mobility is essential in reducing failure outcomes [ 7 ]. This research examines the human resource management (HRM) methods in five Asian countries, highlighting their difficult nature and the desire to realize contextual elements very well. Future research must inspect unique determinants, conduct comparative research, and do longitudinal research to recognize the dynamic nature of human resource management (HRM) in Asia [ 9 ]. This study analyzes the migration styles of younger Europeans in Asia, highlighting their usage of motility to navigate unconventional migration routes. The text underscores the numerous encounters and difficulties these migrants encounter, underscoring the significance of mobility in migration choices [ 10 ]. This article examines the sociological ramifications of the COVID-19 epidemic on the South Asian region, emphasizing its exceptional characteristics and the diverse variety of societal consequences it has engendered.

It emphasizes instant, intermediate, and extended measures for handling and recuperating from the pandemic. Nevertheless, it acknowledges the existence of research deficiencies and advocates for empirical investigations and comparative examinations [ 11 ]. This research investigates Malaysia and Thailand’s economic development, with a selected focus on the middle-income trap and the impact of the Fourth Industrial Revolution in facilitating expanded income levels. The text highlights the importance of establishing institutional frameworks that assist, foster human capital development, and enforce proactive policy interventions [ 12 ]. According to a study, the COVID-19 pandemic has expedited the technique of de-globalization, placing more emphasis on resilience instead of performance. It has had significant implications for international industrial networks, tactics for coping with delivery chains, and patterns of commerce. This transition impacts multinational corporations, necessitating research on the effects of regionalization, government regulations, and the evolution of value chains [ 13 ]. A study conducted in Mexico demonstrates a robust correlation between remote employment and job security during pandemics. However, economically disadvantaged families encounter reduced opportunities due to restricted access to finance. The study recommends further research on coverage interventions, longitudinal research, and comparative tests [ 14 ]. The study examines the kind of social impact bonds (SIBs) that fund active labor market programs in four international European locations. It uncovers political conflicts, the connection between civic and economic interest, and variations due to historic institutional contexts and ongoing welfare state reform [ 15 ]. The current body of research identifies several regions that require additional investigation. These areas encompass occupational health for pregnant women, the outcomes of disasters on labor markets, human resource management practices in Asia, migration patterns, the socio-economic results of pandemics, economic development, globalization trends, and social impact financing. Those gaps highlight the need to develop better state-of-the-art analytical tools to address complicated conditions properly. As described, the FDSS to enhance the risk assessment model is a practical technique for dealing with the deficiencies of existing studies.

The FDSS-ERA model, which integrates decision support methods with fuzzy logic ideas [ 16 ], may effectively manage the complexity and ambiguity shown in different research. The offered framework provides a methodical way to evaluate various risk factors, such as the following: occupational hazards for pregnant women [ 17 ], disaster economic effects [ 18 ], HRM practices in many cultural contexts [ 19 ], migration trends, pandemic responses, economic improvement techniques, globalization advancements, and social effect financing mechanisms [ 20 ]. The FDSS-ERA model incorporates subtle data often missed by other approaches, allowing for a full risk assessment. It equips decision-makers with stronger analytical tools, allowing them to make informed decisions. The FDSS-ERA model can potentially study objectives in numerous fields and promote evidence-primarily based decisions for sustainable development by resolving the limitations of current studies techniques and facilitating a greater complete comprehension of intricate phenomena.

Liu [ 21 ] aimed to tackle the difficulties in accurately capturing similarity by introducing sixteen novel similarity metrics for Fermatean fuzzy sets (FFSs) prompted by Tanimoto and Sørensen coefficients. The processing of fuzzy data gathered from FFSs is more suitable via the study’s methodologies, leading to more significant outcomes in recognizing patterns, medical diagnosis, and analysis of clusters. The computational complexity and scalability problems, though, persist. Kahraman [ 22 ] aimed to simplify the determination of membership, non-member, and indecision degrees by proposing proportional fuzzy sets as a substitute for extensions of ordinary fuzzy sets. More stable value assignment is made possible by proportional relations between parameters. Results show that it is easy to use and that they are legitimate. However, there may be problems with scalability and processing complexity. Addressing the difficulty of clustering incomplete instances, Liu and Letchmunan [ 23 ] provide an upgraded fuzzy clustering algorithm that utilizes Dempster–Shafer theory (DST). Subsets of the dataset are used to fill in missing values by utilizing neighbors with varying weights. Iteratively optimizing an objective function finds each subgroup’s optimal membership and dependability matrices. The adaptive evidence-combining method is designed based on DST to handle uncertainty and imprecision efficiently. This rule combines sub-results with varying reliability levels. The method’s efficacy is proven on real-world datasets by comparing it to existing methodologies.

Liu [ 24 ] suggested two Hellinger-inspired Pythagorean fuzzy distance metrics to quantify the dissimilarities across Pythagorean fuzzy sets (PFSs). Comparative cases demonstrate their higher performance. Moreover, a novel approach to decision-making is created and tested in two scenarios. Sensitivity to parameter choice and computational complexity are two potential limitations. Using the Mamdani model fuzzy inference system logic, Charolina and Fitriyadi [ 25 ] evaluated the level of satisfaction with the service regarding information clarity, officer competency, and facility availability. This is intended to aid agencies’ decisions regarding service quality. Future studies will establish an interface for processing results and increase the number of factors to improve the accuracy of assessments. Putra et al. [ 26 ] suggested that CV. Bangkit Mandiri Sejahtera (BMS) Semarang implemented an automated system to recruit employees to reduce the time and effort spent on manual hiring. Experts are chosen using the AHP approach in a decision support system. The selection process takes less than a month, and applicants receive comments and rankings instantly. Some restrictions may apply, such as the need for precise user input.

Kanj et al. [ 27 ] presented a new method for the secure conveyance of hazardous materials in smart cities utilizing real-time data stored in the cloud. It uses a hybrid approach to risk analysis called AHP–TOPSIS, which stands for fuzzy analytical hierarchy process and technique for preference of order by similarity to ideal solution. While in transit, the values of the criteria—which include time, money, and danger—may change, necessitating dynamic decision-making. The results show that safety was enhanced by reducing risks. One potential limitation is the need for computational resources and precise real-time data for dynamic decision-making. Al-shami et al. [ 28 ] presented “( m , n )-Fuzzy sets,” a general framework on orthopedic fuzzy sets that permits alternative weights for both membership and non-membership classes. In it, we find the ground rules for operations, learn about abstract attributes, find ranking functions, and create aggregation operators. An example using numerical data shows how the suggested method works for decision-making situations with several criteria. Challenges that arise in practical settings can be a limitation.

The goal of the study is to investigate the FDSS-ERA model’s efficacy across a variety of research topics. The study suggests employing the FDSS-ERA model to tackle various complex challenges [ 29 ] in fields including occupational health, disaster impact assessment, procedures for managing human resources, migration patterns, pandemic reactions, economic development tactics, globalization patterns, and social impact financing systems. This approach aims to address the existing gaps in research. The FDSS-ERA model gives a systematic framework for evaluating risk elements and facilitating informed decision-making by merging fuzzy logic notions [ 30 ] with decision support processes. The essential objectives of this study are:

To broaden the FDSS-ERA, a robust model for assessing labor market risk in Southeast Asia, integrating fuzzy logic principles with decision support methods.

To assess labor market risks in Southeast Asia, including employment trends, demographic shifts, skill shortages, and regulatory changes, aim to identify and quantify their significance.

To evaluate the FDSS-ERA model’s effectiveness and superiority with other risk evaluation frameworks like fuzzy rule-based systems, multi-criteria decision making and fuzzy Petri nets.

To provide actionable insights for decision-making within the labor market, figuring out vulnerabilities and risk hotspots, allowing resource allocation and targeted interventions.

To enhance threat assessment methodologies in labor markets by introducing progressive tactics and fuzzy logic concepts, enhancing accuracy, adaptability and comprehensiveness, promoting sustainable development and resilience in the region.

To evaluate the impact of natural disasters on labor markets and explore the FDSS-ERA model that could enhance disaster impact analysis and mitigation techniques.

To investigate the contextual impacts on human resource management practices in numerous Asian nations and analyze the FDSS-ERA model, which could contribute to a higher understanding of HRM dynamics.

Through these goals, the study aims to illustrate the flexibility and effectiveness of the FDSS-ERA model in addressing complex risk assessment-demanding situations and informing evidence-based decision-making across various study domains.

1.1 Revised Major Objectives

Generate a state-of-the-art fuzzy decision support system to enhance risk assessment (FDSS-ERA) specific to the complex dynamics of Southeast Asian labor markets; this model will emphasize risk appraisal and proactive risk mitigation.

Exposing the FDSS-ERA model’s precision, flexibility, and capability to handle the compound problems linked with the Southeast Asian employment market proves that it is more effective than traditional risk assessment protocols.

Concentrating on determining weaknesses, risk hotspots, and opportunities for sustainable growth via advanced risk evaluation approaches to provide evidence-based suggestions and actionable insights into labor market decision-makers.

1.2 Motivation of the Study

The rationale for examining this study arises from the pressing necessity to tackle complex troubles in diverse fields, such as occupational health, catastrophe mitigation, human resource management, migration developments, pandemic responses, economic development, globalization, and social impact finance. Recently, those areas have encountered remarkable uncertainties and intricacies, requiring innovative methodologies for evaluating risks and making informed decisions. Conventional techniques frequently fail to seize the complicated and diverse character of risks in these regions, emphasizing the importance of state-of-the-art analytical tool. The research is driven explicitly by the capacity of FDSS-ERA to offer decision-makers an in-depth and delicate understanding of uncertain and ever-changing conditions. FDSS-ERA uses fuzzy logic concepts to comprehend risk factors better, facilitating proactive change mitigation strategies and informed policy formation. The motivation is more suitable by spotting deficiencies in cutting-edge scholarly works, emphasizing the need for pragmatic remedies and views to address international troubles, including the COVID-19 pandemic, natural calamities, monetary shifts, and social disparities. This study seeks to address the gaps in prevailing studies, facilitate sustainable improvement campaigns, and enhance academic understanding and practical implementation in hazard evaluation and decision-making domains. This endeavors to positively affect resilience, innovation, and development in various industries and areas. The study seeks to showcase the adaptability and effectiveness of the FDSS-ERA model in tackling elaborate risk assessment problems and guiding evidence-based decision-making in various fields of study.

The Southeast Asian labor market is intricate, concerning diverse industries, demographic models, and regulatory frameworks, necessitating effective regulatory control for sustainable development.

Traditional risk assessment methods regularly fail to accurately evaluate and mitigate labor market risks, necessitating extra state-of-the-art and adaptable processes for efficient risk management.

Technological advancements in data analytics, artificial intelligence, and decision support systems can improve labor market risk assessment capabilities, resulting in accurate, efficient, and actionable assessments.

The labor market faces new challenges due to economic changes, demographic shifts, technological advancements, and regulatory reforms, necessitating revolutionary risk evaluation tools for handling complicated and uncertain records.

This study observation aims to offer robust and evidence-based insights to public and private sector decision-makers for labor market control regulations, techniques, and interventions via rigorous evaluation and assessment.

1.3 Revised Motivation

The research is encouraged by the demanding need to undertake the complicated and ever-changing problems afflicting the labor market in Southeast Asia. These comprise the region’s firm economic changes, fluctuating demographics, and regulatory dynamics.

To address the current inadequacies in risk evaluation approaches and decision-making processes, this research presents the novel FDSS-ERA model. This is driven by the need to provide stakeholders with cutting-edge analytical tools to help them deal with hesitations, decrease risks, and promote long-term regional development.

Through its thorough and advanced methodology, this study aims to reinforce varied Southeast Asian industries, enhance awareness of labor market risks, and encourage proactive risk management estimation.

1.4 Contributions of the Study

The study introduces the new concept of FDSS-ERA that appreciably contributes to danger assessment, selection assist structures, and policy improvement. This study provides substantial insights and realistic recommendations for addressing tough barriers in contemporary dynamic surroundings by exploring the implementation of FDSS-ERA in many fields. The FDSS-ERA, a unique approach for comprehensively analyzing risks in several areas, including occupational health, disaster mitigation, and economic growth, is developed and used in this work, which offers a significant addition to the field. The FDSS-ERA framework contributes to a more comprehensive understanding of risk issues by merging fuzzy logic concepts into decision support applications. Employing standard evaluation methods helps decision-makers capture the complexities and uncertainties that are sometimes overlooked accurately. Additionally, this study demonstrates the practical usage of FDSS-ERA in real-life settings, demonstrating its effectiveness in directing decision-making processes and allowing proactive steps to minimize risks earlier. With this study, academic knowledge is significantly advanced, and actual answers are presented to address the complicated difficulties that modern society is currently facing appropriately.

The remaining parts of the paper are organized as follows: Sect.  2 discusses the fundamental ideas associated with this topic. Section  3 provides an in-depth presentation of the recommended technique for this research. Section  4 presents the findings and includes a discussion of the tests conducted to assess the planned work. The conclusion of the study is presented in Sect.  5 .

To make the analysis more thorough and easier to understand, the author can point out where current approaches fall short and explain how the suggested FDSS-ERA method better assesses hazards to the Southeast Asian labor market.

1.5 Limitations of the Existing Methods

The complexity and unpredictability of Southeast Asia’s labor market landscape make it difficult for traditional risk assessment methods to account for it. Incomplete evaluations may also result from their struggles incorporating varied risk factors, demographic shifts, and regulatory dynamics. Because these approaches might not be flexible enough to adjust to new circumstances, decision-makers might not be able to get valuable insights regarding proactive risk control.

1.6 Advantages of the Proposed FDSS-ERA Method

To thoroughly assess hazards in the Southeast Asian labor market, the FDSS-ERA model combines concepts from fuzzy logic with sophisticated analytics and decision support approaches. It facilitates a complete risk assessment by investigating policy efforts and labor market scenarios through simulation and analysis tools. Thanks to validation exercises, real-world execution, and stakeholder input, the model’s feedback system guarantees continual improvement, creating an adaptive and responsive risk evaluation framework. Furthermore, decision-makers can effectively engage stakeholders, convey risk assessment conclusions, and establish strategic strategies based on educated insights thanks to its visualization and reporting tools.

1.7 Comparative Advantages

The FDSS-ERA model offers a data-driven, more advanced approach to labor marketplace risk assessment than previous approaches, leading to more accurate and reliable evaluations. Its flexibility and responsiveness make it ideal for the complex and ever-changing Southeast Asian employment market, where it helps decision-makers manage risks with timely and relevant information. The FDSS-ERA model can provide a more complicated and realistic risk assessment than deterministic or binary methods because it uses fuzzy logic ideas to account for uncertainty and imprecisions in employment data.

2 Basic Concepts

Fuzzy decision support systems are a subset of decision support systems known as fuzzy decision support systems. These systems use concepts from fuzzy logic to deal with imprecision and ambiguity in decision-making. Traditional decision support systems often struggle when accurately portraying complicated and ambiguous data. This is especially true in settings that are characterized by ambiguity and vagueness. The FDSS, on the other hand, provides an adaptable and robust framework, making it suited for modeling and analyzing systems of this kind.

2.1 Definition of FDSS-ERA

One example of an FDSS application adapted for risk assessment is the FDSS to enhance risk assessment model example. This approach systematically examines and controls risks in various industries, including occupational health, disaster management, and economic development. It does this by combining principles from fuzzy logic with decision support systems. The FDSS-ERA system employs fuzzy logic to represent and handle information that is ambiguous and imprecise. In the mathematical formulation of FDSS-ERA, many essential components are incorporated. These components include the following areas:

2.1.1 Fuzzy Sets

Fuzzy sets are used in the FDSS-ERA program to represent linguistic factors and uncertainty. A membership function is what defines a fuzzy set. This function gives a membership degree to every element inside a certain communicative universe set. In the FDSS-ERA framework, fuzzy sets are established across domains of communication X and Y that represent the input and output variables, respectively. The membership function \(\mu_A (x)\) is used to characterize a fuzzy set A on X , where x is an element of X . A membership function \(\mu_B (y)\) characterizes a fuzzy set B on Y , where y denotes an element of Y .

2.1.2 Fuzzy Rules

There is a subcategory of IF–THEN rules known as fuzzy rules. Fuzzy rules are indicative of expert knowledge or decision-making heuristics. An antecedent, which is the “IF” component, and a consequent, which is the “THEN” element, are both components that are included in every rule. The fuzzy propositions specified over the variable inputs make up the antecedent, whereas the fuzzy set related to the variable output is denoted by the consequent. For example, a fuzzy rule included inside the FDSS-ERA framework may be represented by expression ( 1 ).

The input parameters are represented as x 1 and x 2 , whereas A 1 and A 2 , are fuzzy sets that signify the linguistic value. The output variables, y , is linked to a fuzzy set B .

2.1.3 Fuzzy Inference System (FIS)

The FDSS-ERA system uses fuzzy rules within its fuzzy inference systems to make decisions or draw a conclusion by examining input information. The system contains three primary elements: rule assessment, fuzzification, and defuzzification.

Fuzzification : The procedure includes converting the input value \(x_1 , x_2 , \ldots , x_n\) , into fuzzy sets using the membership functions stated in Eq. ( 2 ).

Rule evaluation : The activation degree of every fuzzy rule i can be identified by evaluating the degree of matching between input values and antecedents. Fuzzy logic operators, comprising the minimum (AND) operator, can calculate this rule assessment.

Defuzzification : The output values y can be determined by aggregating the outputs of activated rules. Different policies, such as the centroid or weighted average defuzzification technique, can complete the defuzzification.

2.1.4 Fuzzy Membership Functions

The establishment of the structure and features of fuzzy sets is greatly influenced by membership functions, which are an enormously essential component. Many other shapes may be identified, including triangular, trapezoidal, Gaussian, and sigmoidal forms. Membership functions are used to quantify the degree to which an element is considered to be a member of a fuzzy set. The amount of ambiguity or vagueness that is related to the linguistic variables may be obtained via the use of these functions. The membership function \(\mu_A (x)\) quantifies the extent to which an element x is a member of the fuzzy set A .

2.1.5 Aggregation Methods

The employment of aggregation techniques entails the consolidation of the outputs of separate fuzzy rules into a single aggregated answer. Maximum, average, minimum, and weighted average are the four standard strategies for aggregates. Using Eq. ( 3 ), one may discover how to compute the aggregated output B .

where w i represents the degree of activation of rule i , while \(\mu_{B_i } (y)\) denotes the membership function of the subsequent fuzzy set associated with rule i .

2.2 FDSS-ERA Operation

The FDSS-ERA’s operating technique includes an initial specification of linguistic variables and the membership functions that correspond to those variables. The linguistic variables under examination are input and output elements pertinent to the risk assessment problem. The use of specialized information or heuristics that are particular to the circumstance generates a compilation of intuitive suggestions. The rules that were mentioned previously contain the criteria for risk assessment as well as the arguments that are used to justify making judgments. When determining the membership level after each rule, FDSS-ERA uses fuzzy regulations to analyze the input facts to conclude. First, the input data are fuzzified, then fuzzy rules are assessed, and lastly the rule outputs are consolidated. The process is said to be complete once all these steps are completed. Defuzzification is a procedure that creates a basis for decision-making or risk assessment once completed. The combined fuzzy output is translated into a precise value for the transformation. This is how it is performed. To successfully handle the issues of uncertainty and imprecision typical of data from the real world, a risk assessment approach known as the FDSS-ERA methodology has been identified. This strategy was developed to make the risk assessment process more smooth. Additionally, it has expertise of expert systems and can model very complex risk scenarios. The versatility and efficiency of the approach may be beneficial to a variety of fields, including occupational health and safety, disaster relief, economic development, and environmental risk assessment at the same time.

The FDSS-ERA framework is a powerful instrument for enhancing the number of methods for risk assessment. Fuzzy logic is used to accomplish this goal. Providing a mathematical representation of this idea makes it much easier to explain and evaluate ambiguous and imprecise material. When confronted with challenging situations that require decision-making, those responsible for making judgments can acquire beneficial insights and receive assistance. FDSS-ERA has the potential to be a successful solution when it comes to tackling the complicated challenges connected with risk assessment in various disciplines. This is because it can function in various contexts and is adaptable.

Fuzzy sets : Since fuzzy sets represent ambiguity and uncertainty in data, they are crucial to fuzzy logic and systems that support decisions. Every element in a set can be specified as having varying degrees of membership according to a membership function, which constructs a fuzzy set. The degree to which an element belongs to a set can be indicated by a partial membership value that ranges from 0 to 1 in fuzzy sets, as opposed to classical sets, in which an element can only be either part of the set (1) or not (0).

Mathematical representation of fuzzy sets : Fuzzy sets are used to characterize language variables and uncertainties associated with labor market risk factors within the framework of the FDSS-ERA model. To define a function of membership that describes the degree of participation of every component in the set, the mathematical procedure for creating a fuzzy set is performed. The type of the modeled variable determines the possible shapes of this membership function, which might be triangular, trapezoidal, Gaussian, or sigmoidal.

To illustrate, consider a “Low Risk” fuzzy set for the degree of danger linked to a specific thing in the labor market. As an example of a membership function, consider the following Eq. (4) showing the fuzzy set:

Each value of x is given a level of membership by the function known as \(\mu_{\text{LOW - RISK}} (x)\) . In this example, the value indicates the degree to which it corresponds to the “Low Risk” group. As the value becomes closer to 0, it suggests that the degree of participation within the “Low Risk” group is more significant. As the value gets closer to 1, it indicates that membership is smaller.

Some important characteristics characterize the labor market in Southeast Asia. These qualities include a diverse workforce, rapid economic growth, and dynamic regulatory systems. Considering the intrinsic complexity of the environment, this research aims to evaluate the use of FDSS to improve risk assessment in particular. This endeavor attempts to design comprehensive risk assessment tools that may give considerable insights for decision-making and stimulate sustainable development. This is despite that regional dynamics are always shifting.

3.1 Development of the FDSS-ERA Model

The primary objective of this research is to modify the FDSS-ERA model in such a way that it is specifically tailored to the labor market for Southeast Asian countries. To properly complete this process, it is necessary to include fuzzy logic concepts into well-established decision support systems. With the assistance of several different approaches, the FDSS-ERA model is designed to manage the intricate nature of the labor market dynamics effectively. Alterations in employment patterns, shortages of skills, shifts in demography, and regulatory frameworks are all factors considered. Specifically, to systematically evaluate and mitigate the risks associated with the Southeast Asian labor market, several essential components of the FDSS to enhance the risk assessment model have been developed. It is possible to see a graphical depiction of these components in Fig.  1 , which may be found here.

Architecture of the FDSS-ERA model

The core component of the FDSS-ERA model is the input data layer that consolidates diverse information about the Southeast Asian labor market. The databases encompass a range of pertinent aspects, such as employment patterns, demographic changes, deficiencies in skills, regulatory structures, economic indicators, and other relevant variables—the input data layer functions as the fundamental basis for evaluating risks and making decisions within the model. The input data layer, denoted as X , encompasses a range of datasets about the labor market in Southeast Asia, as depicted in Eq. ( 5 ).

where, X i , denotes specific datasets, such as employment patterns, demographic changes, skill deficiencies, etc. To successfully handle and analyze the incoming data, the FDSS-ERA model employs a fuzzy logic engine. Fuzzy logic facilitates the model’s ability to effectively address imprecise, unclear, and ambiguous information inherent in the labor market dynamics. The ambiguity and complexity of the data on the current state of the labor market may be effectively managed by the engine via fuzzy sets, linguistic elements, and fuzzy rules. Thus, presenting conclusions from risk assessments that are more exact as a result is feasible. “F” is the abbreviation for the fuzzy logic engine applied in FDSS-ERA. Fuzzy sets, linguistic variables, and fuzzy rules are used in processing the input data X by the fuzzy logic engine. This function may be represented by the notation F ( X ).

The primary objective of the risk assessment module is to evaluate possible risks in the labor market scientifically. This is accomplished via the use of processed data and the implementation of fuzzy logic analysis. The system uses predetermined risk indicators and algorithms to determine the possibility of various possible threats. These threats include fluctuations in unemployment, shortages of skilled workers, changes in rules, and modifications in demographic profiles. The software module develops risk profiles and assigns priority levels depending on the severity and urgency of the concerns. This makes it simpler to conduct additional investigations when doing further investigations. The risk assessment module, R , systematically examines labor market hazards using data processed from the fuzzy logic engine F ( X ). The system utilizes predetermined risk indicators and algorithms to evaluate the probability and consequences of different risks.

Using the decision support module, the outcomes of risk assessments may be integrated into decision-making processes more straightforwardly. This tool aims to give decision-makers practical insights, ideas, and scenarios by analyzing the risks that have been uncovered and the potential consequences that these risks may have on the labor market. Instruments for visualization, scenario analysis, and sensitivity testing are some of how the module assists stakeholders in building effective risk reduction strategies and making well-informed decisions. Lettered with a D , D ( R ( X )) allows for integrating the decision support module with the risk assessment outputs. Decision-makers may benefit from this tool by gaining practical insights, suggestions, and scenarios that are based on the identification of risks and the possible repercussions of those risks. S serves as a symbol for the tools used for scenario analysis and simulation. The simulation of various labor market circumstances, policy efforts, and economic scenarios is made possible by S ( D ( R ( X ))), which allows stakeholders to do so. To aid in the proactive identification and reduction of risks, it evaluates their effect on the degree of risk exposure and the capacity to recover.

The FDSS-ERA system also incorporates technologies used for scenario analysis and simulation to study the likelihood of various risk scenarios and their repercussions. Through simulations of various labor market situations, policy efforts, and economic scenarios, decision-makers may be assisted in analyzing the risk exposure and resilience more effectively. Stakeholders may use this ability to assist them in actively anticipating and managing growing risks before they become significant challenges. The model’s capabilities regarding output visualization and reporting operations allow it to explain the risk assessment findings adequately. To deliver short summaries of major discoveries, trends, and risk profiles, those in positions of authority can use dynamic dashboards, reports, and visualizations. Because of these insights, the efficacy of communication, the participation of stakeholders, and the formation of strategic plans at various levels of government and industry have all increased.

The FDSS-ERA’s performance and relevance is continuously improved by its feedback system. Validation exercises, experiences of implementation in the real world, and input from stakeholders that have been obtained systematically are all considered throughout this phase. Since the needs of the labor market in Southeast Asia are always changing, it is presumed that the model will continue to be adaptive, flexible, and in line with those requirements. This is because these techniques are iterative, guaranteeing that the model will remain adaptable. In conclusion, the architectural design of the FDSS-ERA model is an example of a strategy that is both complete and unified in its approach to assessing the risk that is associated with the labor market in the Southeast Asian area. This makes the model an excellent example of a strategy that is both comprehensive and unified. The model uses fuzzy logic rules, cutting-edge analytics, and decision support methods to provide stakeholders with the capacity to make well-informed decisions, efficiently handle risks, and produce sustainable development in the context of the shifting labor market environment in the area.

3.2 Evaluation of Labor Market Risks

Evaluating the risks associated with the labor market within the framework of the FDSS-ERA paradigm is one of the most significant components of this research endeavor. This research aims to comprehensively understand the intricate challenges the Southeast Asian labor market faces. A comprehensive examination of the many risk variables that impact the labor dynamics within the area will be carried out as part of this research project. Fuzzy logic ideas are included in this study, which utilizes advanced procedures. Figure  2 visually depicts the assessment components that affect the labor market dynamics.

Evaluation factors influencing labor market dynamics

The complicated dynamics of the labor situation in Southeast Asia are investigated in this study to understand the issue better. Several elements are considered, including the creation of new jobs, shifts in industry, and unemployment rates. This work studies employment trends to identify developing patterns and possible vulnerabilities within the labor market. The employment level ( E ) at a specific period ( t ) is denoted by Eq. ( 6 ), which calculates the sum of the initial employment level ( E 0 ) and the change in employment over time ( \(\Delta E(t)\) ).

Recognizing developing trends and potential weaknesses within the labor market can be achieved by examining the trend of \(\Delta E(t)\) . The labor market dynamics are suggestively impacted by demographic variations, encompassing alterations in population structure, age distribution, and workforce structure. This research investigates the impact of demographic changes, specifically population aging or young people bulges, on labor supply, dell demand, and stability within the Southeast Asian market. Equation ( 7 ) represents the rate of change of the population ( P ) over time ( t ) that is calculated as the difference between the birth rate ( B ) and the death rate ( D ).

Demographic shifts, such as the aging of the population \(\left( {\frac{{{\text{d}}P}}{{{\text{d}}t}} < 0} \right)\) or the emergence of youth bulges \(\left( {\frac{{{\text{d}}P}}{{{\text{d}}t}} > 0} \right)\) , have the potential to impact labor supply, demand, and the market’s overall stability in Southeast Asia. The Southeast Asian labor market has substantial issues due to skill shortages and mismatches that negatively impact efficiency, competitiveness, and hiring decisions. This study investigates the frequency of skill deficiencies in different sectors and evaluates their consequences for advancing the workforce, economic expansion, and social unity.

The summation of the disparities between the demand for skills determines the skill gap ( \(D_i\) ) and the supply of skills ( \(S_i\) ) across many industries ( n ), as expressed by Eq. ( 8 ). A positive skill gap signifies a deficiency in skills, whereas a negative gap signifies an excess of skills. Examining the skill disparity aids in evaluating the consequences of advancing the labor force, economic expansion, and societal unity.

The influence of regulatory changes on labor circumstances ( Q ) about changes in policy parameters ( P ) is represented by Eq. ( 9 ). Evaluation of the consequences of labor market reforms or policy interventions on worker rights, market effectiveness, and general employment situations in Southeast Asia may be accomplished by decision-makers via the examination of the impact coefficient. Because they affect employment policies, labor laws, and firm practices, regulatory frameworks considerably impact the labor market dynamics. The purpose of this research is to evaluate the influence that regulatory changes, such as labor market consolidations or policy interventions, have had on the workers’ rights, employment conditions, and the market’s general efficiency in Southeast Asia.

In the context of fuzzy logic, the assignment of membership functions ( \(\mu (x)\) ) to variables is described by Eq. ( 10 ), which signifies the extent to which a value ( x ) is considered valid or belongs to a fuzzy set. The model may be able to successfully give fuzzy membership values to reflect the ambiguity and imprecision of the data about the labor market. Because of this, it is possible to conduct a more comprehensive analysis of the hazards that are now in existence. The study uses fuzzy logic methodologies in risk assessment because of the inherent complexity and uncertainties in the labor market data. Incorporating data from the labor market that includes inherent imprecision and ambiguity into the model is made simpler by fuzzy logic. This makes it feasible to undertake a more comprehensive examination of hazards, which conventional risk assessment techniques could ignore.

The summation of the weighted scores determines the risk score ( \(S_i\) ) of different risk factors ( n ), as expressed by Eq. ( 11 ). A weight ( \(w_i\) ) is applied to each risk factor based on its significance. To give credible and comprehensive results in risk assessment, the study undertakes an in-depth investigation of several different datasets and indicators. These findings may be utilized to drive policy-making and strategic measures. A complete analysis of many datasets and indicators is required when an evaluation of the risks connected with the labor market is being conducted. This analysis must take into consideration both quantitative and qualitative risk components. To ensure the generation of risk assessment findings that are dependable and trustworthy, the study makes use of rigorous procedures and analytical tools. Following that, these findings may be used to affect decisions about policy and strategic objectives.

Equation ( 12 ) shows the vulnerability index and is determined by dividing a risk’s potential impact by the labor market’s adaptability. With a higher vulnerability score, the system is more vulnerable to prospective threats and probable risk hotspots. This makes the system more sensitive to potential risks. To increase the market’s resilience in the Southeast Asian region, decision-makers can prioritize the allocation of resources appropriately, carry out focused interventions, and adopt proactive policies when they begin by identifying vulnerabilities. A method of evaluation is used in the study to determine significant vulnerabilities and possible areas of high risk within the labor market of Southeast Asia. Decision-makers can allocate resources effectively, put specific plans into action, and take preventative steps to control risks and build market resilience when they identify areas of concern and then apply those plans.

Evaluating the risks connected with the labor market within the context of the FDSS-ERA paradigm offers significant insights into the complex challenges impacting labor dynamics in Southeast Asia. This evaluation is conducted from a broad perspective. This study significantly contributes to a better understanding of the vulnerabilities within the labor market by using sophisticated analytical approaches and conducting a comprehensive analysis of a wide range of risk factors. It gives helpful insights that may be used in implementing evidence-based policy necessary for achieving sustainable regional development.

3.3 Integration of Fuzzy Logic Principles

Compared to other risk assessment frameworks that are more traditional, the FDSS-ERA model stands out because it combines notions of fuzzy logic as fundamental components. The core of the FDSS-ERA, which ensures dependability, is fuzzy logic, well-known for its ability to handle imprecise, convoluted, and confusing information. Southeast Asia’s labor market is characterized by its complexity, quick changes, and various worker dynamics. This is a well-known fact. When seen in the context of this labor market, integration has several major effects. As a result of its use of fuzzy logic, the FDSS-ERA can deal with the inherent ambiguity and imprecision often seen in data pertinent to the labor market. Fuzzy logic, on the other hand, considers and acknowledges the nuances present in real-world situations. This contrasts with the traditional binary logic systems, which depend on clear and exact differentiations. Within the context of the dynamics of the labor market, fuzzy logic stands out as an approach to analysis that is both more flexible and more sophisticated. There is a possibility that the data are insufficient, unclear, or susceptible to interpretation.

Because it uses fuzzy logic concepts, the FDSS-ERA model can overcome the limitations inherent in conventional risk assessment approaches. It is common for this method to struggle to capture the complex and multifaceted character of labor market dynamics. The model uses fuzzy sets, linguistic variables, and fuzzy rules to define and manage data relevant to the labor market. This method is an alternative to relying only on clear and predictable rules. The FDSS-ERA can generate more sophisticated risk evaluations due to the gradual and hazy transitions between the different states and outcomes. In addition, fuzzy logic provides the FDSS-ERA model with more flexibility and resilience, which helps it to adapt better to the dynamic labor market in Southeast Asia. This is an additional benefit of fuzzy logic. The skill to effectively handle conditions that are ambiguous and are in a state of perpetual change is of the highest significance, especially when taking into consideration the dynamic economic changes, demographic transformations, and regulatory adjustments that are taking place in the area. The FDSS-ERA makes it possible for decision-makers to have access to information that is both timely and relevant. It is feasible to do this via fuzzy logic, which enables real-time dynamic change of its analysis and recommendations.

To further encourage an approach to risk assessment that is more all-encompassing and inclusive, the FDSS-ERA model also incorporates fuzzy logic concepts. Through the use of this strategy, the need for reductionism is removed, and the problem is simplified, which enables a comprehensive understanding of the dangers that are involved. To do this, it acknowledges the preexisting ambiguities and uncertainties in the data about the labor market. By conducting a thorough analysis of the dynamics of the labor market, decision-makers may develop the capacity to make well-informed decisions and improve their odds of success. The varying degrees of ambiguity and complexity inherent in Southeast Asia’s environment are considered to achieve this study investigation’s objectives. To summarize, a considerable advancement has been made by introducing elements of fuzzy logic into the FDSS-ERA model. In the context of the labor market in Southeast Asia, this development makes it possible to conduct risk assessments that are more exhaustive, flexible, and resilient than previously possible. Decision-makers are provided with a powerful tool to negotiate the complexities of the dynamics of the labor market and make well-informed decisions, which this model provides. The model provides Decision-makers with this strong tool rather than attempting to eliminate uncertainty and ambiguity (Table  1 ).

A systematic technique used to assess the dynamics of the labor market using concepts from fuzzy logic is the FDSS-ERA model algorithm, shown in Table  1 . All of the processes that are included in the process are as follows: initialization, preprocessing, fuzzification, evaluation of rules, aggregation, defuzzification, postprocessing, evaluation of the model, iteration and optimization, and output. Using linguistic variables, fuzzy sets, and rules, the algorithm performs the following tasks: the review of past information; the determination of membership in fuzzy sets; the assessment of rules; the computation of fuzzy output values; the conversion of fuzzy output into precise values; the analysis of defuzzified output; the assessment of model performance; the optimization of variables; and the delivery of last risk assessment outcomes and recommendations to stakeholders. All of these features are accomplished through the utilization of fuzzy sets. This technique comprehensively analyzes the dangers associated with the labor market to generate insights that may affect decision-making processes.

3.4 Application of the FDSS-ERA Model

The FDSS-ERA model and its implementation in real-world labor market situations have been developed and applied in Southeast Asian nations. The simulated exercises and case studies used to complete the empirical verification of the utility of FDSS-ERA in recognizing and mitigating risks in the labor market are shown here. Through the use of the model in a wide range of various scenarios, the study demonstrates the flexibility of the idea as well as its practical value. By doing so, the organization proves its ability to influence decision-making processes at various levels of government and industry all around the globe. The results show that FDSS-ERA is a good model for improving risk assessments in the Southeast Asian labor market. Through the use of fuzzy logic techniques, this model was able to successfully capture the inherent complexity and uncertainty that is present in the dynamics of the labor market. Fuzzy logic was used to attain this goal successfully. In addition, as a result, those who make decisions have access to considerable insights that may be used for proactive risk management and the formulation of policies. FDSS-ERA is one of the complex analytical procedures that should be used to overcome the considerable risk assessment challenges. The findings of this study throw even more light on the reason for this need. The implementation of FDSS-ERA has the potential to significantly serve as a guiding principle for the formulation and implementation of policies within the labor market in Southeast Asia. The possibilities are just enormous! Those in charge of making choices are given the power to make better-informed decisions, which allows them to create proactive steps to decrease risks and improve efforts for sustainable growth. There has been an increase in the competence to assess vulnerabilities over this period. Advanced analytical approaches, such as FDSS-ERA, can stimulate an increased capacity to endure and respond to changing circumstances in the labor market and an extra piece of unfavorable news.

To properly manage the complicated issues related to risk assessment within the Southeast Asian labor market, this section highlights the significance of applying innovative approaches such as FDSS-ERAs. The fuzzy decision support system (FDSS-ERA) is a comprehensive framework that integrates fuzzy logic principles with decision support methodologies. Its purpose is to enhance risk assessment abilities and provide information that can be employed to assist decision-making processes based on evidence-based information. The implementation of the FDSS-ERA is an essential component that makes a significant contribution to the promotion of sustainable development and the building of resilience within the labor market of Southeast Asia. Consequently, many stakeholder groups, which include several different types of businesses and sectors, stand to gain from it.

As shown by the study’s findings, assessment has progressed: the results suggest that the FDSS-ERA model may produce more accurate risk assessments by considering various variables and uncertainty intrinsic to the Southeast Asian labor market. This enhanced precision allows for more informed decisions. According to the study’s findings, the FDSS-ERA model provides a more thorough assessment of labor market hazards by using fuzzy logic concepts to identify subtleties and nuances that conventional approaches could miss. A more complete picture of risk variables may emerge from such all-encompassing monitoring. The findings may show that the FDSS-ERA approach improves decision support by giving stakeholders actionable insights, suggestions, and scenarios grounded on the assessed risks. Better decision-making assistance can help people take charge of risk management and develop winning ideas.

The study’s findings may reveal that the FDSS-ERA model’s fuzzy logic engine makes it better suited to deal with complexities and uncertainties in labor market dynamics. The model’s flexibility enables it to process uncertain data and produce improved risk assessment outcomes. A study of the results could reveal that the FDSS-ERA framework is superior to various risk assessment models because of how well it handles noisy data, how supportive it is of decision-making, and how comprehensive its risk assessments are overall. By contrasting the two, we can see how much better the FDSS-ERA model is at enhancing risk assessment. By highlighting these features in the study’s findings, the authors prove that the FDSS-ERA model improves the assessment of risk in Southeast Asian labor markets, which is useful for researchers, stakeholders, and policymakers interested in analyzing the labor market and risk management.

Within the scope of this research, the FDSS-ERA model is compared to several other models. These models include fuzzy rule-based systems (FRBS), fuzzy Petri nets (FPN), and fuzzy multi-criteria decision-making (MDCM), to name a few examples. As part of this study, the performance of the FDSS-ERA model is also tested in great detail and depth. A dataset will be used to carry out the study [ 31 ]. Employment patterns, demographic transitions, regulatory changes, and differences in skilled sets are some of the numerous facets of the labor market covered in this dataset. This research employs several different assessment procedures to determine whether or not the models are effective. Sensitivity to input variables, resilience to noisy data, decision-making supportiveness, uncertainty management, and cost–benefit analysis are some of the processes that fall under this category. In this part, the performance of FDSS-ERA is also reviewed, along with the performance of several other models, including FRBS, FPN, and MCDM. This research aims to investigate the efficacy and application of the FDSS-ERA model in terms of its capacity to deal with the intricate problems related to the job market vulnerability assessment.

The risk assessment coverage (RAC) for the FDSS-ERA model is shown in Fig.  3 , along with three additional models: the FRBS, the FPN, and the MCDM. This statistic is representative of a broad range of age groups combined. After an exhaustive study of the data, it has become abundantly clear that the FDSS-ERA model is superior to all other models in every respect, irrespective of the age group. This is true regardless of whether the evaluation is carried out on adults or if it is carried out on youngsters. Using the currently available information, the FDSS-ERA model can comprehensively investigate the dangers associated with the labor market. It is vital to offer an unambiguous description of the RAC to make it easier to carry out an evaluation that is more complete. As indicated by Eq. ( 13 ), it is possible to define it as the percentage of potentially risky situations in the labor market that have been identified compared to the total number of potential hazards in each age group.

Risk assessment coverage analysis of FDSS-ERA and other models

Based on the conclusions gathered from the data, it was determined that the FDSS-ERA model provided much higher RAC percentages across all age groups than the FRBS, FPN, and MCDM models. For instance, the FDSS-ERA achieves a rate of acceptance and compliance (RAC) of 89% in the age category of 18–24 years old, while its closest competitor, FPN, only hits 80% in the same age range. This is a significant difference. Regarding identifying potential risks in the labor market, the FDSS-ERA model regularly outperforms other models. This is because it is consistent across all age groups. Because it is consistent, this is shown. The overall performance of the FDSS-ERA model has been enhanced due to the incorporation of fuzzy logic principles and decision support methodologies into the model’s design. As a result of this integration, the model can effectively capture distinctions and uncertainties that traditional models normally overlook. Furthermore, the flexibility of the FDSS-ERA model makes it possible to swiftly respond to the dynamic and diversified environment of the labor market. This is a significant advantage. When this is done, it guarantees that a comprehensive risk assessment is carried out across all age groups.

Coverage of risk assessment (RAC): Eq. ( 13 ) states that RAC is the ratio of real risks to all potential risks. Figure  3 shows the RAC analysis by comparing FDSS-ERA to other models across various age groups.

An illustration of the uncertainty score analysis for the FDSS-ERA model and three comparator models, namely FRBS, FPN, and MCDM, is shown in Fig.  4 . This analysis was performed across several different business sectors. After the study’s conclusion, it is evident that the FDSS-ERA model performs far better than other models when effectively addressing uncertainty. This is because the FDSS-ERA model performs significantly better than other models. One evidence supporting this finding is that uncertainty ratings have gradually decreased across all market sectors. To examine the topic, it is necessary to have a general understanding of the uncertainty score. According to Eq. ( 14 ), represented by this metric, the model can appropriately deal with the inherent uncertainties in the data on the labor market.

Uncertainty score analysis of FDSS-ERA and comparative models

The FDSS-ERA model produces rankings of uncertainty that are much lower across all industrial sectors than the FRBS, FPN, and MCDM models, as shown by the results. As a result of its uncertainty, FPN, the firm that is the most direct competitor to FDSS-ERA, has received a rating of 0.72, while FDSS-ERA has received a rating of 0.78. The FDSS-ERA model demonstrates a similar pattern across all sectors, showing its enhanced capability to deal with uncertainty more favorably. The superiority of the FDSS-ERA model may probably be ascribed to the fact that it integrates fuzzy logic standards. This permits the model to have a more desirable ability to gather and analyze uncertainties and obscurities in the labor market. By using decision support mechanisms, the FDSS-ERA model can enhance its resilience in dealing with uncertainty, even when it is presented with data that is either restricted or ambiguous. To achieve this goal, stakeholders are given instructions and insights grounded in reality. By demonstrating that the FDSS-ERA model efficiently addresses uncertainty in various business sectors, our findings illustrate that a model is a practical instrument that can be used to make well-informed decisions when examining the risk linked with the labor market.

The uncertainty score is a measure that shows how well the model handles uncertainty; it is defined in Eq. ( 14 ) as follows. Figure  4 uses the uncertainty score analysis to compare FDSS-ERA with other industry models.

The overall performance of the FDSS-ERA model in evaluating multiple models (FRBS, FPN, and MCDM) across diverse business sectors is displayed in Fig.  5 , which offers the decision-making supportiveness (DMS) analysis. The findings consistently indicate that the FDSS-ERA model can achieve higher DMS scores across all sectors. This proves the model’s enhanced power to give decision-makers practical insights and recommendations. To initiate the communication, let us set up the DMS score as a metric that quantifies the diploma so that the model gives realistic insights and suggestions for people who are responsible when making decisions. The DMS rating can be computed using the formula shown in Eq. ( 15 ).

Decision-making supportiveness analysis of FDSS-ERA and comparative models

The findings imply that the FDSS-ERA model attains superior DMS rankings throughout all commercial sectors compared to opportunity models. In the manufacturing industry, FDSS-ERA has a much higher score of 0.8 than its closest rival, FPN, which has a rating of 0.75. This indicates that FDSS-ERA is better than its competitors. This demonstrates that the FDSS-ERA model effectively provides decision-makers with relevant insights and pointers since the determined sample remains constant across various sectors. The outstanding overall performance of the FDSS-ERA model may be attributed to combining fuzzy logic standards with decision support methodologies. The model’s capacity to efficiently gather and handle data regarding the labor market, characterized by complexity, uncertainty, and imprecision, is improved by incorporating fuzzy logic into the model. In addition, the decision-making assistance techniques included in the FDSS-ERA model enhance its capacity to transform data into actionable recommendations and insights that have been personalized to meet the specific needs of decision-makers.

A decision-making support system, or DMS, is the ratio of practical insights and recommendations to the overall number of suggestions and insights (as shown in Eq.  15 ). Figure  5 shows the DMS study that compares FDSS-ERA to other industry models.

Regarding coping with remarkable contextual changes, the adaptability to contextual changes (ACC) analysis, represented in Fig.  6 , proves that the FDSS-ERA model is superior to other models (FRBS, FPN, and MCDM). Transformations such as economic tendencies, regulatory circumstances, worker dynamics, and technology improvements are included in these changes. Based on the data, it is possible to conclude that the FDSS-ERA model regularly outperforms other models in every area. This demonstrates the model’s greater potential to adapt to changes in the competitive environment of the labor market. Similarly, the ACC score is used to discover the issue matter as a hallmark of the model’s capability to change and respond effectively to swings in the labor market environment, as indicated by Eq. ( 16 ). This is done to find the factor that caused the problem.

Adaptability to contextual changes analysis of FDSS-ERA and other models

FDSS-ERA obtains superior ACC scores in every class compared to different models, as determined by analyzing the consequences. As a result of the changes in the economy, FDSS-ERA received a score of 90%, which is higher than its nearest rival, FPN, which scored 82%. The FDSS-ERA model exhibits its resilience and ability to alter diverse exertions of market factors, as indicated by its consistent performance throughout all contextual changes.

Equation ( 16 ) describes the ACC score as the proportion of right responses to overall replies, which shows how adaptable the model is to context changes. As shown in Fig.  6 , the ACC analysis compares FDSS-ERA against other models across different settings.

It is possible to credit the outstanding performance of the FDSS-ERA model to the introduction of fuzzy logic ideas, which enables the model to efficiently handle and analyze information that is complex, uncertain, and dynamic about the labor market. As an additional point of interest, using decision support approaches in the FDSS-ERA model enhances its capacity to provide practical insights and suggestions specifically crafted to guarantee contextual adjustments.

This method, which is depicted in Fig.  7 , evaluates the impact of various types of input variables, such as economic indicators, changes in demographics, regulatory changes, and skill mismatches, on the performance of the FDSS-ERA model in comparison to the performance of other models, like the FRBS, FPN, and MCDM. The input sensitivity analysis method is depicted in Fig.  7 . Based on the data, the FDSS-ERA model is recommended to perform consistently better than the models presently utilized across all input variables. Based on the findings of this research, it seems that the FDSS-ERA model offers a greater degree of sensitivity and response to fluctuations in the dynamics of the exertion market. One of the measures used to assess how the model reacts to changes in the input variables is called the input sensitivity score. This measure is expressed in Eq. ( 17 ), which may be found at this location. This definition will make examining the more complete issue simpler.

Input sensitivity analysis of FDSS-ERA and comparative models

After analyzing the data, it has become evident that the FDSS-ERA version has greater input sensitivity scores than the alternative model across various input variables. This is the case regardless of the type of input variable being considered. For example, when assessing the responsiveness to economic indicators, FDSS-ERA scores 92%, surpassing its closest competitor, FPN, which receives a score of 85%. Even though the FDSS-ERA model exhibits a consistent pattern across all input variables, it may be susceptible to swings in economic indicators, demographic shifts, regulatory adjustments, and skill mismatches. There is a possibility that fuzzy logic standards are responsible for the increased sensitivity of the FDSS-ERA model. These standards make it easier to analyze problematic fluctuations and refinements within the information about the occupation market. Through the use of decision support techniques inside the framework of the FDSS-ERA model, the capacity of the model to analyze the impact of various input variables on the outcomes of the labor market is effectively expanded.

The input sensitivity rating measures the model’s responsiveness to input variables. It is defined by the ratio of trade in output to change in enter, as shown in Eq. ( 17 ). Figure  7 indicates the consequences of the Input Sensitivity analysis, which compares FDSS-ERA to different models regarding numerous input variables.

The performance evaluation of the FDSS-ERA model in terms of its ability to handle noisy information across a wide range of risk categories, such as financial, environmental, social, and political aspects, is shown in Fig.  8 . The FDSS-ERA model is compared to similar models, including the FRBS, FPN, and MCDM for this assessment. When the data are considered, it is possible to conclude that the FDSS-ERA model performs better than the others across all risk classes. This emphasizes the model’s better power to cope with instances of noisy data, which enables us to develop the robustness score as a measure that analyzes the model’s potential to retain its performance when presented with noisy data, as indicated in Eq. ( 18 ). In addition, this measure ensures that the model can handle instances of noisy data. As a result, we shall gain more comprehensive knowledge of these findings.

Robustness to noisy data analysis of FDSS-ERA and comparative models

Equation ( 18 ) describes a model’s robustness to noisy information as its performance relative to clean information when dealing with noisy information. The robustness analysis compares FDSS-ERA with other models across various risk types (Fig.  8 ).

The data indicate that the FDSS-ERA model generates greater robustness scores across the board for all risk categories. Upon analyzing the results, this was discovered. Regarding economic risk, the FDSS-ERA has a competency level of 90%, greater than the scores of FRBS, FPN, and MCDM, respectively, 85%, 80%, and 78%. As a result of the fact that the trend described above is consistent across all risk categories, it can be deduced that the FDSS-ERA model has a stronger resilience in noisy data across the board for all aspects of risk assessment. The enhanced robustness of the FDSS-ERA model might be attributed to the incorporation of fuzzy logic notions. Because of this inclusion, the model can handle the uncertainties and imprecisions associated with the data about the labor market. Furthermore, decision support mechanisms inside the FDSS-ERA model assist the model in becoming more robust by including strategies to eliminate unneeded data, gain excellent insights from noisy datasets, and help the model become more resilient.

Stakeholder satisfaction : Evaluating the efficacy of the FDSS-ERA method in enhancing labor market chance assessment is based closely on stakeholder satisfaction. It entails talking with stakeholders to decide what they think about the model and how it may help decision-making. Several strategies exist for quantifying this feedback, together with polls, interviews, or ratings according to installed standards. Using the Likert scale is a standard method of measuring stakeholder pleasure. Using Likert scales, stakeholders can indicate how much they agree or are satisfied with particular claims or queries concerning the model’s performance.

Higher values on the Likert scale, generally from 1 to 5 or 1 to 7, imply more agreement or satisfaction. Collecting and analyzing all stakeholder replies is first-rate to get an experience of how satisfied anybody is. Qualitative techniques, awareness businesses, or interviews are other ways to gauge stakeholder satisfaction. These methodologies allow stakeholders to give particular, in-depth feedback on the model’s deserves, shortcomings, and regions for improvement. Thematic and content material analysis are two techniques for studying qualitative statistics that can help display recurring thoughts and styles in stakeholders’ replies.

Figure  9 is a visible depiction of satisfaction profiles that can be used to enhance the presentation with user-friendly images. Net promoter score (NPS) is one metric that can quantify stakeholder satisfaction; others consist of Likert scales and qualitative methods. By aggregating stakeholder satisfaction and advocacy into a single score, NPS determines whether stakeholders will recommend the model to others. Measuring stakeholder satisfaction is essential to information on the FDSS-ERA model’s realistic results and locating methods to improve its capability for manual labor market decisions.

Stakeholder satisfaction with the suggested FDSS-ERA model

4.1 Comparative Study

This section compares the obtained outcomes, inspecting the performance of the FDSS-ERA model versus other comparative models. The objective is to determine the effectiveness of the FDSS-ERA model in dealing with the complicated difficulties associated with labor market risk evaluation. Choosing appropriate assessment metrics and similar models is vital to comprehensively assessing the strengths and boundaries of the FDSS-ERA model.

To assess the risks associated with the labor market, the FDSS-ERA model is an all-encompassing tool that provides a complete view of capacity hazards. To provide decision-makers with a holistic perspective, the risk assessment coverage of the device assigns a numerical value to the proportion of identified hazards compared to the total number of potential risks. Because of this, the reliability of risk assessments is improved by the uncertainty management measure of the model, which evaluates the model’s capacity to effectively deal with the uncertainty inherent in the data about the labor market. By evaluating the model’s capability to provide decision-makers with reasonable insights and suggestions, the decision-making supportiveness measure determines whether the model can allow stakeholders to make well-informed decisions. It is guaranteed that the model will continue to be relevant and effective in changing circumstances because of its flexibility to changes in the setting and adjustments in economic characteristics and dynamics within the group of workers. To make plans and technique systems that are effective over the long term, it is of the utmost necessity to have accurate long-term period preparations. Given that it allows for selecting factors that have a major influence on the outcome, it considers the sensitivity of the components, which helps prioritize interventions and allocate resources. It is ensured that the model’s performance remains constant even when the statistics are not perfect by evaluating its ability to deal with disorganized data.

Because it employs fuzzy logic procedures comparable to those used in FDSS-ERA, FRBS emerges as the favored alternative. The purpose of this model is to serve as a trend for measuring the efficacy of FDSS-ERA when it comes to dealing with data on the labor market that are fuzzy and unclear. FPN offers an approach that is mostly based on fuzzy logic as an opportunity method. When determining the level of risk associated with fuzzy models, this method can provide significant insights into the overall performance of various fuzzy models. In addition to demonstrating that there is potential for enhancement in terms of capabilities, the comparison to FPN demonstrates that the efficacy of FDSS-ERA has been verified. Multi-criteria decision-making (MCDM) models show an opportunity point of view. These models emphasize decision-making techniques and the compromises that may be made between a couple of principles. A comparison of the two frameworks may also be used to evaluate the applicability of FDSS-ERA and MCDM to help decision-making in complicated labor market scenarios. This can be done by comparing the two frameworks. To guarantee that the FDSS-ERA model follows the various requirements of stakeholders in the labor market and contributes to well-informed decision-making and sustainable development, it is possible to evaluate the model’s performance comprehensively. This is made possible by the widespread utilization of these comparative models and assessment metrics.

This section on effects and evaluation presents a complete examination of the usefulness of the FDSS-ERA model in measuring the risk of the labor market. There is also a discussion of the implications of this analysis. Regarding its insurance of chance evaluation, handling of uncertainty, stage of assistance for decision-making, capability to evolve to contextual shifts, accuracy in long-term forecasts, sensitivity to enter variables, and resilience to noisy information, it is abundantly clear that FDSS-ERA outperforms other models. This is the case in several different ways. This result was arrived at after an in-depth discussion and analysis of several different assessment measures and comparing models. When tackling the problematic elements of labor market dynamics and facilitating well-informed decision-making for sustainable growth, the outcomes described above give insight into the efficacy and flexibility of the FDSS-ERA model.

The following benchmark models are added to assess the efficacy and excellence of the FDSS-ERA model in handling labor market risk evaluation problems. The FDSS-ERA paradigm is comparable to the fuzzy rule-based systems (FRBS), a famous model that uses fuzzy logic ideas. It is a benchmark for measuring how well FDSS-ERA handles indistinct and unpredictable labor market data. Petri nets and fuzzy logic are combined inside the modeling technique called fuzzy Petri nets, which allows for the representation and evaluation of complex systems. It can be included if pertinent to the observer’s comparative framework, although it is not always referenced in the text. Models for multi-criteria decision-making (MCDM) consider a couple of criteria or goals while deciding. These models are regularly used in risk evaluation and decision assistance.

4.2 Limitations and Discussion

Making a significant addition to the field, this study examines the FDSS-ERA model to assess labor market hazards. Still, it is critical to note some caveats; further research is needed. The model’s performance may depend on the accuracy and thoroughness of the input data. Misleading or missing data could compromise the model’s predictive and risk assessment capabilities. This is why strengthening the model’s dependability requires a focus on improving data collection techniques and ensuring data integrity. Furthermore, the FDSS-ERA model may not be appropriate in all cases due to the dynamic nature of regulatory landscapes and labor markets. Economic changes, new technologies, and government policies might render the model’s predictions irrelevant or outdated as time passes. Monitoring and updating the version to keep it functional and ensure it represents the conversion circumstances is critical.

Similarly, those without specialized knowledge or technological scalability may find it challenging to adhere to one of the restrictions associated with the difficulty of implementing fuzzy logic principles. The model’s interface may be made more user-friendly and accessible to a wider audience to increase the likelihood of its adoption. While much of the research focuses on the Southeast Asian labor market, it is crucial to remember that different areas have different socio-economic conditions and labor market dynamics; thus, the results may not be applicable everywhere. It is suggested that more studies be conducted to examine the FDSS-ERA model’s flexibility in different settings and to confirm its effectiveness via cross-regional comparisons. It is critical to remove these constraints to optimize the usability and efficacy of the FDSS-ERA model, even if it shows the capacity to compare labor market risks. To make informed decisions in labor markets and advance the risk control area, it is vital to improve, verify, and modify the model continuously.

It is vital to point out how fuzzy logic principles can handle the uncertainty and complexity of the Southeast Asian labor market to justify the usage of FDSS for labor market risk assessment. Data and risk assessment in the Southeast Asian labor market are inherently uncertain because of the market’s diversified and ever-changing characteristics. Because of their strength in dealing with nebulous, obscure, or missing data, fuzzy logic ideas are ideal for simulating unknown and complex systems. Conventional binary decision-making models may oversimplify the complex structure of labor market risks. Because fuzzy logic can describe gradations of actuality, decisions may be made with greater delicacy and context awareness. Because the Southeast Asian labor market is risky, risk assessment methodologies are on the way to adjust to new circumstances quickly. Because of their malleability and flexibility, fuzzy decision structures are well suited for modeling and analyzing fluid systems. The FDSS-ERA mode and other fuzzy decision support structures use a multifaceted technique to assess risk by simultaneously considering various variables, scenarios, and uncertainties. This thorough analysis helps make well-informed decisions about the country of the labor market. According to historical studies, fuzzy logic-based total models are beneficial in several fields, which include risk assessment. The FDSS-ERA modelhas been used for  assessing risks in the labor market by demonstrating its performance, validation, and aggressive advantages.

5 Conclusion

To better understand the risks connected with the labor market in Southeast Asian nations, this study looked at how FDSS was developed and used to enhance the risk assessment model. The FDSS-ERA model provides a well-established framework for assessing many risk factors, including changes in demographics, skill shortages, regulatory frameworks, and employment trends. Fuzzy logic and decision support techniques are combined to form this framework. The main conclusions show that the FDSS-ERA model outperforms comparative models in many assessment metrics, including risk evaluation coverage, uncertainty management, decision-making assistance, adaptability to contextual modifications, sensitivity to input variables, and robustness to noisy data. The findings have shown that the FDSS-ERA model is flexible and effective and can provide detailed and useful information for making smart decisions in the job market. More research could be needed to improve the model’s data quality and integrity, better predict future outcomes by considering the dynamic nature of the labor market, and make it applicable to a wider range of geographic situations. Also, more people, including lawmakers, experts in the labor market, and industry participants, will be able to identify the model if we can make its interface easier to use and more accessible. We may enhance its predictive power and decision-making skills by exploring the potential integration of technologies like AI and device development into the FDSS-ERA model.

The article presents the novel FDSS-ERA model for assessing labor market risk in Southeast Asia and demonstrates its effectiveness compared to other models using various criteria. Contributing to sustainable development, its organized framework assesses different risk variables and helps with well-informed decision-making. Better analytical tools are available to decision-makers due to the model’s use of fuzzy logic concepts. Data quality enhancement, regional applicability, user interface simplification, and the incorporation of new technologies, including machine learning, should be the primary goals of future studies. Academic knowledge will be advanced, labor market difficulties will be addressed, and risk evaluation and decision-making will undergo additional developments due to these endeavors.

Data Availability

The datasets used and/or analyzed during the current study available from the corresponding author on reasonable request.

Kelley, L.C., Peluso, N.L., Carlson, K.M., Afiff, S.: Circular labor migration and land-livelihood dynamics in Southeast Asia’s concession landscapes. J. Rural. Stud. 73 , 21–33 (2020)

Article   Google Scholar  

Novikova, O., Khandii, O., Shamileva, L.: Socio-economic risk assessment and peril analysis in the context of the COVID-19 pandemic and emergencies. Eur. J. Sustain. Dev. 10 (1), 636–636 (2021)

Wang, T., Sun, F.: Integrated drought vulnerability and risk assessment for future scenarios: an indicator based analysis. Sci. Total. Environ. 900 , 165591 (2023)

Halder, S., Patra, U.K.: Status of brick kiln workers in southeast Asia. J. Nat. Remedies 21 (1), 6–16 (2021)

Google Scholar  

Grenčíková, A., Habánik, J., Španková, J., Húževka, M., Šrámka, M.: Current labour market challenges in the light of future economic and demographic developments. Eur. J. Interdiscip. Stud. 14 (1), 100–112 (2022)

Carnegie, G.D., Guthrie, J., Martin-Sardesai, A.: Public universities and impacts of COVID-19 in Australia: risk disclosures and organizational change. Account. Audit. Account. J. 35 (1), 61–73 (2022)

Felsenstein, D., Grinberger, A.Y.: Cascading effects of a disaster on the labor market over the medium to long term. Int. J. Disaster Risk Reduct. 47 , 101524 (2020)

Selander, J.: Physical exposures in the work environment during pregnancy—a challenge for risk assessment. Scand. J. Work Environ. Health 49 (8), 535 (2023)

Malik, A., Pereira, V., Budhwar, P., Froese, F.J., Minbaeva, D., Sun, J., et al.: Multilevel relational influences on HRM practices: a cross-country comparative reflective review of HRM practices in Asia. Asian Bus. Manag. 21 (5), 745–779 (2022)

Hof, H.: Opting out for getting in existential mobility in: European graduates’ migration to Asia. J. Immigr. Refug. Stud. 18 (3), 286–299 (2020)

Karunathilake, K.: Positive and negative impacts of COVID-19, an analysis with special reference to challenges on the supply chain in South Asian countries. J. Soc. Econ. Dev. 23 (Suppl 3), 568–581 (2021)

Lee, K., Wong, C.Y., Intarakumnerd, P., Limapornvanich, C.: Is the fourth industrial revolution a window of opportunity for upgrading or reinforcing the middle-income trap? Asian model of development in Southeast Asia. J. Econ. Policy Reform 23 (4), 408–425 (2020)

Brakman, S., Garretsen, H., van Witteloostuijn, A.: The turn from just-in-time to just-in-case globalization in and after times of COVID-19: an essay on the risk re-appraisal of borders and buffers. Soc. Sci. Humanit. Open 2 (1), 100034 (2020)

Peluffo, C., Viollaz, M.: Intra-household exposure to labor market risk in the time of Covid-19: lessons from Mexico. Rev. Econ. Household 19 , 327–351 (2021)

Fraser, A., Knoll, L., Hevenstone, D.: Contested social impact bonds: welfare conventions, conflicts and compromises in five European active-labor market programs. Int. Public Manag. J. 26 (3), 339–356 (2023)

Jaya, A.S.M., Kadir, N.A., Jarrah, M.I.: Modeling of TiN coating roughness using fuzzy logic approach. Science International 26 (4), 1563–1567 (2014)

Viswanath, S., Mullins, L.B.: Gender responsive budgeting and the COVID-19 pandemic response: a feminist standpoint. Adm. Theory Praxis 43 (2), 230–244 (2021)

Ererdi, C., Nurgabdeshov, A., Kozhakhmet, S., Rofcanin, Y., Demirbag, M.: International HRM in the context of uncertainty and crisis: a systematic review of literature (2000–2018). Int. J. Hum. Resour. Manag. 33 (12), 2503–2540 (2022)

Kharroubi, D.: Global workforce diversity management: challenges across the world. Ekonomicko-manazerske spektrum 15 (1), 28–37 (2021)

Article   MathSciNet   Google Scholar  

Alshanskaya, A.A., Azatbek, T.A.: Assessment of changes in the structure of the labor market under the influence of the corona crisis in Kazakhstan. Econ. Strategy Pract. 17 (2), 146–159 (2022)

Liu, Z.: Fermatean fuzzy similarity measures based on Tanimoto and Sørensen coefficients with applications to pattern classification, medical diagnosis and clustering analysis. Eng. Appl. Artif. Intell. 132 , 107878 (2024). https://doi.org/10.1016/j.engappai.2024.107878

Kahraman, C.: Proportional fuzzy set extensions and imprecise proportions. Informatica 35 (2), 311–339 (2024). https://doi.org/10.15388/24-INFOR550

Liu, Z., Letchmunan, S.: Enhanced fuzzy clustering for incomplete instance with evidence combination. ACM Trans. Knowl. Discov. Data 18 (3), 1–20 (2024). https://doi.org/10.1145/3638061

Liu, Z.: Hellinger distance measures on pythagorean fuzzy environment via their applications. Int. J. Knowl. Based Intell. Eng. Syst. 28 (2):211–229 (2024). https://doi.org/10.3233/KES-230150

Charolina, A., Fitriyadi, F.: Fuzzy mamdani model for assessing the level of service satisfaction for requirements of social welfare services at the “Prof. Dr. Soeharso” integrated center in Surakarta. Int. J. Comput. Inf. Syst. 5 (1), 15–21 (2024)

Putra, Y., Kanafi, K., Fitriana, M.: Decision support system for hiring expert staff at CV BMS using AHP method. Int. J. Comput. Inf. Syst. 4 (2), 63–68 (2023)

Kanj, H., Kotb, Y., Alakkoumi, M., Kanj, S.: Dynamic Decision Making Process for Dangerous Good Transportation Using a Combination of TOPSIS and AHP Methods with Fuzzy Sets. IEEE Access, New York (2024)

Book   Google Scholar  

Al-shami, T.M., Mhemdi, A.: Generalized frame for orthopair fuzzy sets: ( m , n )-fuzzy sets and their applications to multi-criteria decision-making methods. Information 14 (1), 56 (2023)

Azad, M.A.: Typology of international migration in globalization challenges for realizing Bangladeshi migrant worker’s rights and their contribution to social development. Open J. Soc. Sci. 11 (1), 143–164 (2023)

Preeth, S.S.L., Dhanalakshmi, R., Kumar, R., Shakeel, P.M.: An adaptive fuzzy rule based energy efficient clustering and immune-inspired routing protocol for WSN-assisted IoT system. J. Ambient Intell. Human. Comput. 1–13 (2018). https://doi.org/10.1007/s12652-018-1154-z

https://www.kaggle.com/datasets/alenavorushilova/labour-force-world-stats-data-by-oecd?select=ALFS_SUMTAB_26032020094503102.csv

Download references

This work was supported by the Research and Planning Fund for Humanities and Social Sciences of Ministry of Education “Research on the Boundary of Human Resource Management under the Framework of ‘the belt and road initiative’—From the Perspective of Working Conditions Standards” (project number: 18YJA630092).

Author information

Authors and affiliations.

School of Economics and Management, Beijing Jiaotong University, Beijing, 100000, China

Zujie Zhang

You can also search for this author in PubMed   Google Scholar

Contributions

Z.Z. wrote and reviewed the whole paper.

Corresponding author

Correspondence to Zujie Zhang .

Ethics declarations

Conflict of interest.

The authors have declared that no competing interests exist.

Consent for Publication

Written consent to publish this information was obtained from study participants.

Ethics Approval and Consent to Participate

Ethical approval is not required for this study.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Zhang, Z. Application of Fuzzy Decision Support Systems in Risk Assessment of Southeast Asian Labor Market. Int J Comput Intell Syst 17 , 153 (2024). https://doi.org/10.1007/s44196-024-00556-y

Download citation

Received : 28 March 2024

Accepted : 31 May 2024

Published : 19 June 2024

DOI : https://doi.org/10.1007/s44196-024-00556-y

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Rapid economic changes
• Southeast Asian labor market
• Evolving regulatory environments
• Diverse workforce dynamics
• Fuzzy decision support systems
• Risk assessment
• Find a journal
• Publish with us
• Track your research

Unfortunately we don't fully support your browser. If you have the option to, please upgrade to a newer version or use Mozilla Firefox , Microsoft Edge , Google Chrome , or Safari 14 or newer. If you are unable to, and need support, please send us your feedback .

We'd appreciate your feedback. Tell us what you think! opens in new tab/window

CRediT author statement

CRediT (Contributor Roles Taxonomy) was introduced with the intention of recognizing individual author contributions, reducing authorship disputes and facilitating collaboration. The idea came about following a 2012 collaborative workshop led by Harvard University and the Wellcome Trust, with input from researchers, the International Committee of Medical Journal Editors (ICMJE) and publishers, including Elsevier, represented by Cell Press.

CRediT offers authors the opportunity to share an accurate and detailed description of their diverse contributions to the published work.

The corresponding author is responsible for ensuring that the descriptions are accurate and agreed by all authors

The role(s) of all authors should be listed, using the relevant above categories

Authors may have contributed in multiple roles

CRediT in no way changes the journal’s criteria to qualify for authorship

CRediT statements should be provided during the submission process and will appear above the acknowledgment section of the published paper as shown further below.

*Reproduced from Brand et al. (2015), Learned Publishing 28(2), with permission of the authors.

Sample CRediT author statement

Zhang San:  Conceptualization, Methodology, Software  Priya Singh. : Data curation, Writing- Original draft preparation.  Wang Wu : Visualization, Investigation.  Jan Jansen :  Supervision. : Ajay Kumar : Software, Validation.:  Sun Qi:  Writing- Reviewing and Editing,

Read more about CRediT  here opens in new tab/window  or check out this  article from  Authors' Updat e:  CRediT where credit's due .

This paper is in the following e-collection/theme issue:

Published on 18.6.2024 in Vol 26 (2024)

Identification of Ethical Issues and Practice Recommendations Regarding the Use of Robotic Coaching Solutions for Older Adults: Narrative Review

Authors of this article:

• Cécilia Palmier 1, 2 * , MSc   ; 
• Anne-Sophie Rigaud 1, 2 * , Prof Dr Med   ; 
• Toshimi Ogawa 3 , PhD   ; 
• Rainer Wieching 4 , Prof Dr   ; 
• Sébastien Dacunha 1, 2 * , MSc   ; 
• Federico Barbarossa 5 , MEng   ; 
• Vera Stara 5 , PhD   ; 
• Roberta Bevilacqua 5 , MSc   ; 
• Maribel Pino 1, 2 * , PhD  

1 Maladie d’Alzheimer, Université de Paris, Paris, France

2 Service de Gériatrie 1 & 2, Hôpital Broca, Assistance Publique - Hôpitaux de Paris, Paris, France

3 Smart-Aging Research Center, Tohoku University, Sendai, Japan

4 Institute for New Media & Information Systems, University of Siegen, Siegen, Germany

5 Scientific Direction, Istituto Nazionale di Ricovero e Cura per Anziani, Ancona, Italy

*these authors contributed equally

Corresponding Author:

Anne-Sophie Rigaud, Prof Dr Med

Service de Gériatrie 1 & 2

Hôpital Broca

Assistance Publique - Hôpitaux de Paris

54 rue Pascal

Paris, 75013

Phone: 33 144083503

Fax:33 144083510

Email: [email protected]

Background: Technological advances in robotics, artificial intelligence, cognitive algorithms, and internet-based coaches have contributed to the development of devices capable of responding to some of the challenges resulting from demographic aging. Numerous studies have explored the use of robotic coaching solutions (RCSs) for supporting healthy behaviors in older adults and have shown their benefits regarding the quality of life and functional independence of older adults at home. However, the use of RCSs by individuals who are potentially vulnerable raises many ethical questions. Establishing an ethical framework to guide the development, use, and evaluation practices regarding RCSs for older adults seems highly pertinent.

Objective: The objective of this paper was to highlight the ethical issues related to the use of RCSs for health care purposes among older adults and draft recommendations for researchers and health care professionals interested in using RCSs for older adults.

Methods: We conducted a narrative review of the literature to identify publications including an analysis of the ethical dimension and recommendations regarding the use of RCSs for older adults. We used a qualitative analysis methodology inspired by a Health Technology Assessment model. We included all article types such as theoretical papers, research studies, and reviews dealing with ethical issues or recommendations for the implementation of these RCSs in a general population, particularly among older adults, in the health care sector and published after 2011 in either English or French. The review was performed between August and December 2021 using the PubMed, CINAHL, Embase, Scopus, Web of Science, IEEE Explore, SpringerLink, and PsycINFO databases. Selected publications were analyzed using the European Network of Health Technology Assessment Core Model (version 3.0) around 5 ethical topics: benefit-harm balance, autonomy, privacy, justice and equity, and legislation.

Results: In the 25 publications analyzed, the most cited ethical concerns were the risk of accidents, lack of reliability, loss of control, risk of deception, risk of social isolation, data confidentiality, and liability in case of safety problems. Recommendations included collecting the opinion of target users, collecting their consent, and training professionals in the use of RCSs. Proper data management, anonymization, and encryption appeared to be essential to protect RCS users’ personal data.

Conclusions: Our analysis supports the interest in using RCSs for older adults because of their potential contribution to individuals’ quality of life and well-being. This analysis highlights many ethical issues linked to the use of RCSs for health-related goals. Future studies should consider the organizational consequences of the implementation of RCSs and the influence of cultural and socioeconomic specificities of the context of experimentation. We suggest implementing a scalable ethical and regulatory framework to accompany the development and implementation of RCSs for various aspects related to the technology, individual, or legal aspects.

Introduction

Challenges associated to population aging.

Technological and medical advances have led to a demographic shift in the population, with the number of older adults constantly increasing. According to the United Nations [ 1 ], older adults (aged 60-65 years) will represent 16% of the world’s population in 2050. In addition, life expectancy is increasing, from 64.2 years in 1990 to 72.6 years in 2019, and is expected to reach 77.1 years in 2050 [ 1 ]. However, there is a wide diversity of health conditions among older adults. The health status of older adults is dependent on multiple factors, including nonmodifiable genetic factors and environmental factors, such as lifestyle [ 2 ]. Thus, older adults represent a very heterogeneous population with multiple and diverse needs and desires. With advancing age, the loss of functional independence; frailty; and other health diseases such as cardiovascular problems, cancers, osteoarthritis, osteoporosis, or major neurocognitive disorders may appear [ 3 - 5 ]. Among age-related conditions, major neurocognitive disorders (eg, Alzheimer disease) receive particular attention due to the increasing prevalence of these diseases [ 6 ].

The aging population is not only a public health issue but also a socioeconomic one. To face this challenge, it is important to develop preventive measures to support active and healthy aging and to preserve the independent functioning and quality of life of older adults. The adoption of healthy behaviors can help prevent or delay the onset of pathologies or treat them if detected early [ 7 ].

The Use of Technologies for Older Adults

Preventive health measures can be supported through new technologies, such as robotic coaching solutions (RCSs) that promote healthy aging among older adults [ 8 , 9 ]. RCSs have been defined as personalized systems that continuously monitor the activities and environment of the user and provide them with timely health-related advice and interventions [ 10 - 12 ]. These systems can help users define and achieve different health-oriented goals [ 12 ].

RCSs may encompass artificial intelligence (AI) technologies that can analyze user data, personalize coaching programs, and adapt recommendations based on each individual’s needs [ 1 , 13 - 19 ]. RCSs can involve robots equipped with sensors such as cameras, microphones, or motion sensors to collect real-time data about the user, AI, and programming that enables their interaction with users [ 20 , 21 ]. These technologies are often equipped with voice and visual recognition and learning capabilities [ 20 , 21 ]. They can benefit from advanced natural language processing techniques, which allow for understanding of the user’s input, facilitating natural and effective communication [ 22 ]. RCSs can offer guidance, support, and feedback based on preprogrammed information or real-time data analysis. These data can inform coaching strategies and allow RCSs to provide users with relevant feedback [ 8 ].

RCSs can also encompass a virtual agent, which refers to a computer program or an AI system that interacts with users in a manner that simulates human conversation [ 14 , 18 , 23 ]. A virtual agent is an animated character capable of adopting a social behavior mimicking that of humans to encourage the users to make changes in their habits [ 14 ]. Virtual agents might take the form of a chatbot, voice assistant, or other AI-driven communication system [ 14 ]. Biometric monitoring devices to track physiological data such as heart rate, sleep patterns, or stress levels can also be included in RCSs [ 8 , 20 , 21 ]. These data can contribute to the configuration of personalized coaching plans. RCSs can also encompass advanced data analytics that can process large data sets generated by users’ interactions and behaviors. This functionality helps in identifying patterns, trends, and areas for improvement in coaching strategies [ 24 ]. Integrating Internet-of-Things devices in RCSs can provide additional data points about a user’s environment, lifestyle, or habits, thus contributing to a personalized coaching approach [ 25 ].

Health-oriented RCSs could enable users to lead a healthy lifestyle, by identifying needs and goals and providing appropriate risk predictions and individualized recommendations [ 12 , 26 - 28 ]. There are RCSs dedicated to a particular domain, such as physical activity or motor rehabilitation [ 9 , 16 ]. Others may have the objective of promoting independent and healthy aging [ 29 ].

Promoting active and healthy aging can allow older adults to maintain their independence and continue to live at home [ 4 , 30 ], which is a wish of many [ 3 ]. This intervention could also help to reduce the need for assistance, usually provided by informal caregivers and health professionals [ 4 , 19 , 30 - 33 ]. Furthermore, RCSs could lead to a reduction in individual and collective health care expenses [ 4 , 32 , 34 ] by easing access to health and social care interventions to a wide population, including hard-to-reach (eg, geographically isolated) individuals. However, although the use of health-related RCSs could have many benefits, several ethical issues arise with their development and implementation in human environments [ 3 , 35 - 38 ].

An Ethical Framework for the Use of Technologies for Older Adults

For RCSs to contribute to active and healthy aging, it is important that all the stakeholders (engineers, geriatricians, psychologists, etc) involved in their design and implementation refer to an ethical framework [ 3 , 38 ]. It is also important to inform society (politicians and legal experts) about such an extension of technology in people’s lives (private, professional, medicosocial, and commercial context), so that we can create a legal framework for the use of these technologies. An analysis of the way in which ethical and legal dimensions have been addressed by studies, in the field of RCSs for health care, seems useful to support the key actors in their development and implementation. The growing interest in the ethical questions associated with the use of social and assistive robots is evidenced by the volume of literature reviews [ 3 , 12 , 18 , 31 , 32 , 37 , 39 - 51 ] on the topic.

Now, it appears appropriate to systematically examine this body of work, focusing on the ethical analysis, and provide an overview of the literature. Therefore, we performed a review of the literature on RCSs for older adults using the European Network of Health Technology Assessment (EUnetHTA Core Model; version 3.0) model [ 52 ] for analysis. This Health Technology Assessment (HTA) model makes it possible to assess the intended and unintended consequences of the use of a specific technology regarding multiple domains (eg, technological, ethical, clinical, and organizational), providing methods and concepts for this analysis [ 53 ]. Therefore, HTA is a process that informs decision-making about the introduction of new technologies such as RCSs in health care. It also seems necessary to issue guidelines for the development and implementation of health-oriented RCSs [ 54 ].

The objective of this study was to highlight the main ethical questions and corresponding recommendations linked to the use of RCSs for older adults for engineers, researchers, and health professionals in this field. For this purpose, we conducted a narrative literature review using the ethical dimension of the EUnetHTA Core Model to guide the analysis. To the best of our knowledge, such a study has not been conducted so far.

A thematic analysis of the literature was performed to identify publications that describe RCSs for supporting older adults in health care and prevention and those that address ethical issues and recommendations regarding their development and implementation. The methodology used for the narrative review was inspired by the study by Green et al [ 55 ].

Inclusion and Exclusion Criteria

The review encompassed papers focusing on all populations, with particular attention to older adults. It focused on the concept of RCSs for health, while also incorporating publications discussing other health technologies for older adults if the authors have delved into relevant ethical considerations for their development or implementation.

The context of the review revolved around the use of RCSs (or related technologies), especially for older adults, across diverse living environments such as homes, hospitals, and nursing homes. Publications addressing RCSs and related ethical issues within the health care domain were considered, whereas those focusing solely on technical aspects (eg, AI and deep learning) or those outside the health care domain were excluded.

Various types of publications, including theoretical papers, research studies, and reviews, were included if they offered ethical reflections or recommendations for RCS use in health care. These reflections and recommendations were expected to align with the topics and issues of the ethical dimension of the EUnetHTA Core Model.

All publications, regardless of language (English or French), were eligible if published after 2011. This time frame was chosen considering the technological advancements over the past decade, which may have influenced the evolution of ethical issues and recommendations in the field of remote care systems and related technologies. Textbox 1 summarizes the inclusion and exclusion criteria adopted for the selection of papers in this review.

Inclusion criteria

• Types of participants: all populations
• Interventions or phenomena of interest: RCSs or other technologies used in health care, if ethical issues are discussed
• Context: the use of RCSs in the health care sector
• Paper type: all paper types (theoretical papers, research studies, and reviews) that discuss ethical issues
• Language: English or French
• Date of publication: after 2011

Exclusion criteria

• Types of participants: not applicable
• Interventions or phenomena of interest: RCSs or all other types of technology outside the health care sector
• Context: the use of RCSs in non–health care sectors
• Paper type: papers about RCSs and other technologies that are not dealing with ethical issues
• Language: all other languages
• Date of publication: before 2011

Search Strategy and Study Selection

The review was conducted using the following keywords: “seniors,” “older adults,” “social robots,” “assistive robots,” “assistive technology,” “robots,” “virtual coach,” “e-coaching,” “coaching system,” “coaching device,” “ethics,” and “recommendations.”

The review was performed between August 2021 and December 2021 using the PubMed, CINAHL, Embase, Scopus, Web of Science, IEEE Explore, SpringerLink, and PsycINFO databases.

This search allowed us to find 4928 initial publications. Then, secondary research using references from other articles and the same inclusion criteria was conducted. This search allowed us to find 13 additional papers.

In total, 4943 papers were analyzed. The selection of the final publications was performed after reading the title and abstract first and, then, the full article. This selection process helped us to exclude irrelevant papers and duplicates ( Figure 1 ). In total, 0.51% (25/4943) of the papers were included in our review.

Data Analysis Criteria

The selected papers were analyzed using the ethical domain of the EUnetHTA Core Model [ 52 ]. Proper registration of the use of EUnetHTA Core Model for the purpose of this review was made on the HTA Core Model website [ 52 ].

The model was developed for the production and sharing of HTA information, allowing for the support of evidence-based decision-making in health care, but it can also be customized to other research needs. The EUnetHTA Core Model is composed of 9 domains, each including several topics. Each topic also includes different issues (ie, questions that should be considered for the evaluation of health technologies). Thus, the model is structured into 3 levels: domain (level 1), topic (level 2), and issue (level 3). The combination of a domain, topic, and issue is linked to an assessment element ID, which can be identified using a specific code for standardization purposes (B0001, B0002, etc).

The main EUnetHTA model domains include the following: (1) health and current use of the technology, (2) description and technical characteristics of the technology, (3) safety, (4) clinical effectiveness, (5) costs and economic evaluation, (6) ethical aspects, (7) organizational aspects, (8) patient and social aspects, and (9) legal aspects.

The ethical domain (level 1) in the EUnetHTA Core Model [ 52 ] includes 5 topics (level 2): “benefit-harm balance,” “autonomy,” “respect for people,” “justice and equity,” and “legislation.” Each of these topics includes several issues (level 3) [ 52 ].

In this study, 2 authors (CP and ASR) independently analyzed the 25 selected articles. First, they read the articles several times to improve familiarity with the ideas addressing the ethical aspects of RCSs. Then, in each publication (methods, results, and discussion sections), they identified segments of data that were relevant or captured an idea linked to the “ethical” domain of the model. A subsequent exploration of the coded data (sentences or set of statements) was performed to get a more precise classification at the topic level (level 2) and at the issue level (level 3). Then, the coding was performed using the HTA nomenclature. The 2 experts (CP and ASR) compared their results. In a few cases, the coding results showed a lack of consensus between the 2 coding authors, which was resolved through a subsequent discussion between them. Interrater correlation was not calculated.

A thematic analysis using the EUnetHTA framework for conducting a literature review has been described in other studies [ 56 , 57 ]. Furthermore, the use of EUnetHTA to perform an ethical analysis of health technologies has already been proposed [ 58 ]. The 25 selected articles were all coded using this methodology. Some authors have previously emphasized the possibility of overlapping issues between topics in the HTA analysis. They have suggested to assess the overlapping issues in the most relevant topic section [ 59 ].

This review was not registered, and a protocol for the review was not prepared.

Selected articles are presented in Multimedia Appendix 1 [ 3 , 12 , 18 , 31 , 32 , 37 - 51 , 60 - 64 ]. For each topic, we have presented our findings in terms of questions and recommendations according to the EUnetHTA Core Model, wherever possible.

Ethical Issues and Recommendations for the Use of New Technologies

This section aims to summarize the ethical analysis performed regarding the use of RCSs with older adults and to provide recommendations for ethical use of these devices. Table 1 presents a synthetic summary of the elements presented in this section.

Topic 1: Benefit-Harm Balance

RCSs should be developed according to the principles of beneficence (ie, to promote the interest of users) and nonmaleficence (ie, to avoid inflicting harm) [ 39 , 60 , 64 ].

What Are the Known and Estimated Benefits and Harms for Patients When Implementing or Not Implementing the Technology?

Risk of social isolation.

According to Sharkey and Sharkey [ 50 ], technological devices, when used appropriately, could benefit older adults by promoting social interaction and connection with their loved ones [ 4 , 31 , 40 ]. Broadbent et al [ 19 ] have discussed the potential of robots to reduce older adults’ social isolation. However, other authors reported the negative influence of the use of robotic devices on human contact [ 31 , 32 , 65 ]. The use of robots (eg, telepresence robots) to make some cost savings (eg, reducing travel costs and time spent on trips for family and professionals to visit older adults) would reduce face-to-face interactions [ 3 , 36 , 39 , 40 ]. Moreover, according to Körtner [ 47 ], the more people become accustomed to communicating with robots, the less they will be used to communicating with humans. The use of social robots could lead to a reduction of interactions with humans and thus to social isolation and emotional dependence [ 39 ]. However, the influence of technological devices, such as RCSs, on social isolation is still under debate, and the impact of technology would depend on the manner in which it is used.

To avoid exacerbating the users’ social isolation, Portacolone et al [ 38 ] advocate that social robots and similar technologies should be designed with the objective of fostering interactions with other humans, for instance, keeping users informed about the entertainment and socializing activities near their home, connecting them with their loved ones, and so on.

Risk of Deception

Another major risk for users is deception [ 39 , 64 , 66 ]. Portacolone et al [ 38 ] described 3 types of deception that people with neurocognitive disorders may face when interacting with social robotic systems but which may also apply to all users. The first type involves the user’s misconception of what is driving the technological device [ 51 ]. Users may be misled if they think that behind a medical chatbot, there is a real physician who communicates and reads their messages [ 44 ] or, alternatively, if they are not aware that, at some point, there are real humans guiding the technological device [ 38 ]. The second type refers to robotic devices programmed to express feelings or other types of affective communication, which may lead the user to believe that the system’s emotions are authentic. Related to this issue, Körtner [ 47 ] discussed how some older adults may fear that their social robot will forget them during their absence from home. The resemblance with the living in terms of affective behavior (eg, crying, laughing, or expressing concern) can make the user believe that there is a reciprocity between human and robot feelings [ 43 ]. The last type of deception is related to the inadequate interpretations that older adults may have regarding the nature of the robot, for example, thinking that an animal-shaped robot is a real animal or a pet [ 38 ]. Some current developments of social robots tend to make them resemble a living being, in terms of their verbal and nonverbal behaviors [ 34 , 60 ] or by highly anthropomorphizing their design [ 47 ], which may blur the boundary between the real and the artificial [ 45 , 60 ]. These design choices can also impact users’ dignity by infantilizing them as they are led to believe in something that is false [ 50 ].

However, according to some researchers [ 51 , 63 , 64 ], the notion of deception should be considered in terms of the gradation between what is morally acceptable and what is not. Deception would be morally acceptable when it aims to improve a person’s health or quality of life, for example, the use companion robots to calm a person experiencing behavioral disorders linked to dementia [ 51 ].

According to Danaher [ 43 ] and Vandemeulebroucke et al [ 40 ], to avoid deception, it is essential to be transparent to users about the design and operation of devices. As the information given to the participants is the basis for obtaining consent to use the technology, it is essential to offer them documents explaining how the device is built and its advantages and limitations in a clear manner adapted to the user’s knowledge and experience. It is also important to inform users on how to behave with technology [ 12 ]. Researchers should also answer users’ questions, pay attention to their feedback, and use it to improve the device and its documentation [ 60 ]. During experiments with RCSs, it is also important that researchers regularly remind participants of the nature of the technological device to reduce the risk of misinterpretation and to ensure that they still consent to participate in the study [ 38 ].

Biases of Algorithms

An autonomous device does not work without AI or algorithms that allow it to make decisions. However, these technologies are created by humans, and programming biases can be incorporated into them and lead to failures [ 44 ]. A technological device can, for instance, misread a situation and react accordingly, leading to a safety risk for the user [ 18 ]. Thus, it is essential that the researcher scrutinizes the algorithms used in RCSs before their implementation [ 44 ]. Fiske et al [ 44 ] also suggest providing the users with detailed explanations about the algorithms present in the technological device they are using.

What Are the Benefits and Harms of the Technology for Relatives, Other Patients, Organizations, Commercial Entities, Society, Etc?

At the society level, Boada et al [ 39 ] mentioned an ethical consideration related to the ecological impact of robotic devices in the current context of climate crisis and the lack of natural resources. The construction of RCSs requires raw materials, high energy consumption, and the management of their waste. Therefore, it is important for developers to design technologies that consume less energy and can be recycled.

Are There Any Unintended Consequences of the Technology and Its Application for Patients?

Technologies evolving very quickly.

For some older adults, technologies evolve very quickly, which makes it difficult for them to keep up with [ 62 ]. Denning et al [ 67 ] encourage designers to develop products that are intuitive to use or to offer users a simplified training. However, although some technologies are progressing quickly, technological limitations are still present, especially regarding social robotic systems, impacting their performance [ 68 ] and generating frustration among some users [ 69 ].

Unsuitability of Technology

The lack of experience with the technologies and the fact that the systems are not suitable to everyone can reduce the usability and acceptability of RCSs among older adults [ 3 , 60 , 62 ]. Frennert and Östlund [ 62 ] highlighted that some older adults were not confident in their ability to handle a robot because of previous complicated experience with technology. Peek et al [ 70 ] also reported that users had doubts about their ability to use the technology and feared that they would easily forget how to use it. They may also fear false alarms generated by monitoring technologies. For example, a person may decide to sit on the floor, but this behavior can be considered as a fall by the technology, and it could call for an ambulance to be sent to the person’s home in vain [ 70 ].

To promote acceptability and usability of RCSs, it is essential to develop them considering the capabilities, needs, and wishes of various users [ 31 , 47 ]. “User-centered design” approaches should be used for this purpose [ 71 ]. This methodology must be performed in a continuous manner to consider the development, new preferences, and experiences of the users. Technology assessment should also be conducted before deployment in ecological environments to improve the predictability of RCSs and decrease the risk of confusion and accidents [ 40 , 47 ].

Topic 2: Autonomy

According to Anderson and Kamphorst [ 42 ], the notion of autonomy implies the recognition of people, for instance, users of RCSs, as thinking individuals who have their own perspective on matters and are able to judge what is best for them.

Is the Technology Used for Individuals Who Are Especially Vulnerable?

Free and informed consent is a prerequisite for the involvement of an individual in research, regardless of the domain. This aspect is mentioned in numerous codes and declarations such as the Declaration of Helsinki (1964-2008) [ 72 ]. In the context of studies of the use of RCSs, this principle ensures that the person has freely chosen to use a device. However, some older adults, particularly those with cognitive disorders, may have difficulties in understanding and evaluating information related to RCSs and therefore in making appropriate choices [ 3 ]. Moreover, the person may not remember that the RCS is in their environment or how it works [ 38 , 44 ]. The question of how to ensure that the older adult has understood the purpose of RCS and that their choice of using the technology is based solely on their own decision and not that of a relative, caregiver, or institution has also been discussed [ 46 ].

Researchers in the field of RCS should adapt to the cognitive abilities of the populations they work with to facilitate communication and decision-making [ 46 ]. Thus, the observation of the person’s behavior is necessary to identify potential reservations regarding the use of RCSs. When the person is very vulnerable to respond, informed consent could be sought by proxy such as from children, spouse, or partner [ 46 , 64 ]. However, according to Diaz-Orueta et al [ 37 ], the final decision of using RCSs lies with the user. To prevent loss of capacity and to guard against any risk of inducement to participate, advance directives [ 46 , 64 ] or implementation of an advance power of attorney [ 46 ] can be proposed.

Does the Implementation or Use of the Technology Affect the Patient’s Capability and Possibility to Exercise Autonomy?

Dependence on the technology.

Although the main interest of RCSs for older adults is the maintenance of functional independence, it has been claimed that these devices could make people dependent on them. By replacing users in tasks that they can still perform, the use of RCSs could create new forms of vulnerability [ 3 , 31 , 39 , 41 , 51 ].

People could rely entirely on autonomous technological devices, such as RCSs, to guide their behaviors, goals, and actions [ 12 , 73 ]. A questioning of the authenticity of users’ actions has been mentioned by Anderson and Kamphorst [ 42 ]. Users might not feel responsible for the success of their actions if they feel they are completely driven by the guidance of the RCS. People could also develop emotional and psychological feelings toward the technology. This may have negative consequences for the individuals [ 38 , 49 ] and lead to new vulnerabilities [ 39 ].

Loss of Freedom

Another ethical issue relates to the conflict between the user’s safety, encouraged by the technology guidance, and a loss of freedom. The RCS could impose constraints on the user under the pretext that the user’s actions are not good for them [ 39 , 40 , 74 ]. Sharkey and Sharkey [ 50 ] explained that to promote home care, RCS could act as a supervisor (ie, programmed to ensure that no danger is present and, if there is a danger, to implement procedures to stop it and avoid it in the future). For instance, the RCS could prevent the person from eating fatty and high-caloric food because it is harmful to them. To protect users and ensure that they live in good health, individuals using RCSs could end up being deprived of certain actions or being under some type of “house arrest” [ 50 ].

One of the goals of using such RCSs is to support older adults’ independence; therefore, it is essential that developers and researchers in the field take measures to preserve the person’s autonomy [ 75 ]. Furthermore, RCS users must have the opportunity to evaluate and re-evaluate the role given to the device, to assess whether the system is reliable and whether it is serving their interests [ 12 , 42 ].

Creating a New Source of Authority

The use of RCSs could alter human relationships, for example, by creating tensions between older adults and their informal caregivers. Their use could also create some tensions with health care professionals by creating a new source of authority [ 12 ]. Monitoring older adults through RCSs can generate anger in the user, for example, when the device insists that the older adult should take a medication that they do not want to take [ 41 , 75 ].

Topic 3: Respect for Persons

Does the implementation or use of the technology affect human dignity.

Human dignity may be affected by the use of RCSs as these technologies may be perceived as “problem evocators” [ 41 ]. Some RCSs are used to compensate for impaired capacities. However, according to Körtner [ 47 ], their use can make older adults aware of their limitations and lead to negative feelings, anxiety, or exhaustion. RCS use can also lead to a form of stigmatization by making one’s own inabilities visible to others [ 3 , 70 ]. It is important to have positive communication regarding RCSs, to provide a less stigmatizing view of their use.

Does the Technology Invade the Sphere of Privacy of the Patient or User?

To continue living at home, users are increasingly willing to tolerate intrusion in their privacy [ 70 ], but they are not always aware of when and how they are being monitored by RCSs [ 61 ]. Portacolone et al [ 38 ] provided the example of an animal-shaped companion robot, for which the older adults can signal that they no longer wish to interact with it by putting the robot to sleep. However, the animal-shaped robot can record data even when it is sleeping, but users are not always aware of this information. Forgetfulness and the lack of understanding of the device can lead to the risk of manipulation and coercion [ 44 ]. The person who is vulnerable may forget that they are being monitored and reveal personal information [ 50 ].

Technological devices, such as RCSs, must remain under the control of the users [ 47 ]. Users should have the ability to define when and where the device is used—when it collects data—to maintain their privacy, especially in intimate or private care settings.

Security of Data

According to Portacolone et al [ 38 ], remote monitoring technologies are usually controlled by third parties, sometimes even operating in another country, which can lead to cultural biases during the interaction between the older adult and the RCS. This context involves the risk that the person controlling the device (third party) takes advantage of the older adult’s vulnerability to steal their personal information or exposes the user to financial abuse [ 38 ]. Older adults are not always aware or vigilant about the sharing and use of data, which may be personal and sensitive [ 73 ]. Furthermore, RCSs can be connected to internet services that collect, store, and transfer these sensitive data [ 47 ] for commercial use [ 49 , 61 ].

In addition, the use of technologies connected to digital networks involves the risk of hacking and unauthorized surveillance [ 34 , 51 ], which can make people vulnerable [ 62 ]. Denning et al [ 67 ] found that home robots could not only be remotely located and identified but also hacked and controlled. First, users may have either preconceived and erroneous ideas about the capabilities of the device or a lack of knowledge to evaluate the safety, especially regarding data protection [ 3 ]. Second, users do not always configure their technological device correctly or update them [ 67 ].

Encryption or security systems must be put in place to protect users’ personal data captured by the devices at every stage: during collection, storage, transmission, and processing [ 3 ]. Researchers must also give particular attention to data security. In Europe, for instance, researchers and technology providers are required to comply with the General Data Protection Regulation [ 40 , 76 ]. Data collection must be performed legally or approved by the local relevant ethical committees.

To address data security challenges, 3 principles are recommended by Ienca et al [ 46 ] when developing technological devices: transparency, legitimate purpose, and proportionality. Transparency refers to the fact that the user knows that the system is collecting data and has consented to it. The user must also have precise information about when and what type of data are recorded and who has access to them [ 47 ]. Legitimate purpose refers to the notion that the monitoring and collection of data is performed for a specific purpose, (ie, in the best interest of the user or, if applicable, a relative who has consented to it). Finally, the principle of proportionality refers to the fact that the data collected are not disproportionate to the user’s needs.

Topic 4: Justice and Equity

The consequences of the technology implementation on the distribution of health care resources was discussed in the literature.

How Does Implementation or Withdrawal of the Technology Affect the Distribution of Health Care Resources?

Societal pressure.

Socioeconomic issues are also linked to the development and use of RCSs can also be raised. Individual freedom may be hindered by the “incentive” of certain stakeholders or authorities to enforce the use of RCSs [ 37 ]. The use of RCSs and similar systems may also lead to a lesser involvement of relatives, caregivers, and institutions that provide care to older adults and to the reduction of care costs; these perceived economic benefits may pressurize older adults to consent to use these devices [ 40 , 46 ]. It is also possible that older adults may have to agree to use the technological device to receive other health care benefits (eg, aids and subsidies) [ 42 ].

Digital Divide

Different opportunities to access RCSs can result in digital divide, defined by the Organisation for Economic Cooperation and Development [ 77 ] as a gap between those who have access to information and communication technologies and those who do not. This difference can create educational, economic, social, and even health-related disparities among citizens. Some citizens would be able to use these devices and, therefore, could benefit from their advantages, whereas others will not be able to use them and will not enjoy their benefits. The use of technologies in the health care context, through public or private institutions, should be subject to previous authorization by independent ethical committees to ensure that the use of these devices will not harm users in any way.

Inequalities in Resources

Questions about justice, equity, and equality among all citizens also arise [ 12 , 40 , 46 ]. RCSs have relatively high costs [ 64 ] and can generate additional expenses such as an internet subscription [ 3 ] that only a part of the population can afford, and this may be owing to the lack of research allowing to measure the cost-to-benefit ratio of these technologies on health [ 32 ]. It is important to ensure the access to RCSs among different living areas (ie, urban and rural). Therefore, involving municipalities and neighborhood associations seems an interesting way of raising awareness about the opportunities offered by RCSs for older adults and reaching a wider range of people.

To promote justice, equity, and fair distribution, Ienca et al [ 46 ] and Wangmo et al [ 64 ] recommend reducing the development costs of RCSs by promoting an open dissemination of source codes. In addition, RCSs should be distributed in priority to those in greatest need; therefore, measures to ensure access to RCSs under fair conditions should be established [ 51 ]. Joachim [ 78 ] also suggests to cover some of the costs of these health care–oriented technologies through health insurance.

Recommendations have been published by researchers to improve equality of access to technologies, such as using open-source software, providing priority access for individuals with low income, or relying on certain collective financing systems such as retirement or health insurance [ 46 , 51 , 78 ]. Discussions must be conducted among developers, legislators, and private and public organizations to identify viable financing solutions that allow for fair distribution of RCSs.

Replacement of Professionals

Researchers have also reported fears expressed by older adults and caregivers about how the use of technological devices could eliminate care-related jobs or replace humans [ 17 , 34 , 48 , 61 ]. There are also concerns about the use of these technological tools to reduce health care costs by decreasing the number of available health care resources and services, thereby exacerbating social inequalities [ 44 ]. The introduction of health-oriented RCSs requires adapting the contexts of care practices, which may threaten their quality [ 39 ]. Their incorporation into the care work environment can be difficult because the devices are automated and some care situations are unpredictable [ 17 , 62 ]. Furthermore, the gestion of certain tasks by technological devices requires a restructuring of the roles and responsibilities of caregivers [ 39 ]. Fiske et al [ 44 ] highlight that there are currently no recommendations or training to enable health care professionals to adopt RCSs, even though these professionals are increasingly confronted with technological devices in their practice.

The incorporation of RCSs must always be accompanied by a discussion with concerned care professionals regarding the advantages and limits of the technology. Professionals must also be supported in the use of these devices through effective training. Structured training and supervision will contribute to the development of a controlled framework of practice around the use of RCSs and thus avoid potential abuse [ 44 ]. Moreover, to encourage their use among professionals, it is essential to clearly define the role of RCSs as an additional resource for professionals and not a replacement of human care services [ 44 ].

Topic 5: Legislation

The ethical challenges linked to the lack of existing legislations and regulations dedicated to the use of the technology were discussed in the literature.

Can the Use of the Technology Pose Ethical Challenges That Have Not Been Considered in the Existing Legislations and Regulations?

Safety of devices.

The use of RCSs by older adults can result in damage and harm to their environment [ 79 ], especially when the device is still at the prototype stage [ 47 ]. Safety risks linked to the use of RCSs (eg, malfunctioning of the technology and incorrect decisions made by the coaching system) arise when they share a common space with humans and interact with them [ 39 ]. The following questions must be considered: Who is responsible in case of an accident, and who pays for the damages [ 39 , 40 , 48 , 62 , 80 ]? Is it the designer, the device, or the user himself? Currently, the civil code favors the cascade system (ie, first, the liability falls on the designer of the product; then, on the developer; and finally, on the user who has not followed the rules of use) [ 74 ]. However, the more the machine becomes autonomous, the less the existing legal frameworks can answer these questions [ 80 ]. This is a key legal issue regarding the implementation of RCSs in real settings because the person responsible for damage to the user or the environment may incur legal or even penal proceedings.

Damage and prejudice can also be caused by a failure to share authority [ 45 , 49 , 60 ]. Who between the human and the technological device holds the power to make decisions and control a functionality [ 81 ]? According to Grinbaum et al [ 45 ], it is important to specify the circumstances in which the human must take control over the technological device (RCS) and those in which the device should decide autonomously. According to Riek and Howard [ 49 ], it is preferable that in certain cases, the technological device, although autonomous, requires a human validation of its actions to keep the user in control of the device. In addition, Bensoussan and Puigmal [ 80 ] suggested the idea that technological devices must have an emergency stop button, so that the human can switch off the technology at any time.

Regulation of Technology

Currently, there is a gray area between the capabilities of RCSs, the reality of the field, and the regulations in force [ 38 ]. To accompany the researcher during the whole process of development and diffusion of RCSs, an ethical framework should be established [ 18 , 60 ]. Specifically, this can be in the form of an ethical code of conduct illustrating the expectations to all the employees of a company [ 18 ]. The researcher must regularly inform themselves about the ethics to be consistent with the evolution of the regulatory framework [ 60 ]. However, according to Nevejans [ 82 ], these ethical recommendations have no legal value and cannot protect humans from the damage caused by new technologies. Thus, it is necessary to think about a new legal framework to protect the users of RCSs [ 37 ].

The use of technologies, such as RCSs, in the health care field has grown significantly in recent years [ 17 , 18 ]. RCSs are increasingly being used for older adults with the aim of promoting healthy behaviors, quality of life, and well-being. However, the use of RCSs also raises several ethical challenges regarding the cost-to-benefit balance of these new care practices, respect for the autonomy of users, respect for privacy, justice and equity linked to their access, or need for a suitable legal framework. Such challenges could be addressed by establishing relevant recommendations for the development and use of RCSs. Some guidelines regarding the use of robotic systems have been published [ 49 , 83 ]. Moreover, in April 2021, the European Commission unveiled the first legal framework about AI [ 84 ]. However, to the best of our knowledge, no recommendations have been proposed in this field directly linked to an analysis of the literature dealing specifically with these ethical issues and potential solutions to address them.

This narrative review identified 25 articles in which authors highlighted ethical issues and recommendations related to the use of RCSs and similar technologies. The use of the EUnetHTA Core Model for the analysis of these articles made it possible to classify the information retrieved in the publications according to 5 main ethical topics—“benefit-harm balance,” “autonomy,” “respect for persons,” “justice and equity,” and “legislation”—and to provide a detailed analysis of RCS-related ethical issues. Our review also aimed to identify recommendations for better development, diffusion, and use of RCSs by a population of older adults.

Technology devices, such as RCSs, are used with older adults to enable them to live independently; to enhance their quality of life and well-being; and, therefore, to cope with the increasing care demands for older populations. RCSs may be used to encourage a range of health-related goals: physical, cognitive, nutritional, social, and emotional domains. To be effective, RCSs must be able to motivate the user by providing highly personalized care programs [ 85 , 86 ]. However, studies have shown that not all potential target users are included in the development of these devices [ 37 , 87 , 88 ]. Therefore, RCSs design might fail to meet a wide range of users’ needs, capabilities, and wishes. Thus, it is essential to apply “user-centered design” approaches and involve target users with various sociodemographic characteristics and technology experience throughout the development process. A strong involvement of the intended users of these systems in their design process would also improve the quality of the information provided to potential users of RCSs regarding their operation, type of data collected, and potential benefits of the technology. In this way, the involvement of the users would improve the quality of the process of obtaining the consent required from older adults to use the technology.

Another ethical challenge related to the use of RCSs is the fact that their wide implementation for older adults’ care may affect the distribution of health care resources. For instance, it has been found that for some older adults and informal and formal caregivers, the use of RCSs could replace humans in many caregiving tasks, eventually leading to a suppression of jobs or to a degradation of the quality of health care services [ 17 , 34 , 48 , 61 ]. In this regard, the participation of a third person (professional, volunteer, or family member) as a “human coach” could be considered when implementing RCSs in the older adults’ environment. This “human coach” could help build a “chain of trust” by being an intermediary between the RCS and the user. On the one hand, the involvement of a real person in the use of the RCS could reduce the risk of replacement of human assistance by technological assistance. On the other hand, the “human coach” could help enhance the acceptability and usability of the device, while at the same time, reassuring the user and providing recommendations to the developers, so that the RCS is consistent with users’ needs and desires. However, the benefits of involving a “human coach” in the RCS service provision has yet to be evaluated by scientific studies.

According to some studies [ 3 , 39 , 41 , 51 , 65 ], the use of RCSs can have an impact on social relationships, reducing human contact and even altering social relationships by creating tension between older adults and their caregivers. Thus, it would be interesting to identify the repercussions and implications of these devices in older adults’ daily life and in the life of the members of their social environment through new studies. It also seems necessary to evaluate the organizational impact of the implementation of RCSs and to identify potential obstacles to their use in the care professionals’ work context.

Our analysis also confirmed that for RCSs to provide personalized health-related recommendations, the collection of sensitive data is necessary. Data collection in this context also raises several ethical issues. For instance, personal data can be exposed to hacking and misuse. Proper data management, anonymization, and encryption are essential to protect the personal data of RCS users [ 86 ]. In addition, researchers and developers in this field must evaluate RCSs before implementation to ensure that they do not cause physical or moral harm to users. Thus, it has been suggested that stakeholders refer to local and regional regulatory and safety standards to guide their development and use.

Finally, our analysis also discussed how legal and ethical frameworks regarding the use of RCSs need to be adapted to cope with the constant development of new technologies. So far, existing legal frameworks are not yet adequate to respond effectively to the question of liability in case of damage caused by RCSs, particularly because these devices are becoming increasingly autonomous [ 80 ]. The establishment of “operational ethics committees in digital sciences and technologies” could help in the development and conduct of projects in this area [ 60 ]. Guidelines should be established to identify the types of applications and technological devices that require regulatory review and approval [ 44 ]. Research projects and working groups involving users, researchers, and lawyers should be set up to further investigate the legal and ethical issues related to the use of RCSs.

Some countries and regions, such as Europe and Japan have initiated the work of structuring relevant legal and ethical frameworks; however, their orientations and measures may differ culturally [ 78 ]. Future studies in the area of RCSs could consider the influence of cultural and socioeconomic specificities of the contexts of experimentation (countries and regions) regarding the acceptance and use of RCSs by older adults and formal and informal caregivers and regarding the definition of ethical and legal frameworks governing their uses. Therefore, the use of validated and widely applied analysis frameworks, for example, the Western, Educated, Industrialized, Rich and Democratic framework [ 89 ], formulated to measure countries’ commonalities in their approaches to the interpretation of behavioral research findings (eg, regarding technology adoption) could be interesting. The Western, Educated, Industrialized, Rich and Democratic framework [ 89 ] could help not only to explore the differences among countries regarding the validation and adoption of new technologies for older adult care but also to seek greater cultural and demographic diversity in technology research.

This dimension of cross-cultural comparison has received particular attention in the framework of a current international research partnership between Europe and Japan, such as the EU-Japan Virtual Coach for Smart Ageing (e-VITA) project. This project aims to develop a cross-cultural RCS that can be tailored to the needs of healthy older adults to promote aging well. The e-VITA RCS will be made available to older adults in their homes, which raises many of the ethical questions discussed in this paper. Therefore, the study will require the researchers to set up procedures adapted not only to the users but also to the 2 cultures (European and Japanese), respecting the corresponding ethical and legal regulations. Thus, it would be interesting to perform an analysis of the ethical issues raised by users from different countries and cultures within the framework of the e-VITA project.

Limitations

A narrative review of the literature was conducted to provide a nonexhaustive synthesis of the various ethical concerns and recommendations when using RCSs for older adults. This review has some limitations. Only articles in French and English were included. Some articles indicating ethical concerns or recommendations may not have been included when this information was not mentioned in the keywords or abstract.

Conclusions

The use of RCSs in the context of health care, particularly with an older adult population, tends to show many benefits. RCSs have the potential to improve the quality of life of older adults and their independence. When used in an ethical and appropriate manner, RCSs can help improve older adults’ emotions and cognitive and physical abilities and promote social relationships. By helping older adults to continue living at home for as long as possible, the use of health-oriented RCSs could help to address some of the challenges resulting from demographic aging. However, the use of these new health care technologies involves some ethical concerns, with the most cited issues being not only the risk of accidents, lack of reliability, loss of control, risk of deception, and risk of social isolation but also the confidentiality of data and liability in case of safety problems.

Some recommendations have been made in the past regarding the use of social and assistive robotic technologies for older adults, such as considering the opinion of target users; collecting their consent; training the care professionals to use them; and ensuring proper data management, anonymization, and encryption. However, the integration of RCSs in current health practices and, particularly, in the private homes of older adults can be disruptive. It requires the establishment of scalable and adapted ethical and regulatory frameworks that follow the technology progress and the social and digital change of society Thus, studies are needed to identify new ethical concerns arising from the organizational impact of the implementation of RCSs in different contexts, especially in the homes of older adults. The influence of cultural and socioeconomic specificities of the contexts of experimentation (countries and regions) regarding the acceptance and use of RCSs by older adults and formal and informal caregivers is also an area of interest for future studies.

Acknowledgments

This paper is a part of the EU-Japan Virtual Coach for Smart Ageing (e-VITA) project, which aims to develop a robotic coaching system for older adults [ 90 ]. The authors thank the collaborators who made this project possible: European Commission and Assistance Publique–Hôpitaux de Paris (Délégation à la Recherche Clinique et à l’Innovation). This review was based on data collected within the e-Vita project, funded by the European Union H2020 Program (grant 101016453) and the Japanese Ministry of Internal Affairs and Communication (Ministry of Internal Affairs and Communication; grant JPJ000595).

Data Availability

Data sharing is not applicable to this paper as no data sets were generated or analyzed during this review.

Conflicts of Interest

None declared.

The selected relevant articles on ethical issues regarding the use of robotic coaching solutions for older adults.

• United Nations. Croissant à un rythme plus lent, la population mondiale devrait atteindre 9,7 milliards d'habitants en 2050 et pourrait atteindre près de 11 milliards vers 2100 : Rapport de l'ONU. United Nations, Population Division. United Nations; 2019. URL: https://population.un.org/wpp/Publications/Files/WPP2019_PressRelease_FR.pdf [accessed 2022-03-04]
• Rapport mondial sur le vieillissement et la santé. Organisation Mondiale de la Santé. 2016. URL: https://apps.who.int/iris/handle/10665/206556 [accessed 2022-03-04]
• Chung J, Demiris G, Thompson HJ. Ethical considerations regarding the use of smart home technologies for older adults: an integrative review. Annu Rev Nurs Res. 2016;34:155-181. [ CrossRef ] [ Medline ]
• Ganesan B, Gowda T, Al-Jumaily A, Fong KN, Meena SK, Tong RK. Ambient assisted living technologies for older adults with cognitive and physical impairments: a review. Eur Rev Med Pharmacol Sci. Dec 2019;23(23):10470-10481. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Palazzolo J, Baudu C, Quaderi A. Psychothérapies du Sujet Âgé: Prise en Charge des Pathologies du Vieillissement. 2nd edition. Amsterdam, The Netherlands. Elsevier Masson; 2016.
• Principaux repères de l'OMS sur la démence. World Health Organization. URL: https://www.who.int/fr/news-room/fact-sheets/detail/dementia [accessed 2022-03-04]
• Guilbaud A, Mailliez A, Boulanger É. Vieillissement: une approche globale, multidimensionnelle et préventive. Med Sci (Paris). Dec 09, 2020;36(12):1173-1180. [ FREE Full text ] [ CrossRef ] [ Medline ]
• op den Akker H, Klaassen R, op den Akker R, Jones VM, Hermens HJ. Proceedings of the 26th IEEE International Symposium on Computer-Based Medical Systems. 2013. Presented at: CBMS '13; June 20-22, 2013:546-547; Porto, Portugal. URL: https://ieeexplore.ieee.org/document/6627870 [ CrossRef ]
• Görer B, Salah AA, Akın HL. A robotic fitness coach for the elderly. In: Augusto JC, Wichert R, Collier R, Keyson D, Salah AA, Tan AH, editors. Ambient Intelligence: Lecture Notes in Computer Science. Volume 8309. Cham, Switzerland. Springer; 2013.
• Lete N, Beristain A, García-Alonso A. Survey on virtual coaching for older adults. Health Informatics J. Dec 01, 2020;26(4):3231-3249. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Siewiorek D, Smailagic A, Dey A. Architecture and applications of virtual coaches. Proc IEEE. Aug 2012;100(8):2472-2488. [ CrossRef ]
• Kamphorst BA. E-coaching systems: what they are, and what they aren't. Pers Ubiquit Comput. May 23, 2017;21(4):625-632. [ CrossRef ]
• Kyriazakos S, Schlieter H, Gand K, Caprino M, Corbo M, Tropea P, et al. A novel virtual coaching system based on personalized clinical pathways for rehabilitation of older adults-requirements and implementation plan of the vCare project. Front Digit Health. Oct 5, 2020;2:546562. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Wrobel J, Pino M, Wargnier P, Rigaud AS. Robots et agents virtuels au service des personnes âgées : une revue de l’actualité en gérontechnologie. NPG Neurol Psychiatr Geriatr. Aug 2014;14(82):184-193. [ CrossRef ]
• Pérez PJ, Garcia-Zapirain B, Mendez-Zorrilla A. Caregiver and social assistant robot for rehabilitation and coaching for the elderly. Technol Health Care. 2015;23(3):351-357. [ CrossRef ] [ Medline ]
• Fasola J, Mataric MJ. A socially assistive robot exercise coach for the elderly. J Hum-Robot Interact. 2013;2(2):3-32. [ CrossRef ]
• Cresswell K, Cunningham-Burley S, Sheikh A. Health care robotics: qualitative exploration of key challenges and future directions. J Med Internet Res. Jul 04, 2018;20(7):e10410. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Winfield AF, Jirotka M. Ethical governance is essential to building trust in robotics and artificial intelligence systems. Philos Trans A Math Phys Eng Sci. Oct 15, 2018;376(2133):20180085. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Broadbent E, Stafford R, MacDonald B. Acceptance of healthcare robots for the older population: review and future directions. Int J of Soc Robotics. Oct 3, 2009;1(4):319-330. [ CrossRef ]
• Abou Allaban A, Wang M, Padır T. A systematic review of robotics research in support of in-home care for older adults. Information. Jan 30, 2020;11(2):75. [ CrossRef ]
• Cortellessa G, De Benedictis R, Fracasso F, Orlandini A, Umbrico A, Cesta A. AI and robotics to help older adults: revisiting projects in search of lessons learned. Paladyn J Behav Robot. 2021;12(1):356-378. [ CrossRef ]
• Lima MR, Wairagkar M, Gupta M, Rodriguez y Baena F, Barnaghi P, Sharp DJ, et al. Conversational affective social robots for ageing and dementia support. IEEE Trans Cogn Dev Syst. Dec 2022;14(4):1378-1397. [ CrossRef ]
• Sidner CL, Bickmore T, Nooraie B, Rich C, Ring L, Shayganfar M, et al. Creating new technologies for companionable agents to support isolated older adults. ACM Trans Interact Intell Syst. Jul 24, 2018;8(3):1-27. [ CrossRef ]
• Pepito JA, Locsin RC, Constantino RE. Caring for older persons in a technologically advanced nursing future. Health. 2019;11(05):439-463. [ CrossRef ]
• Pinheiro PR, Pinheiro PG, Filho RH, Barrozo JP, Rodrigues JJ, Pinheiro LI, et al. Integration of the mobile robot and internet of things to monitor older people. IEEE Access. 2020;8:138922-138933. [ CrossRef ]
• Chatterjee A, Gerdes MW, Martinez S. eHealth initiatives for the promotion of healthy lifestyle and allied implementation difficulties. In: Proceedings of the 2019 International Conference on Wireless and Mobile Computing, Networking and Communications. 2019. Presented at: WiMOB '19; October 21-23, 2019:1-8; Barcelona, Spain. URL: https://ieeexplore.ieee.org/document/8923324 [ CrossRef ]
• Yousuf H, Reintjens R, Slipszenko E, Blok S, Somsen GA, Tulevski II, et al. Effectiveness of web-based personalised e‑Coaching lifestyle interventions. Neth Heart J. Jan 28, 2019;27(1):24-29. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Bevilacqua R, Casaccia S, Cortellessa G, Astell A, Lattanzio F, Corsonello A, et al. Coaching through technology: a systematic review into efficacy and effectiveness for the ageing population. Int J Environ Res Public Health. Aug 15, 2020;17(16):5930. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Justo R, Ben Letaifa L, Palmero C, Gonzalez-Fraile E, Torp Johansen A, Vázquez A, et al. Analysis of the interaction between elderly people and a simulated virtual coach. J Ambient Intell Human Comput. May 22, 2020;11(12):6125-6140. [ CrossRef ]
• Fiorini L, De Mul M, Fabbricotti I, Limosani R, Vitanza A, D'Onofrio G, et al. Assistive robots to improve the independent living of older persons: results from a needs study. Disabil Rehabil Assist Technol. Jan 22, 2021;16(1):92-102. [ CrossRef ] [ Medline ]
• Sriram V, Jenkinson C, Peters M. Informal carers' experience of assistive technology use in dementia care at home: a systematic review. BMC Geriatr. Jun 14, 2019;19(1):160. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Pilotto A, Boi R, Petermans J. Technology in geriatrics. Age Ageing. Nov 01, 2018;47(6):771-774. [ CrossRef ] [ Medline ]
• Koceska N, Koceski S, Beomonte Zobel P, Trajkovik V, Garcia N. A telemedicine robot system for assisted and independent living. Sensors (Basel). Feb 18, 2019;19(4):834. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Broadbent E. Interactions with robots: the truths we reveal about ourselves. Annu Rev Psychol. Jan 03, 2017;68(1):627-652. [ CrossRef ] [ Medline ]
• Suwa S, Tsujimura M, Ide H, Kodate N, Ishimaru M, Shimamura A, et al. Home-care professionals’ ethical perceptions of the development and use of home-care robots for older adults in Japan. Int J Hum Comput Interact. Mar 13, 2020;36(14):1295-1303. [ CrossRef ]
• Nordgren A. How to respond to resistiveness towards assistive technologies among persons with dementia. Med Health Care Philos. Sep 6, 2018;21(3):411-421. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Diaz-Orueta U, Hopper L, Konstantinidis E. Shaping technologies for older adults with and without dementia: reflections on ethics and preferences. Health Informatics J. Dec 2020;26(4):3215-3230. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Portacolone E, Halpern J, Luxenberg J, Harrison KL, Covinsky KE. Ethical issues raised by the introduction of artificial companions to older adults with cognitive impairment: a call for interdisciplinary collaborations. J Alzheimers Dis. 2020;76(2):445-455. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Boada JP, Maestre BR, Genís CT. The ethical issues of social assistive robotics: a critical literature review. Technol Soc. Nov 2021;67:101726. [ CrossRef ]
• Vandemeulebroucke T, Dierckx de Casterlé B, Gastmans C. The use of care robots in aged care: a systematic review of argument-based ethics literature. Arch Gerontol Geriatr. Jan 2018;74:15-25. [ CrossRef ] [ Medline ]
• Zafrani O, Nimrod G. Towards a holistic approach to studying human-robot interaction in later life. Gerontologist. Jan 09, 2019;59(1):e26-e36. [ CrossRef ] [ Medline ]
• Anderson J, Kamphorst B. Ethics of e-coaching: implications of employing pervasive computing to promote healthy and sustainable lifestyles. In: Proceedings of the 2014 IEEE International Conference on Pervasive Computing and Communication Workshops. 2014. Presented at: PerComW '14; March 24-28, 2014:351-356; Budapest, Hungary. [ CrossRef ]
• Danaher J. Robot betrayal: a guide to the ethics of robotic deception. Ethics Inf Technol. Jan 04, 2020;22(2):117-128. [ CrossRef ]
• Fiske A, Henningsen P, Buyx A. Your robot therapist will see you now: ethical implications of embodied artificial intelligence in psychiatry, psychology, and psychotherapy. J Med Internet Res. May 09, 2019;21(5):e13216. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Grinbaum A, Chatila R, Devillers L, Ganascia JG, Tessier C, Dauchet M. Ethics in robotics research: CERNA mission and context. IEEE Robot Automat Mag. Sep 2017;24(3):139-145. [ CrossRef ]
• Ienca M, Jotterand F, Vică C, Elger B. Social and assistive robotics in dementia care: ethical recommendations for research and practice. Int J of Soc Robotics. Jun 22, 2016;8(4):565-573. [ CrossRef ]
• Körtner T. Ethical challenges in the use of social service robots for elderly people. Z Gerontol Geriatr. Jun 25, 2016;49(4):303-307. [ CrossRef ] [ Medline ]
• Operto F. Ethics in advanced robotics. IEEE Robot Automat Mag. Mar 2011;18(1):72-78. [ CrossRef ]
• Riek LD, Howard DA. A Code of Ethics for the Human-Robot Interaction Profession. In: Proceedings of 2014 Conference on We Robot. 2014. Presented at: WeRobot '14; April 4-5, 2014:1; Coral Gables, FL. URL: https:/​/robots.​law.miami.edu/​2014/​wp-content/​uploads/​2014/​03/​a-code-of-ethics-for-the-human-robot-interaction-profession-riek-howard.​pdf
• Sharkey A, Sharkey N. Granny and the robots: ethical issues in robot care for the elderly. Ethics Inf Technol. Jul 3, 2010;14(1):27-40. [ CrossRef ]
• Yew GC. Trust in and ethical design of carebots: the case for ethics of care. Int J Soc Robot. May 23, 2021;13(4):629-645. [ FREE Full text ] [ CrossRef ] [ Medline ]
• HTA Core Model ® version 3. EUnetHTA. URL: http://www.corehta.info/model/HTACoreModel3.0.pdf [accessed 2022-03-04]
• Health technology assessment process: fundamentals. EUPATI Toolbox. 2015. URL: https://toolbox.eupati.eu/resources/health-technology-assessment-process-fundamentals/ [accessed 2022-03-04]
• Tuli TB, Terefe TO, Rashid MM. Telepresence mobile robots design and control for social interaction. Int J Soc Robot. Jul 13, 2021;13(5):877-886. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Green BN, Johnson CD, Adams A. Writing narrative literature reviews for peer-reviewed journals: secrets of the trade. J Chiropr Med. Sep 2006;5(3):101-117. [ CrossRef ]
• Isabet B, Pino M, Lewis M, Benveniste S, Rigaud AS. Social telepresence robots: a narrative review of experiments involving older adults before and during the COVID-19 pandemic. Int J Environ Res Public Health. Mar 30, 2021;18(7):3597. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Naudé B, Rigaud AS, Pino M. Video calls for older adults: a narrative review of experiments involving older adults in elderly care institutions. Front Public Health. Jan 14, 2021;9:751150. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Saarni S, Hofmann B, Lampe K, Lühmann D, Mäkelä M, Velasco-Garrido M, et al. Ethical analysis to improve decision-making on health technologies. Bull World Health Organ. Aug 01, 2008;86(8):617-623. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Lampe K, Mäkelä M, Garrido MV, Anttila H, Autti-Rämö I, Hicks NJ, et al. The HTA core model: a novel method for producing and reporting health technology assessments. Int J Technol Assess Health Care. Dec 23, 2009;25(S2):9-20. [ CrossRef ]
• Collectif C. Éthique de la recherche en robotique. ALLISTENE. 2014. URL: http://cerna-ethics-allistene.org/digitalAssets/38/38704_Avis_robotique_livret.pdf [accessed 2024-04-02]
• Tisseron S. Introduction. In: Tisseron S, Tordo F, editors. Robots, de Nouveaux Partenaires de Soins Psychiques. Fairfield CA. Érès Publication; 2018.
• Frennert S, Östlund B. How do older people think and feel about robots in health- and elderly care? In: Proceedings of the 2018 INBOTS Conference on Inclusive Robotics for a Better Society. 2018. Presented at: INBOTS '18; October 16-18, 2018:167-174; Pisa, Italy. URL: https://link.springer.com/chapter/10.1007/978-3-030-24074-5_28 [ CrossRef ]
• van Maris A, Zook N, Caleb-Solly P, Studley M, Winfield A, Dogramadzi S. Designing ethical social robots-a longitudinal field study with older adults. Front Robot AI. Jan 24, 2020;7:1. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Wangmo T, Lipps M, Kressig RW, Ienca M. Ethical concerns with the use of intelligent assistive technology: findings from a qualitative study with professional stakeholders. BMC Med Ethics. Dec 19, 2019;20(1):98. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Portet F, Vacher M, Rossato S. Les technologies de la parole et du TALN pour l’assistance à domicile des personnes âgées : un rapide tour d’horizon (quick tour of NLP and speech technologies for ambient assisted living). In: Proceedings of the 2012 Workshop on Interactions Langagières pour personnes Agées Dans les habitats Intelligents. 2012. Presented at: ILADI '12; June 4-8, 2012; Grenoble, France. URL: https://aclanthology.org/W12-1402.pdf
• Bradwell HL, Winnington R, Thill S, Jones RB. Ethical perceptions towards real-world use of companion robots with older people and people with dementia: survey opinions among younger adults. BMC Geriatr. Jul 14, 2020;20(1):244. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Denning T, Matuszek C, Koscher K, Smith JR, Kohno T. A spotlight on security and privacy risks with future household robots: attacks and lessons. In: Proceedings of the 11th international conference on Ubiquitous computing. 2009. Presented at: UbiComp '09; September 30-October 3, 2009:105-114; Orlando, FL. URL: https://dl.acm.org/doi/10.1145/1620545.1620564 [ CrossRef ]
• Papadopoulos I, Koulouglioti C, Lazzarino R, Ali S. Enablers and barriers to the implementation of socially assistive humanoid robots in health and social care: a systematic review. BMJ Open. Jan 09, 2020;10(1):e033096. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Pripfl J, Kortner T, Batko-Klein D. Results of a real world trial with a mobile social service robot for older adults. In: Proceedings of the 11th ACM/IEEE International Conference on Human-Robot Interaction. 2016. Presented at: HRI '16; March 7-10, 2016:497-498; Christchurch, New Zealand. URL: https://ieeexplore.ieee.org/document/7451824 [ CrossRef ]
• Peek ST, Wouters EJ, van Hoof J, Luijkx KG, Boeije HR, Vrijhoef HJ. Factors influencing acceptance of technology for aging in place: a systematic review. Int J Med Inform. Apr 2014;83(4):235-248. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Lespinet-Najib V, Roche A, Chibaudel Q. Santé et handicap: d'une conception centrée «utilisateur» à la conception universelle. Ann Mines Réal Ind. 2017;2:25-27. [ CrossRef ]
• Déclaration d’Helsinki de l’AMM – Principes éthiques applicables à la recherche médicale impliquant des êtres humains. L'Association Médicale Mondiale. URL: https:/​/www.​wma.net/​fr/​policies-post/​declaration-dhelsinki-de-lamm-principes-ethiques-applicables-a-la-recherche-medicale-impliquant-des-etres-humains/​ [accessed 2022-02-03]
• Adams S, Niezen M. Digital ‘solutions’ to unhealthy lifestyle ‘problems’: the construction of social and personal risks in the development of eCoaches. Health Risk Soc. Feb 06, 2016;17(7-8):530-546. [ CrossRef ]
• Saerens P. Le droit des robots, un droit de l’homme en devenir ? Commun Technol Dév. Jun 30, 2020;(8). [ CrossRef ]
• Jenkins S, Draper H. Care, monitoring, and companionship: views on care robots from older people and their carers. Int J of Soc Robotics. Sep 15, 2015;7(5):673-683. [ CrossRef ]
• Règlement (UE) 2016/679 du parlement européen et du Conseil du 27 avril 2016 relatif à la protection des personnes physiques à l'égard du traitement des données à caractère personnel et à la libre circulation de ces données, et abrogeant la directive 95/46/CE (règlement général sur la protection des données) (Texte présentant de l'intérêt pour l'EEE). Le Parlement Européen Et Le Conseil De L'union Européenne. 2016. URL: http://data.europa.eu/eli/reg/2016/679/oj/fra [accessed 2022-02-03]
• Understanding the digital divide. OECD Digital Economy Papers. 2001. URL: https://econpapers.repec.org/paper/oecstiaab/49-en.htm [accessed 2024-04-02]
• Joachim C. Silver economy, robotique et droit - Comparaison franco-japonaise. Centre d’études et de Coopération Juridique Interdisciplinaire – Université de Poitier. Jan 2020. URL: https://hal.science/hal-03147937/document [accessed 2024-04-05]
• Kernaghan K. The rights and wrongs of robotics: ethics and robots in public organizations. Can Public Adm. Dec 12, 2014;57(4):485-506. [ CrossRef ]
• Bensoussan A, Puigmal L. Le droit des robots ? Quelle est l'autonomie de décision d'une machine ? Quelle protection mérite-t-elle ? Arch Philos Droit. 2017;59(1):165-174. [ CrossRef ]
• Tessier C. Robots autonomy: some technical issues. In: Lawless WF, Mittu R, Sofge D, Russell S, editors. Autonomy and Artificial Intelligence: A Threat or Savior? Cham, Switzerland. Springer; 2017:179-194.
• Nevejans N. Comment protéger l'homme face aux robots ? Arch Phil Droit. 2017;59(1):131-163. [ CrossRef ]
• Tambornino L, Lanzerath D, Rodrigues R, Wright D. SIENNA D4.3: survey of REC approaches and codes for artificial intelligence and robotics. Zenodo. Aug 19, 2019. URL: https://zenodo.org/records/4067990 [accessed 2024-04-04]
• Ethics guidelines for trustworthy AI. Shaping Europe's Digital Future. URL: https://digital-strategy.ec.europa.eu/en/library/ethics-guidelines-trustworthy-ai [accessed 2022-07-15]
• Kamali ME, Angelini L, Caon M, Carrino F, Rocke C, Guye S, et al. Virtual coaches for older adults’ wellbeing: a systematic review. IEEE Access. 2020;8:101884-101902. [ CrossRef ]
• Banos O, Nugent C. E-coaching for health. Computer. Mar 2018;51(3):12-15. [ CrossRef ]
• Gustafson DH, McTavish F, Gustafson DH, Mahoney JE, Johnson RA, Lee JD, et al. The effect of an information and communication technology (ICT) on older adults’ quality of life: study protocol for a randomized control trial. Trials. Apr 25, 2015;16(1):3-8. [ CrossRef ]
• Sakaguchi-Tang DK, Cunningham JL, Roldan W, Yip J, Kientz JA. Co-design with older adults: examining and reflecting on collaboration with aging communities. Proc ACM Hum Comput Interact. Oct 18, 2021;5(CSCW2):1-28. [ CrossRef ]
• Henrich J, Heine SJ, Norenzayan A. The weirdest people in the world? Behav Brain Sci. Jun 15, 2010;33(2-3):61-83. [ CrossRef ]
• EU-Japan virtual coach for smart ageing. e-VITA Consortium. URL: https://www.e-vita.coach/ [accessed 2024-04-05]

Abbreviations

Edited by A Mavragani; submitted 12.04.23; peer-reviewed by J Sedlakova, S Liu; comments to author 20.08.23; revised version received 22.12.23; accepted 12.03.24; published 18.06.24.

©Cécilia Palmier, Anne-Sophie Rigaud, Toshimi Ogawa, Rainer Wieching, Sébastien Dacunha, Federico Barbarossa, Vera Stara, Roberta Bevilacqua, Maribel Pino. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 18.06.2024.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.

• DOI: 10.1139/er-2024-0011
• Corpus ID: 269995846

A review of arsenic speciation in freshwater fish: Perspectives on monitoring approaches and analytical methods

• Adam T Lepage , Brian Laird , +2 authors Gretchen L. Lescord
• Published in Environmental Reviews 22 May 2024
• Environmental Science

Related Papers

Showing 1 through 3 of 0 Related Papers

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• v.36(2); 2022 Feb

Risk of bias: why measure it, and how?

Mark r. phillips.

1 Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, ON Canada

Peter Kaiser

2 Cole Eye Institute, Cleveland Clinic, Cleveland, OH USA

Lehana Thabane

3 Biostatistics Unit, St. Joseph’s Healthcare-Hamilton, Hamilton, ON USA

Mohit Bhandari

4 Department of Surgery, McMaster University, Hamilton, ON Canada

Varun Chaudhary

Selection bias.

High quality RCTs will randomize patients, and as important, conceal that randomization. Why? To limit selection bias. Selection bias is best described as a fundamental difference between the patients included in the treatment arms of a study due to the way in which patients were allocated to the treatment groups [ 1 , 4 ]. To assess selection bias, one must consider both the random sequence generation and allocation concealment methods of the RCT [ 3 ].

Sequence generation refers to the method in which patients were randomized to the treatment groups. A truly random sequence for treatment allocation means that the baseline characteristics of both groups will be inherently balanced, but a bias in this allocation can result in systematic differences between the comparison groups [ 1 , 3 ]. Some possible risks of bias due to sequence generation may exist due to non-, quasi-randomized methods of allocation. These methods may allow clinicians to choose which treatment patients’ will get within the study based on their expertize and prior experiences (i.e., A non-random factor). In addition, allocation concealment refers to the methods used to prevent anyone from being able to predict or deduce patient allocations [ 3 ]. Proper allocation concealment can prevent anyone in the research team from determining or predicting which patient’s have gotten which treatment within the trial. In summary, sequence generation refers to how patients are allocated to the comparison groups, and allocation concealment refers to how that allocation is kept secret from all relevant parties.

Performance bias

What about factors that can influence how a patient of clinician performs during the course of an RCT? Performance bias may be present if there are differences between the study groups as a result of systematic differences in performance outside of the study treatment received [ 1 ]. Risks of performance bias can result due to the masking (or blinding) methods of participants and personnel [ 1 ]. If masking is appropriately implemented, one can be assured that there was no additional and undue influence on the outcome of patients that occurred other than the assigned intervention [ 3 , 5 ]. Many vital outcomes in ophthalmology, such as assessments of visual acuity, could be skewed if the patient or assessor are aware of the treatment allocation. When assessing performance bias, it is important to consider if the lack of masking could reasonably impact the outcomes being assessed [ 1 ].

Detection bias

The prior biases have focussed on the methods for randomizing and masking patients and clinicians, but what about biases in the way outcomes are measured? Detection bias can be described as the possibility for differences between the comparison groups with regard to how the outcomes are measured or assessed [ 1 ]. Detection bias also focuses on the concept of masking; however, it is the outcome assessor that should be masked in order to mitigate detection bias [ 3 ]. Masking of outcome assessors ensures that the methods in which an outcome is measured does not differ between patients allocated to the comparison groups—meaning that the outcome measurement is consistent for all participants in the study [ 1 , 3 ].

Attrition bias

After patients have been included in an RCT, there is always a potential for them to withdrawal from the study before completing their follow-ups. Attrition bias can occur as a result of systematic cause of patient withdrawals in a study that disproportionately affect a certain subset of patients [ 1 ]. If a cause for withdrawal is present—or more predominant—in the comparison groups, the withdrawal imbalance could impact the results and conclusions drawn from the study [ 1 , 3 ]. If a specific group of patients were more likely to withdrawal from the study within one of the comparison groups, the imbalance would have clear implications on results [ 1 , 6 ].

Reporting bias

The final form of bias that any clinician should consider when reading an RCT is reporting bias. Reporting bias may occur when there are concerns with regard to the outcomes reported within the results of a study [ 1 ]. Selective outcome reporting is the primary concern in this form of bias, which refers to the reporting of some, but not all, measured outcomes within a study’s results [ 1 , 3 ]. This commonly manifests as a study reporting on significant outcome findings, while omitting outcome findings that are not significant [ 1 , 7 ]. Although this can be difficult to detect, it highlights the importance of a pre-defined study protocol that identifies all outcomes that will be assessed. You, as a reader, should actively seek the confirmation of that important step.

How should we interpret risk of bias assessments?

The next time you read an RCT, consider these risks of bias before making changes to your clinical practice. RCTs that are deemed to have a high risk of bias should be interpreted cautiously, as biases directly impacts the validity of the findings [ 1 ]. Empirical investigations have shown studies with high risk of bias may lead to an exaggeration of treatment effects within trials when compared to studies with a low risk of bias [ 8 , 9 ]. It is common to assess the risks of bias in a study based solely on the reporting in the study manuscript, but poor reporting is not the same as biased conduct [ 3 ]. This is an important distinction to make with regard to risk of bias assessment that requires thoughtful consideration of the potential validity implications of study design decisions. You can refer to the Cochrane Handbook for Systematic Reviews of Interventions, “Chapter 8: Risk of Bias in Randomized Trials” for a comprehensive guide to risk of bias assessment for RCTs [ 1 ].

What is risk of bias

• Clinicians read and interpret randomized controlled trials (RCTs) on a regular basis to inform their practice—but how can they be certain that the RCT is accurate and reliable? Not all RCTs are the same, and thus careful consideration needs to be taken when determining if RCT results are worthy of changing the way you manage future patients. The validity of an RCT can best be evaluated by understanding the possible risks of bias for that particular study.
• Bias exists when a component of the design or execution of a study has systematic impacts on the results of the study that deviate from the truth. When such a bias exists, a study could result in over- or underestimation of the truth, compromising the validity of the study findings or results—even if all other facets of the study were appropriate [ 1 – 3 ].
• Imagine, for example, providing navigation using a compass that was not accurately pointing “North”, but instead had a bias of pointing toward “North East”. Even if you provided thorough navigation steps to a fellow traveller, the end result will not be accurate due to the bias that existed from the inaccuracy of the compass. Similarly, an otherwise robust study that has some form of bias may provide results to clinicians and patients that are not accurate, despite the comprehensiveness of the investigation. With this in mind, it is important to understand the types of bias that may exist within RCTs, how to detect these potential biases, and how to interpret the results of a study in the context of such possible biases.

What types of bias exist, and how can we assess them

Types of Bias Summary.

Author contributions

MP was responsible for conception of idea, writing of paper and review of paper. VC was responsible for conception of idea, writing of paper and review of paper. MB was responsible for conception of idea, writing of paper and review of paper. PK was responsible for critical review and feedback on paper. LT was responsible for critical review and feedback on paper.

Competing interests

MP: Nothing to disclose. LT: Nothing to disclose. VC: Advisory Board Member: Alcon, Roche, Bayer, Novartis; Grants: Bayer, Novartis—unrelated to this study. PK: Consultant; Novartis, Bayer, Regeneron, Kanghong, Allergan, RegenxBio—unrelated to this study. MB: Research funds; Pendopharm, Bioventus, Acumed—unrelated to this study.

Members of the Retina Evidence Trials InterNational Alliance (R.E.T.I.N.A.) Study Group are listed below Author contributions.

The original version of this article was revised: In this article the middle initial in author name Sophie J. Bakri was missing.

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Change history

A Correction to this paper has been published: 10.1038/s41433-021-01913-3

Contributor Information

Charles c. wykoff.

5 Retinal Consultants of Houston, Retina Consultants of America, Blanton Eye Institute, Houston Methodist Hospital, Weill Cornell Medical College, Houston, TX USA

Sobha Sivaprasad

6 NIHR Moorfields Biomedical Research Centre, Moorfields Eye Hospital, London, UK

David Sarraf

7 Retinal Disorders and Ophthalmic Genetics, Stein Eye Institute, University of California, Los Angeles, CA USA

Sophie J. Bakri

8 Department of Ophthalmology, Mayo Clinic, Rochester, MN USA

Sunir J. Garg

9 The Retina Service at Wills Eye Hospital, Philadelphia, PA USA

Rishi P. Singh

10 Center for Ophthalmic Bioinformatics, Cole Eye Institute, Cleveland Clinic, Cleveland, OH USA

11 Cleveland Clinic Lerner College of Medicine, Cleveland, OH USA

Frank G. Holz

12 Department of Ophthalmology, University of Bonn, Boon, Germany

Tien Yin Wong

13 Singapore Eye Research Institute, Singapore, Singapore

14 Singapore National Eye Centre, Duke-NUD Medical School, Singapore, Singapore