risk assessment in research paper

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• My Account Login
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 08 April 2024

A case study on the relationship between risk assessment of scientific research projects and related factors under the Naive Bayesian algorithm

• Xuying Dong 1 &
• Wanlin Qiu 1  

Scientific Reports volume  14 , Article number:  8244 ( 2024 ) Cite this article

185 Accesses

Metrics details

• Computer science
• Mathematics and computing

This paper delves into the nuanced dynamics influencing the outcomes of risk assessment (RA) in scientific research projects (SRPs), employing the Naive Bayes algorithm. The methodology involves the selection of diverse SRPs cases, gathering data encompassing project scale, budget investment, team experience, and other pertinent factors. The paper advances the application of the Naive Bayes algorithm by introducing enhancements, specifically integrating the Tree-augmented Naive Bayes (TANB) model. This augmentation serves to estimate risk probabilities for different research projects, shedding light on the intricate interplay and contributions of various factors to the RA process. The findings underscore the efficacy of the TANB algorithm, demonstrating commendable accuracy (average accuracy 89.2%) in RA for SRPs. Notably, budget investment (regression coefficient: 0.68, P < 0.05) and team experience (regression coefficient: 0.51, P < 0.05) emerge as significant determinants obviously influencing RA outcomes. Conversely, the impact of project size (regression coefficient: 0.31, P < 0.05) is relatively modest. This paper furnishes a concrete reference framework for project managers, facilitating informed decision-making in SRPs. By comprehensively analyzing the influence of various factors on RA, the paper not only contributes empirical insights to project decision-making but also elucidates the intricate relationships between different factors. The research advocates for heightened attention to budget investment and team experience when formulating risk management strategies. This strategic focus is posited to enhance the precision of RAs and the scientific foundation of decision-making processes.

Similar content being viewed by others

A method for managing scientific research project resource conflicts and predicting risks using BP neural networks

Prediction of SMEs’ R&D performances by machine learning for project selection

Machine learning in project analytics: a data-driven framework and case study

Introduction.

Scientific research projects (SRPs) stand as pivotal drivers of technological advancement and societal progress in the contemporary landscape 1 , 2 , 3 . The dynamism of SRP success hinges on a multitude of internal and external factors 4 . Central to effective project management, Risk assessment (RA) in SRPs plays a critical role in identifying and quantifying potential risks. This process not only aids project managers in formulating strategic decision-making approaches but also enhances the overall success rate and benefits of projects. In a recent contribution, Salahuddin 5 provides essential numerical techniques indispensable for conducting RAs in SRPs. Building on this foundation, Awais and Salahuddin 6 delve into the assessment of risk factors within SRPs, notably introducing the consideration of activation energy through an exploration of the radioactive magnetohydrodynamic model. Further expanding the scope, Awais and Salahuddin 7 undertake a study on the natural convection of coupled stress fluids. However, RA of SRPs confronts a myriad of challenges, underscoring the critical need for novel methodologies 8 . Primarily, the intricate nature of SRPs renders precise RA exceptionally complex and challenging. The project’s multifaceted dimensions, encompassing technology, resources, and personnel, are intricately interwoven, posing a formidable challenge for traditional assessment methods to comprehensively capture all potential risks 9 . Furthermore, the intricate and diverse interdependencies among various project factors contribute to the complexity of these relationships, thereby limiting the efficacy of conventional methods 10 , 11 , 12 . Traditional approaches often focus solely on the individual impact of diverse factors, overlooking the nuanced relationships that exist between them—an inherent limitation in the realm of RA for SRPs 13 , 14 , 15 .

The pursuit of a methodology capable of effectively assessing project risks while elucidating the intricate interplay of different factors has emerged as a focal point in SRPs management 16 , 17 , 18 . This approach necessitates a holistic consideration of multiple factors, their quantification in contributing to project risks, and the revelation of their correlations. Such an approach enables project managers to more precisely predict and respond to risks. Marx-Stoelting et al. 19 , current approaches for the assessment of environmental and human health risks due to exposure to chemical substances have served their purpose reasonably well. Additionally, Awais et al. 20 highlights the significance of enthalpy changes in SRPs risk considerations, while Awais et al. 21 delve into the comprehensive exploration of risk factors in Eyring-Powell fluid flow in magnetohydrodynamics, particularly addressing viscous dissipation and activation energy effects. The Naive Bayesian algorithm, recognized for its prowess in probability and statistics, has yielded substantial results in information retrieval and data mining in recent years 22 . Leveraging its advantages in classification and probability estimation, the algorithm presents a novel approach for RA of SRPs 23 . Integrating probability analysis into RA enables a more precise estimation of project risks by utilizing existing project data and harnessing the capabilities of the Naive Bayesian algorithms. This method facilitates a quantitative, statistical analysis of various factors, effectively navigating the intricate relationships between them, thereby enhancing the comprehensiveness and accuracy of RA for SRPs.

This paper seeks to employ the Naive Bayesian algorithm to estimate the probability of risks by carefully selecting distinct research project cases and analyzing multidimensional data, encompassing project scale, budget investment, and team experience. Concurrently, Multiple Linear Regression (MLR) analysis is applied to quantify the influence of these factors on the assessment results. The paper places particular emphasis on exploring the intricate interrelationships between different factors, aiming to provide a more specific and accurate reference framework for decision-making in SRPs management.

This paper introduces several innovations and contributions to the field of RA for SRPs:

Comprehensive Consideration of Key Factors: Unlike traditional research that focuses on a single factor, this paper comprehensively considers multiple key factors, such as project size, budget investment, and team experience. This holistic analysis enhances the realism and thoroughness of RA for SRPs.

Introduction of Tree-Enhanced Naive Bayes Model: The naive Bayes algorithm is introduced and further improved through the proposal of a tree-enhanced naive Bayes model. This algorithm exhibits unique advantages in handling uncertainty and complexity, thereby enhancing its applicability and accuracy in the RA of scientific and technological projects.

Empirical Validation: The effectiveness of the proposed method is not only discussed theoretically but also validated through empirical cases. The analysis of actual cases provides practical support and verification, enhancing the credibility of the research results.

Application of MLR Analysis: The paper employs MLR analysis to delve into the impact of various factors on RA. This quantitative analysis method adds specificity and operability to the research, offering a practical decision-making basis for scientific and technological project management.

Discovery of New Connections and Interactions: The paper uncovers novel connections and interactions, such as the compensatory role of team experience for budget-related risks and the impact of the interaction between project size and budget investment on RA results. These insights provide new perspectives for decision-making in technology projects, contributing significantly to the field of RA for SRPs in terms of both importance and practical value.

The paper is structured as follows: “ Introduction ” briefly outlines the significance of RA for SRPs. Existing challenges within current research are addressed, and the paper’s core objectives are elucidated. A distinct emphasis is placed on the innovative aspects of this research compared to similar studies. The organizational structure of the paper is succinctly introduced, providing a brief overview of each section’s content. “ Literature review ” provides a comprehensive review of relevant theories and methodologies in the domain of RA for SRPs. The current research landscape is systematically examined, highlighting the existing status and potential gaps. Shortcomings in previous research are analyzed, laying the groundwork for the paper’s motivation and unique contributions. “ Research methodology ” delves into the detailed methodologies employed in the paper, encompassing data collection, screening criteria, preprocessing steps, and more. The tree-enhanced naive Bayes model is introduced, elucidating specific steps and the purpose behind MLR analysis. “ Results and discussion ” unfolds the results and discussions based on selected empirical cases. The representativeness and diversity of these cases are expounded upon. An in-depth analysis of each factor’s impact and interaction in the context of RA is presented, offering valuable insights. “ Discussion ” succinctly summarizes the entire research endeavor. Potential directions for further research and suggestions for improvement are proposed, providing a thoughtful conclusion to the paper.

Literature review

A review of ra for srps.

In recent years, the advancement of SRPs management has led to the evolution of various RA methods tailored for SRPs. The escalating complexity of these projects poses a challenge for traditional methods, often falling short in comprehensively considering the intricate interplay among multiple factors and yielding incomplete assessment outcomes. Scholars, recognizing the pivotal role of factors such as project scale, budget investment, and team experience in influencing project risks, have endeavored to explore these dynamics from diverse perspectives. Siyal et al. 24 pioneered the development and testing of a model geared towards detecting SRPs risks. Chen et al. 25 underscored the significance of visual management in SRPs risk management, emphasizing its importance in understanding and mitigating project risks. Zhao et al. 26 introduced a classic approach based on cumulative prospect theory, offering an optional method to elucidate researchers’ psychological behaviors. Their study demonstrated the enhanced rationality achieved by utilizing the entropy weight method to derive attribute weight information under Pythagorean fuzzy sets. This approach was then applied to RA for SRPs, showcasing a model grounded in the proposed methodology. Suresh and Dillibabu 27 proposed an innovative hybrid fuzzy-based machine learning mechanism tailored for RA in software projects. This hybrid scheme facilitated the identification and ranking of major software project risks, thereby supporting decision-making throughout the software project lifecycle. Akhavan et al. 28 introduced a Bayesian network modeling framework adept at capturing project risks by calculating the uncertainty of project net present value. This model provided an effective means for analyzing risk scenarios and their impact on project success, particularly applicable in evaluating risks for innovative projects that had undergone feasibility studies.

A review of factors affecting SRPs

Within the realm of SRPs management, the assessment and proficient management of project risks stand as imperative components. Consequently, a range of studies has been conducted to explore diverse methods and models aimed at enhancing the comprehension and decision support associated with project risks. Guan et al. 29 introduced a new risk interdependence network model based on Monte Carlo simulation to support decision-makers in more effectively assessing project risks and planning risk management actions. They integrated interpretive structural modeling methods into the model to develop a hierarchical project risk interdependence network based on identified risks and their causal relationships. Vujović et al. 30 provided a new method for research in project management through careful analysis of risk management in SRPs. To confirm the hypothesis, the study focused on educational organizations and outlined specific project management solutions in business systems, thereby improving the business and achieving positive business outcomes. Muñoz-La Rivera et al. 31 described and classified the 100 identified factors based on the dimensions and aspects of the project, assessed their impact, and determined whether they were shaping or directly affecting the occurrence of research project accidents. These factors and their descriptions and classifications made significant contributions to improving the security creation of the system and generating training and awareness materials, fostering the development of a robust security culture within organizations. Nguyen et al. concentrated on the pivotal risk factors inherent in design-build projects within the construction industry. Effective identification and management of these factors enhanced project success and foster confidence among owners and contractors in adopting the design-build approach 32 . Their study offers valuable insights into RA in project management and the adoption of new contract forms. Nguyen and Le delineated risk factors influencing the quality of 20 civil engineering projects during the construction phase 33 . The top five risks identified encompass poor raw material quality, insufficient worker skills, deficient design documents and drawings, geographical challenges at construction sites, and inadequate capabilities of main contractors and subcontractors. Meanwhile, Nguyen and Phu Pham concentrated on office building projects in Ho Chi Minh City, Vietnam, to pinpoint key risk factors during the construction phase 34 . These factors were classified into five groups based on their likelihood and impact: financial, management, schedule, construction, and environmental. Findings revealed that critical factors affecting office building projects encompassed both natural elements (e.g., prolonged rainfall, storms, and climate impacts) and human factors (e.g., unstable soil, safety behavior, owner-initiated design changes), with schedule-related risks exerting the most significant influence during the construction phase of Ho Chi Minh City’s office building projects. This provides construction and project management practitioners with fresh insights into risk management, aiding in the comprehensive identification, mitigation, and management of risk factors in office building projects.

While existing research has made notable strides in RA for SRPs, certain limitations persist. These studies limitations in quantifying the degree of influence of various factors and analyzing their interrelationships, thereby falling short of offering specific and actionable recommendations. Traditional methods, due to their inherent limitations, struggle to precisely quantify risk degrees and often overlook the intricate interplay among multiple factors. Consequently, there is an urgent need for a comprehensive method capable of quantifying the impact of diverse factors and revealing their correlations. In response to this exigency, this paper introduces the TANB model. The unique advantages of this algorithm in the RA of scientific and technological projects have been fully realized. Tailored to address the characteristics of uncertainty and complexity, the model represents a significant leap forward in enhancing applicability and accuracy. In comparison with traditional methods, the TANB model exhibits greater flexibility and a heightened ability to capture dependencies between features, thereby elevating the overall performance of RA. This innovative method emerges as a more potent and reliable tool in the realm of scientific and technological project management, furnishing decision-makers with more comprehensive and accurate support for RA.

Research methodology

This paper centers on the latest iteration of ISO 31000, delving into the project risk management process and scrutinizing the RA for SRPs and their intricate interplay with associated factors. ISO 31000, an international risk management standard, endeavors to furnish businesses, organizations, and individuals with a standardized set of risk management principles and guidelines, defining best practices and establishing a common framework. The paper unfolds in distinct phases aligned with ISO 31000:

Risk Identification: Employing data collection and preparation, a spectrum of factors related to project size, budget investment, team member experience, project duration, and technical difficulty were identified.

RA: Utilizing the Naive Bayes algorithm, the paper conducts RA for SRPs, estimating the probability distribution of various factors influencing RA results.

Risk Response: The application of the Naive Bayes model is positioned as a means to respond to risks, facilitating the formulation of apt risk response strategies based on calculated probabilities.

Monitoring and Control: Through meticulous data collection, model training, and verification, the paper illustrates the steps involved in monitoring and controlling both data and models. Regular monitoring of identified risks and responses allows for adjustments when necessary.

Communication and Reporting: Maintaining effective communication throughout the project lifecycle ensures that stakeholders comprehend the status of project risks. Transparent reporting on discussions and outcomes contributes to an informed project environment.

Data collection and preparation

In this paper, a meticulous approach is undertaken to select representative research project cases, adhering to stringent screening criteria. Additionally, a thorough review of existing literature is conducted and tailored to the practical requirements of SRPs management. According to Nguyen et al., these factors play a pivotal role in influencing the RA outcomes of SRPs 35 . Furthermore, research by He et al. underscored the significant impact of team members’ experience on project success 36 . Therefore, in alignment with our research objectives and supported by the literature, this paper identifies variables such as project scale, budget investment, team member experience, project duration, and technical difficulty as the focal themes. To ensure the universality and scientific rigor of our findings, the paper adheres to stringent selection criteria during the project case selection process. After preliminary screening of SRPs completed in the past 5 years, considering factors such as project diversity, implementation scales, and achieved outcomes, five representative projects spanning diverse fields, including engineering, medicine, and information technology, are ultimately selected. These project cases are chosen based on their capacity to represent various scales and types of SRPs, each possessing a typical risk management process, thereby offering robust and comprehensive data support for our study. The subsequent phase involves detailed data collection on each chosen project, encompassing diverse dimensions such as project scale, budget investment, team member experience, project cycle, and technical difficulty. The collected data undergo meticulous preprocessing to ensure data quality and reliability. The preprocessing steps comprised data cleaning, addressing missing values, handling outliers, culminating in the creation of a self-constructed dataset. The dataset encompasses over 500 SRPs across diverse disciplines and fields, ensuring statistically significant and universal outcomes. Particular emphasis is placed on ensuring dataset diversity, incorporating projects of varying scales, budgets, and team experience levels. This comprehensive coverage ensures the representativeness and credibility of the study on RA in SRPs. New influencing factors are introduced to expand the research scope, including project management quality (such as time management and communication efficiency), historical success rate, industry dynamics, and market demand. Detailed definitions and quantifications are provided for each new variable to facilitate comprehensive data processing and analysis. For project management quality, consideration is given to time management accuracy and communication frequency and quality among team members. Historical success rate is determined by reviewing past project records and outcomes. Industry dynamics are assessed by consulting the latest scientific literature and patent information. Market demand is gauged through market research and user demand surveys. The introduction of these variables enriches the understanding of RA in SRPs and opens up avenues for further research exploration.

At the same time, the collected data are integrated and coded in order to apply Naive Bayes algorithm and MLR analysis. For cases involving qualitative data, this paper uses appropriate coding methods to convert it into quantitative data for processing in the model. For example, for the qualitative feature of team member experience, numerical values are used to represent different experience levels, such as 0 representing beginners, 0 representing intermediate, and 2 representing advanced. The following is a specific sample data set example (Table 1 ). It shows the processed structured data, and the values in the table represent the specific characteristics of each project.

Establishment of naive Bayesian model

The Naive Bayesian algorithm, a probabilistic and statistical classification method renowned for its effectiveness in analyzing and predicting multi-dimensional data, is employed in this paper to conduct the RA for SRPs. The application of the Naive Bayesian algorithm to RA for SRPs aims to discern the influence of various factors on the outcomes of RA. The Naive Bayesian algorithm, depicted in Fig.  1 , operates on the principles of Bayesian theorem, utilizing posterior probability calculations for classification tasks. The fundamental concept of this algorithm hinges on the assumption of independence among different features, embodying the “naivety” hypothesis. In the context of RA for SRPs, the Naive Bayesian algorithm is instrumental in estimating the probability distribution of diverse factors affecting the RA results, thereby enhancing the precision of risk estimates. In the Naive Bayesian model, the initial step involves the computation of posterior probabilities for each factor, considering the given RA result conditions. Subsequently, the category with the highest posterior probability is selected as the predictive outcome.

Naive Bayesian algorithm process.

In Fig.  1 , the data collection process encompasses vital project details such as project scale, budget investment, team member experience, project cycle, technical difficulty, and RA results. This meticulous collection ensures the integrity and precision of the dataset. Subsequently, the gathered data undergoes integration and encoding to convert qualitative data into quantitative form, facilitating model processing and analysis. Tailored to specific requirements, relevant features are chosen for model construction, accompanied by essential preprocessing steps like standardization and normalization. The dataset is then partitioned into training and testing sets, with the model trained on the former and its performance verified on the latter. Leveraging the training data, a Naive Bayesian model is developed to estimate probability distribution parameters for various features across distinct categories. Ultimately, the trained model is employed to predict new project features, yielding RA results.

Naive Bayesian models, in this context, are deployed to forecast diverse project risk levels. Let X symbolize the feature vector, encompassing project scale, budget investment, team member experience, project cycle, and technical difficulty. The objective is to predict the project’s risk level, denoted as Y. Y assumes discrete values representing distinct risk levels. Applying the Bayesian theorem, the posterior probability P(Y|X) is computed, signifying the probability distribution of projects falling into different risk levels given the feature vector X. The fundamental equation governing the Naive Bayesian model is expressed as:

In Eq. ( 1 ), P(Y|X) represents the posterior probability, denoting the likelihood of the project belonging to a specific risk level. P(X|Y) signifies the class conditional probability, portraying the likelihood of the feature vector X occurring under known risk level conditions. P(Y) is the prior probability, reflecting the antecedent likelihood of the project pertaining to a particular risk level. P(X) acts as the evidence factor, encapsulating the likelihood of the feature vector X occurring.

The Naive Bayes, serving as the most elementary Bayesian network classifier, operates under the assumption of attribute independence given the class label c , as expressed in Eq. ( 2 ):

The classification decision formula for Naive Bayes is articulated in Eq. ( 3 ):

The Naive Bayes model, rooted in the assumption of conditional independence among attributes, often encounters deviations from reality. To address this limitation, the Tree-Augmented Naive Bayes (TANB) model extends the independence assumption by incorporating a first-order dependency maximum-weight spanning tree. TANB introduces a tree structure that more comprehensively models relationships between features, easing the constraints of the independence assumption and concurrently mitigating issues associated with multicollinearity. This extension bolsters its efficacy in handling intricate real-world data scenarios. TANB employs conditional mutual information \(I(X_{i} ;X_{j} |C)\) to gauge the dependency between attributes \(X_{j}\) and \(X_{i}\) , thereby constructing the maximum weighted spanning tree. In TANB, any attribute variable \(X_{i}\) is permitted to have at most one other attribute variable as its parent node, expressed as \(Pa\left( {X_{i} } \right) \le 2\) . The joint probability \(P_{con} \left( {x,c} \right)\) undergoes transformation using Eq. ( 4 ):

In Eq. ( 4 ), \(x_{r}\) refers to the root node, which can be expressed as Eq. ( 5 ):

TANB classification decision equation is presented below:

In the RA of SRPs, normal distribution parameters, such as mean (μ) and standard deviation (σ), are estimated for each characteristic dimension (project scale, budget investment, team member experience, project cycle, and technical difficulty). This estimation allows the calculation of posterior probabilities for projects belonging to different risk levels under given feature vector conditions. For each feature dimension \({X}_{i}\) , the mean \({mu}_{i,j}\) and standard deviation \({{\text{sigma}}}_{i,j}\) under each risk level are computed, where i represents the feature dimension, and j denotes the risk level. Parameter estimation employs the maximum likelihood method, and the specific calculations are as follows.

In Eqs. ( 7 ) and ( 8 ), \({N}_{j}\) represents the number of projects belonging to risk level j . \({x}_{i,k}\) denotes the value of the k -th item in the feature dimension i . Finally, under a given feature vector, the posterior probability of a project with risk level j is calculated as Eq. ( 9 ).

In Eq. ( 9 ), d represents the number of feature dimensions, and Z is the normalization factor. \(P(Y=j)\) represents the prior probability of category j . \(P({X}_{i}\mid Y=j)\) represents the normal distribution probability density function of feature dimension i under category j . The risk level of a project can be predicted by calculating the posterior probabilities of different risk levels to achieve RA for SRPs.

This paper integrates the probability estimation of the Naive Bayes model with actual project risk response strategies, enabling a more flexible and targeted response to various risk scenarios. Such integration offers decision support to project managers, enhancing their ability to address potential challenges effectively and ultimately improving the overall success rate of the project. This underscores the notion that risk management is not solely about problem prevention but stands as a pivotal factor contributing to project success.

MLR analysis

MLR analysis is used to validate the hypothesis to deeply explore the impact of various factors on RA of SRPs. Based on the previous research status, the following research hypotheses are proposed.

Hypothesis 1: There is a positive relationship among project scale, budget investment, and team member experience and RA results. As the project scale, budget investment, and team member experience increase, the RA results also increase.

Hypothesis 2: There is a negative relationship between the project cycle and the RA results. Projects with shorter cycles may have higher RA results.

Hypothesis 3: There is a complex relationship between technical difficulty and RA results, which may be positive, negative, or bidirectional in some cases. Based on these hypotheses, an MLR model is established to analyze the impact of factors, such as project scale, budget investment, team member experience, project cycle, and technical difficulty, on RA results. The form of an MLR model is as follows.

In Eq. ( 10 ), Y represents the RA result (dependent variable). \({X}_{1}\) to \({X}_{5}\) represent factors, such as project scale, budget investment, team member experience, project cycle, and technical difficulty (independent variables). \({\beta }_{0}\) to \({\beta }_{5}\) are the regression coefficients, which represent the impact of various factors on the RA results. \(\epsilon\) represents a random error term. The model structure is shown in Fig.  2 .

Schematic diagram of an MLR model.

In Fig.  2 , the MLR model is employed to scrutinize the influence of various independent variables on the outcomes of RA. In this specific context, the independent variables encompass project size, budget investment, team member experience, project cycle, and technical difficulty, all presumed to impact the project’s RA results. Each independent variable is denoted as a node in the model, with arrows depicting the relationships between these factors. In an MLR model, the arrow direction signifies causality, illustrating the influence of an independent variable on the dependent variable.

When conducting MLR analysis, it is necessary to estimate the parameter \(\upbeta\) in the regression model. These parameters determine the relationship between the independent and dependent variables. Here, the Ordinary Least Squares (OLS) method is applied to estimate these parameters. The OLS method is a commonly used parameter estimation method aimed at finding parameter values that minimize the sum of squared residuals between model predictions and actual observations. The steps are as follows. Firstly, based on the general form of an MLR model, it is assumed that there is a linear relationship between the independent and dependent variables. It can be represented by a linear equation, which includes regression coefficients β and the independent variable X. For each observation value, the difference between its predicted and actual values is calculated, which is called the residual. Residual \({e}_{i}\) can be expressed as:

In Eq. ( 11 ), \({Y}_{i}\) is the actual observation value, and \({\widehat{Y}}_{i}\) is the value predicted by the model. The goal of the OLS method is to adjust the regression coefficients \(\upbeta\) to minimize the sum of squared residuals of all observations. This can be achieved by solving an optimization problem, and the objective function is the sum of squared residuals.

Then, the estimated value of the regression coefficient \(\upbeta\) that minimizes the sum of squared residuals can be obtained by taking the derivative of the objective function and making the derivative zero. The estimated values of the parameters can be obtained by solving this system of equations. The final estimated regression coefficient can be expressed as:

In Eq. ( 13 ), X represents the independent variable matrix. Y represents the dependent variable vector. \(({X}^{T}X{)}^{-1}\) is the inverse of a matrix, and \(\widehat{\beta }\) is a parameter estimation vector.

Specifically, solving for the estimated value of regression coefficient \(\upbeta\) requires matrix operation and statistical analysis. Based on the collected project data, substitute it into the model and calculate the residual. Then, the steps of the OLS method are used to obtain parameter estimates. These parameter estimates are used to establish an MLR model to predict RA results and further analyze the influence of different factors.

The degree of influence of different factors on the RA results can be determined by analyzing the value of the regression coefficient β. A positive \(\upbeta\) value indicates that the factor has a positive impact on the RA results, while a negative \(\upbeta\) value indicates that the factor has a negative impact on the RA results. Additionally, hypothesis testing can determine whether each factor is significant in the RA results.

The TANB model proposed in this paper extends the traditional naive Bayes model by incorporating conditional dependencies between attributes to enhance the representation of feature interactions. While the traditional naive Bayes model assumes feature independence, real-world scenarios often involve interdependencies among features. To address this, the TANB model is introduced. The TANB model introduces a tree structure atop the naive Bayes model to more accurately model feature relationships, overcoming the limitation of assuming feature independence. Specifically, the TANB model constructs a maximum weight spanning tree to uncover conditional dependencies between features, thereby enabling the model to better capture feature interactions.

Assessment indicators

To comprehensively assess the efficacy of the proposed TANB model in the RA for SRPs, a self-constructed dataset serves as the data source for this experimental evaluation, as outlined in Table 1 . The dataset is segregated into training (80%) and test sets (20%). These indicators cover the accuracy, precision, recall rate, F1 value, and Area Under the Curve (AUC) Receiver Operating Characteristic (ROC) of the model. The following are the definitions and equations for each assessment indicator. Accuracy is the proportion of correctly predicted samples to the total number of samples. Precision is the proportion of Predicted Positive (PP) samples to actual positive samples. The recall rate is the proportion of correctly PP samples among the actual positive samples. The F1 value is the harmonic average of precision and recall, considering the precision and comprehensiveness of the model. The area under the ROC curve measures the classification performance of the model, and a larger AUC value indicates better model performance. The ROC curve suggests the relationship between True Positive Rate and False Positive Rate under different thresholds. The AUC value can be obtained by accumulating the area of each small rectangle under the ROC curve. The confusion matrix is used to display the prediction property of the model in different categories, including True Positive (TP), True Negative (TN), False Positive (FP), and False Negative (FN).

The performance of TANB in RA for SRPs can be comprehensively assessed to understand the advantages, disadvantages, and applicability of the model more comprehensively by calculating the above assessment indicators.

Results and discussion

Accuracy analysis of naive bayesian algorithm.

On the dataset of this paper, Fig.  3 reveals the performance of TANB algorithm under different assessment indicators.

Performance assessment of TANB algorithm on different projects.

From Fig.  3 , the TANB algorithm performs well in various projects, ranging from 0.87 to 0.911 in accuracy. This means that the overall accuracy of the model in predicting project risks is quite high. The precision also maintains a high level in various projects, ranging from 0.881 to 0.923, indicating that the model performs well in classifying high-risk categories. The recall rate ranges from 0.872 to 0.908, indicating that the model can effectively capture high-risk samples. Meanwhile, the AUC values in each project are relatively high, ranging from 0.905 to 0.931, which once again emphasizes the effectiveness of the model in risk prediction. From multiple assessment indicators, such as accuracy, precision, recall, F1 value, and AUC, the TANB algorithm has shown good risk prediction performance in representative projects. The performance assessment results of the TANB algorithm under different feature dimensions are plotted in Figs.  4 , 5 , 6 and 7 .

Prediction accuracy of TANB algorithm on different budget investments.

Prediction accuracy of TANB algorithm on different team experiences.

Prediction accuracy of TANB algorithm at different risk levels.

Prediction accuracy of TANB algorithm on different project scales.

From Figs.  4 , 5 , 6 and 7 , as the level of budget investment increases, the accuracy of most projects also shows an increasing trend. Especially in cases of high budget investment, the accuracy of the project is generally high. This may mean that a higher budget investment helps to reduce project risks, thereby improving the prediction accuracy of the TANB algorithm. It can be observed that team experience also affects the accuracy of the model. Projects with high team experience exhibit higher accuracy in TANB algorithms. This may indicate that experienced teams can better cope with project risks to improve the performance of the model. When budget investment and team experience are low, accuracy is relatively low. This may imply that budget investment and team experience can complement each other to affect the model performance.

There are certain differences in the accuracy of projects under different risk levels. Generally speaking, the accuracy of high-risk and medium-risk projects is relatively high, while the accuracy of low-risk projects is relatively low. This may be because high-risk and medium-risk projects require more accurate predictions, resulting in higher accuracy. Similarly, project scale also affects the performance of the model. Large-scale and medium-scale projects exhibit high accuracy in TANB algorithms, while small-scale projects have relatively low accuracy. This may be because the risks of large-scale and medium-scale projects are easier to identify and predict to promote the performance of the model. In high-risk and large-scale projects, accuracy is relatively high. This may indicate that the impact of project scale is more significant in specific risk scenarios.

Figure  8 further compares the performance of the TANB algorithm proposed here with other similar algorithms.

Performance comparison of different algorithms in RA of SRPs.

As depicted in Fig.  8 , the TANB algorithm attains an accuracy and precision of 0.912 and 0.920, respectively, surpassing other algorithms. It excels in recall rate and F1 value, registering 0.905 and 0.915, respectively, outperforming alternative algorithms. These findings underscore the proficiency of the TANB algorithm in comprehensively identifying high-risk projects while sustaining high classification accuracy. Moreover, the algorithm achieves an AUC of 0.930, indicative of its exceptional predictive prowess in sample classification. Thus, the TANB algorithm exhibits notable potential for application, particularly in scenarios demanding the recognition and comprehensiveness requisite for high-risk project identification. The evaluation results of the TANB model in predicting project risk levels are presented in Table 2 :

Table 2 demonstrates that the TANB model surpasses the traditional Naive Bayes model across multiple evaluation metrics, including accuracy, precision, and recall. This signifies that, by accounting for feature interdependence, the TANB model can more precisely forecast project risk levels. Furthermore, leveraging the model’s predictive outcomes, project managers can devise tailored risk mitigation strategies corresponding to various risk scenarios. For example, in high-risk projects, more assertive measures can be implemented to address risks, while in low-risk projects, risks can be managed more cautiously. This targeted risk management approach contributes to enhancing project success rates, thereby ensuring the seamless advancement of SRPs.

The exceptional performance of the TANB model in specific scenarios derives from its distinctive characteristics and capabilities. Firstly, compared to traditional Naive Bayes models, the TANB model can better capture the dependencies between attributes. In project RA, project features often exhibit complex interactions. The TANB model introduces first-order dependencies between attributes, allowing features to influence each other, thereby more accurately reflecting real-world situations and improving risk prediction precision. Secondly, the TANB model demonstrates strong adaptability and generalization ability in handling multidimensional data. SRPs typically involve data from multiple dimensions, such as project scale, budget investment, and team experience. The TANB model effectively processes these multidimensional data, extracts key information, and achieves accurate RA for projects. Furthermore, the paper explores the potential of using hybrid models or ensemble learning methods to further enhance model performance. By combining other machine learning algorithms, such as random forests and support vector regressors with sigmoid kernel, through ensemble learning, the shortcomings of individual models in specific scenarios can be overcome, thus improving the accuracy and robustness of RA. For example, in the study, we compared the performance of the TANB model with other algorithms in RA, as shown in Table 3 .

Table 3 illustrates that the TANB model surpasses other models in terms of accuracy, precision, recall, F1 value, and AUC value, further confirming its superiority and practicality in RA. Therefore, the TANB model holds significant application potential in SRPs, offering effective decision support for project managers to better evaluate and manage project risks, thereby enhancing the likelihood of project success.

Analysis of the degree of influence of different factors

Table 4 analyzes the degree of influence and interaction of different factors.

In Table 4 , the regression analysis results reveal that budget investment and team experience exert a significantly positive impact on RA outcomes. This suggests that increasing budget allocation and assembling a team with extensive experience can enhance project RA outcomes. Specifically, the regression coefficient for budget investment is 0.68, and for team experience, it is 0.51, both demonstrating significant positive effects (P < 0.05). The P-values are all significantly less than 0.05, indicating a significant impact. The impact of project scale is relatively small, at 0.31, and its P-value is also much less than 0.05. The degree of interaction influence is as follows. The impact of interaction terms is also significant, especially the interaction between budget investment and team experience and the interaction between budget investment and project scale. The P value of the interaction between budget investment and project scale is 0.002, and the P value of the interaction between team experience and project scale is 0.003. The P value of the interaction among budget investment, team experience, and project scale is 0.005. So, there are complex relationships and interactions among different factors, and budget investment and team experience significantly affect the RA results. However, the budget investment and project scale slightly affect the RA results. Project managers should comprehensively consider the interactive effects of different factors when making decisions to more accurately assess the risks of SRPs.

The interaction between team experience and budget investment

The results of the interaction between team experience and budget investment are demonstrated in Table 5 .

From Table 5 , the degree of interaction impact can be obtained. Budget investment and team experience, along with the interaction between project scale and technical difficulty, are critical factors in risk mitigation. Particularly in scenarios characterized by large project scales and high technical difficulties, adequate budget allocation and a skilled team can substantially reduce project risks. As depicted in Table 5 , under conditions of high team experience and sufficient budget investment, the average RA outcome is 0.895 with a standard deviation of 0.012, significantly lower than assessment outcomes under other conditions. This highlights the synergistic effects of budget investment and team experience in effectively mitigating risks in complex project scenarios. The interaction between team experience and budget investment has a significant impact on RA results. Under high team experience, the impact of different budget investment levels on RA results is not significant, but under medium and low team experience, the impact of different budget investment levels on RA results is significantly different. The joint impact of team experience and budget investment is as follows. Under high team experience, the impact of budget investment is relatively small, possibly because high-level team experience can compensate for the risks brought by insufficient budget to some extent. Under medium and low team experience, the impact of budget investment is more significant, possibly because the lack of team experience makes budget investment play a more important role in RA. Therefore, team experience and budget investment interact in RA of SRPs. They need to be comprehensively considered in project decision-making. High team experience can compensate for the risks brought by insufficient budget to some extent, but in the case of low team experience, the impact of budget investment on RA is more significant. An exhaustive consideration of these factors and their interplay is imperative for effectively assessing the risks inherent in SRPs. Merely focusing on budget allocation or team expertise may not yield a thorough risk evaluation. Project managers must scrutinize the project’s scale, technical complexity, and team proficiency, integrating these aspects with budget allocation and team experience. This holistic approach fosters a more precise RA and facilitates the development of tailored risk management strategies, thereby augmenting the project’s likelihood of success. In conclusion, acknowledging the synergy between budget allocation and team expertise, in conjunction with other pertinent factors, is pivotal in the RA of SRPs. Project managers should adopt a comprehensive outlook to ensure sound decision-making and successful project execution.

Risk mitigation strategies

To enhance the discourse on project risk management in this paper, a dedicated section on risk mitigation strategies has been included. Leveraging the insights gleaned from the predictive model regarding identified risk factors and their corresponding risk levels, targeted risk mitigation measures are proposed.

Primarily, given the significant influence of budget investment and team experience on project RA outcomes, project managers are advised to prioritize these factors and devise pertinent risk management strategies.

For risks stemming from budget constraints, the adoption of flexible budget allocation strategies is advocated. This may involve optimizing project expenditures, establishing financial reserves, or seeking additional funding avenues.

In addressing risks attributed to inadequate team experience, measures such as enhanced training initiatives, engagement of seasoned project advisors, or collaboration with experienced teams can be employed to mitigate the shortfall in expertise.

Furthermore, recognizing the impact of project scale, duration, and technical complexity on RA outcomes, project managers are advised to holistically consider these factors during project planning. This entails adjusting project scale as necessary, establishing realistic project timelines, and conducting thorough assessments of technical challenges prior to project commencement.

These risk mitigation strategies aim to equip project managers with a comprehensive toolkit for effectively identifying, assessing, and mitigating risks inherent in SRPs.

This paper delves into the efficacy of the TANB algorithm in project risk prediction. The findings indicate that the algorithm demonstrates commendable performance across diverse projects, boasting high precision, recall rates, and AUC values, thereby outperforming analogous algorithms. This aligns with the perspectives espoused by Asadullah et al. 37 . Particular emphasis was placed on assessing the impact of variables such as budget investment levels, team experience, and project size on algorithmic performance. Notably, heightened budget investment and extensive team experience positively influenced the results, with project size exerting a comparatively minor impact. Regression analysis elucidates the magnitude and interplay of these factors, underscoring the predominant influence of budget investment and team experience on RA outcomes, whereas project size assumes a relatively marginal role. This underscores the imperative for decision-makers in projects to meticulously consider the interrelationships between these factors for a more precise assessment of project risks, echoing the sentiments expressed by Testorelli et al. 38 .

In sum, this paper furnishes a holistic comprehension of the Naive Bayes algorithm’s application in project risk prediction, offering robust guidance for practical project management. The paper’s tangible applications are chiefly concentrated in the realm of RA and management for SRPs. Such insights empower managers in SRPs to navigate risks with scientific acumen, thereby enhancing project success rates and performance. The paper advocates several strategic measures for SRPs management: prioritizing resource adjustments and team training to elevate the professional skill set of team members in coping with the impact of team experience on risks; implementing project scale management strategies to mitigate potential risks by detailed project stage division and stringent project planning; addressing technical difficulty as a pivotal risk factor through assessment and solution development strategies; incorporating project cycle adjustment and flexibility management to accommodate fluctuations and mitigate associated risks; and ensuring the integration of data quality management strategies to bolster data reliability and enhance model accuracy. These targeted risk responses aim to improve the likelihood of project success and ensure the seamless realization of project objectives.

Achievements

In this paper, the application of Naive Bayesian algorithm in RA of SRPs is deeply explored, and the influence of various factors on RA results and their relationship is comprehensively investigated. The research results fully prove the good accuracy and applicability of Naive Bayesian algorithm in RA of science and technology projects. Through probability estimation, the risk level of the project can be estimated more accurately, which provides a new decision support tool for the project manager. It is found that budget input and team experience are the most significant factors affecting the RA results, and their regression coefficients are 0.68 and 0.51 respectively. However, the influence of project scale on the RA results is relatively small, and its regression coefficient is 0.31. Especially in the case of low team experience, the budget input has a more significant impact on the RA results. However, it should also be admitted that there are some limitations in the paper. First, the case data used is limited and the sample size is relatively small, which may affect the generalization ability of the research results. Second, the factors concerned may not be comprehensive, and other factors that may affect RA, such as market changes and policies and regulations, are not considered.

The paper makes several key contributions. Firstly, it applies the Naive Bayes algorithm to assess the risks associated with SRPs, proposing the TANB and validating its effectiveness empirically. The introduction of the TANB model broadens the application scope of the Naive Bayes algorithm in scientific research risk management, offering novel methodologies for project RA. Secondly, the study delves into the impact of various factors on RA for SRPs through MLR analysis, highlighting the significance of budget investment and team experience. The results underscore the positive influence of budget investment and team experience on RA outcomes, offering valuable insights for project decision-making. Additionally, the paper examines the interaction between team experience and budget investment, revealing a nuanced relationship between the two in RA. This finding underscores the importance of comprehensively considering factors such as team experience and budget investment in project decision-making to achieve more accurate RA. In summary, the paper provides crucial theoretical foundations and empirical analyses for SRPs risk management by investigating RA and its influencing factors in depth. The research findings offer valuable guidance for project decision-making and risk management, bolstering efforts to enhance the success rate and efficiency of SRPs.

This paper distinguishes itself from existing research by conducting an in-depth analysis of the intricate interactions among various factors, offering more nuanced and specific RA outcomes. The primary objective extends beyond problem exploration, aiming to broaden the scope of scientific evaluation and research practice through the application of statistical language. This research goal endows the paper with considerable significance in the realm of science and technology project management. In comparison to traditional methods, this paper scrutinizes project risk with greater granularity, furnishing project managers with more actionable suggestions. The empirical analysis validates the effectiveness of the proposed method, introducing a fresh perspective for decision-making in science and technology projects. Future research endeavors will involve expanding the sample size and accumulating a more extensive dataset of SRPs to enhance the stability and generalizability of results. Furthermore, additional factors such as market demand and technological changes will be incorporated to comprehensively analyze elements influencing the risks of SRPs. Through these endeavors, the aim is to provide more precise and comprehensive decision support to the field of science and technology project management, propelling both research and practice in this domain to new heights.

Limitations and prospects

This paper, while employing advanced methodologies like TANB models, acknowledges inherent limitations that warrant consideration. Firstly, like any model, TANB has its constraints, and predictions in specific scenarios may be subject to these limitations. Subsequent research endeavors should explore alternative advanced machine learning and statistical models to enhance the precision and applicability of RA. Secondly, the focus of this paper predominantly centers on the RA for SRPs. Given the unique characteristics and risk factors prevalent in projects across diverse fields and industries, the generalizability of the paper results may be limited. Future research can broaden the scope of applicability by validating the model across various fields and industries. The robustness and generalizability of the model can be further ascertained through the incorporation of extensive real project data in subsequent research. Furthermore, future studies can delve into additional data preprocessing and feature engineering methods to optimize model performance. In practical applications, the integration of research outcomes into SRPs management systems could provide more intuitive and practical support for project decision-making. These avenues represent valuable directions for refining and expanding the contributions of this research in subsequent studies.

Data availability

All data generated or analysed during this study are included in this published article [and its Supplementary Information files].

Moshtaghian, F., Golabchi, M. & Noorzai, E. A framework to dynamic identification of project risks. Smart and sustain. Built. Environ. 9 (4), 375–393 (2020).

Google Scholar  

Nunes, M. & Abreu, A. Managing open innovation project risks based on a social network analysis perspective. Sustainability 12 (8), 3132 (2020).

Article   Google Scholar  

Elkhatib, M. et al. Agile project management and project risks improvements: Pros and cons. Mod. Econ. 13 (9), 1157–1176 (2022).

Fridgeirsson, T. V. et al. The VUCAlity of projects: A new approach to assess a project risk in a complex world. Sustainability 13 (7), 3808 (2021).

Salahuddin, T. Numerical Techniques in MATLAB: Fundamental to Advanced Concepts (CRC Press, 2023).

Book   Google Scholar  

Awais, M. & Salahuddin, T. Radiative magnetohydrodynamic cross fluid thermophysical model passing on parabola surface with activation energy. Ain Shams Eng. J. 15 (1), 102282 (2024).

Awais, M. & Salahuddin, T. Natural convection with variable fluid properties of couple stress fluid with Cattaneo-Christov model and enthalpy process. Heliyon 9 (8), e18546 (2023).

Article   CAS   PubMed   PubMed Central   Google Scholar  

Guan, L., Abbasi, A. & Ryan, M. J. Analyzing green building project risk interdependencies using Interpretive Structural Modeling. J. Clean. Prod. 256 , 120372 (2020).

Gaudenzi, B. & Qazi, A. Assessing project risks from a supply chain quality management (SCQM) perspective. Int. J. Qual. Reliab. Manag. 38 (4), 908–931 (2021).

Lee, K. T., Park, S. J. & Kim, J. H. Comparative analysis of managers’ perception in overseas construction project risks and cost overrun in actual cases: A perspective of the Republic of Korea. J. Asian Archit. Build. Eng. 22 (4), 2291–2308 (2023).

Garai-Fodor, M., Szemere, T. P. & Csiszárik-Kocsir, Á. Investor segments by perceived project risk and their characteristics based on primary research results. Risks 10 (8), 159 (2022).

Senova, A., Tobisova, A. & Rozenberg, R. New approaches to project risk assessment utilizing the Monte Carlo method. Sustainability 15 (2), 1006 (2023).

Tiwari, P. & Suresha, B. Moderating role of project innovativeness on project flexibility, project risk, project performance, and business success in financial services. Glob. J. Flex. Syst. Manag. 22 (3), 179–196 (2021).

de Araújo, F., Lima, P., Marcelino-Sadaba, S. & Verbano, C. Successful implementation of project risk management in small and medium enterprises: A cross-case analysis. Int. J. Manag. Proj. Bus. 14 (4), 1023–1045 (2021).

Obondi, K. The utilization of project risk monitoring and control practices and their relationship with project success in construction projects. J. Proj. Manag. 7 (1), 35–52 (2022).

Atasoy, G. et al. Empowering risk communication: Use of visualizations to describe project risks. J. Constr. Eng. Manage. 148 (5), 04022015 (2022).

Dandage, R. V., Rane, S. B. & Mantha, S. S. Modelling human resource dimension of international project risk management. J. Global Oper. Strateg. Sourcing 14 (2), 261–290 (2021).

Wang, L. et al. Applying social network analysis to genetic algorithm in optimizing project risk response decisions. Inf. Sci. 512 , 1024–1042 (2020).

Marx-Stoelting, P. et al. A walk in the PARC: developing and implementing 21st century chemical risk assessment in Europe. Arch. Toxicol. 97 (3), 893–908 (2023).

Awais, M., Salahuddin, T. & Muhammad, S. Evaluating the thermo-physical characteristics of non-Newtonian Casson fluid with enthalpy change. Thermal Sci. Eng. Prog. 42 , 101948 (2023).

Article   CAS   Google Scholar  

Awais, M., Salahuddin, T. & Muhammad, S. Effects of viscous dissipation and activation energy for the MHD Eyring-Powell fluid flow with Darcy-Forchheimer and variable fluid properties. Ain Shams Eng. J. 15 (2), 102422 (2024).

Yang, L., Lou, J. & Zhao, X. Risk response of complex projects: Risk association network method. J. Manage. Eng. 37 (4), 05021004 (2021).

Acebes, F. et al. Project risk management from the bottom-up: Activity Risk Index. Cent. Eur. J. Oper. Res. 29 (4), 1375–1396 (2021).

Siyal, S. et al. They can’t treat you well under abusive supervision: Investigating the impact of job satisfaction and extrinsic motivation on healthcare employees. Rationality Society 33 (4), 401–423 (2021).

Chen, D., Wawrzynski, P. & Lv, Z. Cyber security in smart cities: A review of deep learning-based applications and case studies. Sustain. Cities Soc. 66 , 102655 (2021).

Zhao, M. et al. Pythagorean fuzzy TODIM method based on the cumulative prospect theory for MAGDM and its application on risk assessment of science and technology projects. Int. J. Fuzzy Syst. 23 , 1027–1041 (2021).

Suresh, K. & Dillibabu, R. A novel fuzzy mechanism for risk assessment in software projects. Soft Comput. 24 , 1683–1705 (2020).

Akhavan, M., Sebt, M. V. & Ameli, M. Risk assessment modeling for knowledge based and startup projects based on feasibility studies: A Bayesian network approach. Knowl.-Based Syst. 222 , 106992 (2021).

Guan, L., Abbasi, A. & Ryan, M. J. A simulation-based risk interdependency network model for project risk assessment. Decis. Support Syst. 148 , 113602 (2021).

Vujović, V. et al. Project planning and risk management as a success factor for IT projects in agricultural schools in Serbia. Technol. Soc. 63 , 101371 (2020).

Muñoz-La Rivera, F., Mora-Serrano, J. & Oñate, E. Factors influencing safety on construction projects (FSCPs): Types and categories. Int. J. Environ. Res. Public Health 18 (20), 10884 (2021).

Article   PubMed   PubMed Central   Google Scholar  

Nguyen, P. T. & Nguyen, P. C. Risk management in engineering and construction: A case study in design-build projects in Vietnam. Eng. Technol. Appl. Sci. Res 10 , 5237–5241 (2020).

Nguyen PT, Le TT. Risks on quality of civil engineering projects-an additive probability formula approach//AIP Conference Proceedings. AIP Publishing, 2798(1) (2023).

Nguyen, P.T., Phu, P.C., Thanh, P.P., et al . Exploring critical risk factors of office building projects. 8 (2), 0309–0315 (2020).

Nguyen, H. D. & Macchion, L. Risk management in green building: A review of the current state of research and future directions. Environ. Develop. Sustain. 25 (3), 2136–2172 (2023).

He, S. et al. Risk assessment of oil and gas pipelines hot work based on AHP-FCE. Petroleum 9 (1), 94–100 (2023).

Asadullah, M. et al. Evaluation of machine learning techniques for hypertension risk prediction based on medical data in Bangladesh. Indones. J. Electr. Eng. Comput. Sci. 31 (3), 1794–1802 (2023).

Testorelli, R., de Araujo, F., Lima, P. & Verbano, C. Fostering project risk management in SMEs: An emergent framework from a literature review. Prod. Plan. Control 33 (13), 1304–1318 (2022).

Download references

Author information

Authors and affiliations.

Institute of Policy Studies, Lingnan University, Tuen Mun, 999077, Hong Kong, China

Xuying Dong & Wanlin Qiu

You can also search for this author in PubMed   Google Scholar

Contributions

Xuying Dong and Wanlin Qiu played a key role in the writing of Risk Assessment of Scientific Research Projects and the Relationship between Related Factors Based on Naive Bayes Algorithm. First, they jointly developed clearly defined research questions and methods for risk assessment using the naive Bayes algorithm at the beginning of the research project. Secondly, Xuying Dong and Wanlin Qiu were responsible for data collection and preparation, respectively, to ensure the quality and accuracy of the data used in the research. They worked together to develop a naive Bayes algorithm model, gain a deep understanding of the algorithm, ensure the effectiveness and performance of the model, and successfully apply the model in practical research. In the experimental and data analysis phase, the collaborative work of Xuying Dong and Wanlin Qiu played a key role in verifying the validity of the model and accurately assessing the risks of the research project. They also collaborated on research papers, including detailed descriptions of methods, experiments and results, and actively participated in the review and revision process, ensuring the accuracy and completeness of the findings. In general, the joint contribution of Xuying Dong and Wanlin Qiu has provided a solid foundation for the success of this research and the publication of high-quality papers, promoted the research on the risk assessment of scientific research projects and the relationship between related factors, and made a positive contribution to the progress of the field.

Corresponding author

Correspondence to Wanlin Qiu .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary information., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Dong, X., Qiu, W. A case study on the relationship between risk assessment of scientific research projects and related factors under the Naive Bayesian algorithm. Sci Rep 14 , 8244 (2024). https://doi.org/10.1038/s41598-024-58341-y

Download citation

Received : 30 October 2023

Accepted : 27 March 2024

Published : 08 April 2024

DOI : https://doi.org/10.1038/s41598-024-58341-y

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Naive Bayesian algorithm
• Scientific research projects
• Risk assessment
• Factor analysis
• Probability estimation
• Decision support
• Data-driven decision-making

By submitting a comment you agree to abide by our Terms and Community Guidelines . If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing: AI and Robotics newsletter — what matters in AI and robotics research, free to your inbox weekly.

Data Science Journal

• Download PDF (English) XML (English)
• Alt. Display

Research Papers

Risk assessment for scientific data.

• Matthew S. Mayernik
• Kelsey Breseman
• Robert R. Downs
• Alexis Garretson
• Chung-Yi (Sophie) Hou
• Environmental Data Governance Initiative (EDGI) and Earth Science Information Partners (ESIP) Data Stewardship Committee
• risk assessment
• data preservation
• data stewardship

Introduction

At 1 the “The Rescue of Data At Risk” workshop held in Boulder, Colorado on September 8th and 9th, 2016, 2 participants were asked the following question: “How would you define ‘at risk’ data?” Discussions on this point ranged widely, and touched on several challenges, including lack of funding or personnel support for data management, natural and political disasters, and metadata loss. One participant’s organization’s definition of risk, however, stood out: “data were considered to be at risk unless they had a dedicated plan to not be at risk.” This simple statement vividly depicts how data’s default state is being in a state of risk. Thus, ongoing stewardship is required to keep data collections and archives in existence.

The risk factors that a given data collection or archive may face vary depending on the data’s characteristics, the data’s current environment, and the priorities and resources available at the time. Many risks can be reduced or eliminated by following best practices codified as certifications and guidelines, such as the CoreTrustSeal Data Repository Certification ( 2018 ) and the ISO 16363:2012. This ISO standard defines audit and certification procedures for trustworthy digital repositories ( ISO 2012b ). Both the CoreTrustSeal certification and ISO 16363:2012 are based on the ISO 14721:2012 standard that defines the Open Archival Information System (OAIS) Reference Model ( ISO 2012a ). But these certifications can be large and complex. Additionally, many of the organizations that hold valuable scientific data collections may not be aware of these standards, even if the organizations are potentially resourced to tackle the challenge ( Maemura, Moles & Becker 2017 ). Further, the attainment of such certifications does not necessarily reduce the risks to data that are outside of the scope of a particular certification instrument.

This paper presents an analysis of data risk factors that scientific data collections and archives may face, and a matrix to support data risk assessments to help ameliorate those risks. The three driving questions for this analysis are:

• How to assess what data are at risk?
• How to characterize what risk factors data collections and/or archives face?
• How to make risks more transparent, internally and/or externally?

The goals of this work are to inform and enable effective data risk assessment by: a) individuals and organizations who manage data collections, and b) individuals and organizations who want to help to reduce the risks associated with data preservation and stewardship. Stakeholders for these two activities include producers, stewards, sponsors, and users of data, as well as the management and staff of the institutions that employ them.

This project has been coordinated through the Data Stewardship Committee within the Earth Science Information Partners (ESIP), a non-profit organization that exists to support collection, stewardship, and use of Earth science data, information, and knowledge. 3 The immediate motivation for the project stemmed from the Data Stewardship Committee members engaging with groups who were undertaking grass-roots “data rescue” initiatives after the 2016 US presidential election. At that time, a number of loosely organized and coordinated efforts were initiated to duplicate data from US government organizations to prevent potential politically motivated data deletion or obfuscation (See for example Dennis 2016 ; Varinsky 2017 ). In many cases, these initiatives specifically focused on duplicating government-hosted Earth science data.

ESIP Data Stewardship Committee members wrote a white paper to provide the Earth science data centers’ perspective on these grass-roots “data rescue” activities ( Mayernik et al. 2017 ). That document described essential considerations within day-to-day work of existing federal and federally-funded Earth science data archiving organizations, including data centers’ constant focus on documentation, traceability, and persistence of scientific data. The white paper also provided suggestions for how the grass-roots efforts might productively engage with the data centers themselves.

One point that was emphasized in the white paper was that the actual risks faced by the data collections may not be transparent from the outside. In other words, “data rescue” activities may have in fact been duplicating data that were at minimal risk of being lost ( Lamdan 2018 ). This point, and the white paper in general, was well received by people inside and outside of these grass-roots initiatives ( Cornelius & Pasquetto 2017 ; McGovern 2017 ). Questions then came back to the ESIP Data Stewardship Committee about how to understand what data held by government agencies were actually at risk.

The analysis presented in this paper was initiated in response to these questions. Since then, these grass-roots “data rescue” initiatives have had mixed success in sustaining and formalizing their efforts ( Allen, Stewart & Wright 2017 ; Chodacki 2018 ; Janz 2018 ). The intention of our paper is to enable more effective data risk assessment broadly. Rescuing data after they have been corrupted, deleted, or lost can be time and effort intensive, and may be impossible ( Pienta & Lyle 2018 ). Thus, we aim to provide guidelines to any individual or organization that manages and provides access to scientific data. In turn, these individuals and organizations can better assess the risks that their data face, and characterize those risks.

When discussing risk and, in particular, data risk, it is useful to ask the question: what is the objective that is being challenged by the possible risk factors? With regard to data, in general, discussions of risk might presume that “risks” threaten the current or future access to data by the potential data users. Currently, continuing public access to and use of scientific data is particularly relevant in light of recent open data and open science initiatives. In this regard, risks for scientific data include factors that could hinder, constrain, or limit current or future data use. Identifying such risk factors to data use offers further analysis opportunities to prevent, mitigate, or eliminate the risks.

Data Risk Assessment

Risk assessment is a regular activity within many organizations. In a general sense, risk management plans are complementary to project management plans ( Cervone 2006 ). Organizational assessment of digital data and information collections is likewise not new ( Maemura, Moles & Becker 2017 ). The analysis presented in this paper builds on prior work in a number of areas: 1) research on data risks, 2) data rescue initiatives within government agencies & specific disciplines, 3) CODATA and RDA working groups & meetings, 4) trusted repository certifications, and 5) knowledge and experience of the ESIP Data Stewardship Committee members. Table 1 summarizes data risk factors that emerge from these knowledge bases. The list of risk factors shown in Table 1 is not meant to be exhaustive. Rather, it provides a useful illustration of the diverse ways in which data sets, collections, and archives might encounter risks to data usability and accessibility. The rest of this section details further key insights from the five areas of prior work noted above.

Risk factors for scientific data collections.

Research on data risks

A range of studies have explored the kinds of risks that scientific data may face, and potential ways to mitigate specific risk factors. Many of these studies touch on practices that are typical of scientific data archives. Metadata, for example, can be considered both a risk factor and a mitigation strategy. Insufficient metadata is itself a potential factor that can reduce the discoverability, usability, and preservability of data, particularly in situations where direct human knowledge of the data is absent ( Michener et al. 1997 ). In fact, many data rescue projects find that the “rescue” efforts must be targeted much more toward metadata than data (see Knapp, Bates & Barkstrom 2007 ; Hsu et al. 2015 ). This might be the case for a couple of reasons. First, insufficient or missing metadata might prevent data from being usable regardless of the condition of the data themselves. Examples include missing column headers in tabular data that prevent a user from knowing what the data are representing, and insufficient provenance metadata that prevent users from trusting the data due to lack of context about data collection and quality control. Second, metadata are also central to documenting and mitigating risks as they manifest while preventing risks from becoming problematic in the future ( Anderson et al. 2011 ). For example, documenting data ownership and usage rights is an essential step in mitigating the risk factor “Legal status for ownership and use” from Table 1 .

Different kinds of metadata might be necessary to reduce specific data risks. For example, specifications of file format structures are a critical type of metadata for mitigating risks associated with digital file format obsolescence. Open specifications complement other critical mitigation practices and tools related to file format obsolescence. As one example, keeping rendering software available is an important way to retain access to particular file formats, but this typically also requires maintaining documentation of how the rendering software works ( Ryan 2014 ).

Other risk factors (listed in Table 1 ) relate to the sustainability and transparency of the archiving organization. These factors are important in ensuring the accessibility of the data and the trustworthiness of the archive. As Yakel et al. ( 2013 ) note, “[t]rust in the repository is a separate and distinct factor from trust in the data” (pg. 154). For people outside of the repository, “institutional reputation appears to be the strongest structural assurance indicator of trust” (pg. 154). Effective communication about data risks and steps taken to eliminate problems is helpful in ensuring users that the archive is trustworthy ( Yoon 2017 ).

Data that face extreme or unusual risks, however, may not be manageable via typical data curation workflows. Downs and Chen ( 2017 ) note that dealing with some data risk factors “may well require divergence from regular data curation procedures as tradeoffs may be necessary” (pg. 273). For example, Gallaher et al. ( 2015 ) undertook an extensive project to recover, reconstruct, and reprocess data from early satellite missions into modern formats that are usable by modern scientists. This project involved dealing with degrading and fragile magnetic tapes, extracting data from the tapes’ unusual format, and recreating documentation for the data. Natural disasters, fires, and floods also present unpredictable risk factors to data collections of all kinds. While these kinds of events can be planned for and steps can be taken to prevent the occurrence of some of them (e.g. fires), they can still cause major data loss and/or require significant recovery effort.

Mitigating risks, of whatever kind, takes effort and resources. The time required to create metadata, re-format files, create contingency plans, and communicate these efforts to user communities can be considerable. This time investment can be the greatest barrier to performing risk assessment and mitigation activities ( Thompson, Robertson & Greenberg, 2014 ). Putting focus on assessment of data risk factors may mean that “certain priorities need to be re-ordered, new skills acquired and taught, resources redirected, and new networks constructed” ( Griffin 2015, pg. 93 ). It can be possible to automate some components of risk assessment ( Graf et al. 2017 ), but most of the steps require human effort. This intensive effort is vividly illustrated by the many data rescue initiatives that have taken place within government agencies and other kinds of organizations over the past few decades.

Data rescue initiatives within government agencies & specific disciplines

Legacy data are data collected in the past with different technologies and data formats than in use today. These data often face the largest numbers of risk factors that could lead to data loss. A wide range of government agencies and other organizations have conducted legacy data rescue initiatives to modernize data and make them more accessible and usable for today’s science. Each data rescue project typically faces many different kinds of data risks. For example, a recent satellite data rescue effort had to address the “loss of datasets, reconciliation of actual media contents with metadata available, deviation of the actual data format from expectations or documentation, and retiring expertise” ( Poli et al. 2017, pg. 1481 ). Data rescue projects typically involve work to prevent future risk factors from manifesting, in addition to modernizing data for accessibility and usability. For example, data rescue projects migrate data to less endangered data formats, and create new metadata and quality control documentation ( Levitus 2012 ).

CODATA/RDA working groups & meetings

Relevant professional organizations, including the International Council for Science (ICSU) Committee on Data for Science and Technology (CODATA) and the Research Data Alliance (RDA), also have been actively identifying improvements for data stewardship practices that can reduce potential risks to data. For example, the former Data At Risk Task Group (DAR-TG), of CODATA, raised awareness about the value of heritage data and described the benefits obtained from several data rescue projects ( Griffin 2015 ). This group also organized the 2016 “Rescue of Data At Risk” workshop mentioned in the introduction of this paper. That workshop led to a document titled, “Guidelines to the Rescue of Data At Risk” ( 2017 ). Subsequently, the Data Rescue Interest Group ( 2018 ) of the Research Data Alliance (RDA), spawned from the CODATA DAR-TG, also focuses on efforts to increase awareness of data rescue projects.

Repository certifications and maturity assessment

Many data repositories have conducted self-assessments and external assessments to document their compliance with the standards for trusted repositories and attain certification of their capabilities and practices for managing data. In addition to emphasizing organizational issues, repository certification instruments, such as ISO 16363 ( 2012b ) and CoreTrustSeal (2018) certification, also focus on digital object management and infrastructure capabilities. Engaging in such assessments offers benefits to repositories and their stakeholders. A key benefit is the identification of areas where improvements have been completed or need to be completed to reduce risks to data (CoreTrustSeal 2018). In an examination of perceptions of repository certification, Donaldson et al. ( 2017 ) found that process improvement was often reported by repository staff as a benefit of repository certification.

In addition to (or complementary to) formal certifications, data repositories may conduct data stewardship maturity assessment exercises to help in identifying data risks and informing data risk mitigation strategies ( Faundeen 2017 ). “Maturity” is used in the sense presented by Peng et al. ( 2015 ), and refers to the level of performance attained to ensure preservability, accessibility, usability, transparency/traceability, and sustainability of data, along with the level of performance in data quality assurance, data quality control/monitoring, data quality assessment, and data integrity checks. Maturity at the institutional (or archive) level in areas such as policy, funding, and infrastructure does not necessarily translate to comprehensive maturity at the dataset level ( Peng 2018 ). Data stewardship maturity assessment should therefore be performed both at the institutional level and at the dataset level. It is recognized that performing stewardship maturity assessments can be time consuming and resource intensive. However, the stewardship organizations are encouraged to perform self-assessment using “stage by stage” or “a la carte” approach (see example in Peng et al. 2019 ). Ultimately, both formal certifications and informal maturity assessments help organizations not only gain self-awareness, but also identify better solutions for their data that might be at risk of being lost or rendered unusable.

Developing a Data Risk Assessment Matrix

Risk assessment is a well-established field, with 30–40 years of history ( National Research Council 1983 ; Aven 2016 ). However, the practice of applying risk assessment methodologies to scientific data collections is less formally established, though regular audits and reviews of data management systems are common in some organizations ( Ramapriyan 2017 ).

The starting point for this project was to establish a process for categorizing the data risk factors shown in Table 1 . The initial idea of our effort was that if data risk factors could be categorized into a logical structure, it would allow data managers to assess the risks to their data collections via a set of predefined and consistent categories. To develop a logical categorization, we held a session to conduct a “card sorting” exercise at the 2018 ESIP Summer Meeting, which took place in July 2018 in Tucson, Arizona. “Card sorting” is an established method for developing categorizations of concepts, vocabulary terms, or web sites ( Zimmerman & Akerelrea 2002 ; Usability.gov   2019 ). Following the card sorting methodology, participants in the 2018 ESIP meeting session were provided the list of data risks in Table 1 , and asked to complete the following task: “Looking at the list of data risk factors, how would you group these factors, based on the categories you would define?”

Approximately 15 attendees engaged in the exercise. We used a combination of an online card sorting tool and hand-written recommendations to collect the completed card sorting categorizations. Following the completion of the exercise, the results were displayed in front of the session participants and a group discussion took place. The outcome of the card sorting exercise and subsequent discussion was a clear recognition that there could be many valid and useful ways of categorizing data risks. No single method for categorizing the risk factors would be sufficient to cover the diverse organizations and situations within which data collections exist. Depending on the situation(s), a data curation organization or individual is facing, they may need to categorize data risks in different ways. This characteristic is common in risk assessments generally, as risk prioritization and categorizations are dependent on the phenomena being assessed, the characteristics of the situation, and the goals of the organizations or people performing the assessment ( Slovic 1999 ).

Through subsequent discussion and analysis of the data risk assessment literature noted above, we identified at least ten different ways that data risk factors could be assessed. Many of these categorization methods are applicable to risk assessments of any kind ( Cervone 2006 ). The list below is not meant to be exhaustive, and some methods are likely related. Data risk factors could be categorized or prioritized according to the methods listed in Table 2 .

Methods for Categorizing Data Risks.

The lists shown in Tables 1 and 2 offer characteristics on which data risk assessments can be built. Combining the categorization methods from Table 2 with the selected risk factors from Table 1 leads to a risk assessment matrix, as shown in Table 3 . This figure shows an example of a selection of specific data risk factors and the categorization methods. Depending on the situation or data collection being assessed, different risk factors and/or categorization methods may be more applicable than the ones shown in Table 3 . Those conducting a data risk assessment can then use the matrix as a way to organize, prioritize, or potentially quantify the selected risks according to the categorization methods that are most relevant for the specific case at hand. The next section provides more detailed illustrations of the use of the data risk assessment matrix. Appendix I shows the full data risk assessment template, with all risks and categorization methods from Tables 1 and 2 .

Example of a blank data risk assessment matrix, after selection of specific risk factors and categorization methods of interest.

Application of the Data Risk Assessment Matrix

Three case studies are described below in which the data risk assessment matrix was used to develop a better understanding of data risks for particular resources. These cases enable evaluation of the data risk assessment framework presented in this paper, clarifying its strengths and weaknesses, and pinpointing the situations in which it can be most useful ( Becker, Maemura & Moles 2020 ).

Case 1 – NCAR Library Analog Data Collection

The National Center for Atmospheric Research (NCAR) Library maintains an analog data collection that consists of about 300 data sets in support of atmospheric and meteorological research conducted by NCAR scientists. These assets are largely compilations of measurements and statistics published by national and international meteorological services and other kinds of government entities. Many of these assets have been in the NCAR Library’s collections for decades, and most were minimally cataloged when they were first brought into the collection. As such, the current usage of the collection is minimal. A prior assessment done by the NCAR Library and a student assistant sought to identify individual assets that were of higher potential value and interest for current science. This assessment effort resulted in a modernization prioritization based on a geographic and temporal framework, and improved metadata records for about 5% of the collection ( Mayernik et al. 2018 ). This effort did not, however, include any kind of risk assessment related to the physical assets themselves.

The data risk assessment matrix was therefore helpful in doing a second-level priority analysis for these NCAR Library analog data assets. We used the matrix as a way to identify which risk factors were most important for these materials, and to characterize the mitigation efforts that were needed for each risk factor. In particular, we focused the risk assessment on the data assets that were previously identified as having high geospatial and temporal interest. The NCAR Library use of the matrix involved a series of steps:

• Step 1 – A number of risk factors listed in the matrix were identified as being of most importance, with the focus being on factors that prevented or impeded the use of these data within current scientific studies. The most immediate risk factors were identified to be the “lack of use” and the “lack of documentation/metadata” for these assets. Other risks that were secondary in immediacy, but still potentially important, were: Data mislabeling, the questionable legal status for ownership and use, media deterioration, lack of planning, and poor data governance.
• Step 2 – The second step was to identify which categorization methods shown in the matrix were most applicable/appropriate for the NCAR Library’s management and maintenance of this collection. The methods selected were: a) Length of recovery time, b) Who is responsible for addressing the problem, c) Nature of mitigation, and d) Resources required for mitigation.
• Step 3 – The third step was to fill in the boxes in the matrix for the risk factors and categorization methods. For example, for the “Length of recovery time” question, we used a simple 1–3 scale to indicate relative differences in how long it would take to mitigate the two most important risk factors: “lack of use” and the “lack of documentation/metadata”. As one example, some data assets were published by international agencies and therefore have title pages and documentation that are not in English. In turn, due to the lack of relevant foreign language expertise in the NCAR Library staff, developing new metadata for these resources will take more effort than for those assets that were published by English-speaking countries. For the “Resources required for mitigation” categorization method, a numerical scale was not as appropriate. Instead, we filled in the matrix with text descriptions of the resources required to mitigate the risk factors. An example entry under the “lack of documentation & metadata” risk factor was: “We would need to create new metadata for the library catalog, then transform to ISO for inclusion in NCAR DASH Search, with added challenge of needing to look at microfilm files (no current working reader in Library).”

In summary, the matrix was very useful as “something to think with.” In other words, it jump-started the process for doing the risk assessment because the NCAR Library staff did not need to spend time developing a comprehensive list of risk factors that may apply for these data, or brainstorm about how to categorize those risks. The risk factor matrix provided a ready-made starting point for the assessment. Because the matrix does not dictate how the cells should be filled in, the NCAR Library staff made decisions about how to apply the matrix for each categorization method that was chosen. The matrix structure could potentially be applied or customized to create a prioritization rubric, by supporting the creation of a numeric scoring process for categories where that is appropriate.

Case 2 – Mohonk Preserve Daniel Smiley Research Library

Mohonk Preserve is a land trust and nature preserve in New Paltz, New York covering more than 8,000 acres of a northern section of the Appalachian Mountains known as the Shawangunk Mountains. Mohonk Preserve’s conservation science division, the Daniel Smiley Research Center (DSCR), is affiliated with the Organization of Biological Field Stations (OBFS) and acts as a NOAA Climate Observation Center. DSRC staff and citizen scientists carry out a variety of long-term monitoring projects and manage an extensive archive of historical observations. The archive houses 60,000 physical items, 9,000 photographs, 86 years of natural history observations, 123 years of daily weather data, and a research library of legacy titles. The physical items include more than 3,000 herbarium specimens, 107 bird specimens, 140 butterfly specimens, 139 mammal specimens, 400 arthropod specimens, and over 14,000 index cards with handwritten and typed observations. The digitization process of the archive holdings is ongoing, but the packaging and publishing of datasets in the Environmental Data Initiative is a priority ( Mohonk Preserve et al. 2018a , 2018b , 2019 ). These data and natural history collections underpin the Mohonk Preserve’s land management and stewardship and have been crucial to an increasing number of scientific publications (e.g., Cook et al. 2009 ; Cook et al. 2008 ; Charifson et al. 2015 ; Richardson et al. 2016 ), but the collections remain underutilized.

The data rescue effort for the archives has largely consisted of digitization and cataloging. Hence, the data risk assessment matrix was used to guide the prioritization of datasets for publication and assess other data rescue needs and considerations for the archives. The most critical risk factors identified through the process were ‘lack of documentation & metadata’, ‘loss of knowledge’, and ‘lack of use.’ In order to address the lack of use, we collaboratively developed a prioritization of the data holdings for publication in a repository, particularly based around the value of data collected for scientific investigations, the temporal coverage of the dataset, and an assessment of the resources required for the digitization, packaging, and publishing of the relevant dataset.

We also realized through the data risk matrix process that many of our risk factors are interdependent – for example, the lack of documentation may not be because the documentation does not exist in the library, but rather that it may not be discoverable in the archives due to incomplete digitization or cataloging of the relevant records or field notes. For example, during the assessment process for our vernal pool monitoring dataset ( Mohonk Preserve et al. 2019 ), we discovered previously unknown environmental quality notes in narrative sections of an undigitized collection of field notes. This supported the current emphasis on the digitization and cataloging of the holdings and suggested areas of high importance, particularly the narrative sections of field notebooks. Additionally, the lack of documentation and metadata is directly related to the loss of knowledge through leadership transitions. Like many long-term ongoing collections projects, metadata and documentation– particularly related to data collection protocols– are held as tacit knowledge by key stakeholders who have been involved with the project for an extended period of time. The loss of those stakeholders or their knowledge, through retirement or employment changes, poses a significant risk to the long-term value of the associated data ( Michener et al. 1997 ).

Because the holdings largely consist of physical items, a subset of the risk factors in the matrix were not directly applicable to the collections but had corollaries in physical collections management. For example, bit rot and data corruption are not a concern for the physical items, but pests present a similar concern that needs mitigation in a physical archive setting. Additionally, storage hardware breakdown is not directly applicable to herbarium collections but ensuring that the mounting sheets are acid-free is key to ensuring the protection of the specimens and preventing deterioration over time. Considering physical risks to the collection media remains a crucial aspect of managing and planning for the future of physical specimen holdings. Through the data assessment process, one of the key risk areas identified through the assessment was the loss of knowledge and documentation due to retirement, so planning for mitigating this risk is ongoing. Overall, the matrix provided a helpful starting point for guiding conversations relating to the stewardship of the archives and proactively planning and allocating resources to make the data more accessible to scientists and researchers.

Case 3 – EDGI Response to the Deer Park Chemical Fire

On March 17 2019, tanks of chemicals at the Intercontinental Terminals Company (ITC) in Deer Park, Texas, caught fire and began a blaze that would last several days, emitting a chemical-laden plume of smoke over surrounding communities. The Environmental Data & Governance Initiative (EDGI) was approached for assistance in rapid-response archival backup of digital environmental data relevant to the fire in case of future tampering or loss of availability.

There were two major causes for concern: (1) evident tampering– the closest air quality monitor was taken down during the fire, and (2) potential conflict of interest– the entity furnishing the data might have some culpability in a future legal case using the data. The approaching organization hopes to use saved data as evidence in legal cases that may take several years (potentially due to the long timespan for benzene-related illnesses to surface in then-students at the local school and workers in nearby factories).

On a limited timeline and with little capacity, EDGI needed to downselect from hundreds to thousands of possibly relevant data sources (including air and water quality monitors affected by the fire’s plume, and plans and response documents surrounding handling of the fire). The primary mission: ensure the data of primary concern is backed up in a legally legitimate (traceable) format that will be usable in a decade or more.

The data risk matrix from this paper was not used at the time

Prioritization.

The approaching organization suggested a few directories of static data to archive. With additional investigation, EDGI also found some API-accessible structured data from the air monitors that was updated daily.

The information proposed for potential rescue included:

• Data from the Deer Park air quality monitor data that was taken down
• Data from other nearby air quality monitors
• Air quality monitors downstream of the plume (potentially very many of them, as the plume traveled more than 20 miles)
• Three years of back-data from any air quality monitors, to establish baseline
• Water quality monitors– local, downstream, and down-plume, in case relevant (no evidence of contamination yet, but the situation still developing), and three years of back data to establish baseline
• Future data from any monitors, to track the still-developing situation and archive it in case of any present risk
• Contextual information: air sampling plans, disaster response plans, air and water quality sampling maps, PDFs of additional air and water quality sampling from different entities than provide the API-callable data

There was no formal review process for deciding what to save. There was some brief discussion internally around technical feasibility and potential environmental justice-focused mapping efforts, but the major use anticipated for the saved data was the legal case. The whole process from request to data backup took just a few days. Ultimately, EDGI’s choices of data to save depended primarily on the abilities and assessment of the two volunteers available. The volunteers used the skills they had and their best intuitions– lacking a clear prioritization between different data that could be saved.

Applying the data risk matrix to this situation, the two major risk factors can be immediately identified as “catastrophe” and “political interference”. Both risk factors are relevant, likely, and potentially catastrophic in effect. This highlights the urgency and source of the risk.

The risk matrix is not as helpful in prioritizing which data to save under capacity and urgency constraints. The risk matrix identifies the type and intensity of risk, but since all of the data is equally high-risk in this use case, the context of the data and its use case (evidence in a far-future legal case) are necessary for the following tasks of identifying, locating, and prioritizing data to save. This was done based on the best assessment and abilities of the available volunteers.

Ultimately, EDGI saved:

• Structured data from the Deer Park and nearby Lynchburg Ferry air quality monitors: saved with metadata to IPFS via Qri ( qri.io ) with script to keep pulling updates
• All of the PDF data (primarily directories of 20–100 links, typically to PDFs, including maps, images, narratives, and tables of data): saved to the Internet Archive as a full site snapshot

Assessing risks to rescued data

Following the data rescue operation, this risk matrix was used to assess ongoing risks to the repositories of rescued data: (1) the PDF data saved to the Internet Archive and (2) the structured data from air quality monitors saved to IPFS. The risk matrix was very effective for identification of vulnerabilities and potential next steps to better secure the data.

The full matrix (all of the categories and all of the risk factors) was applied twice: to PDF data saved to the Internet Archive, and to structured data saved on the decentralized web (IPFS). A scale of numeric values (1 (low) to 3 (high)) was used to rate the category versus the risk factor. For example, the risk factor of Media Deterioration was rated 3 (high) for Severity of Risk, but 1 (low) for Likelihood of Occurrence. This numeric rating was important to use of the full matrix– instead of removing columns as irrelevant, they could be down-rated where the risk was low.

Use of the matrix immediately highlighted the difference in risks important to the data stored on IPFS versus the Internet Archive. For example, data on the Internet Archive is well-governed and reasonably easy to find, but much more susceptible to natural disaster and hardware deterioration than the data on IPFS. IPFS is a new technology designed to store data across many physical locations– so it’s very resilient to location-based risks, but its format may become obsolete as the technology develops.

The risk matrix is particularly useful when combined with spreadsheet tools. For example, a quick to-do list for EDGI as a data manager can be produced using a formula such as:

• Likelihood of occurrence > 1
• Resources for mitigation < 3
• Type of action: proactive
• Responsible party: EDGI
• Print mitigation action for rows where all of the above are true

Overall, the Risk Matrix outlined in this paper is a very useful tool for identifying risks to data and prioritizing next steps for mitigation– as long as the user has or can assume control over the data. However, in a data rescue use case, this risk matrix must be supplemented by additional context in order to prioritize which at-risk data should be saved when capacity is limited.

Conclusions and Lessons Learned

Risk assessments are instrumental for ensuring that existing data collections continue to be useful for scientific research and societal applications. Risk assessments are also an essential component of data rescue efforts in which interventions take place to prevent or minimize data loss. The data risk assessment framework presented in this paper provides a platform from which risk assessments can quickly begin.

To close out this paper, we discuss some observations and lessons learned in developing and applying the data risk assessment matrix. Data risk assessments can get significantly more in-depth and detailed than the basic template presented in Table 3 and Appendix I. As one example, the US Geological Survey (USGS) has undertaken a substantive project to create risk calculations for USGS-held data collections based on a number of criteria ( USGS 2019 ). The USGS process has involved the development of detailed formulas and weighting schemes to produce quantified assessments of data risk. The risk assessment matrix presented in this paper does not provide “out of the box” quantification measures or data risk prioritizations of the level of detail of the USGS project. The data risk matrix does, however, provide the foundations for an individual or organization to develop a more customized risk assessment rubric. The specifics of how risks were quantified or qualified, and how they were prioritized varied across the different uses of the matrix presented in the three case studies.

The three cases did demonstrate a common use pattern for the data risk matrix. The first step in each case was to review Tables 1 and 2 to determine which risk factors and categorization methods were most relevant. Clearly not all of the risk factors are applicable to all cases, and some of the risk factors are closely related, such as the “lack of documentation & metadata” and “lack of provenance information.” Once the risk factors and categorization methods have been filtered down into a smaller matrix, the next step is to determine how to fill in the matrix cells for particular datasets or collections. It may not be obvious how this would work for some data collections. Our cases involved using a mix of quantitative, qualitative, and ordinal rankings (such as using “high, medium, and low” designations for particular cells). This step may take some trial and error by the matrix user(s) to determine ranking approaches that are the most useful.

The third step is then to use the cell values in the matrix to guide conversations and decisions about risk mitigation priorities. In this sense, the matrix exercise can provide a high level overview of data collections, risks they may face, and the relative urgency and challenges that those risks present to the data stewards. The matrix can serve as a common reference point for discussions of resource allocations and stewardship priorities. However, as exemplified in the EDGI use case, prioritization in real-time, as would be required during catastrophic events such as disasters or wars particularly where there may be political interference, is difficult if not impossible. As such, preventing or minimizing data loss requires pre-planning at a scale rarely available.

The goal in creating this data risk assessment matrix has been to provide a light-weight way for data collections to be reviewed, documented, and evaluated against a set of known data risk factors. As the understanding of the value that scientific data have for research and societal uses increases, many initiatives recognize that “old data is the new data” ( NIWA 2019 ). Risk assessments are critical to ensure that “old data” can become “new data,” and are also critical to ensure that new data can continue to be newly useful into the future.

We list EDGI and the ESIP Data Stewardship Committee as authors due to the contributions of many individuals from both organizations to the work described in this paper. The named authors are the individuals involved in each organization who contributed directly to the paper’s text.  

The workshop was organized under the auspices of the Research Data Alliance (RDA) and the Committee on Data (CODATA) within the International Science Council, http://www.codata.org/task-groups/data-at-risk/dar-workshops .  

http://wiki.esipfed.org/index.php/Preservation_and_Stewardship .  

Appendix I – Data Risk Assessment Template

Acknowledgements.

This project was organized and supported by the Data Stewardship Committee within the Earth Science Information Partners (ESIP). We thank ESIP and the committee participants for feedback on the project at numerous points in the past few years.

The work of Robert R. Downs was supported by the National Aeronautics and Space Administration under Contract 80GSFC18C0111 for the Socioeconomic Data and Applications Distributed Active Archive Center (DAAC).

Alexis Garretson acknowledges the support of the Environmental Data Initiative Summer Fellowship program and the Earth Science Information Partners Community Fellows Program. Alexis also acknowledges Mohonk Preserve staff, particularly the staff of the Daniel Smiley Research Center: Elizabeth C. Long, Megan Napoli, and Natalie Feldsine. This material is based upon work supported by the National Science Foundation Graduate Research Fellowship Program under Grant No. 1842191. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.

The work of Chung-Yi (Sophie) Hou was supported by the National Center for Atmospheric Research.

The National Center for Atmospheric Research is sponsored by the U.S. National Science Foundation. Any opinions, findings, and conclusions or recommendations expressed in this publication are those of the author(s) and do not necessarily reflect the views of NCAR or the NSF.

Competing Interests

The authors have no competing interests to declare.

Allen, L, Stewart, C and Wright, S. 2017. Strategic open data preservation. College & Research Libraries News , 78(9). https://crln.acrl.org/index.php/crlnews/article/view/16771/18312 . DOI: https://doi.org/10.5860/crln.78.9.482  

Anderson, WL, Faundeen, JL, Greenberg, J and Taylor, F. 2011. Metadata for data rescue and data at risk. In: Conference on Ensuring Long-Term Preservation in Adding Value to Scientific and Technical Data. http://hdl.handle.net/2152/20056  

Aven, T. 2016. Risk assessment and risk management: Review of recent advances on their foundation. European Journal of Operational Research , 253(1): 1–13. DOI: https://doi.org/10.1016/j.ejor.2015.12.023  

Becker, C, Maemura, E and Moles, N. 2020. The design and use of assessment frameworks in digital curation. Journal of the Association for Information Science and Technology (JASIST) , 71(1): 55–68. DOI: https://doi.org/10.1002/asi.24209  

Cervone, HF. 2006. Project risk management. OCLC Systems & Services: International Digital Library Perspectives , 22(4): 256–62. DOI: https://doi.org/10.1108/10650750610706970  

Charifson, DM, Huth, PC, Thompson, JE, Angyal, RK, Flaherty, MJ and Richardson, DC. 2015. History of Fish Presence and Absence Following Lake Acidification and Recovery in Lake Minnewaska, Shawangunk Ridge, NY. Northeastern Naturalist , 22: 762–781. DOI: https://doi.org/10.1656/045.022.0411  

Chodacki, J. 2018. Data Mirror: Complementing data producers. Against the Grain , 29: 35. https://escholarship.org/uc/item/2n1715ff . DOI: https://doi.org/10.7771/2380-176X.7877  

CoreTrustSeal Data Repository Certification. 2018. https://www.coretrustseal.org/ .  

Cornelius, KB and Pasquetto, IV. 2018. “What data?” Records and data policy coordination during presidential transitions. In: Transforming Digital Worlds , 155–163. Springer International Publishing. DOI: https://doi.org/10.1007/978-3-319-78105-1_20  

Cook, BI, Cook, ER, Anchukaitis, KJ, Huth, PC, Thompson, JE and Smiley, SF. 2009. A Homogeneous Record (1896–2006) of Daily Weather and Climate at Mohonk Lake, New York. Journal of Applied Meteorology and Climatology , 49: 544–555. DOI: https://doi.org/10.1175/2009JAMC2221.1  

Cook, BI, Cook, ER, Huth, PC, Thompson, JE, Forster, A and Smiley, D. 2008. A cross-taxa phenological dataset from Mohonk Lake, NY and its relationship to climate. International Journal of Climatology , 28: 1369–1383. DOI: https://doi.org/10.1002/joc.1629  

Data Rescue Interest Group. 2018. Research Data Alliance. https://rd-alliance.org/groups/data-rescue.html .  

Dennis, B. 2016. Scientists are frantically copying U.S. climate data, fearing it might vanish under Trump. The Washington Post , Dec. 13, 2016. https://www.washingtonpost.com/news/energy-environment/wp/2016/12/13/scientists-are-frantically-copying-u-s-climate-data-fearing-it-might-vanish-under-trump/ .  

Donaldson, DR, Dillo, I, Downs, R and Ramdeen, S. 2017. The perceived value of acquiring Data Seals of Approval. International Journal of Digital Curation , 12(1). DOI: https://doi.org/10.2218/ijdc.v12i1.481  

Downs, RR and Chen, RS. 2017. Curation of scientific data at risk of loss: Data rescue and dissemination. In: Johnston, L (ed.), Curating Research Data. Volume One, Practical Strategies for Your Digital Repository . Association of College and Research Libraries. DOI: https://doi.org/10.7916/D8W09BMQ  

Faundeen, J. 2017. Developing criteria to establish trusted digital repositories. Data Science Journal , 16: 22. DOI: https://doi.org/10.5334/dsj-2017-022  

Gallaher, D, Campbell, GG, Meier, W, Moses, J and Wingo, D. 2015. The process of bringing dark data to light: The rescue of the early Nimbus satellite data. GeoResJ , 6: 124–134. DOI: https://doi.org/10.1016/j.grj.2015.02.013  

Graf, R, Ryan, HM, Houzanme, T and Gordea, S. 2017. A decision support system to facilitate file format selection for digital preservation. Libellarium: Journal for the Research of Writing, Books, and Cultural Heritage Institutions , 9(2). DOI: https://doi.org/10.15291/libellarium.v9i2.274  

Griffin, RE. 2015. When are old data new data? GeoResJ , 6: 92–97. DOI: https://doi.org/10.1016/j.grj.2015.02.004  

Guidelines to the Rescue of Data At Risk. 2017. Research Data Alliance. https://www.rd-alliance.org/guidelines-rescue-data-risk .  

Hsu, L, Lehnert, KA, Goodwillie, A, Delano, JW, Gill, JB, Tivey, MA, Ferrini, VL, Carbotte, SM and Arko, RA. 2015. Rescue of long-tail data from the ocean bottom to the Moon: IEDA Data Rescue Mini-Awards. GeoResJ , 6: 108–114. DOI: https://doi.org/10.1016/j.grj.2015.02.012  

[ISO] International Organization for Standardization. 2012a. ISO 14721:2012 (CCSDS 650.0-M-2): Space data and information transfer systems — Open Archival Information System (OAIS) — Reference model. https://www.iso.org/standard/57284.html .  

[ISO] International Organization for Standardization. 2012b. ISO 16363:2012 (CCSDS 652.0-R-1): Space data and information transfer systems — Audit and certification of trustworthy digital repositories. https://www.iso.org/standard/56510.html .  

Janz, M. 2018. Maintaining access to public data: Lessons from Data Refuge. Against the Grain , 29: 30–33. DOI: https://doi.org/10.31229/osf.io/yavzh  

Knapp, KR, Bates, JJ and Barkstrom, B. 2007. Scientific data stewardship: Lessons learned from a satellite–data rescue effort. Bulletin of the American Meteorological Society , 88(9): 1359–1362. DOI: https://doi.org/10.1175/BAMS-88-9-1359  

Lamdan, S. 2018. Lessons from Datarescue: The limitations of grassroots climate change data preservation and the need for federal records law reform. University of Pennsylvania Law Review Online , 166(1): Article 12. https://scholarship.law.upenn.edu/penn_law_review_online/vol166/iss1/12 .  

Levitus, S. 2012. The UNESCO-IOC-IODE “Global Oceanographic Data Archeology and Rescue” (GODAR) Project and “World Ocean Database” Project. Data Science Journal , 11: 46–71. DOI: https://doi.org/10.2481/dsj.012-014  

Maemura, E, Moles, N and Becker, C. 2017. Organizational assessment frameworks for digital preservation: A literature review and mapping. Journal of the Association for Information Science and Technology , 68(7): 1619–1637. DOI: https://doi.org/10.1002/asi.23807  

Mayernik, MS, Downs, RR, Duerr, R, Hou, C-Y, Meyers, N, Ritchey, N, Thomer, A and Yarmey, L. 2017. Stronger together: The case for cross-sector collaboration in identifying and preserving at-risk data. Figshare. DOI: https://doi.org/10.6084/m9.figshare.4816474.v1  

Mayernik, MS, Huddle, J, Hou, C-Y and Phillips, J. 2018. Modernizing library metadata for historical weather and climate data collections. Journal of Library Metadata , 17(3/4): 219–239. DOI: https://doi.org/10.1080/19386389.2018.1440927  

McGovern, NY. 2017. Data rescue. ACM SIGCAS Computers and Society , 47(2): 19–26. DOI: https://doi.org/10.1145/3112644.3112648  

Michener, WK, et al. 1997. Nongeospatial metadata for the ecological sciences. Ecological Applications , 7(1): 330–342. DOI: https://doi.org/10.1890/1051-0761(1997)007[0330:NMFTES]2.0.CO;2  

Mohonk Preserve, Belardo, C, Feldsine, N, Forester, A, Huth, P, Long, E, Morgan, V, Napoli, M, Pierce, E, Richardson, D, Smiley, D, Smiley, S and Thompson, J. 2018a. History of Acid Precipitation on the Shawangunk Ridge: Mohonk Preserve Precipitation Depths and pH, 1976 to Present. Environmental Data Initiative . DOI: https://doi.org/10.6073/pasta/734ea90749e78613452eacec489f419c  

Mohonk Preserve, Forester, A, Huth, P, Long, E, Morgan, V, Napoli, M, Pierce, E, Smiley, D, Smiley, S and Thompson, J. 2018b. Mohonk Preserve Ground Water Springs Data, 1991 to Present. Environmental Data Initiative . DOI: https://doi.org/10.6073/pasta/928feed7ee748509ab065de7e3791966  

Mohonk Preserve, Feldsine, N, Forester, A, Garretson, A, Huth, P, Long, E, Napoli, M, Pierce, E, Smiley, D, Smiley, S and Thompson, J. 2019. Mohonk Preserve Amphibian and Water Quality Monitoring Dataset at 11 Vernal Pools from 1931–Present. Environmental Data Initiative . DOI: https://doi.org/10.6073/pasta/864aea25998b73c5d1a5b5f36cb6583e  

National Research Council. 1983. Risk Assessment in the Federal Government: Managing the Process. Washington, DC: The National Academies Press. DOI: https://doi.org/10.17226/366  

NIWA. 2019. The week it snowed everywhere. NIWA Media Release , Nov. 21, 2019. https://niwa.co.nz/news/the-week-it-snowed-everywhere .  

Peng, G. 2018. The state of assessing data stewardship maturity – An overview. Data Science Journal , 17: 7. DOI: https://doi.org/10.5334/dsj-2018-007  

Peng, G, Milan, A, Ritchey, NA, Partee, RP, II, Zinn, S, McQuinn, E, Casey, KS, Lemieux, P, III, Ionin, R, Jones, P, Jakositz, A and Collins, D. 2019. Practical application of a data stewardship maturity matrix for the NOAA OneStop project. Data Science Journal , 18: 41. DOI: https://doi.org/10.5334/dsj-2019-041  

Peng, G, Privette, JL, Kearns, EJ, Ritchey, NA and Ansari, S. 2015. A unified framework for measuring stewardship practices applied to digital environmental datasets. Data Science Journal , 13: 231–253. DOI: https://doi.org/10.2481/dsj.14-049  

Pienta, AM and Lyle, J. 2018. Retirement in the 1950s: Rebuilding a longitudinal research database. IASSIST Quarterly , 42(1). DOI: https://doi.org/10.29173/iq19  

Poli, P, Dee, DP, Saunders, R, John, VO, Rayer, P, Schulz, J, Bojinski, S, et al. 2017. Recent advances in satellite data rescue. Bulletin of the American Meteorological Society , 98(7): 1471–1484. DOI: https://doi.org/10.1175/BAMS-D-15-00194.1  

Ramapriyan, HK. 2017. NASA’s EOSDIS: Trust and Certification. Presented at: 2017 ESIP Summer Meeting, Bloomington, IN. Figshare. DOI: https://doi.org/10.6084/m9.figshare.5258047.v1  

Richardson, DC, Charifson, DM, Stanson, VJ, Stern, EM, Thompson, JE and Townley, LA. 2016. Reconstructing a trophic cascade following unintentional introduction of golden shiner to Lake Minnewaska, New York, USA. Inland Waters , 6: 29–33. DOI: https://doi.org/10.5268/IW-6.1.915  

Ryan, H. 2014. Occam’s razor and file format endangerment factors. In: Proceedings of the 11th International Conference on Digital Preservation (iPres), October 6–10, 2014, Melbourne, Australia, 179–188. https://www.nla.gov.au/sites/default/files/ipres2014-proceedings-version_1.pdf .  

Slovic, P. 1999. Trust, emotion, sex, politics, and science: Surveying the risk-assessment battlefield. Risk Analysis , 19(4): 689–701. DOI: https://doi.org/10.1023/A:1007041821623  

Thompson, CA, Robertson, WD and Greenberg, J. 2014. Where have all the scientific data gone? LIS perspective on the data-at-risk predicament. College & Research Libraries , 75(6): 842–861. DOI: https://doi.org/10.5860/crl.75.6.842  

Usability.gov . 2019. Card Sorting. US Department of Health & Human Services. https://www.usability.gov/how-to-and-tools/methods/card-sorting.html .  

USGS. 2019. USGS Data at Risk: Expanding Legacy Data Inventory and Preservation Strategies. US Geological Survey. https://www.sciencebase.gov/catalog/item/58b5ddc3e4b01ccd54fde3fa .  

Varinsky, D. 2017. Scientists across the US are scrambling to save government research in ‘Data Rescue’ events. Business Insider , Feb. 11, 2017. http://www.businessinsider.com/data-rescue-government-data-preservation-efforts-2017-2 .  

Yakel, E, Faniel, I, Kriesberg, A and Yoon, A. 2013. Trust in digital repositories. International Journal of Digital Curation , 8(1). DOI: https://doi.org/10.2218/ijdc.v8i1.251  

Yoon, A. 2017. Data reusers’ trust development. Journal of the Association for Information Science and Technology , 68(4): 946–956. DOI: https://doi.org/10.1002/asi.23730  

Zimmerman, DE and Akerelrea, C. 2002. A group card sorting methodology for developing informational web sites. In: Proceedings IEEE International Professional Communication Conference, 437–445. IEEE. DOI: https://doi.org/10.1109/IPCC.2002.1049127  

• Research article
• Open access
• Published: 12 April 2021

A risk assessment tool for resumption of research activities during the COVID-19 pandemic for field trials in low resource settings

• Suzanne M. Simkovich   ORCID: orcid.org/0000-0003-2462-0856 1 , 2 , 3 ,
• Lisa M. Thompson 4 ,
• Maggie L. Clark 5 ,
• Kalpana Balakrishnan 6 ,
• Alejandra Bussalleu 7 , 8 ,
• William Checkley 1 , 2 ,
• Thomas Clasen 9 ,
• Victor G. Davila-Roman 10 ,
• Anaite Diaz-Artiga 11 ,
• Ephrem Dusabimana 12 ,
• Lisa de las Fuentes 10 ,
• Steven Harvey 2 , 13 ,
• Miles A. Kirby 14 ,
• Amy Lovvorn 9 ,
• Eric D. McCollum 15 ,
• Erick E. Mollinedo 16 ,
• Jennifer L. Peel 5 ,
• Ashlinn Quinn 17 ,
• Ghislaine Rosa 18 ,
• Lindsay J. Underhill 1 , 2 ,
• Kendra N. Williams 1 , 2 ,
• Bonnie N. Young 5 ,
• Joshua Rosenthal 17 &

HAPIN Investigators

BMC Medical Research Methodology volume  21 , Article number:  68 ( 2021 ) Cite this article

5906 Accesses

7 Citations

8 Altmetric

Metrics details

The spread of severe acute respiratory syndrome coronavirus-2 has suspended many non-COVID-19 related research activities. Where restarting research activities is permitted, investigators need to evaluate the risks and benefits of resuming data collection and adapt procedures to minimize risk.

In the context of the multicountry Household Air Pollution Intervention (HAPIN) trial conducted in rural, low-resource settings, we developed a framework to assess the risk of each trial activity and to guide protective measures. Our goal is to maximize the integrity of reseach aims while minimizing infection risk based on the latest scientific understanding of the virus.

We drew on a combination of expert consultations, risk assessment frameworks, institutional guidance and literature to develop our framework. We then systematically graded clinical, behavioral, laboratory and field environmental health research activities in four countries for both adult and child subjects using this framework. National and local government recommendations provided the minimum safety guidelines for our work.

Our framework assesses risk based on staff proximity to the participant, exposure time between staff and participants, and potential viral aerosolization while performing the activity. For each activity, one of four risk levels, from minimal to unacceptable, is assigned and guidance on protective measures is provided. Those activities that can potentially aerosolize the virus are deemed the highest risk.

Conclusions

By applying a systematic, procedure-specific approach to risk assessment for each trial activity, we were able to protect our participants and research team and to uphold our ability to deliver on the research commitments we have made to our staff, participants, local communities, and funders. This framework can be tailored to other research studies conducted in similar settings during the current pandemic, as well as potential future outbreaks with similar transmission dynamics.

The trial is registered with clinicaltrials.gov NCT02944682 on October 26. 2016 .

Peer Review reports

The spread of severe acute respiratory syndrome coronavirus-2 (SARS-CoV-2) and resulting coronavirus disease 2019 (COVID-19) has led to the temporary suspension of many non-COVID-19 related research activities worldwide. Where feasible, studies are considering remote data collection by telephone or web-based conferencing [ 1 , 2 , 3 ]. However, this approach is often not possible when performing anthropometric measurements, specimen collection, or when investigators need to make other direct observations. Even temporary suspension of research activities can potentially cause harm if investigators are evaluating an intervention that is hypothesized to be beneficial. Further, the suspension of data collection could result in loss of study power and potentially introduce bias. Every day, we are gaining a greater understanding of the transmissibility of SARS-CoV-2, and this knowledge increases our ability to safely resume a wide variety of non-COVID-19 related research activities [ 3 , 4 ]. Where local law or institutional regulations permit activities to restart, investigators need to evaluate the risks and benefits to both research staff and participants of resuming data collection. To safely conduct study activities, researchers need to develop standardized procedures that are based on realistic assessment of these risks, provide guidance on where and when they are manageable, as well as how to minimize the risk with physical distance measures and appropriate personal protective equipment (PPE).

Investigators in the Household Air Pollution Intervention Network (HAPIN) trial initially suspended data collection due to the pandemic in March 2020 and have since restarted collection of behavioural, environmental, biological and clinical measurements during the fifth year of a five-year, multi-country trial [ 5 , 6 , 7 , 8 ]. HAPIN is a randomized controlled trial in rural areas of Guatemala, India, Peru, and Rwanda that is assessing the health benefits of providing liquefied petroleum gas (LPG) stoves and an 18-month supply of free LPG to 3200 households that otherwise depend on solid biomass fuel (wood, animal dung, or crop residue) for cooking. Measurements of cooking behavior, personal and in-home exposure to air pollution, biological samples and clinical measurements are being collected longitudinally from pregnant women and their newborns in every household, along with an older, non-pregnant adult woman, if she resides in the house [ 5 , 6 , 7 , 8 ]. Our study involves home visits, as well as visits to health centers and hospitals during the woman’s pregnancy and the first year of the child’s life.

As SARS-CoV-2 spread globally, governments in all four countries implemented public safety restrictions that limited activities to those designated as essential. Essential activities varied across settings and during the initial period of restrictions. Research activities were not considered essential. However, LPG delivery for cooking was considered essential in all four countries. In Guatemala and Rwanda, our research teams were permitted to continue delivering LPG to study households without disruption. In India, the gas companies continued to deliver refill tanks to study participants. In Peru, our team was limited in its ability to deliver gas during the initial weeks of the restrictions, but we were later able to re-establish services with a local gas company for delivery.

With permission to continue delivery of the LPG intervention, we immediately implemented changes in our delivery protocols to minimize SARS-CoV-2 risk. Further, in anticipation of the additional easement of movement restrictions in countries around the world, we reviewed the literature for guidance on strategies researchers have used for assessing the risk of activities during COVID-19 or other pandemics, and found a dearth of available guidance. Perhaps the most relevant existing framework is that proposed by Lumeng and colleagues, which was designed for research focused in clinical settings in the U.S.A. Thus, we developed a risk assessment tool with the guiding principle of ethical research to minimize the potential risks to research staff, participants and rural communities participating in the HAPIN trial research settings. We wanted our risk assessment tool to allow researchers to assess the risk of each study activity utilizing the same general criteria to support management decisions across this large multinational, multi-disciplinary study with both competing and complementary activities. Although our risk assessment tool has been designed within the framework of specific activities of the HAPIN trial, we report here on our approach, which can be applied to other research contexts and questions in similar settings.

In developing our risk assessment tool, we drew on a combination of expert consultations, government regulations, national and local expertise, institutional guidance and review of emerging literature. We queried our multi-national panel of investigators and field team leaders from across the trial with expertise in the disciplines of clinical medicine and imaging, nursing, environmental science, epidemiology, behavioral science, community engagement and statistics, along with the trial funders who provide scientific guidance to the HAPIN trial. We sought input from local community leaders, the Ministries of Health, universities and non-governmental organizations regarding appropriate operations and safety concerns. We consulted with the Institutional Review Boards (IRBs) and Data and Safety Monitoring Board (DSMB) presiding over the trial regarding resumption of activities. We drew upon historical occupational health frameworks for infectious disease biosafety and risk assessment and the most recent peer-reviewed and grey literature about infection dynamics. We also considered staff experience. Using all of these inputs, we built a framework to evaluate risk of exposure to SARS-CoV-2 [ 3 ]. Our intent was to develop criteria that were clear, simple and actionable for field managers and staff to implement, and to recommend appropriate practices and materials, in accordance with the risk level of each procedure and perceived risk threshold.

While SARS-CoV2 research findings are still emerging, our assessment is based on the consensus that aerosolization and droplet carriage of virus, primarily from coughing, sneezing, singing, crying, talking, are the predominant modes of infection. It is unclear how long the virus remains in the air. Fomites from surface contact may also contribute to transmission, but are likely a smaller risk. Evidence of SARS-CoV-2 presence has been detected in urine, stool, breast milk, semen and blood, but we are not aware of documented transmission through these bodily fluids at the time of this publication. Furthermore, the risk of transmission is greatest in the two days preceding onset of symptoms and continues afterward for at least ten days, and up to twenty days in immunosuppressed patients. Because documented asymptomatic carriage has been widely reported, we assumed that any staff member, collaborator or community participant might be shedding the virus. Small children (especially infants) appear to be infected at the same rate as adults, but have more mild disease and thus may be unknowingly spreading disease. We agreed that viral transmissibility and the true prevalence of COVID-19 are not clearly known in any of our study sites due to limited testing. We also note that recent seroprevalence studies have reported that case burdens are likely underreported. As such, we chose to err on the side of caution and assume moderate to substantial incidence of disease in all our settings. Therefore, risk was defined as large-scale, uncontrolled community transmission. When widespread vaccination has been achieved in our settings and/or when there are other indications of lower prevalence of disease in our sites, we may adjust our risk levels accordingly.

We assessed each HAPIN data collection activity among each group of participants (pregnant woman /new mother, infant, or non-pregnant older adult woman) because the risks may vary with each participant group. Data collection activities were graded and agreed upon by our team of scientists. Local site investigators were asked to report perceived concerns by staff and participants in their communities. Risk factors and definitions were presented to the HAPIN steering committee, which met weekly, for feedback before adoption. Even with adoption by the steering committee, if local community risk factors at the sites did not allow continued trial activities, the activity was stopped until safety could be ensured. Standard Operating Procedures were developed for the resumption of study activities and included guidance on screening staff and participants for Covid-19 symptoms, transporting staff in project vehicles, cleaning equipment and surfaces, conducting home visits and health facility survelliance, and quarantining for suspected exposures to the virus. These documents are reviewed monthly by two assigned investigators on the trial to reflect they reflect the most up to date knowledge of transmission dynamics and local risk.

Evaluation of risk criteria for each procedure included the age of participant, location, required physical proximity, exposure time, aerosolization potential, and criteria for use of PPE (Table  1 ). Using these criteria we established a four level schedule that ranged from minimal to unacceptably high risk (Table  2 ). We then proceeded to assess each research activity according to the criteria outlined in Table 1 and assigned a risk level and appropriate PPE to each of these. We assessed research activities that included LPG fuel delivery, administration of tablet-based surveys (e.g. questionnaires asked of mothers about their children’s health), data downloads from environmental monitors, personal exposure assessment to household air pollution, biological sample collection (e.g. urine, nasal swabs, venous blood) and lab processing of biological samples in the field laboratories, clinical measures (e.g. newborn birth weight, lung ultrasound, blood pressure), observations in homes of pregnant women/new mothers, children, and vascular procedures in adults (Additional file 1 ).

Protective measures available in our settings were: a) where feasible, data collection was completed by telephone; b) where possible, face-to-face activities were conducted outside; c) when inside homes, clinics or offices, staff and participants minimized the number of people in the room; d) rigourous hygiene for staff, materials, equipment and surfaces were employed at all times; e) appropriate PPE was used based on the context and activity; f) under very high risk conditions, the visit or the procedure was suspended.

Using this assessment and taking necessary measures for protection, almost all of our research activities were deemed to pose potentially manageable risks. Biological sample collection spanned a range of assigned risk due to differences in participant-staff interaction. The activities with the highest level of risk were those that potentially aerosolize the virus during the procedure. For example, urine collection requires minimal contact (i.e., field workers instruct the participant to collect and store the urine sample until it can be retreived) resulting in low risk to both the participant and the study staff. However, dried blood spot collection from capillary blood draws from infants (who are unable to wear a mask and often cry during the procedure) could feasibly put field workers at high risk (examples of two procedures are provided in Table 1 ; all procedures are described in the Additional file 1 ). To illustrate Level 4 activities, we identified several activities that were not part of our protocol, but that could pose unmanageable risks (e.g. bronchoscopy, sputum inducting procedures, cardiopulmonary resusitation) for routine research in the pandemic context (Table 2 ).

Our risk assessment framework uses a four-level risk schedule that is flexible, allowing adjustment for changes to risk measures and definitions as new evidence emerges about virus transmission. The approach and risk assessment tool we present here can be adapted by other investigators who are assessing and managing the risk posed in their own research during the coronavirus pandemic. However, prior to the deployment of risk assessment tools such as ours, researchers, in association with community members, IRBs, DSMBs, and grant funders need to evaluate the importance of any activity related to the primary aims of a trial weighed against the associated risk of performing the research activity. Obviously, local health regulations related to mobility, home or clinic visits by researchers supersede any of these judgments.

The framework offers a way to systematically evaluate diverse research activities involving different disciplines using the same basic criteria and a scoring system to compare associated risk for a given procedure. It also provides clear guidance for field teams on the appropriate PPE and practices in the context of limited resource environments, and thus appropriately utilizes limited PPE where it may be scarce and expensive. Despite these strengths, there are limitations. Our framework does not make recommendations on whether or not to continue an activity – e.g. through an explict cost-benefit algorithm. Decisions on what research should be continued in the presence of risk also require a careful assessment of benefits. We chose to make the risk-benefit calculation and decisions regarding which activities to suspend an independent process from assessment of risk. In our context, an efficacy trial, we are in equipoise regarding the potential benefits of the intervention to participants. Thus, analysis of benefits can only be honestly assessed in terms of the potential benefit of a given activity to the integrity of the trial, not to trial participants as maybe the case for other kinds of clinical research. Among the criteria we used to examine potential benefits of risk in the HAPIN trial were whether or not the aim of any given procedure supports a primary, secondary or tertiary (exploratory) outcome of the trial protocol. This evaluation is made by the HAPIN Steering Committee.

Furthermore, we do not factor in specific local regulations into the matrix in an a priori fashion, and thus leave it to local investigators and study teams to adjust for these [ 9 ]. Because of this, our framework specifications may need to be adjusted to meet local institutional or government regulations regarding PPE or other safety practices. Finally, our framework is limited by the current evidence regarding transmission risk and should be reevaluated and updated as our understanding grows. Such updates will require evaluation by scientists who are up to date on the current literature and recommendations regarding transmission and appropriate PPE, and must be sensitive to changes in local practices. While the recently discovered variants of the SARS-COV-2 suggests higher transmission risk, we do not have experimental or observation evidence at this time that our framework should be significantly changed [ 10 ]. However, should evidence emerge that for example, cloth masks are less protective or residual survival of the virus on surfaces is greater, we would need to make changes to our protocols [ 11 ].

Of note, the most relevant existing framework we are aware of for resumption of research in the COVID-19 pandemic context is that published by Lumeng and colleagues designed for U.S. clinical research. Our framework was developed independently and for a different context, but their basic approach is similar to ours in that it provides for a high level set of principles, a tiered framework, and risk evaluation that includes factors such as duration and distance of contact between researchers and patients. Our framework adds a great detail to risk evaluation in more complex and varied environments, and outlines how these can apply to specific and diverse research tasks (See Additional file 1 : Tables 1–5). Our framework also differs from Lumeng et al’s in that we have not included an explict benefit analysis, as described in the preceding paragraphs.

This risk analysis takes place in the dynamic context of a global pandemic. We plan to reassess each activity using our tool at least monthly as more information about SARS-CoV-2 transmission and the local epidemics becomes available. While the pandemic has been disruptive to our research, we believe there may also be some benefits from the shift in some data collection methods. For example, collecting data via telephone instead of visiting in-person increases time use efficiency for staff and decreases the burden of household visits on participants. Costs are lower, with less fuel used to travel via truck or motorbike to distant participant homes. On the other hand, telephone surveys may introduce uncertainty, if questions are complex in nature and may lead to poor response rates or lower quality data [ 12 ]. We acknowledge that we have been able to resume study activities in some of our research sites, and attribute this to building relationships with participants over the past several years and the commitment of our local teams and collaborating institutions.

For researchers now facing the need to resume activities that may lead to risk of exposure to staff and participants, we offer the following advice. First, evaluate any scientific developments about risk of infection or severity of disease that might change the calculus fundamentally. Second, convene representatives of your research teams, IRB members, relevant patient groups, stakeholders, and infectious disease experts to evaluate research activities against our framework and risk schedule, and then adapt as necessary with broad input. Third, while the field, clinical and laboratory activities we presented (Tables 1-2, Additional file 1 ) may be similar in scope to other research activities, we have obviously not presented all of the potential scenarios. Research activities should be adapted to fit individual research needs, reviewed repeatedly by stakeholder groups until consensus is reached, and operationalized using Standard Operating Procedures for each activity stream.

Beyond the risk assessment tool we have outlined above, we offer the following brief description of how we deployed these rules for field teams that may have similar needs. At the beginning of the pandemic, we temporarily suspended all activities except for LPG fuel delivery until risk of the measurements could be assessed and procedures put into place to ensure safety. We continued collecting data by telephone when possible. When in-person contact was permitted by local authorities and local institutional IRBs, we used our framework to guide the appropriate protocols. If designated PPE was not available or could not be used properly at any time, we postponed the activity. Similarly, our rules required goggles or face shields for certain procedures, but participants (especially children) may find these terrifying, especially when combined with masks and gloves. In these situations, it may not be possible to complete the work as planned, and local staff had the autonomy and responsibility to decide whether any activity should proceed.

Finally, our guidance is based on expert opinion and has not been empirically verified at this time. Importantly, our framework does not substitute for the need for coordination and approval of IRBs when protocols are modified.

We are optimistic that by applying this systematic, procedure-specific approach to risk assessment for each research activity, we will minimize the disruption in our trial due to the pandemic and support the completion of our primary outcomes. Our framework can be applied to other field trials in low-resource settings to guide investigators in assessing the risk of each trial activity and implementing appropriate safety measures, where the level of risk is acceptable. While no activity in the current context is completely without risk of infection, utilizing a systematic approach is the optimal way to safeguard research activities, protect research staff and participants, and comply with the ethical obligations to those that have agreed to participate in trials, along with the communities and funders that have supported these efforts.

Availability of data and materials

Data sharing is not applicable to this article as no datasets were generated or analyzed during the current study.

Abbreviations

Severe acute respiratory syndrome coronavirus-2

Coronavirus disease 2019

Personal protective equipment

Household Air Pollution Intervention Network

Institutional Review Boards

Data and Safety Monitoring Board

Nicol GE, Piccirillo JF, Mulsant BH, Lenze EJ. Action at a distance: geriatric research during a pandemic. J Am Geriatr Soc. 2020;68(5):922–5.

Article   Google Scholar  

Bikson M, Hanlon CA, Woods AJ, Gillick BT, Charvet L, Lamm C, et al. Guidelines for TMS/tES clinical services and research through the COVID-19 pandemic. Brain Stimul. 2020;13(4):1124–49.

Padala PR, Jendro AM, Gauss CH, Orr LC, Dean KT, Wilson KB, et al. Participant and caregiver perspectives on clinical research during Covid-19 pandemic. J Am Geriatr Soc. 2020;68(6):E14–e8.

Coronavirus disease (COVID-2019) situation reports Geneva: World Health Organization; 2020 [Available from: http://www.who.int/emergencies/diseases/novel-coronavirus-2019/situation-reports/ . Accessed 20 Sept 2020.

Barr DB, Puttaswamy N, Jaacks LM, Steenland K, Rajkumar S, Gupton S, et al. Design and rationale of the biomarker Center of the Household air Pollution Intervention Network (HAPIN) trial. Environ Health Perspect. 2020;128(4):47010.

Article   CAS   Google Scholar  

Clasen T, Checkley W, Peel JL, Balakrishnan K, McCracken JP, Rosa G, et al. Design and rationale of the HAPIN study: a multicountry randomized controlled trial to assess the effect of liquefied petroleum gas stove and continuous fuel distribution. Environ Health Perspect. 2020;128(4):47008.

Simkovich SM, Underhill LJ, Kirby MA, Goodman D, Crocker ME, Hossen S, et al. Design and conduct of facility-based surveillance for severe childhood pneumonia in the Household Air Pollution Intervention Network (HAPIN) trial. ERJ Open Res. 2020;6(1).

Johnson MA, Steenland K, Piedrahita R, Clark ML, Pillarisetti A, Balakrishnan K, et al. Air pollutant exposure and stove use assessment methods for the household air pollution intervention network (HAPIN) trial. Environ Health Perspect. 2020;128(4):47009.

Kigali, Rwanda: Rwanda Biomedical Centre (RBC); [Available from: https://www.rbc.gov.rw/index.php?id=188 . Accessed 1 Oct 2020

Emerging SARS-CoV-2 Variants: Centers for Disease Control and Prevention [updated January 15, 2021. Available from: http://www.gob.pe/minsa/ . Accessed 18 Jan 2021.

Leung K, Shum MH, Leung GM, Lam TT, Wu JT. Early transmissibility assessment of the N501Y mutant strains of SARS-CoV-2 in the United Kingdom, October to November 2020. Euro Surveill. 2021;26(1):2002106.

Carr ECJ, Worth A. The use of the telephone interview for research. NT Res. 2001;6(1):511–24.

Download references

Acknowledgements

A multidisciplinary, independent Data and Safety Monitoring Board (DSMB) appointed by the National Heart, Lung, and Blood Institute (NHLBI) monitors the quality of the data and protects the safety of patients enrolled in the HAPIN trial. NHLBI DSMB: Nancy R. Cook, Sc.D.; Stephen Hecht, Ph.D.; Catherine Karr, M.D., Ph.D.; Katie H. Kavounis, M.P.H.; Dong-Yun Kim, Ph.D.; Joseph Millum, Ph.D.; Lora A. Reineck, M.D., M.S.; Nalini Sathiakumar, M.D., Dr.P.H.; Paul K. Whelton, M.D.; Gail G. Weinmann, M.D.

Program Coordination: Gail Rodgers, M.D., Bill & Melinda Gates Foundation; Claudia L. Thompson, Ph.D., National Institute of Environmental Health Science (NIEHS); Mark J. Parascandola, Ph.D., M.P.H., National Cancer Institute (NCI); Danuta M. Krotoski, Ph.D., Eunice Kennedy Shriver National Institute of Child Health and Human Development (NICHD); Joshua P. Rosenthal, Ph.D., Fogarty International Center (FIC); Conception R. Nierras, Ph.D., NIH Office of Strategic Coordination Common Fund; Antonello Punturieri, M.D., Ph.D. and Barry S. Schmetter, B.S., National Heart, Lung, and Blood Institute (NHLBI).

The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the U.S. National Institutes of Health or Department of Health and Human Services.

HAPIN Investigators: Vigneswari Aravindalochanan, Kalpana Balakrishnan, Dana Boyd Barr, Vanessa Burrowes, Devan Campbell, Julia McPeek Campbell, Eduardo Canuz, Adly Castañaza, Howard Chang, William Checkley, Yunyun Chen, Marilú Chiang, Maggie L. Clark, Thomas Clasen, Rachel Craik, Mary Crocker, Victor Dávila-Román, Lisa de las Fuentes, Oscar De Léon, Anaité Diaz-Artiga, Ephrem Dusabimana, Lisa Elon, Juan Gabriel Espinoza, Irma Sayury Pineda Fuentes, Sarada Garg, Dina Goodman, Savannah Gupton, Meghan Hardison, Stella Hartinger, Steven A. Harvey, Mayari Hengstermann, Phabiola Herrera, Shakir Hossen, Penelope Howards, Lindsay Jaacks, Shirin Jabbarzadeh, Michael A. Johnson, Abigail Jones, Katherine Kearns, Miles Kirby, Jacob Kremer, Margaret Laws, Patricia M. Lenzen, Jiawen Liao, Amy Lovvorn, Fiona Majorin, Eric McCollum, John P. McCracken, Rachel M. Meyers, J. Jaime Miranda, Erick Mollinedo, Lawrence Moulton, Krishnendu Mukhopadhyay, Luke Naeher, Abidan Nambajimana, Florien Ndagijimana, Azhar Nizam, Jean de Dieu Ntivuguruzwa, Aris Papageorghiou, Jennifer Peel, Ricardo Piedrahita, Ajay Pillarisetti, Naveen Puttaswamy, Elisa Puzzolo, Ashlinn Quinn, Sarah Rajkumar, Usha Ramakrishnan, Davis Reardon, Ghislaine Rosa, Joshua Rosenthal, P. Barry Ryan, Zoe Sakas, Sankar Sambandam, Jeremy Sarnat, Suzanne Simkovich, Sheela Sinharoy, Kirk R. Smith, Kyle Steenland, Damien Swearing, Gurusamy Thangavel, Lisa M. Thompson, Ashley K. Toenjes, Lindsay Underhill, Jean Damascene Uwizeyimana, Viviane Valdes, Amit Verma, Lance Waller, Megan Warnock, Kendra Williams, Wenlu Ye, Bonnie N. Young.

The Healthcare Delivery Network at Medstar Health Research Institute supported the submission of this manuscript as Suzanne M. Simkovich transferred her affiliation.

Role of study sponsor

Program officials from all of the above listed organizations participated in regular conference calls, made recommendations about study design and participated in final decision-making on the trial study protocol for the overall HAPIN trial; program officials from the National Heart, Lung and Blood Institute, National Institute of Environmental Health Sciences, and the Bill & Melinda Gates Foundation commented on drafts of this substudy; however, no program officials had a role in the writing of this report or decision to submit it for publication. The corresponding authors share final responsibility for the decision to submit for publication.

This study is funded by the U.S. National Institutes of Health (cooperative agreement 1UM1HL134590) in collaboration with the Bill & Melinda Gates Foundation (OPP1131279). Participating NIH organizations include the National Heart, Lung and Blood Institute, National Institute of Environmental Health Sciences, National Cancer Institute, National Institute of Child Health and Human Development, Fogarty International Center, and the NIH Common Fund. Suzanne M. Simkovich was supported by funding from the National Heart, Lung and Blood Institute U., the National Heart, Lung, and Blood Institute 1F32HL143909–01, the National Heart, Lung, and Blood Institute K12HL137942. Lindsay J. Underhill and Kendra Williams were supported by Research Training Grant D43TW009340 (MPIs: Buekens, Checkley, Chi, Kondwani) funded by United States National Institutes of Health through the following Institutes and Centers: Fogarty International Center, National Institute of Neurological Disorders and Stroke, National Institute of Mental Health, National Heart, Lung, and Blood Institute and the National Institute of Environmental Health Sciences along with the National Heart, Lung, and Blood Institute 1F32HL143909–01.

Author information

Authors and affiliations.

Division of Pulmonary and Critical Care, School of Medicine, Johns Hopkins University, Baltimore, USA

Suzanne M. Simkovich, William Checkley, Lindsay J. Underhill & Kendra N. Williams

Center for Global Non-Communicable Disease Research and Training, School of Medicine, Johns Hopkins University, Baltimore, USA

Suzanne M. Simkovich, William Checkley, Steven Harvey, Lindsay J. Underhill & Kendra N. Williams

MedStar Health Research Institute, Hyattsville, USA

Suzanne M. Simkovich

Nell Hodgson Woodruff School of Nursing, Emory University, Atlanta, USA

Lisa M. Thompson

Department of Environmental and Radiological Health Sciences, Colorado State University, Fort Collins, USA

Maggie L. Clark, Jennifer L. Peel & Bonnie N. Young

Department of Environmental Health Engineering, ICMR Center for Advanced Research on Air Quality, Climate and Health, Sri Ramachandra Institute for Higher Education and Research (Deemed University), Chennai, India

Kalpana Balakrishnan

A.B. PRISMA, San Miguel, Peru

Alejandra Bussalleu

CLIMA – Latin American Center of Excellence in Climate Change and Health; and Intercultural Citizenship and Indigenous Health Unit, Faculty of Public Health and Administration, Universidad Peruana Cayetano Heredia, Lima, Peru

Gangarosa Department of Environmental Health, Rollins School of Public Health, Emory University, Atlanta, USA

Thomas Clasen & Amy Lovvorn

Cardiovascular Division, Department of Medicine, Washington University School of Medicine, St. Louis, USA

Victor G. Davila-Roman & Lisa de las Fuentes

Center for Health Studies, Universidad del Valle de Guatemala, Guatemala City, Guatemala

Anaite Diaz-Artiga

Eagle Research Center, Kigali, Rwanda

Ephrem Dusabimana

Department of International Health, Johns Hopkins Bloomberg School of Public Health, Johns Hopkins University, Baltimore, USA

Steven Harvey

Department of Global Health and Population, Harvard T.H. Chan School of Public Health, Boston, MA, USA

Miles A. Kirby

Global Program for Respiratory Sciences, Eudowood Division of Pediatric Respiratory Sciences, Department of Pediatrics, School of Medicine, Johns Hopkins University, Baltimore, USA

Eric D. McCollum

Department of Environmental Health Science, College of Public Health, University of Georgia, Athens, USA

Erick E. Mollinedo

Division of International Epidemiology and Population Studies, Fogarty International Center, National Institutes of Health, Bethesda, USA

Ashlinn Quinn & Joshua Rosenthal

Faculty of Infectious and Tropical Diseases, London School of Hygiene and Tropical Medicine, London, UK

Ghislaine Rosa

You can also search for this author in PubMed   Google Scholar

• Vigneswari Aravindalochanan
• , Kalpana Balakrishnan
• , Dana Boyd Barr
• , Vanessa Burrowes
• , Devan Campbell
• , Julia Mc Peek Campbell
• , Eduardo Canuz
• , Adly Castañaza
• , Howard Chang
• , William Checkley
• , Yunyun Chen
• , Marilú Chiang
• , Maggie L. Clark
• , Thomas Clasen
• , Rachel Craik
• , Mary Crocker
• , Victor Dávila-Román
• , Lisa de las Fuentes
• , Oscar De Léon
• , Anaité Diaz-Artiga
• , Ephrem Dusabimana
• , Lisa Elon
• , Juan Gabriel Espinoza
• , Irma Sayury Pineda Fuentes
• , Sarada Garg
• , Dina Goodman
• , Savannah Gupton
• , Meghan Hardison
• , Stella Hartinger
• , Steven A. Harvey
• , Mayari Hengstermann
• , Phabiola Herrera
• , Shakir Hossen
• , Penelope Howards
• , Lindsay Jaacks
• , Shirin Jabbarzadeh
• , Michael A. Johnson
• , Abigail Jones
• , Katherine Kearns
• , Miles Kirby
• , Jacob Kremer
• , Margaret Laws
• , Patricia M. Lenzen
• , Jiawen Liao
• , Amy Lovvorn
• , Fiona Majorin
• , Eric McCollum
• , John P. McCracken
• , Rachel M. Meyers
• , J. Jaime Miranda
• , Erick Mollinedo
• , Lawrence Moulton
• , Krishnendu Mukhopadhyay
• , Luke Naeher
• , Abidan Nambajimana
• , Florien Ndagijimana
• , Azhar Nizam
• , Jean de Dieu Ntivuguruzwa
• , Aris Papageorghiou
• , Jennifer Peel
• , Ricardo Piedrahita
• , Ajay Pillarisetti
• , Naveen Puttaswamy
• , Elisa Puzzolo
• , Ashlinn Quinn
• , Sarah Rajkumar
• , Usha Ramakrishnan
• , Davis Reardon
• , Ghislaine Rosa
• , Joshua Rosenthal
• , P. Barry Ryan
• , Zoe Sakas
• , Sankar Sambandam
• , Jeremy Sarnat
• , Suzanne Simkovich
• , Sheela Sinharoy
• , Kirk R. Smith
• , Kyle Steenland
• , Damien Swearing
• , Gurusamy Thangavel
• , Lisa M. Thompson
• , Ashley K. Toenjes
• , Lindsay Underhill
• , Jean Damascene Uwizeyimana
• , Viviane Valdes
• , Amit Verma
• , Lance Waller
• , Megan Warnock
• , Kendra Williams
•  & Bonnie N. Young

Contributions

SS & JR conceptualized design of the tool, built the tool, completed the risk assessment of each activity, provided scientific input to assess each activity and wrote the manuscript. LT and MC built the tool, provided scientific input to assess each activity and participated in the writing of the manuscript.TC, WC, AL, JP oversaw the trial’s decisions in risk and benefit and the building of the risk assessment tool and provided comments to the writing of the manuscript. KB, AB, WC, TC, VDR, ADA, LF, SH, MK, AL, EM, EM, ED, JP, AQ, GR, provided scientific input into the design of the tool, reviewed and provided input on each activity, and provided comments to the writing of the manuscript. LU, KW, BY provided scientific input into the design of the tool, completed the assessment of activities, provided scientific input on the tool and provided comments to the writing of the manuscript. All Authors have reviewed and approved the final manuscript.

Corresponding author

Correspondence to Suzanne M. Simkovich .

Ethics declarations

Ethics approval and consent to participate.

This project is a component of a larger clinical trial. The overall study protocol was reviewed and approved by institutional review boards (IRBs) or Ethics Committees at Emory University (00089799), Johns Hopkins University (00007403), Sri Ramachandra Institute of Higher Education and Research (IEC-N1/16/JUL/54/49) and the Indian Council of Medical Research – Health Ministry Screening Committee (5/8/4–30/(Env)/Indo-US/2016-NCD-I), Universidad del Valle de Guatemala (146–08-2016/11–2016) and Guatemalan Ministry of Health National Ethics Committee (11–2016), A.B. PRISMA (CE29841.17), the London School of Hygiene and Tropical Medicine (11664–5) and the Rwandan National Ethics Committee (No.357/RNEC/2018), and Washington University in St. Louis (201611159). Written consent was obtained from all participants.

Consent for publication

Not applicable.

Competing interests

No authors have competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Suzanne M. Simkovich - Change in institution

Ashlinn Quinn and Joshua Rosenthal - the views expressed in this publication are those of the investigators and do not reflect official statements or policy of the National Institutes of Health.

Supplementary Information

Additional file 1., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Simkovich, S.M., Thompson, L.M., Clark, M.L. et al. A risk assessment tool for resumption of research activities during the COVID-19 pandemic for field trials in low resource settings. BMC Med Res Methodol 21 , 68 (2021). https://doi.org/10.1186/s12874-021-01232-x

Download citation

Received : 22 October 2020

Accepted : 17 February 2021

Published : 12 April 2021

DOI : https://doi.org/10.1186/s12874-021-01232-x

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Risk assessment

BMC Medical Research Methodology

ISSN: 1471-2288

• General enquiries: [email protected]

The Essex website uses cookies. By continuing to browse the site you are consenting to their use. Please visit our cookie policy to find out which cookies we use and why. View cookie policy.

Research risk assessment

It's the responsibility of the principal investigators (PI) and researchers to identify reasonably foreseeable risks associated with their research and control the risks so far as is reasonably practicable.

All participants and research assistants have the right to expect protection from physical, psychological, social, legal and economic harm at all times during an investigation. Certain research may also present reputational, legal and / or economic risks to the University.

As part of the ethical approval process for research involving human participants you are required to identify potential risks associated with your research and the action you will take to mitigate risk. You may be asked to submit your risk assessment.

The risk assessment process is a careful examination of what could cause harm, who/what could be harmed and how. It will help you to determine what risk control measures are needed and whether you are doing enough. 

Risk assessment responsibility

The PI and researchers need to take responsibility for all assessments associated with their projects. Occasionally you may need research workers or students to risk assess an aspect of the work and you will need to check the assessments are adequate and sign them off.

Risk assessors need to be competent and you’ll need to ensure they have adequate training and resource to do the assessments. There is risk assessment training available and help and advice help and advice help and advice from your Health and Safety Advisory Service link advisor and safety specialists (for health and safety risks), or the REO Research Governance team for other risks. In some cases, the hazards are so unique to the research that the PI and their team might be the only people who know the work well enough to make valid judgements about the risk and justify their conclusions.

Risk assessment process

The risk assessment process is a careful examination of what could cause harm, who/what could be harmed and how. It will help you to determine what risk control measures are needed and whether you are doing enough.

To simplify the process you can use the health and safety risk assessment templates, risk estimation tool and guidance for all risks associated with your research project. Please refer to the research risk estimation guidance under how to carry out a risk assessment below to assist you. 

Research risks

Typical risks that need to be considered as part of research ethics are:

• Social risks: disclosures that could affect participants standing in the community, in their family, and their job.
• Legal risks: activities that could result in the participant, researchers and / or University committing an offence; activities that might lead to a participant disclosing criminal activity to a researcher which would necessitate reporting to enforcement authorities; activities that could result in a civil claim for compensation.
• Economic harm: financial harm to participant, researcher and / or University through disclosure or other event.
• Reputational risk: damage to public perception of University or the University/researchers’ reputation in the eyes of funders, the research community and / or the general public. 
• Safeguarding risks:   Risk to young people, vulnerable adults and / or researcher from improper behaviour, abuse or exploitation. Risk to researcher of being in a comprising situation, in which there might be accusations of improper behaviour.
• Health and safety risks: risks of harm to health, physical injury or psychological harm to participants or the researcher. Further information on health and safety risks is given below.

Health and safety risks

The potential hazards and risks in research can be many and varied. You will need to be competent and familiar with the work or know where to obtain expert advice to ensure you have identified reasonably foreseeable risks. Here are some common research hazards and risks:

• Location hazards Location hazards Location hazards and risks are associated with where the research is carried out. For example: fire; visiting or working in participant’s homes; working in remote locations and in high crime areas; overseas travel; hot, cold or extreme weather conditions; working on or by water. Also hazardous work locations, such as construction sites, confined spaces, roofs or laboratories. For overseas travel, you will need to check country / city specific information, travel health requirements and consider emergency arrangements as part of your research planning, by following the University’s overseas travel  health and safety standard .  
• Activity hazards Activity hazards Activity hazards and risks associated with the tasks carried out. For example: potentially mentally harmful activities; distressing and stressful work and content; driving; tripping, or slipping; falling from height; physically demanding work; lifting, carrying, pushing and pulling loads; night time and weekend working.
• Machinery and equipment Machinery and equipment Machinery and equipment . For example: ergonomic hazards, including computer workstations and equipment; contact with electricity; contact with moving, rotating, ejecting or cutting parts in machinery and instruments; accidental release of energy from machines and instruments.
• Chemicals and other hazardous substances . The use, production, storage, waste, transportation and accidental release of chemicals and hazardous substances; flammable, dangerous and explosive substances; asphyxiating gases; allergens; biological agents, blood and blood products. You’ll need to gather information about the amount, frequency and duration of exposure and carry out a COSHH or DSEAR assessment which will inform whether you may need health surveillance for yourself and / or your research participants.
• Physical agents Physical agents Physical agents . For example: excessive noise exposure, hand-arm vibration and whole body vibration; ionising radiation; lasers; artificial optical radiation and electromagnetic fields. You’ll need to gather information about the amount, frequency and duration of exposure inform whether you may need health surveillance for yourself and / or your research participants.

When to carry out a risk assessment

Carrying out initial risk assessments as part of the planning process will help you identify whether existing resources and facilities are adequate to ensure risk control, or if the project needs to be altered accordingly. It will also help you to identify potential costs that need to be considered as part of the funding bid.

Once the project is approved, research specific risk assessments need to be carried out before work starts.

The research may need ethical approval if there is significant risk to participants, researchers or the University.

How to carry out a risk assessment

The University standard on risk assessments provides guidance, tips on getting it right, as well as resources and the forms to help you produce suitable and sufficient risk assessments and must be used.

• Risk assessment template (.dotx)
• Flow chart to research risk assessment (.pdf)
• Research risk assessment: Risk estimation tool (.pdf)
• Example of a Social Science research risk assessment (.pdf)

Refer to carrying out a risk assessment carrying out a risk assessment carrying out a risk assessment for step by step guidance.

Risk assessments must relate to the actual work and must be monitored by the PI. If there are significant changes to the activities, locations, equipment or substances used, the risk assessment will need to reviewed, updated and the old version archived. Risk assessments should also consider the end of projects, arrangements for waste disposal, equipment, controlled area decommission and emergencies. 

Things to consider:

• The risks may be specialist in nature or general. Information can found from legislation, sector guidance, safety data sheets, manufacturers equipment information, research documents, forums and health and safety professionals.
• Practical research might involve less well-known hazards. Do you or your team have the expertise to assess the risk adequately? Do you know who to go to for expert advice?
• The capabilities, training, knowledge, skills and experience of the project team members. Are they competent or are there gaps?
• In fast changing research environments, is there a need to carry out dynamic risk assessments? Are they understood and recorded?
• The right personal protective equipment for the hazards identified and training in how to use it.
• Specific Occupational Health vaccinations, health surveillance and screening requirements identified and undertaken. With physical agents and substances you’ll need to make an informed decision about the amount, frequency and duration of exposure. If you need help with this contact HSAS.
• Associated activities: storage, transport/travel, cleaning, maintenance, foreseeable emergencies (eg spillages), decommissioning and disposal.
• The safe design, testing and maintenance of the facilities and equipment.
• Planned and preventative maintenance of general plant and specialist equipment.

These risk assessments relate to the actual work and must be monitored by the PI. If there are significant changes to the activities, locations, equipment or substances used, the risk assessment will need to reviewed, updated and the old version archived. Risk assessments should also consider the end of projects, arrangements for waste disposal, equipment and controlled area decommission and emergencies.

Training 

If you would like training on risk assessment, please book onto one of our courses:

• Research Risk Assessment (for research staff and students in Humanities and Social Sciences)
• Research Risk Assessment (for research staff and students in Faculty of Science of Health only)
• Carrying out a risk assessment Carrying out a risk assessment Carrying out a risk assessment
• People especially at risk People especially at risk People especially at risk
• IOSH/USHA/UCEA guidance on managing health and safety in research (.pdf) 
• Research governance: Ethical approval

• For enquiries contact your Student Services Hub
• University of Essex
• Wivenhoe Park
• Colchester CO4 3SQ
• Accessibility
• Privacy and Cookie Policy

• Open access
• Published: 09 December 2020

Data of risk analysis management in university campuses

• Alireza Dehdashti 1 ,
• Farin Fatemi 1 ,
• Muhammadreza Janati 2 ,
• Fatemeh Asadi 2 &
• Marzieh Belji Kangarloo 2  

BMC Research Notes volume  13 , Article number:  554 ( 2020 ) Cite this article

3555 Accesses

1 Citations

Metrics details

This data paper aims to provide the data set of a practical method to health, safety, and environmental risk assessment to assess and rank potential threats/hazards and to prevent and decrease the accidents and harmful consequences at an academic setting. Descriptive data on type of hazards, places, and persons at risk were collected. Quantitative data on risk probability and severity of identified hazards were determined. Additionally, the descriptive statistics and analytical tests were applied to create a concise perspective on health, safety and environmental hazards/threats situation in research location under study. The dataset further provides information on the prioritization of determined risks according to the relevant scores and levels for doing the relevant control measures to remove and mitigate the related risks.

Data description

This paper provides data of comprehensive risk assessment of health, safety and, environmental hazards of academic setting. For each identified hazard, the descriptive and numeric data are available. The information about the risk level and prevention or mitigation measure related to each hazard is provided. Additionally, the statistical tests are applied for determining the relations among the variables under study. The data and methodology on risk assessment in this article may be used to manage variety of risks in higher education institutions.

Risk assessment is the process of evaluating risks to persons’ safety and health from workplace hazards [ 1 ]. This data note aims to provide comprehensive information about the Health, Safety and Environment (HSE) risk assessment. Such these data sets are necessary for accident prevention and decrease the harmful impacts such as death, injuries and damages to structures and equipment at academic settings [ 2 , 3 ]. Also, such these risk assessment results promote information sharing across the university systems about best practices to mitigate the vulnerability of hazards [ 4 ]. Corresponding mitigation measures for each campus’s highest risks is beneficial to create a University-wide relative risk ranking of all threat events and to summarize the status of campus mitigation measures [ 5 ]. Taking steps to either eliminate or to reduce risks (as far as reasonably practicable) by introducing control measures should be done in the risk management phase [ 6 ]. To achieve these goals, we developed two checklists to collect descriptive and quantitative data based on the evidence review [ 7 , 8 ]. The laboratories, public offices, and peripheral areas of the academic setting were included in this study. Besides, we planned to determine the hazard mitigation strategies for each identified hazard in the assessed areas of the academic setting in the applied checklist [ 9 ]. In the current paper, we described the collected data during the HSE risk assessment in this study. A research paper was written up and published based on our data and methodology to further describe and prioritize risks in terms of varying risk impacts in the university environment [ 10 ]. These data may help researchers to assess probability, impact, and mitigation risks in detail and find a proactive approach toward risks in higher education institutions.

This cross-sectional study was conducted from June to July 2018 at the School of Public Health, Semnan, University of Medical Sciences, Semnan, Iran. For implementing a successful HSE risk assessment, we needed to identify the potential hazards into three individual domains of health, safety, and environment in different areas of the academic setting. First, a list of hazards in three sections of health, safety, and environment was developed in a checklist. The reviewed literature on concerning recent accidents or incidents within universities was much helpful in defining the list of HSE hazards. The hazards checklist was completed via on-site walking and interviews by three trained students in the 39 locations of the academic setting. Additionally, the persons at risk including students, staff, or faculties were determined for each hazard in the assessed location. The descriptive analysis indicated that the most frequency of hazards was related to health hazards (50.3%). The safety and environmental hazards were 44%, and 5.7%, respectively.

Second, the research team analyzed the risks based on the risk matrix in ISO 31000 [ 8 ] (see Table  1 ). Risk analysis was done by estimating the probability and severity of risks for identified hazards. The probability of occurrence metrics was measured on a five-point scale from “not applicable” to “inevitable”. The interpretation of each scale has been mentioned in data file 1 [ 11 ]. The novelty of this study is determining the severity rate in terms of human, equipment, and institution. To estimate the effect of each identified hazard on three mentioned terms, we applied two items with a five-point score response [ 12 ]. Responses were scored and averaged to obtain an overall severity score. The severity metrics were measured on a five-point scale from “very low” to “very high”. The final risk scores were calculated from multiplying probability by severity and categorized into 3 risk levels, risks rated in the top category (red) are larger than those rated in moderate category (yellow) and low category (green).

Based on the results, hazards with high risk level were belonged to safety category that they were included 5.7%, required immediate mitigation measures. From all identified health, safety and environmental hazards, 6.5%, 36% and 43% were categorized in moderate risk level, respectively. These hazards need to be corrected in the near future. The rest of identified HSE hazards had acceptable risk score and categorized in low risk hazards. Therefore, the mitigation measures were recommended in appropriate to determined risk levels for identified hazards.

In data file 2, in order to create a more detailed view on health, safety and environmental hazards/threats situation, the analytical tests were applied to examine associations among the main variables under study [ 13 ]. Type of analytical tests (Chi square or Kruskal–Wallis)was used based on type of variable that is qualitative or quantitative. The most important finding is related to significant relation between the calculated risk scores and the type of hazards. The safety-related hazards indicated a statistically higher contribution to the total risk score when compared to health and environmental hazards. The descriptive analysis and providing analytical information helps to withstand and cope with the adverse effects of accidents and emergencies [ 14 , 15 ].

Limitations

Although risk matrices can indeed be very useful in risk analysis, but can mistakenly assign higher qualitative ratings to smaller risks that leading to wrong risk management decision. It means that we might have quantitatively overestimated or underestimated the risk scores for identified hazards. Probably, applying new approaches to risk matrix such as new scaling and scoring methods to risk matrix extensions depends on the location of risk assessment in future studies would minimize the mentioned concern.

The data of this study were collected in a typical higher education institution with its exclusive hazards, which may limit its interpretation of safety, health, and environmental risks. However, the authors believe that an integrated approach for obtaining data will be beneficial to compare various domains of hazards in a workplace.

Availability of data and materials

The data described in this Data note can be freely and openly accessed via http://figshare.com . Please see Table  1 and Refs. [ 11 , 13 ] for details and links to the data.

Abbreviations

Health, safety, environment

Sarker MN, Yang B, Lv Y, Huq ME, Kamruzzaman MM. Climate change adaptation and resilience through big data. Int J Adv Comput Sci Appl. 2020;11:533–9. https://doi.org/10.14569/IJACSA.2020.0110368 .

Article   Google Scholar  

Farrokhi M, Khankeh H, Poursadeqiyan M. The necessity to establish health, safety and environment management major at the University of Social Welfare and Rehabilitation Sciences. Health Emerg Disasters Quarterly. 2019;4(3):119–26. http://hdq.uswr.ac.ir/article-1-205-en.html .

Salazar-Escoboza MA, Laborin-Alvarez JF, Alvarez-Chavez CR, Noriega-Orozco L, Borbon-Morales C. Safety climate perceived by users of academic laboratories in higher education institutes. Saf Sci. 2020;121:93–9. http://www.sciencedirect.com/science/article/pii/S0925753519306198 .

Skurka C, Quick BL, Reynolds-Tylus T, Short T, Bryan AL. An evaluation of a college campus emergency preparedness intervention. J Saf Res. 2018;65(67–72):000434745100008.

Google Scholar  

Yang N, Wu YM. University public emergency plan formulation: Current situation and countermeasures. Jun H, Hong C, Jianbin J, editors 2010. 40–3 p.000289654800010.

Marincioni F, Fraboni R. A baseline assessment of emergency planning and preparedness in Italian universities. Disasters. 2012;36(2):291–315. https://doi.org/10.1111/j.1467-7717.2011.01259.x .

Article   PubMed   Google Scholar  

Example risk assessments United Kingdom: Aberystwyth University; 2018. https://www.aber.ac.uk/en/hse/proc-prac/risk-assessment/#template-and-guidance .

Risk management framework. Second ed: Goverment of South Australia; 2012. https://dhs.sa.gov.au/data/assets/pdf_file/0008/9782/risk-management-framework.pdf .

Ilo PI, Izuagbe R, Mole AJC, Ekwueme L. Measuring disaster preparedness and response practices in university libraries in Nigeria: the role of disaster equipment. Int J Disaster Risk Reduct. 2018;31:85–91.

Dehdashti A, Fatemi F, Jannati M, Asadi F, Kangarloo MB. Applying health, safety, and environmental risk assessment at academic settings. BMC Public Health; 2020. https://bmcpublichealth.biomedcentral.com/articles/10.1186/s12889-020-09419-5 .

Fatemi F. Risk assessment data file. figshare. Dataset. 2020. https://doi.org/10.6084/m9figshare.13106825.v1 .

Nouri J, Mansouri N, Abbaspour M, Karbasi AR, Omidvari M. Designing a developed model for assessing the disaster induced vulnerability value in educational centers. Saf Sci. 2011;49(5):679–85. https://doi.org/10.1016/j.ssci.2011.01.002 .

Fatemi F. Analytical data. figshare. Dataset. 2020. https://doi.org/10.6084/m9.figshare.13106843.v1 .

Sarker MNI, Peng Y, Yiran C, Shouse RC. Disaster resilience through big data: Way to environmental sustainability. Int J Disaster Risk Reduct. 2020;51 March:101769. https://doi.org/10.1016/j.ijdrr.2020.101769 .

Tajdinan S, Afshari D. Checking of human errors in Ancoiler Device Control Room of Ahvaz Pipe Mill using SHERPA and HET methods at year 1390. Iran Occup Health. 2013;10(3). http://ioh.iums.ac.ir/article-1-771-en.html .

Download references

Acknowledgements

We thank all colleagues and staff of the School of Public Health who supported and assisted for conducting this research.

There is no funding source for this research.

Author information

Authors and affiliations.

Research Center for Health Sciences and Technologies, Semnan University of Medical Sciences, Semnan, Iran

Alireza Dehdashti & Farin Fatemi

Student Research Committee, Semnan University of Medical Siences, Semnan, Iran

Muhammadreza Janati, Fatemeh Asadi & Marzieh Belji Kangarloo

You can also search for this author in PubMed   Google Scholar

Contributions

MJ, FA and MB collected data and contributed to entering data into dataset. FF and AD designed study, analyzed data, and prepared the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Farin Fatemi .

Ethics declarations

Ethics approval and consent to participate.

This study was approved by the Ethics Committee Review Board at Semnan university of Medical Sciences (IR.SEMUMS.REC.1398.131). All the participants signed a consent form and were informed on the purpose of the study prior to interview as per local protocol on research ethics.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Dehdashti, A., Fatemi, F., Janati, M. et al. Data of risk analysis management in university campuses. BMC Res Notes 13 , 554 (2020). https://doi.org/10.1186/s13104-020-05397-4

Download citation

Received : 17 October 2020

Accepted : 24 November 2020

Published : 09 December 2020

DOI : https://doi.org/10.1186/s13104-020-05397-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Risk assessment
• Academic settings

BMC Research Notes

ISSN: 1756-0500

• Submission enquiries: [email protected]
• General enquiries: [email protected]

• Global directory Global directory
• Product logins Product logins
• Contact us Contact us

Our Privacy Statement & Cookie Policy

All Thomson Reuters websites use cookies to improve your online experience. They were placed on your computer when you launched this website. You can change your cookie settings through your browser.

• Privacy Statement
• Cookie Policy

Join our community

Sign up for industry-leading insights, updates, and all things AI @ Thomson Reuters.

The growing importance of adverse media searches

False positives and false negatives: How best to leverage adverse media searches in the battle against financial crime.

Charting a path forward with AI adoption in compliance: Reducing uncertainty and embracing change

Industry experts discuss the best examples of AI being used for compliance, the key risks, what this means for compliance analysts, and how leaders can make themselves and regulators comfortable with putting AI to use.

Mitigate risk, detect fraudulent activity, and streamline investigations

In today’s digital world, risk and fraud detection is even more important than ever before

Related posts

How businesses can responsibly use AI and address ethical and security challenges

Fraud investigation: An overview

Fraud detection: An overview

More answers.

Driving change and becoming a data-informed legal team

5 compelling reasons to adore your external lawyers

Quid pro quo social engineering [infographic and explanation]

This paper is in the following e-collection/theme issue:

Published on 29.4.2024 in Vol 26 (2024)

The Applications of Artificial Intelligence for Assessing Fall Risk: Systematic Review

Authors of this article:

• Ana González-Castro 1 , PT, MSc   ; 
• Raquel Leirós-Rodríguez 2 , PT, PhD   ; 
• Camino Prada-García 3 , MD, PhD   ; 
• José Alberto Benítez-Andrades 4 , PhD  

1 Nursing and Physical Therapy Department, Universidad de León, Ponferrada, Spain

2 SALBIS Research Group, Nursing and Physical Therapy Department, Universidad de León, Ponferrada, Spain

3 Department of Preventive Medicine and Public Health, Universidad de Valladolid, Valladolid, Spain

4 SALBIS Research Group, Department of Electric, Systems and Automatics Engineering, Universidad de León, León, Spain

Corresponding Author:

Ana González-Castro, PT, MSc

Nursing and Physical Therapy Department

Universidad de León

Astorga Ave

Ponferrada, 24401

Phone: 34 987442000

Email: [email protected]

Background: Falls and their consequences are a serious public health problem worldwide. Each year, 37.3 million falls requiring medical attention occur. Therefore, the analysis of fall risk is of great importance for prevention. Artificial intelligence (AI) represents an innovative tool for creating predictive statistical models of fall risk through data analysis.

Objective: The aim of this review was to analyze the available evidence on the applications of AI in the analysis of data related to postural control and fall risk.

Methods: A literature search was conducted in 6 databases with the following inclusion criteria: the articles had to be published within the last 5 years (from 2018 to 2024), they had to apply some method of AI, AI analyses had to be applied to data from samples consisting of humans, and the analyzed sample had to consist of individuals with independent walking with or without the assistance of external orthopedic devices.

Results: We obtained a total of 3858 articles, of which 22 were finally selected. Data extraction for subsequent analysis varied in the different studies: 82% (18/22) of them extracted data through tests or functional assessments, and the remaining 18% (4/22) of them extracted through existing medical records. Different AI techniques were used throughout the articles. All the research included in the review obtained accuracy values of >70% in the predictive models obtained through AI.

Conclusions: The use of AI proves to be a valuable tool for creating predictive models of fall risk. The use of this tool could have a significant socioeconomic impact as it enables the development of low-cost predictive models with a high level of accuracy.

Trial Registration: PROSPERO CRD42023443277; https://tinyurl.com/4sb72ssv

Introduction

According to alarming figures reported by the World Health Organization in 2021, falls cause 37.3 million injuries annually that require medical attention and result in 684,000 deaths [ 1 ]. These figures indicate a significant impact of falls on the health care system and on society, both directly and indirectly [ 2 , 3 ].

Life expectancy has progressively increased over the years, leading to an aging population [ 4 ]. By 2050, it is estimated that 16% of the population will be >65 years of age. In this group, the incidence of falls has steadily risen, becoming the leading cause of accidental injury and death (accounting for 55.8% of such deaths, according to some research) [ 5 , 6 ]. It is estimated that 30% of this population falls at least once a year, negatively impacting their physical and psychological well-being [ 7 , 8 ].

Physically, falls are often associated with severe complications that can lead to extended hospitalizations [ 9 ]. These hospitalizations are usually due to serious injuries, often cranioencephalic trauma, fractures, or soft tissue injuries [ 10 , 11 ]. Psychologically, falls among the older adult population tend to result in self-imposed limitations due to the fear of falling again [ 10 , 12 ]. These limitations lead to social isolation as individuals avoid participating in activities or even individual mobility [ 13 ]. Consequently, falls can lead to psychological conditions such as anxiety and depression [ 14 , 15 ]. Numerous research studies on the risk of falls are currently underway, with ongoing investigations into various innovations and intervention ideas [ 16 - 19 ]. These studies encompass the identification of fall risk factors [ 20 , 21 ], strategies for prevention [ 22 , 23 ], and the outcomes following rehabilitation [ 23 , 24 ].

In the health care field, artificial intelligence (AI) is characterized by data management and processing, offering new possibilities to the health care paradigm [ 24 ]. Some applications of AI in the health care domain include assessing tumor interaction processes [ 25 ], serving as a tool for image-based diagnostics [ 26 , 27 ], participating in virus detection [ 28 ], and, most importantly, as a statistical and predictive method [ 29 - 32 ].

Several publications have combined AI techniques to address health care issues [ 33 - 35 ]. Within the field of predictive models, it is important to understand certain differentiations. In AI, we have machine learning and deep learning [ 36 - 38 ]. Machine learning encompasses a set of techniques applied to data and can be done in a supervised or unsupervised manner [ 39 , 40 ]. On the other hand, deep learning is typically used to work with larger data sets compared to machine learning, and its computational cost is higher [ 41 , 42 ].

Some examples of AI techniques include the gradient boosting machine [ 43 ], learning method, and the long short-term memory (LSTM) [ 44 ] and the convolutional neural network (CNN) [ 45 ], all of them are deep learning methods.

For all the reasons mentioned in the preceding section, it was considered necessary to conduct a systematic review to analyze the scientific evidence of AI applications in the analysis of data related to postural control and the risk of falls.

Data Sources and Searches

This systematic review and meta-analysis were prospectively registered on PROSPERO (ID CRD42023443277) and followed the Meta-Analyses of Observational Studies in Epidemiology checklist [ 46 ] and the recommendations of the Cochrane Collaboration [ 47 ].

The search was conducted in January 2024 on the following databases: PubMed, Scopus, ScienceDirect, Web of Science, CINAHL, and Cochrane Library. The Medical Subject Headings (MeSH) terms used for the search included machine learning , artificial intelligent , accidental falls , rehabilitation , and physical therapy specialty . The terms “predictive model” and “algorithms” were also used. These terms were combined using the Boolean operators AND and OR ( Textbox 1 ).

• (“machine learning”[MeSH Terms] OR “artificial intelligent”[MeSH Terms]) AND “accidental falls”[MeSH Terms]
• (“machine learning”[MeSH Terms] OR “artificial intelligent”) AND (“rehabilitation”[MeSH Terms] OR “physical therapy specialty”[MeSH Terms])
• “accidental falls” [Title/Abstract] AND “algorithms” [Title/Abstract]
• “accidental falls”[Title/Abstract] AND “predictive model” [Title/Abstract]
• TITLE-ABS-KEY (“machine learning” OR “artificial intelligent”) AND TITLE-ABS-KEY (“accidental falls”)
• TITLE-ABS-KEY (“machine learning” OR “artificial intelligent”) AND TITLE-ABS-KEY (“rehabilitation” OR “physical therapy specialty”)
• TITLE-ABS-KEY (“accidental falls” AND “algorithms”)
• TITLE-ABS-KEY (“accidental falls” AND “predictive model”)

ScienceDirect

• Title, abstract, keywords: (“machine learning” OR “artificial intelligent”) AND “accidental falls”
• Title, abstract, keywords: (“machine learning” OR “artificial intelligent”) AND (“rehabilitation” OR “physical therapy specialty”)
• Title, abstract, keywords: (“accidental falls” AND “algorithms”)
• Title, abstract, keywords: (“accidental falls” AND “predictive model”)

Web of Science

• TS=(“machine learning” OR “artificial intelligent”) AND TS=“accidental falls”
• TS=(“machine learning” OR “artificial intelligent”) AND TS= (“rehabilitation” OR “physical therapy specialty”)
• AB= (“accidental falls” AND “algorithms”)
• AB= (“accidental falls” AND “predictive model”)
• (MH “machine learning” OR MH “artificial intelligent”) AND MH “accidental falls”
• (MH “machine learning” OR MH “artificial intelligent”) AND (MH “rehabilitation” OR MH “physical therapy specialty”)
• (AB “accidental falls”) AND (AB “algorithms”)
• (AB “accidental falls”) AND (AB “predictive model”)

Cochrane Library

• (“machine learning” OR “artificial intelligent”) in Title Abstract Keyword AND “accidental falls” in Title Abstract Keyword
• (“machine learning” OR “artificial intelligent”) in Title Abstract Keyword AND (“rehabilitation” OR “physical therapy specialty”) in Title Abstract Keyword
• “accidental falls” in Title Abstract Keyword AND “algorithms” in Title Abstract Keyword
• “accidental falls” in Title Abstract Keyword AND “predictive model” in Title Abstract Keyword

Study Selection

After removing duplicates, 2 reviewers (AGC and RLR) independently screened articles for eligibility. In the case of disagreement, a third reviewer (JABA) finally decided whether the study should be included or not. We calculated the κ coefficient and percentage agreement scores to assess reliability before any consensus and estimated the interrater reliability using κ. Interrater reliability was estimated using κ>0.7 indicating a high level of agreement between the reviewers, κ of 0.5 to 0.7 indicating a moderate level of agreement, and κ<0.5 indicating a low level of agreement [ 48 ].

For the selection of results, the inclusion criteria were established as follows: (1) articles should have been published in the last 5 years (from 2018 to the present); (2) they must apply some AI method; (3) AI analyses should be applied to data from samples of humans; and (4) the sample analyzed should consist of people with independent walking, with or without the use of external orthopedic devices.

After screening the data, extracting, obtaining, and screening the titles and abstracts for inclusion criteria, the selected abstracts were obtained in full texts. Titles and abstracts lacking sufficient information regarding inclusion criteria were also obtained as full texts. Full-text articles were selected in case of compliance with inclusion criteria by the 2 reviewers using a data extraction form.

Data Extraction and Quality Assessment

The 2 reviewers mentioned independently extracting data from the included studies using a customized data extraction table in Excel (Microsoft Corporation). In case of disagreement, both reviewers debated until an agreement was reached.

The data extracted from the included articles for further analysis were: demographic information (title, authors, journal, and year), characteristics of the sample (age, inclusion and exclusion criteria, and number of participants), study-specific parameters (study type, AI techniques applied, and data analyzed), and the results obtained. Tables were used to describe both the studies’ characteristics and the extracted data.

Assessment of Risk of Bias

The methodological quality of the selected articles was evaluated using the Critical Review Form for Quantitative Studies [ 49 ]. The ROBINS-E (Risk of Bias in Nonrandomized Studies of Exposures) tool was used to evaluate the risk of bias [ 50 ].

Characteristics of the Selected Studies

A total of 3858 articles were initially retrieved, with 1563 duplicates removed. From the remaining 2295 articles, 2271 were excluded based on the initial selection criteria, leaving 24 articles for the subsequent analysis. In this second analysis, 2 articles were removed as they were systematic reviews, and 22 articles were finally selected [ 51 - 72 ] ( Figure 1 ). After the first reading of all candidate full texts, the kappa score for inclusion of the results of reviewers 1 and 2 was 0.98, indicating a very high level of agreement.

The methodological quality of the 22 analyzed studies (Table S1 in Multimedia Appendix 1 [ 51 , 52 , 54 , 56 , 58 , 59 , 61 , 63 , 64 , 69 , 70 , 72 ]) ranged from 11 points in 2 (9.1%) studies [ 52 , 65 ] to 16 points in 7 (32%) studies [ 53 , 54 , 56 , 63 , 69 - 71 ].

Study Characteristics and Risk of Bias

All the selected articles were cross-sectional observational studies ( Table 1 ).

In total, 34 characteristics affecting the risk of falls were extracted and classified into high fall-risk and low fall-risk groups with the largest sample sizes significantly differing from the rest. Studies based on data collected from various health care systems had larger sample sizes, ranging from 22,515 to 265,225 participants [ 60 , 65 , 67 ]. In contrast, studies that applied some form of evaluation test had sample sizes ranging from 8 participants [ 56 ] to 746 participants [ 55 ].

It is worth noting the various studies conducted by Dubois et al [ 54 , 72 ], whose publications on fall risk and machine learning started in 2018 and progressed until 2021. A total of 9.1% (2/22) of the articles by this author were included in the final selection [ 54 , 72 ]. Both articles used samples with the same characteristics, even though the first one was composed of 43 participants [ 54 ] and the last one had 30 participants [ 72 ]. All 86.4% (19/22) of the articles used samples of individuals aged ≥65 years [ 51 - 60 , 62 - 65 , 68 - 72 ]. In the remaining 13.6% (3/22) of the articles, the ages ranged between 16 and 62 years [ 61 , 66 , 67 ].

Althobaiti et al [ 61 ] used a sample of participants between the ages of 19 and 35 years for their research, where these participants had to reproduce examples of falls for subsequent analysis. In 2022, Ladios-Martin et al [ 67 ] extracted medical data from participants aged >16 years for their research. Finally, in 2023, the study by Maray et al [ 66 ] used 3 types of samples, with ages ranging from 21 to 62 years. Among the 22 selected articles, only 1 (4.5%) of them did not describe the characteristics of its sample [ 52 ].

Finally, regarding the sex of the samples, 13.6% (3/22) of the articles specified in the characteristics of their samples that only female individuals were included among their participants [ 53 , 59 , 70 ].

a AI: artificial intelligence.

b ML: machine learning.

c nd: none described.

d ADL: activities of daily living.

e TUG: Timed Up and Go.

f BBS: Berg Balance Scale.

g ASM: associative skill memories.

h CNN: convolutional neural network.

i FP: fall prevention.

j IMU: inertial measurement unit.

k AUROC: area under the receiver operating characteristic curve.

l AUPR: area under the precision-recall curve.

m MFS: Morse Fall Scale.

n XGB: extreme gradient boosting.

o MCT: motor control test.

p GBM: gradient boosting machine.

q RF: random forest.

r LOOCV: leave-one-out cross-validation.

s LSTM: long short-term memory.

Applied Assessment Procedures

All articles initially analyzed the characteristics of their samples to subsequently create a predictive model of the risk of falls. However, they did not all follow the same evaluation process.

Regarding the applied assessment procedures, 3 main options stood out: studies with tests or assessments accompanied by sensors or accelerometers [ 51 - 57 , 59 , 61 - 63 , 66 , 70 - 72 ], studies with tests or assessments accompanied by cameras [ 68 , 69 ], or studies based on medical records [ 58 , 60 , 65 , 67 ] ( Figure 2 ). Guillan et al [ 64 ] performed a physical and functional evaluation of the participants. In their study, they evaluated parameters such as walking speed, stride frequency and length, and the minimum space between the toes. Afterward, they asked them to record the fall events they had during the past 2 years in a personal diary.

In total, 22.7% (5/22) of the studies used the Timed Up and Go test [ 53 , 54 , 69 , 71 , 72 ]. In 18.2% (4/22) of them, the participants performed the test while wearing a sensor to collect data [ 53 , 54 , 71 , 72 ]. In 1 (4.5%) study, the test was recorded with a camera for later analysis [ 69 ]. Another commonly used method in studies was to ask participants to perform everyday tasks or activities of daily living while a sensor collected data for subsequent analysis. Specifically, 18.2% (4/22) of the studies used this method to gather data [ 51 , 56 , 61 , 62 ].

A total of 22.7% (5/22) of the studies asked participants to simulate falls and nonfalls while a sensor collected data [ 52 , 61 - 63 , 66 ]. In this way, the data obtained were used to create the predictive model of falls. As for the tests used, Eichler et al [ 68 ] asked participants to perform the Berg Balance Scale while a camera recorded their performance.

Finally, other authors created their own battery of tests for data extraction [ 55 , 59 , 64 , 70 ]. Gillain et al [ 64 ] used gait records to analyze speed, stride length, frequency, symmetry, regularity, and foot separation. Hu et al [ 59 ] asked their participants to perform normal walking, the postural reflexive response test, and the motor control test. In the study by Noh et al [ 55 ], gait tests were conducted, involving walking 20 m at different speeds. Finally, Greene et al [ 70 ] created a 12-question questionnaire and asked their participants to maintain balance while holding a mobile phone in their hand.

AI Techniques

The selected articles used various techniques within AI. They all had the same objective in applying these techniques, which was to achieve a predictive and classification model for the risk of falls [ 51 - 72 ].

In chronological order, in 2018, Nait Aicha et al [ 51 ] compared single-task learning models with multitask learning, obtaining better evaluation results through multitask learning. In the same year, Dubois et al [ 54 ] applied AI techniques that analyzed multiple parameters to classify the risk of falls in their sample. Qiu et al [ 53 ], also in the same year, used 6 machine learning models (logistic regression, naïve Bayes, decision tree, RF, boosted tree, and support vector machine) in their research.

In contrast, in 2019, Ferrete et al [ 52 ] compared the applicability of 2 different deep learning models: the classifier based on associative skill memories and a CNN classifier. In the same year, after confirming the applicability of AI as a predictive method for the risk of falls, various authors investigated through methods such as the RF to identify factors that can predict and quantify the risk of falls [ 63 , 65 ].

Among the selected articles, 5 (22.7%) were published in 2020 [ 58 - 62 ]. The research conducted by Tunca et al [ 62 ] compared the applicability of deep learning LSTM networks with traditional machine learning applied to the risk of falls. Hu et al [ 59 ] first used cross-validation, where algorithms were trained randomly, and then used the gradient boosting machine algorithm to classify participants as high or low risk. Ye et al [ 60 ] and Hsu et al [ 58 ] both used the extreme gradient boosting (XGBoost) algorithm based on machine learning to create their predictive model. In the same year, Althobaiti et al [ 61 ] trained machine learning models for their research.

In 2021, Lockhart et al [ 57 ] started using 3 deep learning techniques simultaneously with the same goal as before: to create a predictive model for the risk of falls. Specifically, they used the RF, RF with feature engineering, and RF with feature engineering and linear and nonlinear variables. Noh et al [ 55 ], again in the same year, used the XGBoost algorithm, while Roshdibenam et al [ 71 ], on the other hand, used the CNN algorithm for each location of the wearable sensors used in their research. Various machine learning techniques were used for classifying the risk of falls and for balance loss events in the research by Hauth et al [ 56 ]. Dubois et al [ 72 ] used the following algorithms: decision tree, adaptive boosting, neural net, naïve Bayes, k-nearest neighbors, linear support vector machine, radial basis function support vector machine, RF, and quadratic discriminant analysis. Hauth et al [ 56 ], on the other hand, used regularized logistic regression and bidirectional LSTM networks. In the research conducted by Greene et al [ 70 ], AI was used, but the specific procedure that they followed is not described.

Tang et al [ 69 ] published their research with innovation up to that point. In their study, they used a smart gait analyzer with the help of deep learning techniques to assess the diagnostic accuracy of fall risk through vision. Months later, in August 2022, Ladios-Martin et al [ 67 ] published their research, in which they compared 2 deep learning models to achieve the best results in terms of specificity and sensitivity in detecting fall risk. The first model used the Bayesian Point Machine algorithm with a fall prevention variable, and the second one did not use the variable. They obtained better results when using that variable, a mitigating factor defined as a set of care interventions carried out by professionals to prevent the patient from experiencing a fall during hospitalization. Particularly controversial, as its exclusion could obscure the model’s performance. Eichler et al [ 68 ], on the other hand, used machine learning–based classifier training and later tested the performance of RFs in score predictions.

Finally, in January 2023, Maray et al [ 66 ] published their research, linking the previously mentioned terms (AI and fall risk) with 3 wearable devices that are commonly used today. They collected data through these devices and applied transfer learning to generalize the model across heterogeneous devices.

The results of the 22 articles provided promising data, and all of them agreed on the feasibility of applying various AI techniques as a method for predicting and classifying the risk of falls. Specifically, the accuracy values obtained in the studies exceed 70%. Noh et al [ 55 ] achieved the “lowest” accuracy among the studies conducted, with a 70% accuracy rate. Ribeiro et al [ 52 ] obtained an accuracy of 92.7% when using CNN to differentiate between normal gait and fall events. Hsu et al [ 58 ] further demonstrated that the XGBoost model is more sensitive than the Morse Fall Scale. Similarly, in their comparative study, Nait Aicha et al [ 51 ] also showed that a predictive model created from accelerometer data with AI is comparable to conventional models for assessing the risk of falls. More specifically, Dubois et al [ 54 ] concluded that using 1 gait-related parameter (excluding velocity) in combination with another parameter related to seated position allowed for the correct classification of individuals according to their risk of falls.

Principal Findings

The aim of this research was to analyze the scientific evidence regarding the applications of AI in the analysis of data related to postural control and the risk of falls. On the basis of the analysis of results, it can be asserted that the following risk factors were identified in the analyzed studies: age [ 65 ], daily habits [ 65 ], clinical diagnoses [ 65 ], environmental and hygiene factors [ 65 ], sex [ 64 ], stride length [ 55 , 72 ], gait speed [ 55 ], and posture [ 55 ]. This aligns with other research that also identifies sex [ 73 , 74 ], age [ 73 ], and gait speed [ 75 ].

On the other hand, the “fear of falling” has been identified in various studies as a risk factor and a predictor of falls [ 73 , 76 ], but it was not identified in any of the studies included in this review.

As for the characteristics of the analyzed samples, only 9.1% (2/22) of the articles used a sample composed exclusively of women [ 53 , 59 ], and no article used a sample composed exclusively of men. This fact is incongruent with reality, as women have a longer life expectancy than men, and therefore, the number of women aged >65 years is greater than the number of men of the same age [ 77 ]. Furthermore, women experience more falls than men [ 78 ]. The connection between menopause and its consequences, including osteopenia, suggests a higher risk of falls among older women than among men of the same age [ 79 , 80 ].

Within the realm of analysis tools, the most frequently used devices to analyze participants were accelerometers [ 51 - 57 , 59 , 61 - 63 , 66 , 70 - 72 ]. However, only 36.4% (8/22) of the studies provided all the information regarding the characteristics of these devices [ 51 , 53 , 59 , 61 , 63 , 66 , 70 , 72 ]. On the other hand, 18.2% (4/22) of the studies used the term “inertial measurement unit” as the sole description of the devices used [ 55 - 57 , 71 ].

The fact that most of the analyzed procedures involved the use of inertial sensors reflects the current widespread use of these devices for postural control analysis. These sensors, in general (and triaxial accelerometers in particular), have demonstrated great diagnostic capacity for balance [ 81 ]. In addition, they exhibit good sensitivity and reliability, combined with their portability and low economic cost [ 82 ]. Another advantage of triaxial accelerometers is their versatility in both adult and pediatric populations [ 83 - 86 ], although the studies included in this review did not address the pediatric population.

The remaining studies extracted data from cameras [ 68 , 69 ], medical records [ 58 , 60 , 65 , 67 ], and other functional and clinical tests [ 59 , 64 , 70 ]. Regarding the AI techniques used, out of the 18.2% (4/22) of articles that used deep learning techniques [ 52 , 57 , 62 , 71 ], only 4.5% (1/22) did not provide a description of the sample characteristics [ 52 ]. In this case, the authors focused on the AI landscape, while the rest of the articles strike a balance between AI and health sciences.

Regarding the validity of the generated models, only 40.9% (9/22) of the articles assessed this characteristic [ 52 , 53 , 55 , 61 - 64 , 68 , 69 ]. The authors of these 9 (N=22, 40.9%) articles evaluated the validity of the models through accuracy. All the results obtained reflected accuracies exceeding 70%, with Ribeiro et al [ 52 ] achieving a notable accuracy of 92.7% and 100%. Specifically, they obtained a 92.7% accuracy through the CNN model for distinguishing normal gait, the prefall condition, and the falling situation, considering the step before the fall, and 100% when not considering it [ 52 ].

The positive results of sensitivity and specificity can only be compared between the studies of Qiu et al [ 53 ] and Gillain et al [ 64 ], as they were the only ones to take them into account, and in both investigations, they were very high. Similarly, in the case of the F 1 -score, only Althobaiti et al [ 61 ] examined this validity measure. This measure is the result of combining precision and recall into a single figure, and the outcome obtained by these researchers was promising.

Despite these differences, the 22 studies obtained promising results in the health care field [ 51 - 72 ]. Specifically, their outcomes highlight the potential of AI integration into clinical settings. However, further research is necessary to explore how health care professionals can effectively use these predictive models. Consequently, future research should focus on studying the application and integration of the already-developed models. In this context, fall prevention plans could be implemented for the target populations identified by the predictive models. This approach would allow for a retrospective analysis to determine if the combination of predictive models with prevention programs effectively reduces the prevalence of falls in the population.

Limitations

Regarding limitations, the articles showed significant variation in the sample sizes selected. Moreover, even in the study with the largest sample size (with 265,225 participants [ 60 ]), the amount of data analyzed was relatively small. In addition, several of the databases used were not generated specifically for the published research but rather derived from existing medical records [ 58 , 60 , 65 , 67 ]. This could explain the significant variability in the variables analyzed across different studies.

Despite the limitations, this research has strengths, such as being the first systematic review on the use of AI as a tool to analyze postural control and the risk of falls. Furthermore, a total of 6 databases were used for the literature search, and a comprehensive article selection process was carried out by 3 researchers. Finally, only cross-sectional observational studies were selected, and they shared the same objective.

Conclusions

The use of AI in the analysis of data related to postural control and the risk of falls proves to be a valuable tool for creating predictive models of fall risk. It has been identified that most AI studies analyze accelerometer data from sensors, with triaxial accelerometers being the most frequently used.

For future research, it would be beneficial to provide more detailed descriptions of the measurement procedures and the AI techniques used. In addition, exploring larger databases could lead to the development of more robust models.

Conflicts of Interest

None declared.

Quality scores of reviewed studies (Critical Review Form for Quantitative Studies tool results).

PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) checklist.

• Step safely: strategies for preventing and managing falls across the life-course. World Health Organization. 2021. URL: https://www.who.int/publications/i/item/978924002191-4 [accessed 2024-04-02]
• Keall MD, Pierse N, Howden-Chapman P, Guria J, Cunningham CW, Baker MG. Cost-benefit analysis of fall injuries prevented by a programme of home modifications: a cluster randomised controlled trial. Inj Prev. Feb 2017;23(1):22-26. [ CrossRef ] [ Medline ]
• Almada M, Brochado P, Portela D, Midão L, Costa E. Prevalence of falls and associated factors among community-dwelling older adults: a cross-sectional study. J Frailty Aging. 2021;10(1):10-16. [ CrossRef ] [ Medline ]
• Menéndez-González L, Izaguirre-Riesgo A, Tranche-Iparraguirre S, Montero-Rodríguez Á, Orts-Cortés MI. [Prevalence and associated factors of frailty in adults over 70 years in the community]. Aten Primaria. Dec 2021;53(10):102128. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Guirguis-Blake JM, Michael YL, Perdue LA, Coppola EL, Beil TL. Interventions to prevent falls in older adults: updated evidence report and systematic review for the US preventive services task force. JAMA. Apr 24, 2018;319(16):1705-1716. [ CrossRef ] [ Medline ]
• Pereira CB, Kanashiro AM. Falls in older adults: a practical approach. Arq Neuropsiquiatr. May 2022;80(5 Suppl 1):313-323. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Byun M, Kim J, Kim M. Physical and psychological factors affecting falls in older patients with arthritis. Int J Environ Res Public Health. Feb 09, 2020;17(3):1098. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Goh HT, Nadarajah M, Hamzah NB, Varadan P, Tan MP. Falls and fear of falling after stroke: a case-control study. PM R. Dec 04, 2016;8(12):1173-1180. [ CrossRef ] [ Medline ]
• Alanazi FK, Lapkin S, Molloy L, Sim J. The impact of safety culture, quality of care, missed care and nurse staffing on patient falls: a multisource association study. J Clin Nurs. Oct 12, 2023;32(19-20):7260-7272. [ CrossRef ] [ Medline ]
• Hossain A, Lall R, Ji C, Bruce J, Underwood M, Lamb SE. Comparison of different statistical models for the analysis of fracture events: findings from the Prevention of Falls Injury Trial (PreFIT). BMC Med Res Methodol. Oct 02, 2023;23(1):216. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Williams CT, Whyman J, Loewenthal J, Chahal K. Managing geriatric patients with falls and fractures. Orthop Clin North Am. Jul 2023;54(3S):e1-12. [ CrossRef ] [ Medline ]
• Gadhvi C, Bean D, Rice D. A systematic review of fear of falling and related constructs after hip fracture: prevalence, measurement, associations with physical function, and interventions. BMC Geriatr. Jun 23, 2023;23(1):385. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Lohman MC, Fallahi A, Mishio Bawa E, Wei J, Merchant AT. Social mediators of the association between depression and falls among older adults. J Aging Health. Aug 12, 2023;35(7-8):593-603. [ CrossRef ] [ Medline ]
• Smith AD, Silva AO, Rodrigues RA, Moreira MA, Nogueira JD, Tura LF. Assessment of risk of falls in elderly living at home. Rev Lat Am Enfermagem. Apr 06, 2017;25:e2754. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Koh V, Matchar DB, Chan A. Physical strength and mental health mediate the association between pain and falls (recurrent and/or injurious) among community-dwelling older adults in Singapore. Arch Gerontol Geriatr. Sep 2023;112:105015. [ CrossRef ] [ Medline ]
• Soh SE, Morgan PE, Hopmans R, Barker AL, Ackerman IN. The feasibility and acceptability of a falls prevention e-learning program for physiotherapists. Physiother Theory Pract. Mar 18, 2023;39(3):631-640. [ CrossRef ] [ Medline ]
• Morat T, Snyders M, Kroeber P, De Luca A, Squeri V, Hochheim M, et al. Evaluation of a novel technology-supported fall prevention intervention - study protocol of a multi-centre randomised controlled trial in older adults at increased risk of falls. BMC Geriatr. Feb 18, 2023;23(1):103. [ FREE Full text ] [ CrossRef ] [ Medline ]
• You T, Koren Y, Butts WJ, Moraes CA, Yeh GY, Wayne PM, et al. Pilot studies of recruitment and feasibility of remote Tai Chi in racially diverse older adults with multisite pain. Contemp Clin Trials. May 2023;128:107164. [ CrossRef ] [ Medline ]
• Aldana-Benítez D, Caicedo-Pareja MJ, Sánchez DP, Ordoñez-Mora LT. Dance as a neurorehabilitation strategy: a systematic review. J Bodyw Mov Ther. Jul 2023;35:348-363. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Jawad A, Baattaiah BA, Alharbi MD, Chevidikunnan MF, Khan F. Factors contributing to falls in people with multiple sclerosis: the exploration of the moderation and mediation effects. Mult Scler Relat Disord. Aug 2023;76:104838. [ CrossRef ] [ Medline ]
• Warren C, Rizo E, Decker E, Hasse A. A comprehensive analysis of risk factors associated with inpatient falls. J Patient Saf. Oct 01, 2023;19(6):396-402. [ CrossRef ] [ Medline ]
• Gross M, Roigk P, Schoene D, Ritter Y, Pauly P, Becker C, et al. Bundesinitiative Sturzprävention. [Update of the recommendations of the federal falls prevention initiative-identification and prevention of the risk of falling in older people living at home]. Z Gerontol Geriatr. Oct 11, 2023;56(6):448-457. [ CrossRef ] [ Medline ]
• Li S, Li Y, Liang Q, Yang WJ, Zi R, Wu X, et al. Effects of tele-exercise rehabilitation intervention on women at high risk of osteoporotic fractures: study protocol for a randomised controlled trial. BMJ Open. Nov 07, 2022;12(11):e064328. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Jiang F, Jiang Y, Zhi H, Dong Y, Li H, Ma S, et al. Artificial intelligence in healthcare: past, present and future. Stroke Vasc Neurol. Dec 2017;2(4):230-243. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Ye Y, Wu X, Wang H, Ye H, Zhao K, Yao S, et al. Artificial intelligence-assisted analysis for tumor-immune interaction within the invasive margin of colorectal cancer. Ann Med. Dec 2023;55(1):2215541. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Kuwahara T, Hara K, Mizuno N, Haba S, Okuno N, Fukui T, et al. Current status of artificial intelligence analysis for the treatment of pancreaticobiliary diseases using endoscopic ultrasonography and endoscopic retrograde cholangiopancreatography. DEN Open. Apr 30, 2024;4(1):e267. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Yokote A, Umeno J, Kawasaki K, Fujioka S, Fuyuno Y, Matsuno Y, et al. Small bowel capsule endoscopy examination and open access database with artificial intelligence: the SEE-artificial intelligence project. DEN Open. Apr 22, 2024;4(1):e258. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Ramalingam M, Jaisankar A, Cheng L, Krishnan S, Lan L, Hassan A, et al. Impact of nanotechnology on conventional and artificial intelligence-based biosensing strategies for the detection of viruses. Discov Nano. Dec 01, 2023;18(1):58. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Yerukala Sathipati S, Tsai MJ, Shukla SK, Ho SY. Artificial intelligence-driven pan-cancer analysis reveals miRNA signatures for cancer stage prediction. HGG Adv. Jul 13, 2023;4(3):100190. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Liu J, Dan W, Liu X, Zhong X, Chen C, He Q, et al. Development and validation of predictive model based on deep learning method for classification of dyslipidemia in Chinese medicine. Health Inf Sci Syst. Dec 06, 2023;11(1):21. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Carou-Senra P, Ong JJ, Castro BM, Seoane-Viaño I, Rodríguez-Pombo L, Cabalar P, et al. Predicting pharmaceutical inkjet printing outcomes using machine learning. Int J Pharm X. Dec 2023;5:100181. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Li X, Zhu Y, Zhao W, Shi R, Wang Z, Pan H, et al. Machine learning algorithm to predict the in-hospital mortality in critically ill patients with chronic kidney disease. Ren Fail. Dec 2023;45(1):2212790. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Bonnin M, Müller-Fouarge F, Estienne T, Bekadar S, Pouchy C, Ait Si Selmi T. Artificial intelligence radiographic analysis tool for total knee arthroplasty. J Arthroplasty. Jul 2023;38(7 Suppl 2):S199-207.e2. [ CrossRef ] [ Medline ]
• Kao DP. Intelligent artificial intelligence: present considerations and future implications of machine learning applied to electrocardiogram interpretation. Circ Cardiovasc Qual Outcomes. Sep 2019;12(9):e006021. [ FREE Full text ] [ CrossRef ] [ Medline ]
• van der Stigchel B, van den Bosch K, van Diggelen J, Haselager P. Intelligent decision support in medical triage: are people robust to biased advice? J Public Health (Oxf). Aug 28, 2023;45(3):689-696. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Jakhar D, Kaur I. Artificial intelligence, machine learning and deep learning: definitions and differences. Clin Exp Dermatol. Jan 09, 2020;45(1):131-132. [ CrossRef ] [ Medline ]
• Ghosh M, Thirugnanam A. Introduction to artificial intelligence. In: Srinivasa KG, Siddesh GM, Sekhar SR, editors. Artificial Intelligence for Information Management: A Healthcare Perspective. Cham, Switzerland. Springer; 2021;88-44.
• Taulli T. Artificial Intelligence Basics: A Non-Technical Introduction. Berkeley, CA. Apress Berkeley; 2019.
• Patil S, Joda T, Soffe B, Awan KH, Fageeh HN, Tovani-Palone MR, et al. Efficacy of artificial intelligence in the detection of periodontal bone loss and classification of periodontal diseases: a systematic review. J Am Dent Assoc. Sep 2023;154(9):795-804.e1. [ CrossRef ] [ Medline ]
• Quek LJ, Heikkonen MR, Lau Y. Use of artificial intelligence techniques for detection of mild cognitive impairment: a systematic scoping review. J Clin Nurs. Sep 10, 2023;32(17-18):5752-5762. [ CrossRef ] [ Medline ]
• Tan D, Mohd Nasir NF, Abdul Manan H, Yahya N. Prediction of toxicity outcomes following radiotherapy using deep learning-based models: a systematic review. Cancer Radiother. Sep 2023;27(5):398-406. [ CrossRef ] [ Medline ]
• Rabilloud N, Allaume P, Acosta O, De Crevoisier R, Bourgade R, Loussouarn D, et al. Deep learning methodologies applied to digital pathology in prostate cancer: a systematic review. Diagnostics (Basel). Aug 14, 2023;13(16):2676. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Li K, Yao S, Zhang Z, Cao B, Wilson C, Kalos D, et al. Efficient gradient boosting for prognostic biomarker discovery. Bioinformatics. Mar 04, 2022;38(6):1631-1638. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Chen T, Chen Y, Li H, Gao T, Tu H, Li S. Driver intent-based intersection autonomous driving collision avoidance reinforcement learning algorithm. Sensors (Basel). Dec 16, 2022;22(24):9943. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Huynh QT, Nguyen PH, Le HX, Ngo LT, Trinh NT, Tran MT, et al. Automatic acne object detection and acne severity grading using smartphone images and artificial intelligence. Diagnostics (Basel). Aug 03, 2022;12(8):1879. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Brooke BS, Schwartz TA, Pawlik TM. MOOSE reporting guidelines for meta-analyses of observational studies. JAMA Surg. Aug 01, 2021;156(8):787-788. [ CrossRef ] [ Medline ]
• Scholten RJ, Clarke M, Hetherington J. The Cochrane collaboration. Eur J Clin Nutr. Aug 28, 2005;59 Suppl 1(S1):S147-S196. [ CrossRef ] [ Medline ]
• Warrens MJ. Kappa coefficients for dichotomous-nominal classifications. Adv Data Anal Classif. Apr 07, 2020;15(1):193-208. [ CrossRef ]
• Law M, Stewart D, Letts L, Pollock N, Bosch J. Guidelines for critical review of qualitative studies. McMaster University Occupational Therapy Evidence-Based Practice Research Group. URL: https://www.canchild.ca/system/tenon/assets/attachments/000/000/360/original/qualguide.pdf [accessed 2024-04-05]
• Higgins JP, Morgan RL, Rooney AA, Taylor KW, Thayer KA, Silva RA, et al. Risk of bias in non-randomized studies - of exposure (ROBINS-E). ROBINS-E tool. URL: https://www.riskofbias.info/welcome/robins-e-tool [accessed 2024-04-02]
• Nait Aicha A, Englebienne G, van Schooten KS, Pijnappels M, Kröse B. Deep learning to predict falls in older adults based on daily-life trunk accelerometry. Sensors (Basel). May 22, 2018;18(5):1654. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Ribeiro NF, André J, Costa L, Santos CP. Development of a strategy to predict and detect falls using wearable sensors. J Med Syst. Apr 04, 2019;43(5):134. [ CrossRef ] [ Medline ]
• Qiu H, Rehman RZ, Yu X, Xiong S. Application of wearable inertial sensors and a new test battery for distinguishing retrospective fallers from non-fallers among community-dwelling older people. Sci Rep. Nov 05, 2018;8(1):16349. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Dubois A, Bihl T, Bresciani JP. Automatic measurement of fall risk indicators in timed up and go test. Inform Health Soc Care. Sep 13, 2019;44(3):237-245. [ CrossRef ] [ Medline ]
• Noh B, Youm C, Goh E, Lee M, Park H, Jeon H, et al. XGBoost based machine learning approach to predict the risk of fall in older adults using gait outcomes. Sci Rep. Jun 09, 2021;11(1):12183. [ CrossRef ] [ Medline ]
• Hauth J, Jabri S, Kamran F, Feleke EW, Nigusie K, Ojeda LV, et al. Automated loss-of-balance event identification in older adults at risk of falls during real-world walking using wearable inertial measurement units. Sensors (Basel). Jul 07, 2021;21(14):4661. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Lockhart TE, Soangra R, Yoon H, Wu T, Frames CW, Weaver R. Prediction of fall risk among community-dwelling older adults using a wearable system. Sci Rep. 2021;11(1):20976. [ CrossRef ]
• Hsu YC, Weng HH, Kuo CY, Chu TP, Tsai YH. Prediction of fall events during admission using eXtreme gradient boosting: a comparative validation study. Sci Rep. Oct 08, 2020;10(1):16777. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Hu Y, Bishnoi A, Kaur R, Sowers R, Hernandez ME. Exploration of machine learning to identify community dwelling older adults with balance dysfunction using short duration accelerometer data. In: Proceedings of the 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society. 2020. Presented at: EMBC '20; July 20-24, 2020;812-815; Montreal, QC. URL: https://ieeexplore.ieee.org/document/9175871 [ CrossRef ]
• Ye C, Li J, Hao S, Liu M, Jin H, Zheng L, et al. Identification of elders at higher risk for fall with statewide electronic health records and a machine learning algorithm. Int J Med Inform. May 2020;137:104105. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Althobaiti T, Katsigiannis S, Ramzan N. Triaxial accelerometer-based falls and activities of daily life detection using machine learning. Sensors (Basel). Jul 06, 2020;20(13):3777. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Tunca C, Salur G, Ersoy C. Deep learning for fall risk assessment with inertial sensors: utilizing domain knowledge in spatio-temporal gait parameters. IEEE J Biomed Health Inform. Jul 2020;24(7):1994-2005. [ CrossRef ]
• Kim K, Yun G, Park SK, Kim DH. Fall detection for the elderly based on 3-axis accelerometer and depth sensor fusion with random forest classifier. In: Proceedings of the 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society. 2019. Presented at: EMBC '19; July 23-27, 2019;4611-4614; Berlin, Germany. URL: https://ieeexplore.ieee.org/document/8856698 [ CrossRef ]
• Gillain S, Boutaayamou M, Schwartz C, Brüls O, Bruyère O, Croisier JL, et al. Using supervised learning machine algorithm to identify future fallers based on gait patterns: a two-year longitudinal study. Exp Gerontol. Nov 2019;127:110730. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Lo Y, Lynch SF, Urbanowicz RJ, Olson RS, Ritter AZ, Whitehouse CR, et al. Using machine learning on home health care assessments to predict fall risk. Stud Health Technol Inform. Aug 21, 2019;264:684-688. [ CrossRef ] [ Medline ]
• Maray N, Ngu AH, Ni J, Debnath M, Wang L. Transfer learning on small datasets for improved fall detection. Sensors (Basel). Jan 18, 2023;23(3):1105. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Ladios-Martin M, Cabañero-Martínez MJ, Fernández-de-Maya J, Ballesta-López FJ, Belso-Garzas A, Zamora-Aznar FM, et al. Development of a predictive inpatient falls risk model using machine learning. J Nurs Manag. Nov 30, 2022;30(8):3777-3786. [ CrossRef ] [ Medline ]
• Eichler N, Raz S, Toledano-Shubi A, Livne D, Shimshoni I, Hel-Or H. Automatic and efficient fall risk assessment based on machine learning. Sensors (Basel). Feb 17, 2022;22(4):1557. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Tang YM, Wang YH, Feng XY, Zou QS, Wang Q, Ding J, et al. Diagnostic value of a vision-based intelligent gait analyzer in screening for gait abnormalities. Gait Posture. Jan 2022;91:205-211. [ CrossRef ] [ Medline ]
• Greene BR, McManus K, Ader LG, Caulfield B. Unsupervised assessment of balance and falls risk using a smartphone and machine learning. Sensors (Basel). Jul 13, 2021;21(14):4770. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Roshdibenam V, Jogerst GJ, Butler NR, Baek S. Machine learning prediction of fall risk in older adults using timed up and go test kinematics. Sensors (Basel). May 17, 2021;21(10):3481. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Dubois A, Bihl T, Bresciani JP. Identifying fall risk predictors by monitoring daily activities at home using a depth sensor coupled to machine learning algorithms. Sensors (Basel). Mar 11, 2021;21(6):1957. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Vo MT, Thonglor R, Moncatar TJ, Han TD, Tejativaddhana P, Nakamura K. Fear of falling and associated factors among older adults in Southeast Asia: a systematic review. Public Health. Sep 2023;222:215-228. [ CrossRef ] [ Medline ]
• Torun E, Az A, Akdemir T, Solakoğlu GA, Açiksari K, Güngörer B. Evaluation of the risk factors for falls in the geriatric population presenting to the emergency department. Ulus Travma Acil Cerrahi Derg. Aug 2023;29(8):897-903. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Son NK, Ryu YU, Jeong HW, Jang YH, Kim HD. Comparison of 2 different exercise approaches: Tai Chi versus Otago, in community-dwelling older women. J Geriatr Phys Ther. 2016;39(2):51-57. [ CrossRef ] [ Medline ]
• Sawa R, Doi T, Tsutsumimoto K, Nakakubo S, Kurita S, Kiuchi Y, et al. Overlapping status of frailty and fear of falling: an elevated risk of incident disability in community-dwelling older adults. Aging Clin Exp Res. Sep 11, 2023;35(9):1937-1944. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Calazans JA, Permanyer I. Levels, trends, and determinants of cause-of-death diversity in a global perspective: 1990-2019. BMC Public Health. Apr 05, 2023;23(1):650. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Kakara R, Bergen G, Burns E, Stevens M. Nonfatal and fatal falls among adults aged ≥65 years - United States, 2020-2021. MMWR Morb Mortal Wkly Rep. Sep 01, 2023;72(35):938-943. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Dostan A, Dobson CA, Vanicek N. Relationship between stair ascent gait speed, bone density and gait characteristics of postmenopausal women. PLoS One. Mar 22, 2023;18(3):e0283333. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Zheng Y, Wang X, Zhang ZK, Guo B, Dang L, He B, et al. Bushen Yijing Fang reduces fall risk in late postmenopausal women with osteopenia: a randomized double-blind and placebo-controlled trial. Sci Rep. Feb 14, 2019;9(1):2089. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Woelfle T, Bourguignon L, Lorscheider J, Kappos L, Naegelin Y, Jutzeler CR. Wearable sensor technologies to assess motor functions in people with multiple sclerosis: systematic scoping review and perspective. J Med Internet Res. Jul 27, 2023;25:e44428. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Abdollah V, Dief TN, Ralston J, Ho C, Rouhani H. Investigating the validity of a single tri-axial accelerometer mounted on the head for monitoring the activities of daily living and the timed-up and go test. Gait Posture. Oct 2021;90:137-140. [ CrossRef ] [ Medline ]
• Mielke GI, de Almeida Mendes M, Ekelund U, Rowlands AV, Reichert FF, Crochemore-Silva I. Absolute intensity thresholds for tri-axial wrist and waist accelerometer-measured movement behaviors in adults. Scand J Med Sci Sports. Sep 12, 2023;33(9):1752-1764. [ CrossRef ] [ Medline ]
• Löppönen A, Delecluse C, Suorsa K, Karavirta L, Leskinen T, Meulemans L, et al. Association of sit-to-stand capacity and free-living performance using Thigh-Worn accelerometers among 60- to 90-yr-old adults. Med Sci Sports Exerc. Sep 01, 2023;55(9):1525-1532. [ FREE Full text ] [ CrossRef ] [ Medline ]
• García-Soidán JL, Leirós-Rodríguez R, Romo-Pérez V, García-Liñeira J. Accelerometric assessment of postural balance in children: a systematic review. Diagnostics (Basel). Dec 22, 2020;11(1):8. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Leirós-Rodríguez R, García-Soidán JL, Romo-Pérez V. Analyzing the use of accelerometers as a method of early diagnosis of alterations in balance in elderly people: a systematic review. Sensors (Basel). Sep 09, 2019;19(18):3883. [ FREE Full text ] [ CrossRef ] [ Medline ]

Abbreviations

Edited by A Mavragani; submitted 28.11.23; peer-reviewed by E Andrade, M Behzadifar, A Suárez; comments to author 09.01.24; revised version received 30.01.24; accepted 13.02.24; published 29.04.24.

©Ana González-Castro, Raquel Leirós-Rodríguez, Camino Prada-García, José Alberto Benítez-Andrades. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 29.04.2024.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.

NTRS - NASA Technical Reports Server

Available downloads, related records.

Assessment of Societal Health Risks: Spatial Distribution and Potential Hazards of Toxic Metals in Street Dust Across Diverse Communities

• Open access
• Published: 30 April 2024
• Volume 235 , article number  302 , ( 2024 )

Cite this article

You have full access to this open access article

• Oznur Isinkaralar   ORCID: orcid.org/0000-0001-9774-5137 1 ,
• Kaan Isinkaralar 2 &
• Balram Ambade 3  

On a global scale, the urban design of city centers is a topic of discussion concerning various aspects such development and its impact on public health. This research examines the health effects of urban agglomeration in city centers with compact, close development. In this work, the potentially toxic metals in street dust were studied by collecting and measuring street dust samples, measuring trace metal concentrations, and using index assessment, spatial analysis, correlation analysis, and health risk assessment models. Eskişehir, located in the part of Central Anatolia close to the Aegean region, west Türkiye, has been widely recognized as one of the most popular, known for having many narrow and old buildings in urban environments. The present paper investigates the atmospheric dust-related chemical speciation, urban environmental pollution, and human health risks in Eskişehir City by studying 66 dust samples collected at 11 points in the selected streets in August 2023. the study found that the concentrations of trace elements followed the order Cr > Ni > Pb > Cd > Cu. The primary source of these high levels is believed to be traffic-related contamination involving Cd, Pb, and Ni. The assessment of non-carcinogenic health risks has shown that the significant sources of potential toxic metals exposure for both children and adults are i) through ingestion and ii) dermal contact. The Hazard index (HI) for selected metals decreased in the order Cr > Pb > Ni > Cd > Cu for both children and adults without imposing possible non-carcinogenic risk (HI<1). On the contrary, Cr posed cancer risks above the safety threshold (> 10 -4 ) through ingestion. Based on the available findings, Eskişehir still suffers from considerable environmental and ecological degradation and severe health risks due to street dust contamination. However, while high pollution was detected in the city center, where there is commercial land use, low values were observed in the region, rich with the water surface, bicycle paths, landscape design, and where traffic is slowed down.

Avoid common mistakes on your manuscript.

1 Introduction

Urban environments face mounting challenges due to population growth, industrialization, vehicular emissions, and energy demands. These stressors negatively impact air quality, urban expansion, and sustainable development in cities (Bernardino et al., 2019 ; Wang et al., 2018 ). This issue has transcended regional, city, or country boundaries and has become a global concern (Goudie, 2014 ). As climate change intensifies, it has been reported that over 7 million lives have been lost, primarily due to respiratory infections, heart and lung diseases, and lung cancer (An et al., 2018 ; Combes & Franchineau, 2019 ). Exposure to urban pollutants has become inevitable in societies that fail to prioritize and implement planned urbanization (Zeng et al., 2020 ). Urban pollution affecting air, water, and noise has emerged as a significant threat to public health (Jaafari et al., 2021 ). Human health and ecosystems are exposed to hazardous substances (Aguilera et al., 2021 ). Numerous advanced techniques and integrated risk functions have been developed to assess the global disease burden of fine particulate matter (PM 2.5 ) from cities' anthropogenic sources (Adewumi, 2022 ). However, their adoption remains limited (Fang et al., 2015 ; Liu et al., 2016 ). Many of these pollutants can be attributed to traffic and related emissions, adversely affecting city air quality through carbon emissions, aerosols, particulate matter, and toxic metals (Boloorani et al., 2021 ).

City centers, characterized by concentrated services and populations, serve as commercial hubs (Trujillo-González et al., 2016 ). These areas are influenced by intricate dynamics interconnected at various levels (Liang et al., 2019 ). Consequently, the development of sustainable city centers has gained significant attention (Yu et al., 2013 ). The sustainability challenge has rekindled interest in urban land use patterns and city structures' compactness or sprawling nature. As a result, urban planning research now focuses on managing growth following sustainability goals and maintaining the relationships established within city centers (Weber et al., 2014 ).

Pollutants in the urban environment are predominantly anthropogenic, such as vehicular exhaust, industrial emissions, and coal-burning activities. However, they can originate from both natural and anthropogenic sources, either directly or indirectly (Zhao et al., 2021 ). They were released into the outdoor environment, attached to PMs, and traveled long distances with them. Among them are elements that bioaccumulate and have high toxicity on ecosystems and human health, depending on the concentration. As the release of pollutants continues to increase, their concentrations increase in the region, posing a net health risk to residents (Xie et al., 2019 ). The combined effects on human health due to the increased diversity of pollutants and their non-dispersive presence is a reason, among many others, for the migration of residents to a more tranquil and pollutant-free area. Toxic metals can be found in street dust in urban environments that have deteriorated the urban air quality and endangered living organisms and human health via inhalation, ingestion, and skin contact (Acosta et al., 2015 ).

During the summer months, when there is little or no rainfall, the accumulated dust increases, and the amount of toxic metals increases; they are carried to a considerable distance by the wind due to the durability of the gaps (Qadeer et al., 2020 ). Particularly in Eskişehir, intense accumulation and transport occur throughout the year in cities where precipitation is low but urbanization is intense. According to data from Türkiye’s's Meteorology Offices, during 2023, the lowest and highest PM 10 values recorded were 6.92 and 146.31 μg/m 3 , respectively. Some days, they demonstrated that the daily mean value surpassed 336 μg/m 3 , with over 10% of the days exceeding the hazardous health limit set by the United States Environmental Protection Agency (US EPA, 2008 ). During the summer, when there is little or no rainfall, dust and toxic metals increase. They are carried to a considerable distance by the wind due to the durability of the gaps. This is why the rates of cardiovascular and respiratory diseases and lung cancer are increasing in survivors. The street dust particles are highly connected with toxic metals. This problem has attracted global attention through the local governments, which need to continuously monitor the region's toxic air pollutants level to assess ecological and health hazards. Effective spatial planning and sustainable land management can have significant impacts on the potential transfer of heavy metals from contaminated soils to humans. In this context, while the compact or sprawling structure of the city center is being discussed, the kind of threat posed to health by single-centered, compact settlements should be known. The current literature review shows that road dust is widely used for heavy metal monitoring. The network between identifying heavy metal sources and health risk assessment is strong in Fig. 1 . Therefore, road dust was chosen as an indicator for health risk assessment.

"Visualizing Networks: Street Dust Representation through Link-based Analysis using Vosviewer Software in Bibliographic Studies"

This study comprehensively investigates street dust trace metal concentrations, contamination levels, and pollution sources in Eskişehir, assessing ecological risks through multiple pollution/ecological indices. It further evaluates potential non-carcinogenic and carcinogenic risks for children and adults due to long-term exposure to toxic metals in street dust. Focusing on high-traffic streets, this research offers unique insights into pollution levels and ecological/human health risks associated with metal pollution. It distinguishes itself from prior Eskişehir studies and establishes a vital database on environmental pollution and its impact on inhabitants' health. Analyzing the city center's spatial structure and discussing land use and health risks across various age groups completes the research.

2 Literature Review

The framework of potentially toxic metals in street dust by vehicular emission, engine corrosion, and interruption of air streams is used to research environmental literary endeavors. Potentially toxic metals in street dust are a cognitive extension of the standard urban air quality that assumes various elements influencing environmental health (Li et al., 2016 ). Urbanization, population, wealth, and technology are commonly considered arguments for examining their contribution to urban air quality (Dehghani et al., 2017 ; Jiang et al., 2018 ). As a result, this current research aims to confirm the effects of population growth, economic growth, green economy, and energy consumption on environmental degradation. Overall, this paper aims to affirm the effects of population growth, urbanization, economic growth, and energy consumption on urban air quality degradation.

The potentially toxic metals in street dust explain how urban emissions impact urban air quality deterioration. Saeedi et al. ( 2012 ) revealed the high ecological risk of street dust samples by traffic and related activities. After that, a sizable body of studies appeared to examine the occurrence of ecological and health risk assessment (Acosta et al., 2014 ; Musa et al., 2019 ; Tang et al., 2013 ; Urrutia-Goyes et al., 2018 ). Their experimental evidence demonstrated that income has various effects on urban air degradation. Zheng et al. ( 2010 ), Wang et al. ( 2012 ), and Rajaram et al. ( 2014 ) have shown that rapid urbanization and traffic emission leads to urban air degradation in several regions. This phenomenon supports the potentially toxic metals, demonstrating a deposition on biomass. Their findings revealed an accumulation, showing rapid urbanization and its emission are incredibly connected. Increased urban activities will boost existing emissions mobility and worsen ecology and health.

Many scholars have explored the correlation between densely populated streets and health risks (Du et al., 2013 ; Duong & Lee, 2011 ; Han et al., 2014 ). Apeagyei et al. ( 2011 ) highlighted that motor vehicle traffic significantly elevates heavy metal concentrations, increasing metal pollution and declining human health. Similarly, Li et al. ( 2013 ) and Elom et al. ( 2014 ) investigated the relationship between toxic metal deposition in the human body via street dust. They found that urban street dust poses substantial health risks to humans. The growing human and vehicular populations in cities, alongside the expansion of industrialization, have resulted in increased waste generation. Although alternative road routes used by urban centers to combat overcrowding have reduced traffic, the accumulation of traffic-induced toxic substances could not be prevented. Although the accumulation of toxic substances has been reduced on the roads intended to be pedestrianized, they remain in the environment for long periods without decomposing. Many studies have been conducted on the adverse effects of toxic substances on human health by Han et al. ( 2016 ), and Ladonin and Mikhaylova ( 2020 ). In the results, the concentration, source, distribution pattern, degree of pollution, and risk assessment of heavy metals from various anthropogenic sources pointed to the use of motor vehicles, especially traffic, leakages in industrial processes, and various other activities. Table 1 emphasizes the aspects those studies focus on determining the impact of toxic metals on ecology and human health.

3 Material and Methods

This study comprises a five-phase approach, illustrated in Fig. 2 . The research design was initially established, followed by land use assessment and sample collection in the study area. Subsequently, the findings were assessed through comprehensive analyses at three distinct levels: heavy metal concentrations, correlation examination, and health risk evaluation. The risk assessment was tailored to consider land use, statistical data, and age group demographics.

Stages of the research

3.1 Study Area

Eskişehir is situated in the northwestern of Türkiye, covering approximately 2678 km 2 and a population of 807,068 habitats accroding to Turkish Statistical Institute (TUİK), ( 2022 ) in Fig. 3 . This city is the most populated area because it has an Organized Industrial Zone. Agricultural activities are relatively limited, so their contribution to atmospheric pollution is insignificant. Its climate is severe continental and extremely fragile to wind erosion/dust emissions; also, average rainfall is generally low during the summer and autumn. The characteristics of annual average temperature and precipitation of 21.9 °C and 15 mm typically occur in August.

The location of the sampling points in Eskişehir

3.2 Street Dust Sampling

In August 2023, a total of 66 street dust samples were collected from 11 bustling, high-traffic streets with dense populations. These areas experience heavy vehicular congestion at traffic lights. A 0.5 square meter frame, polyethylene brush (5.6 cm), and a plastic hand shovel were used at each site to gather the samples. The collected samples were then placed in bags for transportation to the laboratory.

Upon reaching the lab, the street dust samples underwent drying at 50°C for 48 hours, followed by sieving with 100-micrometer sieves. This process yielded 0.5 grams of each dust sample, which were then separated for the analysis of toxic metal concentrations. Since background values were unavailable, the continental upper crust values Rudnick and Gao ( 2003 ) reported were employed as references. An aqua regia extraction method was used in conjunction with SpectroBlue atomic emission spectrometry (ICP-OES, Germany), which features a SpectroBlue plasma source to analyze the toxic metal concentrations.

3.3 Chemical Analysis

This research concentrated on the determination of 5 toxic elements, including Cd, Cr, Cu, Ni, and Pb. All chemicals and reagents were purchased from Sigma-Aldsrich and used as analytical grades. The wet digestion technique was applied with aqua regia–HCl (37%) / HNO 3 (69%), 3:1 (v/v) for 60 min at 80°C and then, they were washed with deionized water, filtered through a Whatman filter, were dried and analyzed by Inductively Coupled Plasma Optical Emission Spectroscopy (ICP-OES) (Ishtiaq et al., 2018 ). A blank sample was prepared for each group during digestion for quality assurance/quality control (QA/QC). At the same time, the calibration curve was plotted by using five standards for the accuracy of the extraction and dust assessment method. All samples were processed and inspected in three replicates.

3.4 Statistical Analysis

In order to establish the relationship between toxic metals in street dust and their potential origins, a comprehensive analysis was conducted using the Pearson correlation coefficient and cluster analysis. These methods were implemented with the help of the statistical software package SPSS version 22.0. The Pearson correlation coefficient is a widely recognized metric in environmental research, as it measures the relative strength of the association between two trace metals. During this analysis, the standard deviation (S.D.) was calculated for each dust sample. The results demonstrated good precision, as the relative standard deviation was within 5% (S.D. < 5%). This indicates that the detection limits for the concentration of metals such as Cadmium (Cd), Chromium (Cr), Copper (Cu), Nickel (Ni), and Lead (Pb) were accurately measured at 0.02, 0.09, 0.020, 0.015, and 0.09 mg/kg, respectively.

3.5 Pollution Indices

Here, the potential ecological and human health risks caused by toxic metals in street dust gathered from several points in the Eskişehir were analyzed through standard indexes, which are explained as follows Hakanson ( 1980 ): i) Enrichment Factor ( EF ) is commonly applied to evaluate the contamination level and to trace the origin and sources of certain elements contained in street dust, ii) Ecological Risk Index ( RI ) describes the degree of contamination of each metal based on their adverse environmental risk (Laniyan & Adewumi, 2019 ), and iii) Health Risk Assessment model by US EPA ( 1989 , 2001 , 2011 ) associated with toxic metals for both adults and children was calculated through Hazard Index ( HI ) and hazard quotient ( HQ ).

Here C i is the concentration of heavy metals ( i , mg·kg -1 ), and C r is the concentration of the reference metal ( r , mg·kg -1 ). The EF value is smaller than one, which means that the element mainly comes from the crust and other natural sources, while an EF larger than 1 implies that it is affected by both human and natural factors. The EFs are categorized for enrichment level as the minimum (1-2), moderate (2-5), significant (5-20), very high (20-40), and extremely strong (>40) (Ekwere & Edet, 2021 ). E i is the potential ecological risk factor of metal i, and T i is the metal toxic factor. ƒ i is the metal pollution factor of metal i, which equals the amount of metal i in the sample ( C i ) divided by its B i : reference value for metals, C i is the content of metals in street dust. Classification levels were determined as low (<150), moderate (150-300), high (300-600), severe (>600) for RI and low (<40), moderate (40-80), high (80-160), serious (160-320), severe (>320) for E i . The Eq. ( 5 - 7 ) were calculated health risk by exposure patways which were used some standard values as flows: IngR child , and IngR adult are 100 and 200 mg·day -1 for the children and adult ingestion rate (US EPA, 2011 ); Inh child , and Ingh adult are 20 and 7.6 m 3 day -1 for the children and adult inhalation rate (US EPA, 2009 ); PEF child and PEF adult are 1.36E+09 m 3 ·kg -1 for the children and adults particle emission factor (US EPA, 2002 ); SL child and SL adult are 0.2 and 0.7 mg·cm -2 ·day -1 for the children and adult skin adherence factor (US EPA, 2002 ); SA child and SA adult are 2800 and 5700 cm 2 for the children and adult exposed skin area (US EPA, 2004 ); ABS child and ABS adult are the dermal absorption factor and same value as 0.001 for Cd (US EPA, 2004 ); EF child and EF adult in Eq. ( 5 ) are the exposure frequency as 350 day·yr -1 (); ED child and ED adult are 6 and 24 year for the children and adult exposure duration; BW child and BW adult are 15 and 70 kg for the children and adult body weight (US EPA, 2001 ); AT child and AT adult are the same value as ED × 365 day for non-cancer the averaging time and 70 × 365 day for cancer the averaging time. The HQ is expressed as the ratio between the average daily dosage ( ADD ) received through several pathways ( D ing , D inh , and D der ) and the reference dose ( RfD ) mg/kg/day for a given toxic metal. The HI > 1 means probable non-carcinogenic activity of toxic metals; HI < 1 suggests no health risk. However, carcinogenic risk (CR>1×10 -4 ) and acceptable level (1×10 -6 < CR< 1×10 -4 ) indicate the risk of toxic metal in street dust that 1 in 10,000 people can get any cancer as a consequence of a lifetime of exposure to carcinogenic hazards. Also, all variables are used as a guide in the human health risk assessment model based on US EPA ( 1989 , 2001 , 2007 ). Eq. ( 8 ) expresses reference dose by different exposure pathways, in which R f D Ing , R f D Inh , and R f D Der are varied, and the range value from 6.00E-05 to 1.20E-02 for the toxic metal.

4.1 Toxic Metal Concentration in Street Dust

Table 2 gives the descriptive statistics of Cd, Cr, Cu, Ni, and Pb concentrations obtained from 66 street dust samples gathered from eleven points on Eskisehir Street. The trace elements examined in previous studies for Cr, Ni, Cu, Pb, and Cd ranged from 224 mg/kg for Cr to 0.51 mg/kg for Cd. The overall high Pb levels are due to the high traffic density and industrial activities in Eskişehir compared to other cities. A recent study revealed higher average Cd, Cu, and Pb concentrations in Eskişehir than outdoors. The mean concentrations of Cd, Ni, and Pb have exceeded the values recorded for the Upper Continental Crust. There was no considerable variation in concentrations between streets and sampling days, suggesting a higher contamination risk from anthropogenic activities. Eskişehir has a unique characteristic combination of elemental compositions. In these selected points, observed values may not reflect actual natural and anthropogenic diversities for all sites of Eskişehir. The skewness values for the Cr, Ni, Cu, Pb, and Cd were largely positive, indicating that the means were higher than the median, suggesting the presence of high pollution events and the temporal nature of the highest concentrations between sampling points. The arithmetic mean contents of Cr, Ni, Cu, Pb, and Cd in the street dust of Eskişehir street dust decrease in the order Cr > Ni > Pb > Cu > Cd. Except for Cd, all trace metal levels exhibited major standard deviations. It demonstrates the great diversity of amounts in street dust. The skewness values of Cr, Cu, Pb, and Cd, except Ni, are higher than unity, which indicates that these elements are highly positively skewed towards low concentrations.

Spatial distribution assessment is an assisted tool for determining the polluted and non-polluted zones in a defined field on ArcGIS spatial map. In the current work site, spatial distribution patterns of Cr, Ni, Cu, Pb, and Cd in road dust are depicted in Fig. 4 .

Spatio-temporal distribution of ( a ): Cr, ( b ): Cu, ( c ): Cd, ( d ): Ni, and ( e ): Pb in the street dust for different land-use types in Eskişehir

4.2 Correlation Analysis

Figure 5 a displays dendrogram results in four clusters: i) Ni-Pb ii) Pb and Cu; iii) Cr; and iv) Cu and Cd, which are fully consistent with the correlation results. However, clusters 3 and 4 seem to come together relatively higher, probably indicating a common source. Pearson's correlation coefficients between metals represent common origin with potential natural and anthropogenic sources in Fig. 5 b. It can be seen that Cr, Ni, Cu, Pb, and Cd had significant positive correlations with each other. Pb-Cr and Pb-Ni showed significantly strong correlations, r: 0.56** and r: 0.51** because they represented vehicular traffic-related emissions. Similarly, Ni and Cu had positive correlations with r: 0.54**, whereas Cu only had a slight correlation with Cd (r: 0.11) due to Cd and Cu having different sources. This suggests that Cd and Cu are partly derived from a natural source (local soil), whereas Pb, Ni, and Cr are mainly impacted by traffic and industrial operations.

( a ): Dendrogram showing clustering and ( b ): Matrix of Pearson correlation coefficient values of toxic metals concentrations in street dust. **. Correlation is significant at the 0.01 level (2-tailed)

According to EF values, EF value of Cr, and Cu smaller than 1 implies no enrichment, while EF value of Cd is 1-2, representing deficiency to minimal enrichment. The EF values of Pb and Ni were determined as moderate enrichment due to their values calculated between 2 and 5.

4.3 Health Risk Assessment

The study of street dust environmental risks and pollution factors is of major importance, giving insight into emission sources and assisting decision-making for resilient cities. The EF of selected toxic metals was calculated to assess the degree of contamination of street dust, which obtained mean EF values for Cu and Cd below 1.5. Their values represent deficiency to minimal enrichment based on their EF values, whereas the EF values showed moderate enrichment for Cd and Pb and significant enrichment for Ni. Traffic-generated emissions in Eskişehir City and other urban facilities, such as construction and household activities, can be the main sources of anthropogenic emissions. Cd and Cu could be partly released from vehicular, although fossil-fuel combustion may also contribute to their levels. Table 3 shows that non-carcinogenic health risks were calculated for Cr, Ni, Cu, Pb, and Cd. The highest HQ ing  was estimated for Ni and Pb in children (1.67E+00 and 2.67E+00), although these values were found for adults (1.41E-02 and 2.27E-01), which means that low potential to cause non-carcinogenic risk (HQ < 1). The HQ ing values for Cu were also high and comparable to Cu's, with averages of 1.15E-01 for children and Pb 2.27E-01 for adults. Ni presented the highest risk values regarding the inhalation and dermal contact pathways, thus being recognized as the most hazardous element.

5 Discussion

In terms of the spatial distribution of toxic metals, the sampling sites were chosen from urban environments to represent diverse global locations. The research by Mehmood et al. ( 2019 ), Ciarkowska et al. ( 2019 ), and Silva et al. ( 2021 ) has focused on toxic metals from industrial or factory sources and vehicular traffic in cities. Pb, Ni, and Cd exhibited higher concentrations in this context than other metals. However, these concentrations were still lower than those in the Upper Continental Crust. On the contrary, the concentrations of anthropogenic-related elements such as Zn, Cu, Co, and Pb in the street dust samples were comparatively lower than those detected in atmospheric or street dust samples collected in other cities. This can be attributed to Eskişehir's lower population density, reduced vehicular emissions, and the absence of significant industrial zones (as mentioned in Shabanda et al., 2019 and Hanfi et al., 2020 ). The street dust was significantly enriched with Pb and Ni and moderately enriched with Cd due to heavier traffic density, higher population, and industrial activity (Pan et al., 2018 ). Although Pb has been stopped being used as an additive in petrol and gasoline worldwide, Pb emissions may still be related to the industrial sector and traffic, similar to other urban sites (Huang et al., 2022 ). Pan et al. ( 2017 ) noted that the current toxic metal concentrations for outdoor and indoor dust were significantly higher than those recently reported in the city.

Trace elements in street dust are considerably higher than in airborne dust because street dust is an important sink for pollutants and heavy metals in the urban environment (Mahato et al., 2023 ; Nezat et al., 2017 ). As such, while suspended dust polluted with urban pollution and heavy metals accumulates in the streets, only a proportion can be re-suspended into the atmosphere due to wind, traffic, and other human activities (Bisht et al., 2022 ; Luo et al., 2019 ). Additionally, the highly relevant relationship between Ni and Cr indicates that they presumably derive from similar geogenic sources and are influenced by anthropogenic sources, such as the traffic sector tire and brake wear (Rahman et al., 2019 ; Wahab et al., 2020 ). Wei et al. (2021) exhibited that brake materials contained Cu elements and existed in the atmosphere. Related research defined by Shen et al. ( 2018 ) and Chen et al. ( 2019 ) that Cu can be emitted into the urban environment due to wear of the car's oil pump, corrosion of metal parts in contact with the oil, and engine wear. Many health risk assessment investigations about street dust deposition, a considerable part of Ni, Fe, Cr, and Co in the atmosphere comes from lithogenic sinks by Bartholomew et al. ( 2020 ), Dat et al. ( 2021 ), and Delgado-Iniesta et al. ( 2022 ). Similar to this paper, Marín Sanleandro et al. ( 2018 ) reported that the EF values for Pb and Ni show significant enrichment due to traffic-related contamination. Previous research investigating pollution and ecological risk indices for street dust in urban environments has also reported higher pollution levels and risk index values due to the more significant deposition of hazardous metals in the street rather than in airborne dust by Budai and Clement ( 2018 ); and Nargis et al. ( 2022 ). The seasonal variation of the content of toxic and poisonous metals, multiannual trends, and different sizes of street dust require further research. Roy et al. ( 2022 ) emphasized that although general trends for certain toxic metals in street dust are maintained, individual indices show slightly different pollution levels. The problem of improving the method of calculating the index requires further work. Urban centers contain urban functions that serve the city as a whole. Therefore, traditional urban centers have a high density of buildings and population. This density increases towards the city center and decreases towards the periphery. However, city centers have frequently contained connections and nodes designed with human scale in the past. Due to this road network and urban morphology, the road patterns are connected at short intervals.

Due to vehicular traffic frequently slowing down and stopping, the exhaust emissions of fossil fuel-consuming motor vehicles increase due to their stop-and-go movements. This leads to the release of toxic elements into the external environment, as technical components are responsible for enabling vehicles to stop and start wearing out. These stop-and-go movements also contribute to adverse effects on public health. Moreover, studies have indicated that such traffic patterns can indirectly decrease gross national income.

6 Conclusions

In the research area, residential zones with low densities featured water surfaces, parks, and alternative transportation systems to fossil fuels, exhibiting lower pollution risks. In contrast, commercial areas with high motor vehicle usage presented a higher risk of urban pollution based on heavy metals and health hazards. Extensive research has focused on distributing common trace metals in street dust, such as Cr, Ni, Cu, Pb, Cd, and Ni. A total of 66 samples were collected from diverse locations, including parks, traffic lights, and high-traffic areas. Strong correlations between Ni and Pb suggest similar geogenic origins influenced by anthropogenic factors like traffic, residential, and commercial emissions. Enrichment factor calculations highlight significant enrichment of Cd and moderate enrichment of Ni and Pb in street dust, posing a moderate ecological risk due to mean Ni and Pb contents. The primary contributors to severe pollution and ecological risks are Pb and Ni, accounting for 68% of the total potential ecological risk. Moderate risks are associated with Cd, while vehicular emissions in the city center and the usage of brakes and engines providing stop-and-go movements contribute to the consistently high concentrations of Cd and Pb. Assessment of the non-carcinogenic human health risk demonstrated that the ingestion exposure route posed much greater health risks than inhalation and skin contact. Since the HI values of the Cr, Ni, Cu, Pb, and Cd were lower than the safe level (HI<1), the non-carcinogenic health risks of targeted PTEs in Zabol atmospheric dust were generally low. Among the Ni, it exhibited the highest cancer risk through the inhalation pathway, although it was still below the safe limit of 10 -4 —the carcinogenic risk related to street dust, considering the quantities of poisonous metals. There is an urgent need for urban management actions to mitigate the consequences of serious Pb, Ni, and Cd pollution, as well as the risk of Cr cancer. In urban areas, a detailed urban air quality control/plan is needed to assess the situation and select effective strategies to deal with the ecological and human health hazards of dust pollution. The findings provide a reference for urban management and the protection of residents' health in similar urban governments surrounding heavily trafficked streets.

Future research on urban traffic emissions is more comprehensive than previously, but certain strategies should be followed to ensure sustainable urban development. Based on the principle of sustainability, monocentric-multicentric urban growth models should be discussed separately. Alternative modes of transportation and public transport should be studied. Transportation systems powered by new energy sources should be targeted for a healthier urban model. Pedestrian-oriented designs should be presented to reduce vehicle and pedestrian density in the city center. Local governments can pursue policies such as taxation or pricing to limit the entry of vehicles using fossil fuels into the city center.

Data Availability

The data that support the findings of this study are available from the corresponding author, upon reasonable request.

Acosta, J. A., Faz, A., Kalbitz, K., Jansen, B., & Martínez-Martínez, S. (2014). Partitioning of heavy metals over different chemical fraction in street dust of Murcia (Spain) as a basis for risk assessment. Journal of Geochemical Exploration, 144 , 298–305. https://doi.org/10.1016/j.gexplo.2014.02.004

Article   CAS   Google Scholar  

Acosta, J. A., Gabarrón, M., Faz, A., Martínez-Martínez, S., Zornoza, R., & Arocena, J. M. (2015). Influence of population density on the concentration and speciation of metals in the soil and street dust from urban areas. Chemosphere, 134 , 328–337. https://doi.org/10.1016/j.chemosphere.2015.04.038

Adewumi, A. J. (2022). Heavy metals in soils and road dust in Akure City, Southwest Nigeria: pollution, sources, and ecological and health risks. Exposure and Health, 14 (2), 375–392. https://doi.org/10.1007/s12403-021-00456-y

Aguilera, A., Bautista-Hernández, D., Bautista, F., Goguitchaichvili, A., & Cejudo, R. (2021). Is the urban form a driver of heavy metal pollution in road dust? Evidence from Mexico City. Atmosphere, 12 (2), 266. https://doi.org/10.3390/atmos12020266

Alsbou, E. M. E., & Al-Khashman, O. A. (2018). Heavy metal concentrations in roadside soil and street dust from Petra region, Jordan. Environmental Monitoring and Assessment, 190 , 1–13. https://doi.org/10.1007/s10661-017-6409-1

An, L., Che, H., Xue, M., Zhang, T., Wang, H., Wang, Y., et al. (2018). Temporal and spatial variations in sand and dust storm events in East Asia from 2007 to 2016: Relationships with surface conditions and climate change. Science of The Total Environment, 633 , 452–462. https://doi.org/10.1016/j.scitotenv.2018.03.068

Apeagyei, E., Bank, M. S., & Spengler, J. D. (2011). Distribution of heavy metals in road dust along an urban-rural gradient in Massachusetts. Atmospheric Environment, 45 (13), 2310–2323. https://doi.org/10.1016/j.atmosenv.2010.11.015

Bartholomew, C. J., Li, N., Li, Y., Dai, W., Nibagwire, D., & Guo, T. (2020). Characteristics and health risk assessment of heavy metals in street dust for children in Jinhua, China. Environmental Science and Pollution Research, 27 , 5042–5055. https://doi.org/10.1007/s11356-019-07144-0

Bernardino, C. A., Mahler, C. F., Santelli, R. E., Freire, A. S., Braz, B. F., & Novo, L. A. (2019). Metal accumulation in roadside soils of Rio de Janeiro, Brazil: impact of traffic volume, road age, and urbanization level. Environmental Monitoring and Assessment, 191 , 1–14. https://doi.org/10.1007/s10661-019-7265-y

Bisht, L., Gupta, V., Singh, A., Gautam, A. S., & Gautam, S. (2022). Heavy metal concentration and its distribution analysis in urban road dust: A case study from most populated city of Indian state of Uttarakhand. Spatial and Spatio-Temporal Epidemiology, 40 , 100470. https://doi.org/10.1016/j.sste.2021.100470

Article   Google Scholar  

Boloorani, A. D., Shorabeh, S. N., Samany, N. N., Mousivand, A., Kazemi, Y., Jaafarzadeh, N., et al. (2021). Vulnerability mapping and risk analysis of sand and dust storms in Ahvaz, IRAN. Environmental Pollution, 279 , 116859. https://doi.org/10.1016/j.envpol.2021.116859

Budai, P., & Clement, A. (2018). Spatial distribution patterns of four traffic-emitted heavy metals in urban road dust and the resuspension of brake-emitted particles: Findings of a field study. Transportation Research Part D: Transport and Environment, 62 , 179–185. https://doi.org/10.1016/j.trd.2018.02.014

Castillo-Nava, D., Elias-Santos, M., López-Chuken, U. J., Valdés-González, A., de La Riva-Solis, L. G., Vargas-Pérez, M. P., et al. (2020). Heavy metals (lead, cadmium and zinc) from street dust in Monterrey, Mexico: ecological risk index. International Journal of Environmental Science and Technology, 17 , 3231–3240. https://doi.org/10.1007/s13762-020-02649-5

Chen, Y., Ma, J., Duan, H., & Miao, C. (2019). Occurrence, source apportionment, and potential human health risks of metal (loid) s and PAHs in dusts from driving school campuses in an urban area of Henan, China. Environmental Science and Pollution Research, 26 , 30029–30043. https://doi.org/10.1007/s11356-019-06044-7

Ciarkowska, K., Gambus, F., Antonkiewicz, J., & Koliopoulos, T. (2019). Polycyclic aromatic hydrocarbon and heavy metal contents in the urban soils in southern Poland. Chemosphere, 229 , 214–226. https://doi.org/10.1016/j.chemosphere.2019.04.209

Combes, A., & Franchineau, G. (2019). Fine particle environmental pollution and cardiovascular diseases. Metabolism, 100 , 153944. https://doi.org/10.1016/j.metabol.2019.07.008

Dat, N. D., Nguyen, V. T., Vo, T. D. H., Bui, X. T., Bui, M. H., Nguyen, L. S. P., et al. (2021). Contamination, source attribution, and potential health risks of heavy metals in street dust of a metropolitan area in Southern Vietnam. Environmental Science and Pollution Research, 28 (36), 50405–50419. https://doi.org/10.1007/s11356-021-14246-1

Dehghani, S., Moore, F., Keshavarzi, B., & Beverley, A. H. (2017). Health risk implications of potentially toxic metals in street dust and surface soil of Tehran, Iran. Ecotoxicology and Environmental Safety, 136 , 92–103. https://doi.org/10.1016/j.ecoenv.2016.10.037

Delgado-Iniesta, M. J., Marín-Sanleandro, P., Díaz-Pereira, E., Bautista, F., Romero-Muñoz, M., & Sánchez-Navarro, A. (2022). Estimation of ecological and human health risks posed by heavy metals in street dust of Madrid City (Spain). International Journal of Environmental Research and Public Health, 19 (9), 5263. https://doi.org/10.3390/ijerph19095263

Du, Y., Gao, B., Zhou, H., Ju, X., Hao, H., & Yin, S. (2013). Health risk assessment of heavy metals in road dusts in urban parks of Beijing, China. Procedia Environmental Sciences, 18 , 299–309. https://doi.org/10.1016/j.proenv.2013.04.039

Duong, T. T., & Lee, B. K. (2011). Determining contamination level of heavy metals in road dust from busy traffic areas with different characteristics. Journal of Environmental Management, 92 (3), 554–562. https://doi.org/10.1016/j.jenvman.2010.09.010

Ekwere, A. S., & Edet, B. B. (2021). Temporal variations of heavy metals in sediment, soil and dust particulates across the rock quarrying districts of the Oban Massif, Southeastern Nigeria. Environmental Nanotechnology, Monitoring & Management, 15 , 100431. https://doi.org/10.1016/j.enmm.2021.100431

Elom, N. I., Entwistle, J., & Dean, J. R. (2014). Human health risk from Pb in urban street dust in northern U.K. cities. Environmental Chemistry Letters, 12 , 209–218. https://doi.org/10.1007/s10311-013-0436-0

USEPA, (2008). Integrated review plan for the national ambient air quality standards for particulate matter. https://www3.epa.gov/ttn/naaqs/standards/pm/data/2008_03_final_integrated_review_plan.pdf . Accessed 26 June 2023

Fang, S., Qiao, Y., Yin, C., Yang, X., & Li, N. (2015). Characterizing the physical and demographic variables associated with heavy metal distribution along urban-rural gradient. Environmental Monitoring and Assessment, 187 , 1–14. https://doi.org/10.1007/s10661-015-4780-3

Gope, M., Masto, R. E., George, J., Hoque, R. R., & Balachandran, S. (2017). Bioavailability and health risk of some potentially toxic elements (Cd, Cu, Pb and Zn) in street dust of Asansol, India. Ecotoxicology and Environmental Safety, 138 , 231–241. https://doi.org/10.1016/j.ecoenv.2017.01.008

Goudie, A. S. (2014). Desert dust and human health disorders. Environment International, 63 , 101–113. https://doi.org/10.1016/j.envint.2013.10.011

Hakanson, L. (1980). An ecological risk index for aquatic pollution control. A sedimentological approach. Water Research, 14 (8), 975–1001. https://doi.org/10.1016/0043-1354(80)90143-8

Han, N. M. I. M., Latif, M. T., Othman, M., Dominick, D., Mohamad, N., Juahir, H., & Tahir, N. M. (2014). Composition of selected heavy metals in road dust from Kuala Lumpur city centre. Environmental Earth Sciences, 72 , 849–859. https://doi.org/10.1007/s12665-013-3008-5

Han, X., Lu, X., Zhang, Q., Wuyuntana, Hai, Q., & Pan, H. (2016). Grain-size distribution and contamination characteristics of heavy metal in street dust of Baotou, China. Environmental Earth Sciences, 75 , 1–10. https://doi.org/10.1007/s12665-016-5316-z

Hanfi, M. Y., Mostafa, M. Y., & Zhukovsky, M. V. (2020). Heavy metal contamination in urban surface sediments: sources, distribution, contamination control, and remediation. Environmental Monitoring and Assessment, 192 , 1–21. https://doi.org/10.1007/s10661-019-7947-5

Huang, C., Zhang, L., Meng, J., Yu, Y., Qi, J., Shen, P., et al. (2022). Characteristics, source apportionment and health risk assessment of heavy metals in urban road dust of the Pearl River Delta, South China. Ecotoxicology and Environmental Safety, 236 , 113490. https://doi.org/10.1016/j.ecoenv.2022.113490

Ishtiaq, M., Jehan, N., Khan, S. A., Muhammad, S., Saddique, U., Iftikhar, B., & Zahidullah. (2018). Potential harmful elements in coal dust and human health risk assessment near the mining areas in Cherat, Pakistan. Environmental Science and Pollution Research, 25 (15), 14666–14673. https://doi.org/10.1007/s11356-018-1655-5

Isinkaralar, K., Isinkaralar, O., & Bayraktar, E. P. (2024). Ecological and Health Risk Assessment in Road Dust Samples from Various Land Use of Düzce City Center: Towards the Sustainable Urban Development. Water, Air, & Soil Pollution, 235 , 84. https://doi.org/10.1007/s11270-023-06879-4

Jaafari, J., Naddafi, K., Yunesian, M., Nabizadeh, R., Hassanvand, M. S., Shamsipour, M., et al. (2021). Associations between short term exposure to ambient particulate matter from dust storm and anthropogenic sources and inflammatory biomarkers in healthy young adults. Science of The Total Environment, 761 , 144503. https://doi.org/10.1016/j.scitotenv.2020.144503

Jiang, Y., Shi, L., Guang, A. L., Mu, Z., Zhan, H., & Wu, Y. (2018). Contamination levels and human health risk assessment of toxic heavy metals in street dust in an industrial city in Northwest China. Environmental Geochemistry and Health, 40 , 2007–2020. https://doi.org/10.1007/s10653-017-0028-1

Kamani, H., Ashrafi, S. D., Isazadeh, S., Jaafari, J., Hoseini, M., Mostafapour, F. K., et al. (2015). Heavy metal contamination in street dusts with various land uses in Zahedan, Iran. Bulletin of Environmental Contamination and Toxicology, 94 , 382–386. https://doi.org/10.1007/s00128-014-1453-9

Kim, J. A., Park, J. H., & Hwang, W. J. (2016). Heavy metal distribution in street dust from traditional markets and the human health implications. International Journal of Environmental Research and Public Health, 13 (8), 820. https://doi.org/10.3390/ijerph13080820

Ladonin, D. V., & Mikhaylova, A. P. (2020). Heavy metals and arsenic in soils and street dust of the southeastern administrative district of Moscow: long-term data. Eurasian Soil Science, 53 , 1635–1644. https://doi.org/10.1134/S1064229320110095

Laniyan, T. A., & Adewumi, A. J. (2019). Health risk assessment of heavy metals pollution in groundwater around an exposed dumpsite in Southwestern Nigeria. Journal of Health and Pollution, 9 (24), 191–210. https://doi.org/10.5696/2156-9614-9.24.191210

Li, F., Zhang, J., Huang, J., Huang, D., Yang, J., Song, Y., & Zeng, G. (2016). Heavy metals in road dust from Xiandao District, Changsha City, China: characteristics, health risk assessment, and integrated source identification. Environmental Science and Pollution Research, 23 , 13100–13113. https://doi.org/10.1007/s11356-016-6458-y

Li, H., Qian, X., Hu, W., Wang, Y., & Gao, H. (2013). Chemical speciation and human health risk of trace metals in urban street dusts from a metropolitan city, Nanjing, S.E. China. Science of the Total Environment, 456 , 212–221. https://doi.org/10.1016/j.scitotenv.2013.03.094

Liang, L., Wang, Z., & Li, J. (2019). The effect of urbanization on environmental pollution in rapidly developing urban agglomerations. Journal of Cleaner Production, 237 , 117649. https://doi.org/10.1016/j.jclepro.2019.117649

Liu, R., Wang, M., Chen, W., & Peng, C. (2016). Spatial pattern of heavy metals accumulation risk in urban soils of Beijing and its influencing factors. Environmental Pollution, 210 , 174–181. https://doi.org/10.1016/j.envpol.2015.11.044

Luo, L., Mei, K., Qu, L., Zhang, C., Chen, H., Wang, S., et al. (2019). Assessment of the Geographical Detector Method for investigating heavy metal source apportionment in an urban watershed of Eastern China. Science of the Total Environment, 653 , 714–722. https://doi.org/10.1016/j.scitotenv.2018.10.424

Mahato, M. K., Singh, A. K., & Giri, S. (2023). Assessment of metal pollution and human health risks in road dust from mineral rich zone of East Singhbhum, India. Environmental Geochemistry and Health, 45 (5), 2291–2308. https://doi.org/10.1007/s10653-022-01341-6

Malakootian, M., Mohammadi, A., Nasiri, A., Asadi, A. M. S., Conti, G. O., & Faraji, M. (2021). Spatial distribution and correlations among elements in smaller than 75 μm street dust: ecological and probabilistic health risk assessment. Environmental Geochemistry and Health, 43 , 567–583. https://doi.org/10.1007/s10653-020-00694-0

Marín Sanleandro, P., Sánchez Navarro, A., Díaz-Pereira, E., Bautista Zuñiga, F., Romero Muñoz, M., & Delgado Iniesta, M. J. (2018). Assessment of heavy metals and color as indicators of contamination in street dust of a city in SE Spain: Influence of traffic intensity and sampling location. Sustainability, 10 (11), 4105. https://doi.org/10.3390/su10114105

Mehmood, K., Ahmad, H. R., Abbas, R., & Murtaza, G. (2019). Heavy metals in urban and peri-urban soils of a heavily-populated and industrialized city: Assessment of ecological risks and human health repercussions. Human and Ecological Risk Assessment: An International Journal . https://doi.org/10.1080/10807039.2019.1601004

Musa, A. A., Hamza, S. M., & Kidak, R. (2019). Street dust heavy metal pollution implication on human health in Nicosia, North Cyprus. Environmental Science and Pollution Research, 26 (28), 28993–29002. https://doi.org/10.1007/s11356-019-06028-7

Nargis, A., Habib, A., Islam, M. N., Chen, K., Sarker, M. S. I., Al-Razee, A. N. M., et al. (2022). Source identification, contamination status and health risk assessment of heavy metals from road dusts in Dhaka, Bangladesh. Journal of Environmental Sciences, 121 , 159–174. https://doi.org/10.1016/j.jes.2021.09.011

Nezat, C. A., Hatch, S. A., & Uecker, T. (2017). Heavy metal content in urban residential and park soils: A case study in Spokane, Washington, USA. Applied Geochemistry, 78 , 186–193. https://doi.org/10.1016/j.apgeochem.2016.12.018

Pan, H., Lu, X., & Lei, K. (2017). A comprehensive analysis of heavy metals in urban road dust of Xi'an, China: contamination, source apportionment and spatial distribution. Science of the Total Environment, 609 , 1361–1369. https://doi.org/10.1016/j.scitotenv.2017.08.004

Pan, L., Wang, Y., Ma, J., Hu, Y., Su, B., Fang, G., et al. (2018). A review of heavy metal pollution levels and health risk assessment of urban soils in Chinese cities. Environmental Science and Pollution Research, 25 , 1055–1069. https://doi.org/10.1007/s11356-017-0513-1

Qadeer, A., Saqib, Z. A., Ajmal, Z., Xing, C., Khalil, S. K., Usman, M., et al. (2020). Concentrations, pollution indices and health risk assessment of heavy metals in road dust from two urbanized cities of Pakistan: Comparing two sampling methods for heavy metals concentration. Sustainable Cities and Society, 53 , 101959. https://doi.org/10.1016/j.scs.2019.101959

Rahman, M. S., Khan, M. D. H., Jolly, Y. N., Kabir, J., Akter, S., & Salam, A. (2019). Assessing risk to human health for heavy metal contamination through street dust in the Southeast Asian Megacity: Dhaka, Bangladesh. Science of the Total Environment, 660 , 1610–1622. https://doi.org/10.1016/j.scitotenv.2018.12.425

Rajaram, B. S., Suryawanshi, P. V., Bhanarkar, A. D., & Rao, C. V. C. (2014). Heavy metals contamination in road dust in Delhi city, India. Environmental Earth Sciences, 72 , 3929–3938. https://doi.org/10.1007/s12665-014-3281-y

Roy, S., Gupta, S. K., Prakash, J., Habib, G., & Kumar, P. (2022). A global perspective of the current state of heavy metal contamination in road dust. Environmental Science and Pollution Research, 1–22. https://doi.org/10.1007/s11356-022-18583-7

Rudnick, R. L., & Gao, S. (2003). Vol. 3: The Crust, 3.01–The Composition of the Continental Crust (pp. 1–64). Treatise on Geochemistry.

Google Scholar  

Saeedi, M., Li, L. Y., & Salmanzadeh, M. (2012). Heavy metals and polycyclic aromatic hydrocarbons: pollution and ecological risk assessment in street dust of Tehran. Journal of Hazardous Materials, 227 , 9–17. https://doi.org/10.1016/j.jhazmat.2012.04.047

Shabanda, I. S., Koki, I. B., Low, K. H., Zain, S. M., Khor, S. M., & Abu Bakar, N. K. (2019). Daily exposure to toxic metals through urban road dust from industrial, commercial, heavy traffic, and residential areas in Petaling Jaya, Malaysia: a health risk assessment. Environmental Science and Pollution Research, 26 , 37193–37211. https://doi.org/10.1007/s11356-019-06718-2

Shen, M. H., Dong, W. J., Wang, M. L., Yang, S., Yang, T. F., Tang, L. F., et al. (2018). Pollution characteristics of heavy metals in road dust and its relationship with road levels: Taking Beijing and Zhengzhou as examples. Environmental Chemistry, 37 (5), 942–951.

CAS   Google Scholar  

Silva, H. F., Silva, N. F., Oliveira, C. M., & Matos, M. J. (2021). Heavy metals contamination of urban soils—A decade study in the city of lisbon, portugal. Soil Systems, 5 (2), 27. https://doi.org/10.3390/soilsystems5020027

Tang, R., Ma, K., Zhang, Y., & Mao, Q. (2013). The spatial characteristics and pollution levels of metals in urban street dust of Beijing, China. Applied Geochemistry, 35 , 88–98. https://doi.org/10.1016/j.apgeochem.2013.03.016

Trujillo-González, J. M., Torres-Mora, M. A., Keesstra, S., Brevik, E. C., & Jiménez-Ballesta, R. (2016). Heavy metal accumulation related to population density in road dust samples taken from urban sites under different land uses. Science of the Total Environment, 553 , 636–642. https://doi.org/10.1016/j.scitotenv.2016.02.101

TUİK (2022) Address-based population registration system results. Retrieved November 20, 2023. https://data.tuik.gov.tr/Bulten/Index?p=49685

Urrutia-Goyes, R., Hernandez, N., Carrillo-Gamboa, O., Nigam, K. D. P., & Ornelas-Soto, N. (2018). Street dust from a heavily-populated and industrialized city: Evaluation of spatial distribution, origins, pollution, ecological risks and human health repercussions. Ecotoxicology and Environmental Safety, 159 , 198–204. https://doi.org/10.1016/j.ecoenv.2018.04.054

US EPA. (1989). Risk assessment guidance for superfund Human Health Evaluation Manual. Office of Soild Waste and Emergency Response, 1 .

US EPA. (2001). Risk Assessment Guidance for Superfund: Volume III-Part A, Process for Conducting Probabilistic Risk Assessment Office of Emergency and Remedial Response . U.S. Environmental Protection Agency.

US EPA (2002). Supplemental Guidance for Developing Soil Screening Levels for Superfund Sites

US EPA. (2004). Risk Assessment Guidance for Superfund Volume I: Human Health Evaluation Manual (Part E, Supplemental Guidance for Dermal Risk Assessment) . Office of Superfund Remediation and Technology Innovation.

US EPA. (2007). Estimation of Relative Bioavailablity of Lead in Soil and Soil-like Materials Using In Vivo and In Vitro Methods . OSWER 9285.7–77 Office of Solid Waste and Emergency Response, U.S. Environmental Protection Agency.

US EPA. (2009). Risk Assessment Guidance for Superfund Volume I: Human Health Evaluation Manual Office of Superfund Remediation and Technology Innovation . Washington.

US EPA. (2011). Exposure factors handbook National Center for Environmental Assessment . Office of Research and Development.

Wahab, M. I. A., Abd Razak, W. M. A., Sahani, M., & Khan, M. F. (2020). Characteristics and health effect of heavy metals on non-exhaust road dusts in Kuala Lumpur. Science of the Total Environment, 703 , 135535. https://doi.org/10.1016/j.scitotenv.2019.135535

Wang, G., Oldfield, F., Xia, D., Chen, F., Liu, X., & Zhang, W. (2012). Magnetic properties and correlation with heavy metals in urban street dust: a case study from the city of Lanzhou, China. Atmospheric Environment, 46 , 289–298. https://doi.org/10.1016/j.atmosenv.2011.09.059

Wang, M., Liu, R., Chen, W., Peng, C., & Markert, B. (2018). Effects of urbanization on heavy metal accumulation in surface soils, Beijing. Journal of Environmental Sciences, 64 , 328–334. https://doi.org/10.1016/j.jes.2016.11.026

Weber, N., Haase, D., & Franck, U. (2014). Assessing modelled outdoor traffic-induced noise and air pollution around urban structures using the concept of landscape metrics. Landscape and Urban Planning, 125 , 105–116. https://doi.org/10.1016/j.landurbplan.2014.02.018

Wei, X., Gao, B., Wang, P., Zhou, H., & Lu, J. (2015). Pollution characteristics and health risk assessment of heavy metals in street dusts from different functional areas in Beijing, China. Ecotoxicology and Environmental Safety, 112 , 186–192. https://doi.org/10.1016/j.ecoenv.2014.11.005

Wei L, Choy YS, Cheung CS, Chu HK (2021) Comparison of tribology performance, particle emissions and brake squeal noise between Cu-containing and Cu-free brake materials. Wear 466:203577. https://doi.org/10.1016/j.wear.2020.203577

Xie, T., Wang, M., Chen, W., & Uwizeyimana, H. (2019). Impacts of urbanization and landscape patterns on the accumulation of heavy metals in soils in residential areas in Beijing. Journal of Soils and Sediments, 19 , 148–158. https://doi.org/10.1007/s11368-018-2011-6

Yu, X., Zhang, W., Zhang, L., Li, V. O., Yuan, J., & You, I. (2013). Understanding urban dynamics based on pervasive sensing: An experimental study on traffic density and air pollution. Mathematical and Computer Modelling, 58 (5-6), 1328–1339. https://doi.org/10.1016/j.mcm.2013.01.002

Zeng, Y., Bi, C., Jia, J., Deng, L., & Chen, Z. (2020). Impact of intensive land use on heavy metal concentrations and ecological risks in an urbanized river network of Shanghai. Ecological Indicators, 116 , 106501. https://doi.org/10.1016/j.ecolind.2020.106501

Zgłobicki, W., Telecka, M., & Skupiński, S. (2019). Assessment of short-term changes in street dust pollution with heavy metals in Lublin (E Poland)—levels, sources and risks. Environmental Science and Pollution Research, 26 , 35049–35060. https://doi.org/10.1007/s11356-019-06496-x

Zhao, X., Zhou, W., & Han, L. (2021). The spatial and seasonal complexity of PM2. 5 pollution in cities from a social-ecological perspective. Journal of Cleaner Production, 309 , 127476. https://doi.org/10.1016/j.jclepro.2021.127476

Zheng, N., Liu, J., Wang, Q., & Liang, Z. (2010). Health risk assessment of heavy metal exposure to street dust in the zinc smelting district, Northeast of China. Science of the Total Environment, 408 (4), 726–733. https://doi.org/10.1016/j.scitotenv.2009.10.075

Download references

Acknowledgements

Not applicable.

Availability of Data and Materials

Not applicable

Open access funding provided by the Scientific and Technological Research Council of Türkiye (TÜBİTAK). There is no financial support and commercial support.

Author information

Authors and affiliations.

Department of Landscape Architecture, Faculty of Engineering and Architecture, Kastamonu University, 37150, Kastamonu, Türkiye

Oznur Isinkaralar

Department of Environmental Engineering, Faculty of Engineering and Architecture, Kastamonu University, 37150, Kastamonu, Türkiye

Kaan Isinkaralar

Department of Chemistry, National Institute of Technology, Jamshedpur, Jharkhand, 831014, India

Balram Ambade

You can also search for this author in PubMed   Google Scholar

Contributions

Oznur Isinkaralar: Processing analysis, data interpretation, writing original draft, data curation, processing analysis, and interpretation. Kaan Isinkaralar: Processing analysis, data interpretation, writing original draft, and data curation. Balram Ambade: Data interpretation, and review.

Corresponding author

Correspondence to Oznur Isinkaralar .

Ethics declarations

Competing interest.

The authors declare that they have no competing interests.

Ethics Approval

All authors have read, understood, and have complied as applicable with the statement on “Ethical responsibilities of Authors” as found in the Instructions for Authors.

Consent to Participate

Consent to publish, additional information, publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Isinkaralar, O., Isinkaralar, K. & Ambade, B. Assessment of Societal Health Risks: Spatial Distribution and Potential Hazards of Toxic Metals in Street Dust Across Diverse Communities. Water Air Soil Pollut 235 , 302 (2024). https://doi.org/10.1007/s11270-024-07104-6

Download citation

Received : 12 December 2023

Accepted : 19 April 2024

Published : 30 April 2024

DOI : https://doi.org/10.1007/s11270-024-07104-6

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Human exposure
• urban modeling
• spatial distribution
• land management
• Find a journal
• Publish with us
• Track your research

Help | Advanced Search

Computer Science > Computation and Language

Title: bored to death: artificial intelligence research reveals the role of boredom in suicide behavior.

Abstract: Background: Recent advancements in Artificial Intelligence (AI) contributed significantly to suicide assessment, however, our theoretical understanding of this complex behavior is still limited. Objective: This study aimed to harness AI methodologies to uncover hidden risk factors that trigger or aggravate suicide behaviors. Method: The primary dataset included 228,052 Facebook postings by 1,006 users who completed the gold-standard Columbia Suicide Severity Rating Scale. This dataset was analyzed using a bottom-up research pipeline without a-priory hypotheses and its findings were validated using a top-down analysis of a new dataset. This secondary dataset included responses by 1,062 participants to the same suicide scale as well as to well-validated scales measuring depression and boredom. Results: An almost fully automated, AI-guided research pipeline resulted in four Facebook topics that predicted the risk of suicide, of which the strongest predictor was boredom. A comprehensive literature review using APA PsycInfo revealed that boredom is rarely perceived as a unique risk factor of suicide. A complementing top-down path analysis of the secondary dataset uncovered an indirect relationship between boredom and suicide, which was mediated by depression. An equivalent mediated relationship was observed in the primary Facebook dataset as well. However, here, a direct relationship between boredom and suicide risk was also observed. Conclusions: Integrating AI methods allowed the discovery of an under-researched risk factor of suicide. The study signals boredom as a maladaptive 'ingredient' that might trigger suicide behaviors, regardless of depression. Further studies are recommended to direct clinicians' attention to this burdening, and sometimes existential experience.

Submission history

Access paper:.

• Other Formats

References & Citations

• Google Scholar
• Semantic Scholar

BibTeX formatted citation

Bibliographic and Citation Tools

Code, data and media associated with this article, recommenders and search tools.

• Institution

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs .