data analysis in experimental research

• Skip to main content
• Skip to primary sidebar
• Skip to footer
• QuestionPro

• Solutions Industries Gaming Automotive Sports and events Education Government Travel & Hospitality Financial Services Healthcare Cannabis Technology Use Case NPS+ Communities Audience Contactless surveys Mobile LivePolls Member Experience GDPR Positive People Science 360 Feedback Surveys
• Resources Blog eBooks Survey Templates Case Studies Training Help center

Home Market Research

Data Analysis in Research: Types & Methods

Content Index

Why analyze data in research?

Types of data in research, finding patterns in the qualitative data, methods used for data analysis in qualitative research, preparing data for analysis, methods used for data analysis in quantitative research, considerations in research data analysis, what is data analysis in research.

Definition of research in data analysis: According to LeCompte and Schensul, research data analysis is a process used by researchers to reduce data to a story and interpret it to derive insights. The data analysis process helps reduce a large chunk of data into smaller fragments, which makes sense. 

Three essential things occur during the data analysis process — the first is data organization . Summarization and categorization together contribute to becoming the second known method used for data reduction. It helps find patterns and themes in the data for easy identification and linking. The third and last way is data analysis – researchers do it in both top-down and bottom-up fashion.

LEARN ABOUT: Research Process Steps

On the other hand, Marshall and Rossman describe data analysis as a messy, ambiguous, and time-consuming but creative and fascinating process through which a mass of collected data is brought to order, structure and meaning.

We can say that “the data analysis and data interpretation is a process representing the application of deductive and inductive logic to the research and data analysis.”

Researchers rely heavily on data as they have a story to tell or research problems to solve. It starts with a question, and data is nothing but an answer to that question. But, what if there is no question to ask? Well! It is possible to explore data even without a problem – we call it ‘Data Mining’, which often reveals some interesting patterns within the data that are worth exploring.

Irrelevant to the type of data researchers explore, their mission and audiences’ vision guide them to find the patterns to shape the story they want to tell. One of the essential things expected from researchers while analyzing data is to stay open and remain unbiased toward unexpected patterns, expressions, and results. Remember, sometimes, data analysis tells the most unforeseen yet exciting stories that were not expected when initiating data analysis. Therefore, rely on the data you have at hand and enjoy the journey of exploratory research. 

Create a Free Account

Every kind of data has a rare quality of describing things after assigning a specific value to it. For analysis, you need to organize these values, processed and presented in a given context, to make it useful. Data can be in different forms; here are the primary data types.

• Qualitative data: When the data presented has words and descriptions, then we call it qualitative data . Although you can observe this data, it is subjective and harder to analyze data in research, especially for comparison. Example: Quality data represents everything describing taste, experience, texture, or an opinion that is considered quality data. This type of data is usually collected through focus groups, personal qualitative interviews , qualitative observation or using open-ended questions in surveys.
• Quantitative data: Any data expressed in numbers of numerical figures are called quantitative data . This type of data can be distinguished into categories, grouped, measured, calculated, or ranked. Example: questions such as age, rank, cost, length, weight, scores, etc. everything comes under this type of data. You can present such data in graphical format, charts, or apply statistical analysis methods to this data. The (Outcomes Measurement Systems) OMS questionnaires in surveys are a significant source of collecting numeric data.
• Categorical data: It is data presented in groups. However, an item included in the categorical data cannot belong to more than one group. Example: A person responding to a survey by telling his living style, marital status, smoking habit, or drinking habit comes under the categorical data. A chi-square test is a standard method used to analyze this data.

Learn More : Examples of Qualitative Data in Education

Data analysis in qualitative research

Data analysis and qualitative data research work a little differently from the numerical data as the quality data is made up of words, descriptions, images, objects, and sometimes symbols. Getting insight from such complicated information is a complicated process. Hence it is typically used for exploratory research and data analysis .

Although there are several ways to find patterns in the textual information, a word-based method is the most relied and widely used global technique for research and data analysis. Notably, the data analysis process in qualitative research is manual. Here the researchers usually read the available data and find repetitive or commonly used words. 

For example, while studying data collected from African countries to understand the most pressing issues people face, researchers might find  “food”  and  “hunger” are the most commonly used words and will highlight them for further analysis.

LEARN ABOUT: Level of Analysis

The keyword context is another widely used word-based technique. In this method, the researcher tries to understand the concept by analyzing the context in which the participants use a particular keyword.  

For example , researchers conducting research and data analysis for studying the concept of ‘diabetes’ amongst respondents might analyze the context of when and how the respondent has used or referred to the word ‘diabetes.’

The scrutiny-based technique is also one of the highly recommended  text analysis  methods used to identify a quality data pattern. Compare and contrast is the widely used method under this technique to differentiate how a specific text is similar or different from each other. 

For example: To find out the “importance of resident doctor in a company,” the collected data is divided into people who think it is necessary to hire a resident doctor and those who think it is unnecessary. Compare and contrast is the best method that can be used to analyze the polls having single-answer questions types .

Metaphors can be used to reduce the data pile and find patterns in it so that it becomes easier to connect data with theory.

Variable Partitioning is another technique used to split variables so that researchers can find more coherent descriptions and explanations from the enormous data.

LEARN ABOUT: Qualitative Research Questions and Questionnaires

There are several techniques to analyze the data in qualitative research, but here are some commonly used methods,

• Content Analysis:  It is widely accepted and the most frequently employed technique for data analysis in research methodology. It can be used to analyze the documented information from text, images, and sometimes from the physical items. It depends on the research questions to predict when and where to use this method.
• Narrative Analysis: This method is used to analyze content gathered from various sources such as personal interviews, field observation, and  surveys . The majority of times, stories, or opinions shared by people are focused on finding answers to the research questions.
• Discourse Analysis:  Similar to narrative analysis, discourse analysis is used to analyze the interactions with people. Nevertheless, this particular method considers the social context under which or within which the communication between the researcher and respondent takes place. In addition to that, discourse analysis also focuses on the lifestyle and day-to-day environment while deriving any conclusion.
• Grounded Theory:  When you want to explain why a particular phenomenon happened, then using grounded theory for analyzing quality data is the best resort. Grounded theory is applied to study data about the host of similar cases occurring in different settings. When researchers are using this method, they might alter explanations or produce new ones until they arrive at some conclusion.

LEARN ABOUT: 12 Best Tools for Researchers

Data analysis in quantitative research

The first stage in research and data analysis is to make it for the analysis so that the nominal data can be converted into something meaningful. Data preparation consists of the below phases.

Phase I: Data Validation

Data validation is done to understand if the collected data sample is per the pre-set standards, or it is a biased data sample again divided into four different stages

• Fraud: To ensure an actual human being records each response to the survey or the questionnaire
• Screening: To make sure each participant or respondent is selected or chosen in compliance with the research criteria
• Procedure: To ensure ethical standards were maintained while collecting the data sample
• Completeness: To ensure that the respondent has answered all the questions in an online survey. Else, the interviewer had asked all the questions devised in the questionnaire.

Phase II: Data Editing

More often, an extensive research data sample comes loaded with errors. Respondents sometimes fill in some fields incorrectly or sometimes skip them accidentally. Data editing is a process wherein the researchers have to confirm that the provided data is free of such errors. They need to conduct necessary checks and outlier checks to edit the raw edit and make it ready for analysis.

Phase III: Data Coding

Out of all three, this is the most critical phase of data preparation associated with grouping and assigning values to the survey responses . If a survey is completed with a 1000 sample size, the researcher will create an age bracket to distinguish the respondents based on their age. Thus, it becomes easier to analyze small data buckets rather than deal with the massive data pile.

LEARN ABOUT: Steps in Qualitative Research

After the data is prepared for analysis, researchers are open to using different research and data analysis methods to derive meaningful insights. For sure, statistical analysis plans are the most favored to analyze numerical data. In statistical analysis, distinguishing between categorical data and numerical data is essential, as categorical data involves distinct categories or labels, while numerical data consists of measurable quantities. The method is again classified into two groups. First, ‘Descriptive Statistics’ used to describe data. Second, ‘Inferential statistics’ that helps in comparing the data .

Descriptive statistics

This method is used to describe the basic features of versatile types of data in research. It presents the data in such a meaningful way that pattern in the data starts making sense. Nevertheless, the descriptive analysis does not go beyond making conclusions. The conclusions are again based on the hypothesis researchers have formulated so far. Here are a few major types of descriptive analysis methods.

Measures of Frequency

• Count, Percent, Frequency
• It is used to denote home often a particular event occurs.
• Researchers use it when they want to showcase how often a response is given.

Measures of Central Tendency

• Mean, Median, Mode
• The method is widely used to demonstrate distribution by various points.
• Researchers use this method when they want to showcase the most commonly or averagely indicated response.

Measures of Dispersion or Variation

• Range, Variance, Standard deviation
• Here the field equals high/low points.
• Variance standard deviation = difference between the observed score and mean
• It is used to identify the spread of scores by stating intervals.
• Researchers use this method to showcase data spread out. It helps them identify the depth until which the data is spread out that it directly affects the mean.

Measures of Position

• Percentile ranks, Quartile ranks
• It relies on standardized scores helping researchers to identify the relationship between different scores.
• It is often used when researchers want to compare scores with the average count.

For quantitative research use of descriptive analysis often give absolute numbers, but the in-depth analysis is never sufficient to demonstrate the rationale behind those numbers. Nevertheless, it is necessary to think of the best method for research and data analysis suiting your survey questionnaire and what story researchers want to tell. For example, the mean is the best way to demonstrate the students’ average scores in schools. It is better to rely on the descriptive statistics when the researchers intend to keep the research or outcome limited to the provided  sample  without generalizing it. For example, when you want to compare average voting done in two different cities, differential statistics are enough.

Descriptive analysis is also called a ‘univariate analysis’ since it is commonly used to analyze a single variable.

Inferential statistics

Inferential statistics are used to make predictions about a larger population after research and data analysis of the representing population’s collected sample. For example, you can ask some odd 100 audiences at a movie theater if they like the movie they are watching. Researchers then use inferential statistics on the collected  sample  to reason that about 80-90% of people like the movie. 

Here are two significant areas of inferential statistics.

• Estimating parameters: It takes statistics from the sample research data and demonstrates something about the population parameter.
• Hypothesis test: I t’s about sampling research data to answer the survey research questions. For example, researchers might be interested to understand if the new shade of lipstick recently launched is good or not, or if the multivitamin capsules help children to perform better at games.

These are sophisticated analysis methods used to showcase the relationship between different variables instead of describing a single variable. It is often used when researchers want something beyond absolute numbers to understand the relationship between variables.

Here are some of the commonly used methods for data analysis in research.

• Correlation: When researchers are not conducting experimental research or quasi-experimental research wherein the researchers are interested to understand the relationship between two or more variables, they opt for correlational research methods.
• Cross-tabulation: Also called contingency tables,  cross-tabulation  is used to analyze the relationship between multiple variables.  Suppose provided data has age and gender categories presented in rows and columns. A two-dimensional cross-tabulation helps for seamless data analysis and research by showing the number of males and females in each age category.
• Regression analysis: For understanding the strong relationship between two variables, researchers do not look beyond the primary and commonly used regression analysis method, which is also a type of predictive analysis used. In this method, you have an essential factor called the dependent variable. You also have multiple independent variables in regression analysis. You undertake efforts to find out the impact of independent variables on the dependent variable. The values of both independent and dependent variables are assumed as being ascertained in an error-free random manner.
• Frequency tables: The statistical procedure is used for testing the degree to which two or more vary or differ in an experiment. A considerable degree of variation means research findings were significant. In many contexts, ANOVA testing and variance analysis are similar.
• Analysis of variance: The statistical procedure is used for testing the degree to which two or more vary or differ in an experiment. A considerable degree of variation means research findings were significant. In many contexts, ANOVA testing and variance analysis are similar.
• Researchers must have the necessary research skills to analyze and manipulation the data , Getting trained to demonstrate a high standard of research practice. Ideally, researchers must possess more than a basic understanding of the rationale of selecting one statistical method over the other to obtain better data insights.
• Usually, research and data analytics projects differ by scientific discipline; therefore, getting statistical advice at the beginning of analysis helps design a survey questionnaire, select data collection methods , and choose samples.

LEARN ABOUT: Best Data Collection Tools

• The primary aim of data research and analysis is to derive ultimate insights that are unbiased. Any mistake in or keeping a biased mind to collect data, selecting an analysis method, or choosing  audience  sample il to draw a biased inference.
• Irrelevant to the sophistication used in research data and analysis is enough to rectify the poorly defined objective outcome measurements. It does not matter if the design is at fault or intentions are not clear, but lack of clarity might mislead readers, so avoid the practice.
• The motive behind data analysis in research is to present accurate and reliable data. As far as possible, avoid statistical errors, and find a way to deal with everyday challenges like outliers, missing data, data altering, data mining , or developing graphical representation.

LEARN MORE: Descriptive Research vs Correlational Research The sheer amount of data generated daily is frightening. Especially when data analysis has taken center stage. in 2018. In last year, the total data supply amounted to 2.8 trillion gigabytes. Hence, it is clear that the enterprises willing to survive in the hypercompetitive world must possess an excellent capability to analyze complex research data, derive actionable insights, and adapt to the new market needs.

LEARN ABOUT: Average Order Value

QuestionPro is an online survey platform that empowers organizations in data analysis and research and provides them a medium to collect data by creating appealing surveys.

MORE LIKE THIS

I Am Disconnected – Tuesday CX Thoughts

May 21, 2024

20 Best Customer Success Tools of 2024

May 20, 2024

AI-Based Services Buying Guide for Market Research (based on ESOMAR’s 20 Questions) 

Data Information vs Insight: Essential differences

May 14, 2024

Other categories

• Academic Research
• Artificial Intelligence
• Assessments
• Brand Awareness
• Case Studies
• Communities
• Consumer Insights
• Customer effort score
• Customer Engagement
• Customer Experience
• Customer Loyalty
• Customer Research
• Customer Satisfaction
• Employee Benefits
• Employee Engagement
• Employee Retention
• Friday Five
• General Data Protection Regulation
• Insights Hub
• Life@QuestionPro
• Market Research
• Mobile diaries
• Mobile Surveys
• New Features
• Online Communities
• Question Types
• Questionnaire
• QuestionPro Products
• Release Notes
• Research Tools and Apps
• Revenue at Risk
• Survey Templates
• Training Tips
• Uncategorized
• Video Learning Series
• What’s Coming Up
• Workforce Intelligence

Effective Experiment Design and Data Analysis in Transportation Research (2012)

Chapter: chapter 3 - examples of effective experiment design and data analysis in transportation research.

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

10 Examples of Effective Experiment Design and Data Analysis in Transportation Research About this Chapter This chapter provides a wide variety of examples of research questions. The examples demon- strate varying levels of detail with regard to experiment designs and the statistical analyses required. The number and types of examples were selected after consulting with many practitioners. The attempt was made to provide a couple of detailed examples in each of several areas of transporta- tion practice. For each type of problem or analysis, some comments also appear about research topics in other areas that might be addressed using the same approach. Questions that were briefly introduced in Chapter 2 are addressed in considerably more depth in the context of these examples. All the examples are organized and presented using the outline below. Where applicable, ref- erences to the two-volume primer produced under NCHRP Project 20-45 have been provided to encourage the reader to obtain more detail about calculation techniques and more technical discussion of issues. Basic Outline for Examples The numbered outline below is the model for the structure of all of the examples that follow. 1. Research Question/Problem Statement: A simple statement of the research question is given. For example, in the maintenance category, does crack sealant A perform better than crack sealant B? 2. Identification and Description of Variables: The dependent and independent variables are identified and described. The latter includes an indication of whether, for example, the variables are discrete or continuous. 3. Data Collection: A hypothetical scenario is presented to describe how, where, and when data should be collected. As appropriate, reference is made to conventions or requirements for some types of data (e.g., if delay times at an intersection are being calculated before and after some treatment, the data collected need to be consistent with the requirements in the Highway Capacity Manual). Typical problems are addressed, such as sample size, the need for control groups, and so forth. 4. Specification of Analysis Technique and Data Analysis: The links between successfully framing the research question, fully describing the variables that need to be considered, and the specification of the appropriate analysis technique are highlighted in each example. Refer- ences to NCHRP Project 20-45 are provided for additional detail. The appropriate types of statistical test(s) are described for the specific example. 5. Interpreting the Results: In each example, results that can be expected from the analysis are discussed in terms of what they mean from a statistical perspective (e.g., the t-test result from C h a p t e r 3

examples of effective experiment Design and Data analysis in transportation research 11 a comparison of means indicates whether the mean values of two distributions can be con- sidered to be equal with a specified degree of confidence) as well as an operational perspective (e.g., judging whether the difference is large enough to make an operational difference). In each example, the typical results and their limitations are discussed. 6. Conclusion and Discussion: This section recaps how the early steps in the process lead directly to the later ones. Comments are made regarding how changes in the early steps can affect not only the results of the analysis but also the appropriateness of the approach. 7. Applications in Other Areas of Transportation Research: Each example includes a short list of typical applications in other areas of transportation research for which the approach or analysis technique would be appropriate. Techniques Covered in the Examples The determination of what kinds of statistical techniques to include in the examples was made after consulting with a variety of professionals and examining responses to a survey of research- oriented practitioners. The examples are not exhaustive insofar as not every type of statistical analysis is covered. However, the attempt has been made to cover a representative sample of tech- niques that the practitioner is most likely to encounter in undertaking or supervising research- oriented projects. The following techniques are introduced in one or more examples: • Descriptive statistics • Fitting distributions/goodness of fit (used in one example) • Simple one- and two-sample comparison of means • Simple comparisons of multiple means using analysis of variance (ANOVA) • Factorial designs (also ANOVA) • Simple comparisons of means before and after some treatment • Complex before-and-after comparisons involving control groups • Trend analysis • Regression • Logit analysis (used in one example) • Survey design and analysis • Simulation • Non-parametric methods (used in one example) Although the attempt has been made to make the examples as readable as possible, some tech- nical terms may be unfamiliar to some readers. Detailed definitions for most applicable statistical terms are available in the glossary in NCHRP Project 20-45, Volume 2, Appendix A. Most defini- tions used here are consistent with those contained in NCHRP Project 20-45, which contains useful information for everyone from the beginning researcher to the most accomplished statistician. Some variations appear in the notations used in the examples. For example, in statistical analy- sis an alternate hypothesis may be represented by Ha or by H1, and readers will find both notations used in this report. The examples were developed by several authors with differing backgrounds, and latitude was deliberately given to the authors to use the notations with which they are most familiar. The variations have been included purposefully to acquaint readers with the fact that the same concepts (e.g., something as simple as a mean value) may be noted in various ways by different authors or analysts. Finally, the more widely used techniques, such as analysis of variance (ANOVA), are applied in more than one example. Readers interested in ANOVA are encouraged to read all the ANOVA examples as each example presents different aspects of or perspectives on the approach, and computational techniques presented in one example may not be repeated in later examples (although a citation typically is provided).

12 effective experiment Design and Data analysis in transportation research Areas Covered in the Examples Transportation research is very broad, encompassing many fields. Based on consultation with many research-oriented professionals and a survey of practitioners, key areas of research were identified. Although these areas have lots of overlap, explicit examples in the following areas are included: • Construction • Environment • Lab testing and instrumentation • Maintenance • Materials • Pavements • Public transportation • Structures/bridges • Traffic operations • Traffic safety • Transportation planning • Work zones The 21 examples provided on the following pages begin with the most straightforward ana- lytical approaches (i.e., descriptive statistics) and progress to more sophisticated approaches. Table 1 lists the examples along with the area of research and method of analysis for each example. Example 1: Structures/Bridges; Descriptive Statistics Area: Structures/bridges Method of Analysis: Descriptive statistics (exploring and presenting data to describe existing conditions and develop a basis for further analysis) 1. Research Question/Problem Statement: An engineer for a state agency wants to determine the functional and structural condition of a select number of highway bridges located across the state. Data are obtained for 100 bridges scheduled for routine inspection. The data will be used to develop bridge rehabilitation and/or replacement programs. The objective of this analysis is to provide an overview of the bridge conditions, and to present various methods to display the data in a concise and meaningful manner. Question/Issue Use collected data to describe existing conditions and prepare for future analysis. In this case, bridge inspection data from the state are to be studied and summarized. 2. Identification and Description of Variables: Bridge inspection generally entails collection of numerous variables that include location information, traffic data, structural elements’ type and condition, and functional characteristics. In this example, the variables are: bridge condition ratings of the deck, superstructure, and substructure; and overall condition of the bridge. Based on the severity of deterioration and the extent of spread through a bridge component, a condition rating is assigned on a discrete scale from 0 (failed) to 9 (excellent). These ratings (in addition to several other factors) are used in categorization of a bridge in one of three overall conditions: not deficient; structurally deficient; or functionally obsolete.

examples of effective experiment Design and Data analysis in transportation research 13 Example Area Method of Analysis 1 Structures/bridges Descriptive statistics (exploring and presenting data to describe existing conditions) 2 Public transport Descriptive statistics (organizing and presenting data to describe a system or component) 3 Environment Descriptive statistics (organizing and presenting data to explain current conditions) 4 Traffic operations Goodness of fit (chi-square test; determining if observed/collected data fit a certain distribution) 5 Construction Simple comparisons to specified values (t-test to compare the mean value of a small sample to a standard or other requirement) 6 Maintenance Simple two-sample comparison (t-test for paired comparisons; comparing the mean values of two sets of matched data) 7 Materials Simple two-sample comparisons (t-test for paired comparisons and the F-test for comparing variances) 8 Laboratory testing and/or instrumentation Simple ANOVA (comparing the mean values of more than two samples using the F-test) 9 Materials Simple ANOVA (comparing more than two mean values and the F-test for equality of means) 10 Pavements Simple ANOVA (comparing the mean values of more than two samples using the F-test) 11 Pavements Factorial design (an ANOVA approach exploring the effects of varying more than one independent variable) 12 Work zones Simple before-and-after comparisons (exploring the effect of some treatment before it is applied versus after it is applied) 13 Traffic safety Complex before-and-after comparisons using control groups (examining the effect of some treatment or application with consideration of other factors) 14 Work zones Trend analysis (examining, describing, and modeling how something changes over time) 15 Structures/bridges Trend analysis (examining a trend over time) 16 Transportation planning Multiple regression analysis (developing and testing proposed linear models with more than one independent variable) 17 Traffic operations Regression analysis (developing a model to predict the values that a dependent variable can take as a function of one or more independent variables) 18 Transportation planning Logit and related analysis (developing predictive models when the dependent variable is dichotomous) 19 Public transit Survey design and analysis (organizing survey data for statistical analysis) 20 Traffic operations Simulation (using field data to simulate or model operations or outcomes) 21 Traffic safety Non-parametric methods (methods to be used when data do not follow assumed or conventional distributions) Table 1. Examples provided in this report.

14 effective experiment Design and Data analysis in transportation research 3. Data Collection: Data are collected at 100 scheduled locations by bridge inspectors. It is important to note that the bridge condition rating scale is based on subjective categories, and there may be inherent variability among inspectors in their assignment of ratings to bridge components. A sample of data is compiled to document the bridge condition rating of the three primary structural components and the overall condition by location and ownership (Table 2). Notice that the overall condition of a bridge is not necessarily based only on the condition rating of its components (e.g., they cannot just be added). 4. Specification of Analysis Technique and Data Analysis: The two primary variables of inter- est are bridge condition rating and overall condition. The overall condition of the bridge is a categorical variable with three possible values: not deficient; structurally deficient; and functionally obsolete. The frequencies of these values in the given data set are calculated and displayed in the pie chart below. A pie chart provides a visualization of the relative proportions of bridges falling into each category that is often easier to communicate to the reader than a table showing the same information (Figure 1). Another way to look at the overall bridge condition variable is by cross-tabulation of the three condition categories with the two location categories (urban and rural), as shown in Table 3. A cross-tabulation provides the joint distribution of two (or more) variables such that each cell represents the frequency of occurrence of a specific combination of pos- sible values. For example, as seen in Table 3, there are 10 structurally deficient bridges in rural areas, which represent 11.4% of all rural area bridges inspected. The numbers in the parentheses are column percentages and add up to 100%. Table 3 also shows that 88 of the bridges inspected were located in rural areas, whereas 12 were located in urban areas. The mean values of the bridge condition rating variable for deck, superstructure, and sub- structure are shown in Table 4. These have been calculated by taking the sum of all the values and then dividing by the total number of cases (100 in this example). Generally, a condition rating Bridge No. Owner Location Bridge Condition Rating Overall Condition Deck Superstructure Substructure 1 State Rural 8 8 8 ND* 7 Local agency Rural 6 6 6 FO* 39 State Urban 6 6 2 SD* 69 State park Rural 7 5 5 SD 92 City Urban 5 6 6 ND *ND = not deficient; FO: functionally obsolete; SD: structurally deficient. Table 2. Sample bridge inspection data. Structurally Deficient (SD), 13% Functionally Obsolete (FO), 10% Neither SD/FO, 77% Figure 1. Highway bridge conditions.

examples of effective experiment Design and Data analysis in transportation research 15 of 4 or below indicates deficiency in a structural component. For the purpose of comparison, the mean bridge condition rating of the 13 structurally deficient bridges also is provided. Notice that while the rating scale for the bridge conditions is discrete with values ranging from 0 (failure) to 9 (excellent), the average bridge condition variable is continuous. Therefore, an average score of 6.47 would indicate overall condition of all bridges to be between 6 (satisfactory) and 7 (good). The combined bridge condition rating of deck, superstructure, and substructure is not defined; therefore calculating the mean of the three components’ average rating would make no sense. Also, the average bridge condition rating of functionally obsolete bridges is not calculated because other functional characteristics also accounted for this designation. The distributions of the bridge condition ratings for deck, superstructure, and substructure are shown in Figure 2. Based on the cut-off point of 4, approximately 7% of all bridge decks, 2% of all superstructures, and 5% of all substructures are deficient. 5. Interpreting the Results: The results indicate that a majority of bridges (77%) are not struc- turally or functionally deficient. The inspections were carried out on bridges primarily located in rural areas (88 out of 100). The bridge condition variable may also be cross-tabulated with the ownership variable to determine distribution by jurisdiction. The average condition ratings for the three bridge components for all bridges lies between 6 (satisfactory, some minor problems) and 7 (good, no problems noted). 6. Conclusion and Discussion: This example illustrates how to summarize and present quan- titative and qualitative data on bridge conditions. It is important to understand the mea- surement scale of variables in order to interpret the results correctly. Bridge inspection data collected over time may also be analyzed to determine trends in the condition of bridges in a given area. Trend analysis is addressed in Example 15 (structures). 7. Applications in Other Areas of Transportation Research: Descriptive statistics could be used to present data in other areas of transportation research, such as: • Transportation Planning—to assess the distribution of travel times between origin- destination pairs in an urban area. Overall averages could also be calculated. • Traffic Operations—to analyze the average delay per vehicle at a railroad crossing. Rating Category Mean Value Overall average bridge condition rating (deck) 6.20 Overall average bridge condition rating (superstructure) 6.47 Overall average bridge condition rating (substructure) 6.08 Average bridge condition rating of structurally deficient bridges (deck) 4.92 Average bridge condition rating of structurally deficient bridges (superstructure) 5.30 Average bridge condition rating of structurally deficient bridges (substructure) 4.54 Table 4. Bridge condition ratings. Rural Urban Total Structurally deficient 10 (11.4%) 3 (25.0%) 13 Functionally obsolete 6 (6.8%) 4 (33.3%) 10 Not deficient 72 (81.8%) 5 (41.7%) 77 Total 88 (100%) 12 (100%) 100 Table 3. Cross-tabulation of bridge condition by location.

16 effective experiment Design and Data analysis in transportation research • Traffic Operations/Safety—to examine the frequency of turning violations at driveways with various turning restrictions. • Work Zones, Environment—to assess the average energy consumption during various stages of construction. Example 2: Public Transport; Descriptive Statistics Area: Public transport Method of Analysis: Descriptive statistics (organizing and presenting data to describe a system or component) 1. Research Question/Problem Statement: The manager of a transit agency would like to present information to the board of commissioners on changes in revenue that resulted from a change in the fare. The transit system provides three basic types of service: local bus routes, express bus routes, and demand-responsive bus service. There are 15 local bus routes, 10 express routes, and 1 demand-responsive system. 0 5 10 15 20 25 30 35 40 45 9 8 7 6 5 4 3 2 1 0 Condition Ratings Pe rc en ta ge o f S tru ctu re s Deck Superstructure Substructure Figure 2. Bridge condition ratings. Question/Issue Use data to describe some change over time. In this instance, data from 2008 and 2009 are used to describe the change in revenue on each route/part of a transit system when the fare structure was changed from variable (per mile) to fixed fares. 2. Identification and Description of Variables: Revenue data are available for each route on the local and express bus system and the demand-responsive system as a whole for the years 2008 and 2009. 3. Data Collection: Revenue data were collected on each route for both 2008 and 2009. The annual revenue for the demand-responsive system was also collected. These data are shown in Table 5. 4. Specification of Analysis Technique and Data Analysis: The objective of this analysis is to present the impact of changing the fare system in a series of graphs. The presentation is intended to show the impact on each component of the transit system as well as the impact on overall system revenue. The impact of the fare change on the overall revenue is best shown with a bar graph (Figure 3). The variation in the impact across system components can be illustrated in a similar graph (Figure 4). A pie chart also can be used to illustrate the relative impact on each system component (Figure 5).

examples of effective experiment Design and Data analysis in transportation research 17 Bus Route 2008 Revenue 2009 Revenue Local Route 1 $350,500 $365,700 Local Route 2 $263,000 $271,500 Local Route 3 $450,800 $460,700 Local Route 4 $294,300 $306,400 Local Route 5 $173,900 $184,600 Local Route 6 $367,800 $375,100 Local Route 7 $415,800 $430,300 Local Route 8 $145,600 $149,100 Local Route 9 $248,200 $260,800 Local Route 10 $310,400 $318,300 Local Route 11 $444,300 $459,200 Local Route 12 $208,400 $205,600 Local Route 13 $407,600 $412,400 Local Route 14 $161,500 $169,300 Local Route 15 $325,100 $340,200 Express Route 1 $85,400 $83,600 Express Route 2 $110,300 $109,200 Express Route 3 $65,800 $66,200 Express Route 4 $125,300 $127,600 Express Route 5 $90,800 $90,400 Express Route 6 $125,800 $123,400 Express Route 7 $87,200 $86,900 Express Route 8 $68.300 $67,200 Express Route 9 $110,100 $112,300 Express Route 10 $73,200 $72,100 Demand-Responsive System $510,100 $521,300 Table 5. Revenue by route or type of service and year. 6.02 6.17 0 1 2 3 4 5 6 7 8 2008 2009 Total System Revenue Re ve nu e (M illi on $ ) Figure 3. Impact of fare change on overall revenue.

18 effective experiment Design and Data analysis in transportation research Express Buses, 15.7% Express Buses, 15.2% Local Buses, 76.3% Local Buses, 75.8% Demand Responsive, 8.5% Demand Responsive, 8.5% 2008 2009 Figure 5. Pie charts illustrating percent of revenue from each component of a transit system. If it is important to display the variability in the impact within the various bus routes in the local bus or express bus operations, this also can be illustrated (Figure 6). This type of diagram shows the maximum value, minimum value, and mean value of the percent increase in revenue across the 15 local bus routes and the 10 express bus routes. 5. Interpreting the results: These results indicate that changing from a variable fare based on trip length (2008) to a fixed fare (2009) on both the local bus routes and the express bus routes had little effect on revenue. On the local bus routes, there was an average increase in revenue of 3.1%. On the express bus routes, there was an average decrease in revenue of 0.4%. These changes altered the percentage of the total system revenue attributed to the local bus routes and the express bus routes. The local bus routes generated 76.3% of the revenue in 2009, compared to 75.8% in 2008. The percentage of revenue generated by the express bus routes dropped from 15.7% to 15.2%, and the demand-responsive system generated 8.5% in both 2008 and 2009. 6. Conclusion and Discussion: The total revenue increased from $6.02 million to $6.17 mil lion. The cost of operating a variable fare system is greater than that of operating a fixed fare system— hence, net income probably increased even more (more revenue, lower cost for fare collection), and the decision to modify the fare system seems reasonable. Notice that the entire discussion Figure 4. Variation in impact of fare change across system components. 0.94 0.51 0.94 0.52 4.57 4.71 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 5.5 6 Local Buses Express Buses Demand Responsive Re ve nu e (M illi on $ ) 2008 2009

examples of effective experiment Design and Data analysis in transportation research 19 also is based on the assumption that no other factors changed between 2008 and 2009 that might have affected total revenues. One of the implicit assumptions is that the number of riders remained relatively constant from 1 year to the next. If the ridership had changed, the statistics reported would have to be changed. Using the measure revenue/rider, for example, would help control (or normalize) for the variation in ridership. 7. Applications in Other Areas in Transportation Research: Descriptive statistics are widely used and can convey a great deal of information to a reader. They also can be used to present data in many areas of transportation research, including: • Transportation Planning—to display public response frequency or percentage to various alternative designs. • Traffic Operations—to display the frequency or percentage of crashes by route type or by the type of traffic control devices present at an intersection. • Airport Engineering—to display the arrival pattern of passengers or flights by hour or other time period. • Public Transit—to display the average load factor on buses by time of day. Example 3: Environment; Descriptive Statistics Area: Environment Method of Analysis: Descriptive statistics (organizing and presenting data to explain current conditions) 1. Research Question/Problem Statement: The planning and programming director in Envi- ronmental City wants to determine the current ozone concentration in the city. These data will be compared to data collected after the projects included in the Transportation Improvement Program (TIP) have been completed to determine the effects of these projects on the environ- ment. Because the terrain, the presence of hills or tall buildings, the prevailing wind direction, and the sample station location relative to high volume roads or industrial sites all affect the ozone level, multiple samples are required to determine the ozone concentration level in a city. For this example, air samples are obtained each weekday in the month of July (21 days) at 14 air-sampling stations in the city: 7 in the central city and 7 in the outlying areas of the city. The objective of the analysis is to determine the ozone concentration in the central city, the outlying areas of the city, and the city as a whole. Figure 6. Graph showing variation in revenue increase by type of bus route. -0.4 -1.3 -2.1 3.1 6.2 2.0 -3 -2 -1 0 1 2 3 4 5 6 7 Local Bus Routes Express Bus Routes Percent Increase in Revenue

20 effective experiment Design and Data analysis in transportation research 2. Identification and Description of Variables: The variable to be analyzed is the 8-hour average ozone concentration in parts per million (ppm) at each of the 14 air-sampling stations. The 8-hour average concentration is the basis for the EPA standard, and July is selected because ozone levels are temperature sensitive and increase with a rise in the temperature. 3. Data Collection: Ozone concentrations in ppm are recorded for each hour of the day at each of the 14 air-sampling stations. The highest average concentration for any 8-hour period during the day is recorded and tabulated. This results in 294 concentration observations (14 stations for 21 days). Table 6 and Table 7 show the data for the seven central city locations and the seven outlying area locations. 4. Specification of Analysis Technique and Data Analysis: Much of the data used in analyzing transportation issues has year-to-year, month-to-month, day-to-day, and even hour-to-hour variations. For this reason, making only one observation, or even a few observations, may not accurately describe the phenomenon being observed. Thus, standard practice is to obtain several observations and report the mean value of all observations. In this example, the phenomenon being observed is the daily ozone concentration at a series of air-sampling locations. The statistic to be estimated is the mean value of this variable over Question/Issue Use collected data to describe existing conditions and prepare for future analysis. In this example, air pollution levels in the central city, the outlying areas, and the overall city are to be described. Day Station 1 2 3 4 5 6 7 ∑ 1 0.079 0.084 0.081 0.083 0.088 0.086 0.089 0.590 2 0.082 0.087 0.088 0.086 0.086 0.087 0.081 0.597 3 0.080 0.081 0.077 0.072 0.084 0.083 0.081 0.558 4 0.083 0.086 0.082 0.079 0.086 0.087 0.089 0.592 5 0.082 0.087 0.080 0.075 0.090 0.089 0.085 0.588 6 0.075 0.084 0.079 0.076 0.080 0.083 0.081 0.558 7 0.078 0.079 0.080 0.074 0.078 0.080 0.075 0.544 8 0.081 0.077 0.082 0.081 0.076 0.079 0.074 0.540 9 0.088 0.084 0.083 0.085 0.083 0.083 0.088 0.594 10 0.085 0.087 0.086 0.089 0.088 0.087 0.090 0.612 11 0.079 0.082 0.082 0.089 0.091 0.089 0.090 0.602 12 0.078 0.080 0.081 0.086 0.088 0.089 0.089 0.591 13 0.081 0.079 0.077 0.083 0.084 0.085 0.087 0.576 14 0.083 0.080 0.079 0.081 0.080 0.082 0.083 0.568 15 0.084 0.083 0.080 0.085 0.082 0.086 0.085 0.585 16 0.086 0.087 0.085 0.087 0.089 0.090 0.089 0.613 17 0.082 0.085 0.083 0.090 0.087 0.088 0.089 0.604 18 0.080 0.081 0.080 0.087 0.085 0.086 0.088 0.587 19 0.080 0.083 0.077 0.083 0.085 0.084 0.087 0.579 20 0.081 0.084 0.079 0.082 0.081 0.083 0.088 0.578 21 0.082 0.084 0.080 0.081 0.082 0.083 0.085 0.577 ∑ 1.709 1.744 1.701 1.734 1.773 1.789 1.793 12.243 Table 6. Central city 8-hour ozone concentration samples (ppm).

examples of effective experiment Design and Data analysis in transportation research 21 the test period selected. The mean value of any data set (x _ ) equals the sum of all observations in the set divided by the total number of observations in the set (n): x x n i i n = = ∑ 1 The variables of interest stated in the research question are the average ozone concentration for the central city, the outlying areas, and the total city. Thus, there are three data sets: the first table, the second table, and the sum of the two tables. The first data set has a sample size of 147; the second data set also has a sample size of 147, and the third data set contains 294 observations. Using the formula just shown, the mean value of the ozone concentration in the central city is calculated as follows: x xi i = = = = ∑ 147 12 243 147 0 083 1 147 . . ppm The mean value of the ozone concentration in the outlying areas of the city is: x xi i = = = = ∑ 147 10 553 147 0 072 1 147 . . ppm The mean value of the ozone concentration for the entire city is: x xi i = = = = ∑ 294 22 796 294 0 078 1 294 . . ppm Day Station 8 9 10 11 12 13 14 ∑ 1 0.072 0.074 0.073 0.071 0.079 0.070 0.074 0.513 2 0.074 0.075 0.077 0.075 0.081 0.075 0.077 0.534 3 0.070 0.072 0.074 0.074 0.083 0.078 0.080 0.531 4 0.067 0.070 0.071 0.077 0.080 0.077 0.081 0.523 5 0.064 0.067 0.068 0.072 0.079 0.078 0.079 0.507 6 0.069 0.068 0.066 0.070 0.075 0.079 0.082 0.509 7 0.071 0.069 0.070 0.071 0.074 0.071 0.077 0.503 8 0.073 0.072 0.074 0.072 0.076 0.073 0.078 0.518 9 0.072 0.075 0.077 0.074 0.078 0.074 0.080 0.530 10 0.074 0.077 0.079 0.077 0.080 0.076 0.079 0.542 11 0.070 0.072 0.075 0.074 0.079 0.074 0.078 0.522 12 0.068 0.067 0.068 0.070 0.074 0.070 0.075 0.492 13 0.065 0.063 0.067 0.068 0.072 0.067 0.071 0.473 14 0.063 0.062 0.067 0.069 0.073 0.068 0.073 0.475 15 0.064 0.064 0.066 0.067 0.070 0.066 0.070 0.467 16 0.061 0.059 0.062 0.062 0.067 0.064 0.069 0.434 17 0.065 0.061 0.060 0.064 0.069 0.066 0.073 0.458 18 0.067 0.063 0.065 0.068 0.073 0.069 0.076 0.499 19 0.069 0.067 0.068 0.072 0.077 0.071 0.078 0.502 20 0.071 0.069 0.070 0.074 0.080 0.074 0.077 0.515 21 0.070 0.065 0.072 0.076 0.079 0.073 0.079 0.514 ∑ 1.439 1.431 1.409 1.497 1.598 1.513 1.606 10.553 Table 7. Outlying area 8-hour ozone concentration samples (ppm).

22 effective experiment Design and Data analysis in transportation research Using the same equation, the mean value for each air-sampling location can be found by summing the value of the ozone concentration in the column representing that location and dividing by the 21 observations at that location. For example, considering Sample Station 1, the mean value of the ozone concentration is 1.709/21 = 0.081 ppm. Similarly, the mean value of the ozone concentrations for any specific day can be found by summing the ozone concentration values in the row representing that day and dividing by the number of stations. For example, for Day 1, the mean value of the ozone concentration in the central city is 0.590/7=0.084. In the outlying areas of the city, it is 0.513/7=0.073, and for the entire city it is 1.103/14=0.079. The highest and lowest values of the ozone concentration can be obtained by searching the two tables. The highest ozone concentration (0.091 ppm) is logged as having occurred at Station 5 on Day 11. The lowest ozone concentration (0.059 ppm) occurred at Station 9 on Day 16. The variation by sample location can be illustrated in the form of a frequency diagram. A graph can be used to show the variation in the average ozone concentration for the seven sample stations in the central city (Figure 7). Notice that all of these calculations (and more) can be done very easily if all the data are put in a spreadsheet and various statistical functions used. Graphs and other displays also can be made within the spreadsheet. 5. Interpreting the Results: In this example, the data are not tested to determine whether they fit a known distribution or whether one average value is significantly higher or lower than another. It can only be reported that, as recorded in July, the mean ozone concentration in the central city was greater than the concentration in the outlying areas of the city. (For testing to see whether the data fit a known distribution or comparing mean values, see Example 4 on fitting distribu- tions and goodness of fit. For comparing mean values, see examples 5 through 7.) It is known that ozone concentration varies by day and by location of the air-sampling equipment. If there is some threshold value of importance, such as the ozone concentration level considered acceptable by the EPA, these data could be used to determine the number of days that this level was exceeded, or the number of stations that recorded an ozone concentration above this threshold. This is done by comparing each day or each station with the threshold 0.081 0.083 0.081 0.083 0.084 0.085 0.085 0.070 0.072 0.074 0.076 0.078 0.080 0.082 0.084 0.086 1 2 3 4 5 6 7 Station A ve ra ge o zo ne c on ce nt ra tio n Figure 7. Average ozone concentration for seven central city sampling stations (ppm).

examples of effective experiment Design and Data analysis in transportation research 23 value. It must be noted that, as presented, this example is not a statistical comparison per se (i.e., there has been no significance testing or formal statistical comparison). 6. Conclusion and Discussion: This example illustrates how to determine and present quanti- tative information about a data set containing values of a varying parameter. If a similar set of data were captured each month, the variation in ozone concentration could be analyzed to describe the variation over the year. Similarly, if data were captured at these same locations in July of every year, the trend in ozone concentration over time could be determined. 7. Applications in Other Areas in Transportation: These descriptive statistics techniques can be used to present data in other areas of transportation research, such as: • Traffic Operations/Safety and Transportation Planning – to analyze the average speed of vehicles on streets with a speed limit of 45 miles per hour (mph) in residential, commercial, and industrial areas by sampling a number of streets in each of these area types. – to examine the average emergency vehicle response time to various areas of the city or county, by analyzing dispatch and arrival times for emergency calls to each area of interest. • Pavement Engineering—to analyze the average number of potholes per mile on pavement as a function of the age of pavement, by sampling a number of streets where the pavement age falls in discrete categories (0 to 5 years, 5 to 10 years, 10 to 15 years, and greater than 15 years). • Traffic Safety—to evaluate the average number of crashes per month at intersections with two-way STOP control versus four-way STOP control by sampling a number of intersections in each category over time. Example 4: Traffic Operations; Goodness of Fit Area: Traffic operations Method of Analysis: Goodness of fit (chi-square test; determining if observed distributions of data fit hypothesized standard distributions) 1. Research Question/Problem Statement: A research team is developing a model to estimate travel times of various types of personal travel (modes) on a path shared by bicyclists, in-line skaters, and others. One version of the model relies on the assertion that the distribution of speeds for each mode conforms to the normal distribution. (For a helpful definition of this and other statistical terms, see the glossary in NCHRP Project 20-45, Volume 2, Appendix A.) Based on a literature review, the researchers are sure that bicycle speeds are normally distributed. However, the shapes of the speed distributions for other users are unknown. Thus, the objective is to determine if skater speeds are normally distributed in this instance. Question/Issue Do collected data fit a specific type of probability distribution? In this example, do the speeds of in-line skaters on a shared-use path follow a normal distribution (are they normally distributed)? 2. Identification and Description of Variables: The only variable collected is the speed of in-line skaters passing through short sections of the shared-use path. 3. Data Collection: The team collects speeds using a video camera placed where most path users would not notice it. The speed of each free-flowing skater (i.e., each skater who is not closely following another path user) is calculated from the times that the skater passes two benchmarks on the path visible in the camera frame. Several days of data collection allow a large sample of 219 skaters to be measured. (An implicit assumption is made that there is no

24 effective experiment Design and Data analysis in transportation research variation in the data by day.) The data have a familiar bell shape; that is, when graphed, they look like they are normally distributed (Figure 8). Each bar in the figure shows the number of observations per 1.00-mph-wide speed bin. There are 10 observations between 6.00 mph and 6.99 mph. 4. Specification of Analysis Technique and Data Analysis: This analysis involves several pre- liminary steps followed by two major steps. In the preliminaries, the team calculates the mean and standard deviation from the data sample as 10.17 mph and 2.79 mph, respectively, using standard formulas described in NCHRP Project 20-45, Volume 2, Chapter 6, Section C under the heading “Frequency Distributions, Variance, Standard Deviation, Histograms, and Boxplots.” Then the team forms bins of observations of sufficient size to conduct the analysis. For this analysis, the team forms bins containing at least four observations each, which means forming a bin for speeds of 5 mph and lower and a bin for speeds of 17 mph or higher. There is some argument regarding the minimum allowable cell size. Some analysts argue that the minimum is five; others argue that the cell size can be smaller. Smaller numbers of observations in a bin may distort the results. When in doubt, the analysis can be done with different assumptions regarding the cell size. The left two columns in Table 8 show the data ready for analysis. The first major step of the analysis is to generate the theoretical normal distribution to compare to the field data. To do this, the team calculates a value of Z, the standard normal variable for each bin i, using the following equation: Z xi = − µ σ where x is the speed in miles per hour (mph) corresponding to the bin, µ is the mean speed, and s is the standard deviation of all of the observations in the speed sample in mph. For example (and with reference to the data in Table 8), for a speed of 5 mph the value of Z will be (5 - 10.17)/2.79 = -1.85 and for a speed of 6 mph, the value of Z will be (6 - 10.17)/2.79 = -1.50. The team then consults a table of standard normal values (i.e., NCHRP Project 20-45, Volume 2, Appendix C, Table C-1) to convert these Z values into A values representing the area under the standard normal distribution curve. The A value for a Z of -1.85 is 0.468, while the A value for a Z of -1.50 is 0.432. The difference between these two A values, representing the area under the standard normal probability curve corresponding to the speed of 6 mph, is 0.036 (calculated 0.468 - 0.432 = 0.036). The team multiplies 0.036 by the total sample size (219), to estimate that there should be 7.78 skaters with a speed of 6 mph if the speeds follow the standard normal distribution. The team follows Figure 8. Distribution of observed in-line skater speeds. 0 5 10 15 20 25 30 35 40 1 3 5 7 9 11 13 15 17 232119 Speed, mph Nu m be r o f o bs er va tio ns

examples of effective experiment Design and Data analysis in transportation research 25 a similar procedure for all speeds. Notice that the areas under the curve can also be calculated in a simple Excel spreadsheet using the “NORMDIST” function for a given x value and the average speed of 10.17 and standard deviation of 2.79. The values shown in Table 8 have been estimated using the Excel function. The second major step of the analysis is to use the chi-square test (as described in NCHRP Project 20-45, Volume 2, Chapter 6, Section F) to determine if the theoretical normal distribution is significantly different from the actual data distribution. The team computes a chi-square value for each bin i using the formula: χi i i i O E E 2 2 = −( ) where Oi is the number of actual observations in bin i and Ei is the expected number of obser- vations in bin i estimated by using the theoretical distribution. For the bin of 6 mph speeds, O = 10 (from the table), E = 7.78 (calculated), and the ci2 contribution for that cell is 0.637. The sum of the ci2 values for all bins is 19.519. The degrees of freedom (df) used for this application of the chi-square test are the number of bins minus 1 minus the number of variables in the distribution of interest. Given that the normal distribution has two variables (see May, Traffic Flow Fundamentals, 1990, p. 40), in this example the degrees of freedom equal 9 (calculated 12 - 1 - 2 = 9). From a standard table of chi-square values (NCHRP Project 20-45, Volume 2, Appendix C, Table C-2), the team finds that the critical value at the 95% confidence level for this case (with df = 9) is 16.9. The calculated value of the statistic is ~19.5, more than the tabular value. The results of all of these observations and calculations are shown in Table 8. 5. Interpreting the Results: The calculated chi-square value of ~19.5 is greater than the criti- cal chi-square value of 16.9. The team concludes, therefore, that the normal distribution is significantly different from the distribution of the speed sample at the 95% level (i.e., that the in-line skater speed data do not appear to be normally distributed). Larger variations between the observed and expected distributions lead to higher values of the statistic and would be interpreted as it being less likely that the data are distributed according to the Speed (mph) Number of Observations Number Predicted by Normal Distribution Chi-Square Value Under 5.99 6 6.98 0.137 6.00 to 6.99 10 7.78 0.637 7.00 to 7.99 18 13.21 1.734 8.00 to 8.99 24 19.78 0.902 9.00 to 9.99 37 26.07 4.585 10.00 to 10.99 38 30.26 1.980 11.00 to 11.99 24 30.93 1.554 12.00 to 12.99 21 27.85 1.685 13.00 to 13.99 15 22.08 2.271 14.00 to 14.99 13 15.42 0.379 15.00 to 15.99 4 9.48 3.169 16.00 to 16.99 4 5.13 0.251 17.00 and over 5 4.03 0.234 Total 219 219 19.519 Table 8. Observations, theoretical predictions, and chi-square values for each bin.

26 effective experiment Design and Data analysis in transportation research hypothesized distribution. Conversely, smaller variations between observed and expected distributions result in lower values of the statistic, which would suggest that it is more likely that the data are normally distributed because the observed values would fit better with the expected values. 6. Conclusion and Discussion: In this case, the results suggest that the normal distribution is not a good fit to free-flow speeds of in-line skaters on shared-use paths. Interestingly, if the 23 mph observation is considered to be an outlier and discarded, the results of the analysis yield a different conclusion (that the data are normally distributed). Some researchers use a simple rule that an outlier exists if the observation is more than three standard deviations from the mean value. (In this example, the 23 mph observation is, indeed, more than three standard deviations from the mean.) If there is concern with discarding the observation as an outlier, it would be easy enough in this example to repeat the data collection exercise. Looking at the data plotted above, it is reasonably apparent that the well-known normal distribution should be a good fit (at least without the value of 23). However, the results from the statistical test could not confirm the suspicion. In other cases, the type of distribution may not be so obvious, the distributions in question may be obscure, or some distribution parameters may need to be calibrated for a good fit. In these cases, the statistical test is much more valuable. The chi-square test also can be used simply to compare two observed distributions to see if they are the same, independent of any underlying probability distribution. For example, if it is desired to know if the distribution of traffic volume by vehicle type (e.g., automobiles, light trucks, and so on) is the same at two different freeway locations, the two distributions can be compared to see if they are similar. The consequences of an error in the procedure outlined here can be severe. This is because the distributions chosen as a result of the procedure often become the heart of predictive models used by many other engineers and planners. A poorly-chosen distribution will often provide erroneous predictions for many years to come. 7. Applications in Other Areas of Transportation Research: Fitting distributions to data samples is important in several areas of transportation research, such as: • Traffic Operations—to analyze shapes of vehicle headway distributions, which are of great interest, especially as a precursor to calibrating and using simulation models. • Traffic Safety—to analyze collision frequency data. Analysts often assume that the Poisson distribution is a good fit for collision frequency data and must use the method described here to validate the claim. • Pavement Engineering—to form models of pavement wear or otherwise compare results obtained using different designs, as it is often required to check the distributions of the parameters used (e.g., roughness). Example 5: Construction; Simple Comparisons to Specified Values Area: Construction Method of Analysis: Simple comparisons to specified values—using Student’s t-test to compare the mean value of a small sample to a standard or other requirement (i.e., to a population with a known mean and unknown standard deviation or variance) 1. Research Question/Problem Statement: A contractor wants to determine if a specified soil compaction can be achieved on a segment of the road under construction by using an on-site roller or if a new roller must be brought in.

examples of effective experiment Design and Data analysis in transportation research 27 The cost of obtaining samples for many construction materials and practices is quite high. As a result, decisions often must be made based on a small number of samples. The appropri- ate statistical technique for comparing the mean value of a small sample with a standard or requirement is Student’s t-test. Formally, the working, or null, hypothesis (Ho) and the alternative hypothesis (Ha) can be stated as follows: Ho: The soil compaction achieved using the on-site roller (CA) is less than a specified value (CS); that is, (CA < CS). Ha: The soil compaction achieved using the on-site roller (CA) is greater than or equal to the specified value (CS); that is, (CA ≥ CS). Question/Issue Determine whether a sample mean exceeds a specified value. Alternatively, deter- mine the probability of obtaining a sample mean (x _ ) from a sample of size n, if the universe being sampled has a true mean less than or equal to a population mean with an unknown variance. In this example, is an observed mean of soil compaction samples equal to or greater than a specified value? 2. Identification and Description of Variables: The variable to be used is the soil density results of nuclear densometer tests. These values will be used to determine whether the use of the on-site roller is adequate to meet the contract-specified soil density obtained in the laboratory (Proctor density) of 95%. 3. Data Collection: A 125-foot section of road is constructed and compacted with the on-site roller, and four samples of the soil density are obtained (25 feet, 50 feet, 75 feet, and 100 feet from the beginning of the test section). 4. Specification of Analysis Technique and Data Analysis: For small samples (n < 30) where the population mean is known but the population standard deviation is unknown, it is not appropriate to describe the distribution of the sample mean with a normal distribution. The appropriate distribution is called Student’s distribution (t-distribution or t-statistic). The equation for Student’s t-statistic is: t x x S n = − ′ where x _ is the sample mean, x _ ′ is the population mean (or specified standard), S is the sample standard deviation, and n is the sample size. The four nuclear densometer readings were 98%, 97%, 93% and 99%. Then, showing some simple sample calculations, X X S X i i i n = = + + + = = = = = ∑ 4 98 97 93 99 4 387 4 96 75 1 4 1 . % Σ i X n S −( ) − = = 2 1 20 74 3 2 63 . . %

28 effective experiment Design and Data analysis in transportation research and using the equation for t above, t = − = = 96 75 95 00 2 63 2 1 75 1 32 1 33 . . . . . . The calculated value of the t-statistic (1.33) is most typically compared to the tabularized values of the t-statistic (e.g., NCHRP Project 20-45, Volume 2, Appendix C, Table C-4) for a given significance level (typically called t critical or tcrit). For a sample size of n = 4 having 3 (n - 1) degrees of freedom (df), the values for tcrit are: 1.638 for a = 0.10 and 2.353 for a = 0.05 (two common values of a for testing, the latter being most common). Important: The specification of the significance level (a level) for testing should be done before actual testing and interpretation of results are done. In many instances, the appropriate level is defined by the agency doing the testing, a specified testing standard, or simply common practice. Generally speaking, selection of a smaller value for a (e.g., a = 0.05 versus a = 0.10) sets a more stringent standard. In this example, because the calculated value of t (1.33) is less than the critical value (2.353, given a = 0.05), the null hypothesis is accepted. That is, the engineer cannot be confident that the mean value from the densometer tests (96.75%) is greater than the required specifica- tion (95%). If a lower confidence level is chosen (e.g., a = 0.15), the value for tcrit would change to 1.250, which means the null hypothesis would be rejected. A lower confidence level can have serious implications. For example, there is an approximately 15% chance that the standard will not be met. That level of risk may or may not be acceptable to the contractor or the agency. Notice that in many standards the required significance level is stated (typically a = 0.05). It should be emphasized that the confidence level should be chosen before calculations and testing are done. It is not generally permissible to change the confidence level after calculations have been performed. Doing this would be akin to arguing that standards can be relaxed if a test gives an answer that the analyst doesn’t like. The results of small sample tests often are sensitive to the number of samples that can be obtained at a reasonable cost. (The mean value may change considerably as more data are added.) In this example, if it were possible to obtain nine independent samples (as opposed to four) and the mean value and sample standard deviation were the same as with the four samples, the calculation of the t-statistic would be: t = − = 96 75 95 00 2 63 3 1 99 . . . . Comparing the value of t (with a larger sample size) to the appropriate tcrit (for n - 1 = 8 df and a = 0.05) of 1.860 changes the outcome. That is, the calculated value of the t-statistic is now larger than the tabularized value of tcrit, and the null hypothesis is rejected. Thus, it is accepted that the mean of the densometer readings meets or exceeds the standard. It should be noted, however, that the inclusion of additional tests may yield a different mean value and standard deviation, in which case the results could be different. 5. Interpreting the Results: By themselves, the results of the statistical analysis are insufficient to answer the question as to whether a new roller should be brought to the project site. These results only provide information the contractor can use to make this decision. The ultimate decision should be based on these probabilities and knowledge of the cost of each option. What is the cost of bringing in a new roller now? What is the cost of starting the project and then determining the current roller is not adequate and then bringing in a new roller? Will this decision result in a delay in project completion—and does the contract include an incentive for early completion and/or a penalty for missing the completion date? If it is possible to conduct additional independent densometer tests, what is the cost of conducting them?

examples of effective experiment Design and Data analysis in transportation research 29 If there is a severe penalty for missing the deadline (or a significant reward for finishing early), the contractor may be willing to incur the cost of bringing in a new roller rather than accepting a 15% probability of being delayed. 6. Conclusion and Discussion: In some cases the decision about which alternative is preferable can be expressed in the form of a probability (or level of confidence) required to make a deci- sion. The decision criterion is then expressed in a hypothesis and the probability of rejecting that hypothesis. In this example, if the hypothesis to be tested is “Using the on-site roller will provide an average soil density of 95% or higher” and the level of confidence is set at 95%, given a sample of four tests the decision will be to bring in a new roller. However, if nine independent tests could be conducted, the results in this example would lead to a decision to use the on-site roller. 7. Applications in Other Areas in Transportation Research: Simple comparisons to specified values can be used in a variety of areas of transportation research. Some examples include: • Traffic Operations—to compare the average annual number of crashes at intersections with roundabouts with the average annual number of crashes at signalized intersections. • Pavement Engineering—to test the comprehensive strength of concrete slabs. • Maintenance—to test the results of a proposed new deicer compound. Example 6: Maintenance; Simple Two-Sample Comparisons Area: Maintenance Method of Analysis: Simple two-sample comparisons (t-test for paired comparisons; com- paring the mean values of two sets of matched data) 1. Research Question/Problem Statement: As a part of a quality control and quality assurance (QC/QA) program for highway maintenance and construction, an agency engineer wants to compare and identify discrepancies in the contractor’s testing procedures or equipment in making measurements on materials being used. Specifically, compacted air voids in asphalt mixtures are being measured. In this instance, the agency’s test results need to be compared, one-to-one, with the contractor’s test results. Samples are drawn or made and then literally split and tested—one by the contractor, one by the agency. Then the pairs of measurements are analyzed. A paired t-test will be used to make the comparison. (For another type of two-sample comparison, see Example 7.) Question/Issue Use collected data to test if two sets of results are similar. Specifically, do two test- ing procedures to determine air voids produce the same results? Stated in formal terms, the null and alternative hypotheses are: Ho: There is no mean difference in air voids between agency and contractor test results: H Xo d: = 0 Ha: There is a mean difference in air voids between agency and contractor test results: H Xa d: ≠ 0 (For definitions and more discussion about the formulation of formal hypotheses for test- ing, see NCHRP Project 20-45, Volume 2, Appendix A and Volume 1, Chapter 2, “Hypothesis.”) 2. Identification and Description of Variables: The testing procedure for laboratory-compacted air voids in the asphalt mixture needs to be verified. The split-sample test results for laboratory-

30 effective experiment Design and Data analysis in transportation research compacted air voids are shown in Table 9. Twenty samples are prepared using the same asphalt mixture. Half of the samples are prepared in the agency’s laboratory and the other half in the contractor’s laboratory. Given this arrangement, there are basically two variables of concern: who did the testing and the air void determination. 3. Data Collection: A sufficient quantity of asphalt mix to make 10 lots is produced in an asphalt plant located on a highway project. Each of the 10 lots is collected, split into two samples, and labeled. A sample from each lot, 4 inches in diameter and 2 inches in height, is prepared in the contractor’s laboratory to determine the air voids in the compacted samples. A matched set of samples is prepared in the agency’s laboratory and a similar volumetric procedure is used to determine the agency’s lab-compacted air voids. The lab-compacted air void contents in the asphalt mixture for both the contractor and agency are shown in Table 9. 4. Specification of Analysis Technique and Data Analysis: A paired (two-sided) t-test will be used to determine whether a difference exists between the contractor and agency results. As noted above, in a paired t-test the null hypothesis is that the mean of the differences between each pair of two tests is 0 (there is no difference between the means). The null hypothesis can be expressed as follows: H Xo d: = 0 The alternate hypothesis, that the two means are not equal, can be expressed as follows: H Xa d: ≠ 0 The t-statistic for the paired measurements (i.e., the difference between the split-sample test results) is calculated using the following equation: t X s n d d = − 0 Using the actual data, the value of the t-statistic is calculated as follows: t = − = 0 88 0 0 7 10 4 . . Sample Air Voids (%) DifferenceContractor Agency 1 4.37 4.15 0.21 2 3.76 5.39 -1.63 3 4.10 4.47 -0.37 4 4.39 4.52 -0.13 5 4.06 5.36 -1.29 6 4.14 5.01 -0.87 7 3.92 5.23 -1.30 8 3.38 4.97 -1.60 9 4.12 4.37 -0.25 10 3.68 5.29 -1.61 X 3.99 4.88 dX = -0.88 S 0.31 0.46 ds = 0.70 Table 9. Laboratory-compacted air voids in split samples.

examples of effective experiment Design and Data analysis in transportation research 31 For n - 1 (10 - 1 = 9) degrees of freedom and a = 0.05, the tcrit value can be looked up using a t-table (e.g., NCHRP Project 20-45, Volume 2, Appendix C, Table C-4): t0 025 9 2 262. , .= For a more detailed description of the t-statistic, see the glossary in NCHRP Project 20-45, Volume 2, Appendix A. 5. Interpreting the Results: Given that t = 4 > t0.025, 9 = 2.685, the engineer would reject the null hypothesis and conclude that the results of the paired tests are different. This means that the contractor and agency test results from paired measurements indicate that the test method, technicians, and/or test equipment are not providing similar results. Notice that the engineer cannot conclude anything about the material or production variation or what has caused the differences to occur. 6. Conclusion and Discussion: The results of the test indicate that a statistically significant difference exists between the test results from the two groups. When making such comparisons, it is important that random sampling be used when obtaining the samples. Also, because sources of variability influence the population parameters, the two sets of test results must have been sampled over the same time period, and the same sampling and testing procedures must have been used. It is best if one sample is drawn and then literally split in two, then another sample drawn, and so on. The identification of a difference is just that: notice that a difference exists. The reason for the difference must still be determined. A common misinterpretation is that the result of the t-test provides the probability of the null hypothesis being true. Another way to look at the t-test result in this example is to conclude that some alternative hypothesis provides a better description of the data. The result does not, however, indicate that the alternative hypothesis is true. To ensure practical significance, it is necessary to assess the magnitude of the difference being tested. This can be done by computing confidence intervals, which are used to quantify the range of effect size and are often more useful than simple hypothesis testing. Failure to reject a hypothesis also provides important information. Possible explanations include: occurrence of a type-II error (erroneous acceptance of the null hypothesis); small sample size; difference too small to detect; expected difference did not occur in data; there is no difference/effect. Proper experiment design and data collection can minimize the impact of some of these issues. (For a more comprehensive discussion of this topic, see NCHRP Project 20-45, Volume 2, Chapter 1.) 7. Applications in Other Areas of Transportation Research: The application of the t-test to compare two mean values in other areas of transportation research may include: • Traffic Operations—to evaluate average delay in bus arrivals at various bus stops. • Traffic Operations/Safety—to determine the effect of two enforcement methods on reduction in a particular traffic violation. • Pavement Engineering—to investigate average performance of two pavement sections. • Environment—to compare average vehicular emissions at two locations in a city. Example 7: Materials; Simple Two-Sample Comparisons Area: Materials Method of Analysis: Simple two-sample comparisons (using the t-test to compare the mean values of two samples and the F-test for comparing variances) 1. Research Question/Problem Statement: As a part of dispute resolution during quality control and quality assurance, a highway agency engineer wants to validate a contractor’s test results concerning asphalt content. In this example, the engineer wants to compare the results

32 effective experiment Design and Data analysis in transportation research of two sets of tests: one from the contractor and one from the agency. Formally, the (null) hypothesis to be tested, Ho, is that the contractor’s tests and the agency’s tests are from the same population. In other words, the null hypothesis is that the means of the two data sets will be equal, as will the standard deviations. Notice that in the latter instance the variances are actually being compared. Test results were also compared in Example 6. In that example, the comparison was based on split samples. The same test specimens were tested by two different analysts using different equipment to see if the same results could be obtained by both. The major difference between Example 6 and Example 7 is that, in this example, the two samples are randomly selected from the same pavement section. Question/Issue Use collected data to test if two measured mean values are the same. In this instance, are two mean values of asphalt content the same? Stated in formal terms, the null and alternative hypotheses can be expressed as follows: Ho: There is no difference in asphalt content between agency and contractor test results: H m mo c a: − =( )0 Ha: There is a difference in asphalt content between agency and contractor test results: H m ma c a: − ≠( )0 2. Identification and Description of Variables: The contractor runs 12 asphalt content tests and the agency engineer runs 6 asphalt content tests over the same period of time, using the same random sampling and testing procedures. The question is whether it is likely that the tests have come from the same population based on their variability. 3. Data Collection: If the agency’s objective is simply to identify discrepancies in the testing procedures or equipment, then verification testing should be done on split samples (as in Example 6). Using split samples, the difference in the measured variable can more easily be attributed to testing procedures. A paired t-test should be used. (For more information, see NCHRP Project 20-45, Volume 2, Chapter 4, Section A, “Analysis of Variance Methodology.”) A split sample occurs when a physical sample (of whatever is being tested) is drawn and then literally split into two testable samples. On the other hand, if the agency’s objective is to identify discrepancies in the overall material, process, sampling, and testing processes, then validation testing should be done on independent samples. Notice the use of these terms. It is important to distinguish between testing to verify only the testing process (verification) versus testing to compare the overall production, sampling, and testing processes (validation). If independent samples are used, the agency test results still can be compared with contractor test results (using a simple t-test for comparing two means). If the test results are consistent, then the agency and contractor tests can be combined for contract compliance determination. 4. Specification of Analysis Technique and Data Analysis: When comparing the two data sets, it is important to compare both the means and the variances because the assumption when using the t-test requires equal variances for each of the two groups. A different test is used in each instance. The F-test provides a method for comparing the variances (the standard devia- tion squared) of two sets of data. Differences in means are assessed by the t-test. Generally, construction processes and material properties are assumed to follow a normal distribution.

examples of effective experiment Design and Data analysis in transportation research 33 In this example, a normal distribution is assumed. (The assumption of normality also can be tested, as in Example 4.) The ratios of variances follow an F-distribution, while the means of relatively small samples follow a t-distribution. Using these distributions, hypothesis tests can be conducted using the same concepts that have been discussed in prior examples. (For more information about the F-test and the t-distribution, see NCHRP Project 20-45, Volume 2, Chapter 4, Section A, “Compute the F-ratio Test Statistic.” For more information about the t-distribution, see NCHRP Project 20-45, Volume 2, Chapter 4, Section A.) For samples from the same normal population, the statistic F (the ratio of the two-sample variances) has a sampling distribution called the F-distribution. For validation and verification testing, the F-test is based on the ratio of the sample variance of the contractor’s test results (sc 2) and the sample variance of the agency’s test results (sa 2). Similarly, the t-test can be used to test whether the sample mean of the contractor’s tests, X _ c, and the agency’s tests, X _ a, came from populations with the same mean. Consider the asphalt content test results from the contractor samples and agency samples (Table 10). In this instance, the F-test is used to determine whether the variance observed for the contractor’s tests differs from the variance observed for the agency’s tests. Using the F-test Step 1. Compute the variance (s2), for each set of tests: sc 2 = 0.064 and sa 2 = 0.092. As an example, sc 2 can be calculated as: s x X n c i c i2 2 2 2 1 6 4 6 1 11 6 2 6 1 11 = −( ) − = −( ) + −( )∑ . . . . + + −( ) + −( ) =. . . . . . . 6 6 1 11 5 7 6 1 11 0 0645 2 2 Step 2. Compute F s s calc a c = = = 2 2 0 092 0 064 1 43 . . . . Contractor Samples Agency Samples 1 6.4 1 5.4 2 6.2 2 5.8 3 6.0 3 6.2 4 6.6 4 5.4 5 6.1 5 5.6 6 6.0 6 5.8 7 6.3 8 6.1 9 5.9 10 5.8 11 6.0 12 5.7 Descriptive Statistics = 6.1cX Descriptive Statistics = 5.7aX = 0.0642cs = 0.0922as = 0.25cs = 0.30as = 12cn = 6an Table 10. Asphalt content test results from independent samples.

34 effective experiment Design and Data analysis in transportation research Step 3. Determine Fcrit from the F-distribution table, making sure to use the correct degrees of freedom (df) for the numerator (the number of observations minus 1, or na - 1 = 6 - 1 = 5) and the denominator (nc - 1 = 12 - 1 = 11). For a = 0.01, Fcrit = 5.32. The critical F-value can be found from tables (see NCHRP Project 20-45, Volume 2, Appendix C, Table C-5). Read the F-value for 1 - a = 0.99, numerator and denominator degrees of freedom 5 and 11, respectively. Interpolation can be used if exact degrees of freedom are not available in the table. Alternatively, a statistical function in Microsoft Excel™ can be used to determine the F-value. Step 4. Compare the two values to determine if Fcalc < Fcrit. If Fcalc < Fcrit is true, then the variances are equal; if not, they are unequal. In this example, Fcalc (1.43) is, in fact, less than Fcrit (5.32) and, thus, there is no evidence of unequal variances. Given this result, the t-test for the case of equal variances is used to determine whether to declare that the mean of the contractor’s tests differs from the mean of the agency’s tests. Using the t-test Step 1. Compute the sample means (X _ ) for each set of tests: X _ c = 6.1 and X _ a = 5.7. Step 2. Compute the pooled variance sp 2 from the individual sample variances: s s n s n n n p c c a a c a 2 2 21 1 2 0 064 12 1 = −( )+ −( ) + − = −( )+. 0 092 6 1 12 6 2 0 0731 . . −( ) + − = Step 3. Compute the t-statistic using the following equation for equal variance: t X X s n s n c a p c p a = − + = − + = 2 2 6 1 5 7 0 0731 12 0 0731 6 . . . . 2 9. t0 005 16 2 921. , .= (For more information, see NCHRP Project 20-45, Volume 2, Appendix C, Table C-4 for A v= − =1 2 16 α and .) 5. Interpreting the Results: Given that F < Fcrit (i.e., 1.43 < 5.32), there is no reason to believe that the two sets of data have different variances. That is, they could have come from the same population. Therefore, the t-test can be used to compare the means using equal variance. Because t < tcrit (i.e., 2.9 < 2.921), the engineer does not reject the null hypothesis and, thus, assumes that the sample means are equal. The final conclusion is that it is likely that the contractor and agency test results represent the same process. In other words, with a 99% confidence level, it can be said that the agency’s test results are not different from the contrac- tor’s and therefore validate the contractor tests. 6. Conclusion and Discussion: The simple t-test can be used to validate the contractor’s test results by conducting independent sampling from the same pavement at the same time. Before conducting a formal t-test to compare the sample means, the assumption of equal variances needs to be evaluated. This can be accomplished by comparing sample variances using the F-test. The interpretation of results will be misleading if the equal variance assumption is not validated. If the variances of two populations being compared for their means are different, the mean comparison will reflect the difference between two separate populations. Finally, based on the comparison of means, one can conclude that the construction materials have consistent properties as validated by two independent sources (contractor and agency). This sort of comparison is developed further in Example 8, which illustrates tests for the equality of more than two mean values.

examples of effective experiment Design and Data analysis in transportation research 35 7. Applications in Other Areas of Transportation Research: The simple t-test can be used to compare means of two independent samples. Applications for this method in other areas of transportation research may include: • Traffic Operations – to compare average speeds at two locations along a route. – to evaluate average delay times at two intersections in an urban area. • Pavement Engineering—to investigate the difference in average performance of two pavement sections. • Maintenance—to determine the effects of two maintenance treatments on average life extension of two pavement sections. Example 8: Laboratory Testing/Instrumentation; Simple Analysis of Variance (ANOVA) Area: Laboratory testing and/or instrumentation Method of Analysis: Simple analysis of variance (ANOVA) comparing the mean values of more than two samples and using the F-test 1. Research Question/Problem Statement: An engineer wants to test and compare the com- pressive strength of five different concrete mix designs that vary in coarse aggregate type, gradation, and water/cement ratio. An experiment is conducted in a laboratory where five different concrete mixes are produced based on given specifications, and tested for com- pressive strength using the ASTM International standard procedures. In this example, the comparison involves inference on parameters from more than two populations. The purpose of the analysis, in other words, is to test whether all mix designs are similar to each other in mean compressive strength or whether some differences actually exist. ANOVA is the statistical procedure used to test the basic hypothesis illustrated in this example. Question/Issue Compare the means of more than two samples. In this instance, compare the compres- sive strengths of five concrete mix designs with different combinations of aggregates, gradation, and water/cement ratio. More formally, test the following hypotheses: Ho: There is no difference in mean compressive strength for the various (five) concrete mix types. Ha: At least one of the concrete mix types has a different compressive strength. 2. Identification and Description of Variables: In this experiment, the factor of interest (independent variable) is the concrete mix design, which has five levels based on differ- ent coarse aggregate types, gradation, and water/cement ratios (denoted by t and labeled A through E in Table 11). Compressive strength is a continuous response (dependent) variable, measured in pounds per square inch (psi) for each specimen. Because only one factor is of interest in this experiment, the statistical method illustrated is often called a one-way ANOVA or simple ANOVA. 3. Data Collection: For each of the five mix designs, three replicates each of cylinders 4 inches in diameter and 8 inches in height are made and cured for 28 days. After 28 days, all 15 specimens are tested for compressive strength using the standard ASTM International test. The compres- sive strength data and summary statistics are provided for each mix design in Table 11. In this example, resource constraints have limited the number of replicates for each mix design to

36 effective experiment Design and Data analysis in transportation research three. (For a discussion on sample size determination based on statistical power requirements, see NCHRP Project 20-45, Volume 2, Chapter 1, “Sample Size Determination.”) 4. Specification of Analysis Technique and Data Analysis: To perform a one-way ANOVA, pre- liminary calculations are carried out to compute the overall mean (y _ P), the sample means (y _ i.), and the sample variances (si 2) given the total sample size (nT = 15) as shown in Table 11. The basic strategy for ANOVA is to compare the variance between levels or groups—specifically, the variation between sample means—to the variance within levels. This comparison is used to determine if the levels explain a significant portion of the variance. (Details for perform- ing a one-way ANOVA are given in NCHRP Project 20-45, Volume 2, Chapter 4, Section A, “Analysis of Variance Methodology.”) ANOVA is based on partitioning of the total sum of squares (TSS, a measure of overall variability) into within-level and between-levels components. The TSS is defined as the sum of the squares of the differences of each observation (yij) from the overall mean (y _ P). The TSS, between-levels sum of squares (SSB), and within-level sum of squares (SSE) are computed as follows. TSS y y SSB y y ij i j i = −( ) = = −( ) ∑ .. , . .. . 2 2 4839620 90 = = −( ) = ∑ 4331513 60 508107 30 2 . . , . , i j ij i i j SSE y y∑ The next step is to compute the between-levels mean square (MSB) and within-levels mean square (MSE) based on respective degrees of freedom (df). The total degrees of freedom (dfT), between-levels degrees of freedom (dfB), and within-levels degrees of freedom (dfE) for one- way ANOVA are computed as follows: df n df t df n t T T B E T = − = − = = − = − = = − = − = 1 15 1 14 1 5 1 4 15 5 10 where nT = the total sample size and t = the total number of levels or groups. The next step of the ANOVA procedure is to compute the F-statistic. The F-statistic is the ratio of two variances: the variance due to interaction between the levels, and the variance due to differences within the levels. Under the null hypothesis, the between-levels mean square (MSB) and within-levels mean square (MSE) provide two independent estimates of the variance. If the means for different levels of mix design are truly different from each other, the MSB will tend Replicate Mix Design A B C D E 1 y11 = 5416 y21 = 5292 y31 = 4097 y41 = 5056 y51 = 4165 2 y12 = 5125 y22 = 4779 y32 = 3695 y42 = 5216 y52 = 3849 3 y13 = 4847 y23 = 4824 y33 = 4109 y43 = 5235 y53 = 4089 Mean y– 1. = 5129 y– 2. = 4965 y– 3. = 3967 y– 4. = 5169 y– 5. = 4034 Standard deviation s1 = 284.52 s2 = 284.08 s3 = 235.64 s4 = 98.32 s5 = 164.94 Overall mean y–.. = 4653 Table 11. Concrete compressive strength (psi) after 28 days.

examples of effective experiment Design and Data analysis in transportation research 37 to be larger than the MSE, such that it will be more likely to reject the null hypothesis. For this example, the calculations for MSB, MSE, and F are as follows: MSB SSB df MSE SSE df F M B E = = = = = 1082878 40 50810 70 . . SB MSE = 21 31. If there are no effects due to level, the F-statistic will tend to be smaller. If there are effects due to level, the F-statistic will tend to be larger, as is the case in this example. ANOVA computations usually are summarized in the form of a table. Table 12 summarizes the computations for this example. The final step is to determine Fcrit from the F-distribution table (e.g., NCHRP Project 20-45, Volume 2, Appendix C, Table C-5) with t - 1 (5 - 1 = 4) degrees of freedom for the numerator and nT - t (15 - 5 = 10) degrees of freedom for the denominator. For a significance level of a = 0.01, Fcrit is found (in Table C-5) to be 5.99. Given that F > Fcrit (21.31 > 5.99), the null hypothesis that all mix designs have equal compressive strength is rejected, supporting the conclusion that at least two mix designs are different from each other in their mean effect. Table 12 also shows the p-value calculated using a computer program. The p-value is the probability that a sample would result in the given statistic value if the null hypothesis were true. The p-value of 0.0000698408 is well below the chosen significance level of 0.01. 5. Interpreting the Results: The ANOVA results in rejection of the null hypothesis at a = 0.01. That is, the mean values are judged to be statistically different. However, the ANOVA result does not indicate where the difference lies. For example, does the compressive strength of mix design A differ from that of mix design C or D? To carry out such multiple mean comparisons, the analyst must control the experiment-wise error rate (EER) by employing more conservative methods such as Tukey’s test, Bonferroni’s test, or Scheffe’s test, as appropriate. (Details for ANOVA are given in NCHRP Project 20-45, Volume 2, Chapter 4, Section A, “Analysis of Variance Methodology.”) The coefficient of determination (R2) provides a rough indication of how well the statistical model fits the data. For this example, R2 is calculated as follows: R SSB TSS 2 4331513 60 4839620 90 0 90= = = . . . For this example, R2 indicates that the one-way ANOVA classification model accounts for 90% of the total variation in the data. In the controlled laboratory experiment demonstrated in this example, R2 = 0.90 indicates a fairly acceptable fit of the statistical model to the data. 6. Conclusion and Discussion: This example illustrates a simple one-way ANOVA where infer- ence regarding parameters (mean values) from more than two populations or treatments was Source Sum of Squares (SS) Degrees of Freedom (df) Mean Square (MS) F Probability > F (Significance) Between 4331513.60 4 1082878.40 21.31 0.0000698408 Within 508107.30 10 50810.70 Total 4839620.90 14 Table 12. ANOVA results.

38 effective experiment Design and Data analysis in transportation research desired. The focus of computations was the construction of the ANOVA table. Before pro- ceeding with ANOVA, however, an analyst must verify that the assumptions of common vari- ance and data normality are satisfied within each group/level. The results do not establish the cause of difference in compressive strength between mix designs in any way. The experimental setup and analytical procedure shown in this example may be used to test other properties of mix designs such as flexure strength. If another factor (for example, water/cement ratio with levels low or high) is added to the analysis, the classification will become a two-way ANOVA. (In this report, two-way ANOVA is demonstrated in Example 11.) Notice that the equations shown in Example 8 may only be used for one-way ANOVA for balanced designs, meaning that in this experiment there are equal numbers of replicates for each level within a factor. (For a discussion of computations on unbalanced designs and multifactor designs, see NCHRP Project 20-45.) 7. Applications in Other Areas of Transportation Research: Examples of applications of one-way ANOVA in other areas of transportation research include: • Traffic Operations—to determine the effect of various traffic calming devices on average speeds in residential areas. • Traffic Operations/Safety—to study the effect of weather conditions on accidents in a given time period. • Work Zones—to compare the effect of different placements of work zone signs on reduction in highway speeds at some downstream point. • Materials—to investigate the effect of recycled aggregates on compressive and flexural strength of concrete. Example 9: Materials; Simple Analysis of Variance (ANOVA) Area: Materials Method of Analysis: Simple analysis of variance (ANOVA) comparing more than two mean values and using the F-test for equality of means 1. Research Question/Problem Statement: To illustrate how increasingly detailed analysis may be appropriate, Example 9 is an extension of the two-sample comparison presented in Exam- ple 7. As a part of dispute resolution during quality control and quality assurance, let’s say the highway agency engineer from Example 7 decides to reconfirm the contractor’s test results for asphalt content. The agency hires an independent consultant to verify both the contractor- and agency-measured asphalt contents. It now becomes necessary to compare more than two mean values. A simple one-way analysis of variance (ANOVA) can be used to analyze the asphalt contents measured by three different parties. Question/Issue Extend a comparison of two mean values to compare three (or more) mean values. Specifically, use data collected by several (>2) different parties to see if the results (mean values) are the same. Formally, test the following null (Ho) and alternative (Ha) hypotheses, which can be stated as follows: Ho: There is no difference in asphalt content among three different parties: H m m mo contractor agency: = =( )consultant Ha: At least one of the parties has a different measured asphalt content.

examples of effective experiment Design and Data analysis in transportation research 39 2. Identification and Description of Variables: The independent consultant runs 12 additional asphalt content tests by taking independent samples from the same pavement section as the agency and contractor. The question is whether it is likely that the tests came from the same population, based on their variability. 3. Data Collection: The descriptive statistics (mean, standard deviation, and sample size) for the asphalt content data collected by the three parties are shown in Table 13. Notice that 12 measurements each have been taken by the contractor and the independent consultant, while the agency has only taken six measurements. The data for the contractor and the agency are the same as presented in Example 7. For brevity, the consultant’s raw observations are not repeated here. The mean value and standard deviation for the consultant’s data are calculated using the same formulas and equations that were used in Example 7. 4. Specification of Analysis Technique and Data Analysis: The agency engineer can use one-way ANOVA to resolve this question. (Details for one-way ANOVA are available in NCHRP Project 20-45, Volume 2, Chapter 4, Section A, “Analysis of Variance Methodology.”) The objective of the ANOVA is to determine whether the variance observed in the depen- dent variable (in this case, asphalt content) is due to the differences among the samples (different from one party to another) or due to the differences within the samples. ANOVA is basically an extension of two-sample comparisons to cases when three or more samples are being compared. More formally, the technician is testing to see whether the between- sample variability is large relative to the within-sample variability, as stated in the formal hypothesis. This type of comparison also may be referred to as between-groups versus within-groups variance. Rejection of the null hypothesis (that the mean values are the same) gives the engineer some information concerning differences among the population means; however, it does not indicate which means actually differ from each other. Rejection of the null hypothesis tells the engineer that differences exist, but it does not specify that X _ 1 differs from X _ 2 or from X _ 3. To control the experiment-wise error rate (EER) for multiple mean comparisons, a con- servative test—Tukey’s procedure for unplanned comparisons—can be used for unplanned comparisons. (Information about Tukey’s procedure can be found in almost any good statistics textbook, such as those by Freund and Wilson [2003] and Kutner et al. [2005].) The F-statistic calculated for determining the effect of who (agency, contractor, or consultant) measured Party Type Asphalt Content Percent Contractor 1 1 1 X s n = 6.1 = 0.254 = 12 Agency 2 2 2 X s n = 5.7 = 0.303 = 6 Consultant 3 3 3 X s n = 5.12 = 0.186 = 12 Table 13. Asphalt content data summary.

40 effective experiment Design and Data analysis in transportation research the asphalt content is given in Table 14. (See Example 8 for a more detailed discussion of the calculations necessary to create Table 14.) Although the ANOVA results reveal whether there are overall differences, it is always good practice to visually examine the data. For example, Figure 9 shows the mean and associated 95% confidence intervals (CI) of the mean asphalt content measured by each of the three parties involved in the testing. 5. Interpreting the Results: A simple one-way ANOVA is conducted to determine whether there is a difference in mean asphalt content as measured by the three different parties. The analysis shows that the F-statistic is significant (p-value < 0.05), meaning that at least two of the means are significantly different from each other. The engineer can use Tukey’s procedure for com- parisons of multiple means, or he or she can observe the plotted 95% confidence intervals to figure out which means are actually (and significantly) different from each other (see Figure 9). Because the confidence intervals overlap, the results show that the asphalt content measured by the contractor and the agency are somewhat different. (These same conclusions were obtained in Example 7.) However, the mean asphalt content obtained by the consultant is significantly different from (and lower than) that obtained by both of the other parties. This is evident because the confidence interval for the consultant doesn’t overlap with the confidence interval of either of the other two parties. Source Sum of Squares (SS) Degrees of Freedom (df) Mean Square (MS) F Significance Between groups 5.6 2 2.8 49.1 0.000 Within groups 1.5 27 0.06 Total 7.2 29 Table 14. ANOVA results. Figure 9. Mean and confidence intervals for asphalt content data.

examples of effective experiment Design and Data analysis in transportation research 41 6. Conclusion and Discussion: This example uses a simple one-way ANOVA to compare the mean values of three sets of results using data drawn from the same test section. The error bar plots for data from the three different parties visually illustrate the statistical differences in the multiple means. However, the F-test for multiple means should be used to formally test the hypothesis of the equality of means. The interpretation of results will be misleading if the variances of populations being compared for their mean difference are not equal. Based on the comparison of the three means, it can be concluded that the construction material in this example may not have consistent properties, as indicated by the results from the independent consultant. 7. Applications in Other Areas of Transportation Research: Simple one-way ANOVA is often used when more than two means must be compared. Examples of applications in other areas of transportation research include: • Traffic Safety/Operations—to evaluate the effect of intersection type on the average number of accidents per month. Three or more types of intersections (e.g., signalized, non-signalized, and rotary) could be selected for study in an urban area having similar traffic volumes and vehicle mix. • Pavement Engineering – to investigate the effect of hot-mix asphalt (HMA) layer thickness on fatigue cracking after 20 years of service life. Three HMA layer thicknesses (5 inches, 6 inches, and 7 inches) are to be involved in this study, and other factors (i.e., traffic, climate, and subbase/base thicknesses and subgrade types) need to be similar. – to determine the effect of climatic conditions on rutting performance of flexible pavements. Three or more climatic conditions (e.g., wet-freeze, wet-no-freeze, dry-freeze, and dry-no-freeze) need to be considered while other factors (i.e., traffic, HMA, and subbase/ base thicknesses and subgrade types) need to be similar. Example 10: Pavements; Simple Analysis of Variance (ANOVA) Area: Pavements Method of Analysis: Simple analysis of variance (ANOVA) comparing the mean values of more than two samples and using the F-test 1. Research Question/Problem Statement: The aggregate coefficient of thermal expansion (CTE) in Portland cement concrete (PCC) is a critical factor affecting thermal behavior of PCC slabs in concrete pavements. In addition, the interaction between slab curling (caused by the thermal gradient) and axle loads is assumed to be a critical factor for concrete pavement performance in terms of cracking. To verify the effect of aggregate CTE on slab cracking, a pavement engineer wants to conduct a simple observational study by collecting field pave- ment performance data on three different types of pavement. For this example, three types of aggregate (limestone, dolomite, and gravel) are being used in concrete pavement construction and yield the following CTEs: • 4 in./in. per °F • 5 in./in. per °F • 6.5 in./in. per °F It is necessary to compare more than two mean values. A simple one-way ANOVA is used to analyze the observed slab cracking performance by the three different concrete mixes with different aggregate types based on geology (limestone, dolomite, and gravel). All other factors that might cause variation in cracking are assumed to be held constant.

42 effective experiment Design and Data analysis in transportation research 2. Identification and Description of Variables: The engineer identifies 1-mile sections of uni- form pavement within the state highway network with similar attributes (aggregate type, slab thickness, joint spacing, traffic, and climate). Field performance, in terms of the observed percentage of slab cracked (“% slab cracked,” i.e., how cracked is each slab) for each pavement section after about 20 years of service, is considered in the analysis. The available pavement data are grouped (stratified) based on the aggregate type (CTE value). The % slab cracked after 20 years is the dependent variable, while CTE of aggregates is the independent variable. The question is whether pavement sections having different types of aggregate (CTE values) exhibit similar performance based on their variability. 3. Data Collection: From the data stratified by CTE, the engineer randomly selects nine pave- ment sections within each CTE category (i.e., 4, 5, and 6.5 in./in. per °F). The sample size is based on the statistical power (1-b) requirements. (For a discussion on sample size determina- tion based on statistical power requirements, see NCHRP Project 20-45, Volume 2, Chapter 1, “Sample Size Determination.”) The descriptive statistics for the data, organized by three CTE categories, are shown in Table 15. The engineer considers pavement performance data for 9 pavement sections in each CTE category. 4. Specification of Analysis Technique and Data Analysis: Because the engineer is concerned with the comparison of more than two mean values, the easiest way to make the statistical comparison is to perform a one-way ANOVA (see NCHRP Project 20-45, Volume 2, Chapter 4). The comparison will help to determine whether the between-section variability is large relative to the within-section variability. More formally, the following hypotheses are tested: HO: All mean values are equal (i.e., m1 = m2 = m3). HA: At least one of the means is different from the rest. Although rejection of the null hypothesis gives the engineer some information concerning difference among the population means, it doesn’t tell the engineer anything about how the means differ from each other. For example, does m1 differ from m2 or m3? To control the experiment-wise error rate (EER) for multiple mean comparisons, a conservative test— Tukey’s procedure for unplanned comparisons—can be used. (Information about Tukey’s procedure can be found in almost any good statistics textbook, such as those by Freund and Wilson [2003] and Kutner et al. [2005].)The F-statistic calculated for determining the effect of CTE on % slab cracked after 20 years is shown in Table 16. Question/Issue Compare the means of more than two samples. Specifically, is the cracking perfor- mance of concrete pavements designed using more than two different types of aggregates the same? Stated a bit differently, is the performance of three different types of concrete pavement statistically different (are the mean performance measures different)? CTE (in./in. per oF) % Slab Cracked After 20 Years 4 1 1 137, 4.8, 9X s n= = = 5 2 2 253.7, 6.1, 9X s n= = = 6.5 3 3 372.5, 6.3, 9X s n= = = Table 15. Pavement performance data.

examples of effective experiment Design and Data analysis in transportation research 43 The data in Table 16 have been produced by considering the original data and following the procedures presented in earlier examples. The emphasis in this example is on understanding what the table of results provides the researcher. Also in this example, the test for homogeneity of variances (Levene test) shows no significant difference among the standard deviations of % slab cracked for different CTE values. Figure 10 presents the mean and associated 95% confi- dence intervals of the average % slab cracked (also called the mean and error bars) measured for the three CTE categories considered. 5. Interpreting the Results: A simple one-way ANOVA is conducted to determine if there is a difference among the mean values for % slab cracked for different CTE values. The analysis shows that the F-statistic is significant (p-value < 0.05), meaning that at least two of the means are statistically significantly different from each other. To gain more insight, the engineer can use Tukey’s procedure to specifically compare the mean values, or the engineer may simply observe the plotted 95% confidence intervals to ascertain which means are significantly different from each other (see Figure 10). The plotted results show that the mean % slab cracked varies significantly for different CTE values—there is no overlap between the different mean/error bars. Figure 10 also shows that the mean % slab cracked is significantly higher for pavement sections having a higher CTE value. (For more information about Tukey’s procedure, see NCHRP Project 20-45, Volume 2, Chapter 4.) 6. Conclusion and Discussion: In this example, simple one-way ANOVA is used to assess the effect of CTE on cracking performance of rigid pavements. The F-test for multiple means is used to formally test the (null) hypothesis of mean equality. The confidence interval plots for data from pavements having three different CTE values visually illustrate the statistical differ- ences in the three means. The interpretation of results will be misleading if the variances of Source Sum of Squares (SS) Degrees of Freedom (df) Mean Square (MS) F Significance Between groups 5652.7 2 0.0002826.3 84.1 Within groups 806.9 24 33.6 Total 6459.6 26 Table 16. ANOVA results. Figure 10. Error bars for % slab cracked with different CTE.

44 effective experiment Design and Data analysis in transportation research populations being compared for their mean difference are not equal or if a proper multiple mean comparisons procedure is not adopted. Based on the comparison of the three means in this example, the engineer can conclude that the pavement slabs having aggregates with a higher CTE value will exhibit more cracking than those with lower CTE values, given that all other variables (e.g., climate effects) remain constant. 7. Applications in Other Areas of Transportation Research: Simple one-way ANOVA is widely used and can be employed whenever multiple means within a factor are to be compared with one another. Potential applications in other areas of transportation research include: • Traffic Operations—to evaluate the effect of commuting time on level of service (LOS) of an urban highway. Mean travel times for three periods (e.g., morning, afternoon, and evening) could be selected for specified highway sections to collect the traffic volume and headway data in all lanes. • Traffic Safety—to determine the effect of shoulder width on accident rates on rural highways. More than two shoulder widths (e.g., 0 feet, 6 feet, 9 feet, and 12 feet) should be selected in this study. • Pavement Engineering—to investigate the impact of air void content on flexible pavement fatigue performance. Pavement sections having three or more air void contents (e.g., 3%, 5%, and 7%) in the surface HMA layer could be selected to compare their average fatigue cracking performance after the same period of service (e.g., 15 years). • Materials—to study the effect of aggregate gradation on the rutting performance of flexible pavements. Three types of aggregate gradations (fine, intermediate, and coarse) could be adopted in the laboratory to make different HMA mix samples. Performance testing could be conducted in the laboratory to measure rut depths for a given number of load cycles. Example 11: Pavements; Factorial Design (ANOVA Approach) Area: Pavements Method of Analysis: Factorial design (an ANOVA approach used to explore the effects of varying more than one independent variable) 1. Research Question/Problem Statement: Extending the information from Example 10 (a simple ANOVA example for pavements), the pavement engineer has verified that the coefficient of thermal expansion (CTE) in Portland cement concrete (PCC) is a critical factor affecting thermal behavior of PCC slabs in concrete pavements and significantly affects concrete pave- ment performance in terms of cracking. The engineer now wants to investigate the effects of another factor, joint spacing (JS), in addition to CTE. To study the combined effects of PCC CTE and JS on slab cracking, the engineer needs to conduct a factorial design study by collect- ing field pavement performance data. As before, three CTEs will be considered: • 4 in./in. per °F, • 5 in./in. per °F, and • 6.5 in./in. per °F. Now, three different joint spacings (12 ft, 16 ft, and 20 ft) also will be considered. For this example, it is necessary to compare multiple means within each factor (main effects) and the interaction between the two factors (interactive effects). The statistical technique involved is called a multifactorial two-way ANOVA. 2. Identification and Description of Variables: The engineer identifies uniform 1-mile pavement sections within the state highway network with similar attributes (e.g., slab thickness, traffic, and climate). The field performance, in terms of observed percentage of each slab cracked (% slab cracked) after about 20 years of service for each pavement section, is considered the

examples of effective experiment Design and Data analysis in transportation research 45 dependent (or response) variable in the analysis. The available pavement data are stratified based on CTE and JS. CTE and JS are considered the independent variables. The question is whether pavement sections having different CTE and JS exhibit similar performance based on their variability. Question/Issue Use collected data to determine the effects of varying more than one independent variable on some measured outcome. In this example, compare the cracking perfor- mance of concrete pavements considering two independent variables: (1) coefficients of thermal expansion (CTE) as measured using more than two types of aggregate and (2) differing joint spacing (JS). More formally, the hypotheses can be stated as follows: Ho : ai = 0, No difference in % slabs cracked for different CTE values. Ho : gj = 0, No difference in % slabs cracked for different JS values. Ho : (ag)ij = 0, for all i and j, No difference in % slabs cracked for different CTE and JS combinations. 3. Data Collection: The descriptive statistics for % slab cracked data by three CTE and three JS categories are shown in Table 17. From the data stratified by CTE and JS, the engineer has randomly selected three pavement sections within each of nine combinations of CTE values. (In other words, for each of the nine pavement sections from Example 10, the engineer has selected three JS.) 4. Specification of Analysis Technique and Data Analysis: The engineer can use two-way ANOVA test statistics to determine whether the between-section variability is large relative to the within-section variability for each factor to test the following null hypotheses: • Ho : ai = 0 • Ho : gj = 0 • Ho : (ag)ij = 0 As mentioned before, although rejection of the null hypothesis does give the engineer some information concerning differences among the population means (i.e., there are differences among them), it does not clarify which means differ from each other. For example, does µ1 differ from µ2 or µ3? To control the experiment-wise error rate (EER) for the comparison of multiple means, a conservative test—Tukey’s procedure for an unplanned comparison—can be used. (Information about two-way ANOVA is available in NCHRP Project 20-45, Volume 2, CTE (in/in per oF) Marginal µ & σ 4 5 6.5 Joint spacing (ft) 12 1,1 = 32.4 s1,1 = 0.1 1,2 = 46.8 s1,2 = 1.8 1,3 = 65.3 s 1,3 = 3.2 1,. = 48.2 s1,. = 14.4 16 2,1 = 36.0 s2,1 = 2.4 2,2 = 54 s2,2 = 2.9 2,3 = 73 s2,3 = 1.1 2,. = 54.3 s2,. = 16.1 20 3,1 = 42.7 s3,1 = 2.4 3,2 = 60.3 s3,2 = 0.5 3,3 = 79.1 s3,3 = 2.0 3,. = 60.7 s3,. = 15.9 Marginal µ & σ .,1 = 37.0 x– x– x– x– x– x– x– x– x– x– x– x– x– x– x– x– s.,1 = 4.8 .,2 = 53.7 s.,2 = 6.1 .,3 = 72.5 s.,3 = 6.3 .,. = 54.4 s.,. = 15.8 Note: n = 3 in each cell; values are cell means and standard deviations. Table 17. Summary of cracking data.

46 effective experiment Design and Data analysis in transportation research Chapter 4. Information about Tukey’s procedure can be found in almost any good statistics textbook, such as those by Freund and Wilson [2003] and Kutner et al. [2005].) The results of the two-way ANOVA are shown in Table 18. From the first line it can be seen that both of the main effects, CTE and JS, are significant in explaining cracking behavior (i.e., both p-values < 0.05). However, the interaction (CTE × JS) is not significant (i.e., the p-value is 0.999, much greater than 0.05). Also, the test for homogeneity of variances (Levene statistic) shows that there is no significant difference among the standard deviations of % slab cracked for different CTE and JS values. Figure 11 illustrates the main and interactive effects of CTE and JS on % slabs cracked. 5. Interpreting the Results: A two-way (multifactorial) ANOVA is conducted to determine if difference exists among the mean values for “% slab cracked” for different CTE and JS values. The analysis shows that the main effects of both CTE and JS are significant, while the inter- action effect is insignificant (p-value > 0.05). These results show that when CTE and JS are considered jointly, they significantly impact the slab cracking separately. Given these results, the conclusions from the results will be based on the main effects alone without considering interaction effects. In fact, if the interaction effect had been significant, the conclusions would be based on them. To gain more insight, the engineer can use Tukey’s procedure to compare specific multiple means within each factor, or the engineer can simply observe the plotted means in Figure 11 to ascertain which means are significantly different from each other. The plotted results show that the mean % slab cracked varies significantly for different CTE and JS values; that is, the CTE seems to be more influential than JS. All lines are almost parallel to Source Sum of Squares (SS) Degrees of Freedom (df) Mean Square (MS) F Significance CTE 5677.74 2 2838.87 657.16 0.000 JS 703.26 2 351.63 81.40 0.000 CTE × JS 0.12 4 0.03 0.007 0.999 Residual/error 77.76 18 4.32 Total 6458.88 26 Table 18. ANOVA results. M ea n % s la bs c ra ck ed 6.55.04.0 75 70 65 60 55 50 45 40 35 201612 CTE JS Main Effects Plot (data means) for Cracking Joint Spacing (ft) M ea n % s la bs c ra ck ed 201612 80 70 60 50 40 30 CTE 6.5 4.0 5.0 Interaction Plot (data means) for Cracking Figure 11. Main and interaction effects of CTE and JS on slab cracking.

examples of effective experiment Design and Data analysis in transportation research 47 each other when plotted for both factors together, showing no interactive effects between the levels of two factors. 6. Conclusion and Discussion: The two-way ANOVA can be used to verify the combined effects of CTE and JS on cracking performance of rigid pavements. The marginal mean plot for cracking having three different CTE and JS levels visually illustrates the differences in the multiple means. The plot of cell means for cracking within the levels of each factor can indicate the presence of interactive effect between two factors (in this example, CTE and JS). However, the F-test for multiple means should be used to formally test the hypothesis of mean equality. Finally, based on the comparison of three means within each factor (CTE and JS), the engineer can conclude that the pavement slabs having aggregates with higher CTE and JS values will exhibit more cracking than those with lower CTE and JS values. In this example, the effect of CTE on concrete pavement cracking seems to be more critical than that of JS. 7. Applications in Other Areas of Transportation Research: Multifactorial designs can be used when more than one factor is considered in a study. Possible applications of these methods can extend to all transportation-related areas, including: • Pavement Engineering – to determine the effects of base type and base thickness on pavement performance of flexible pavements. Two or more levels can be considered within each factor; for exam- ple, two base types (aggregate and asphalt-treated bases) and three base thicknesses (8 inches, 12 inches, and 18 inches). – to investigate the impact of pavement surface conditions and vehicle type on fuel con- sumption. The researcher can select pavement sections with three levels of ride quality (smooth, rough, and very rough) and three types of vehicles (cars, vans, and trucks). The fuel consumptions can be measured for each vehicle type on all surface conditions to determine their impact. • Materials – to study the effects of aggregate gradation and surface on tensile strength of hot-mix asphalt (HMA). The engineer can evaluate two levels of gradation (fine and coarse) and two types of aggregate surfaces (smooth and rough). The samples can be prepared for all the combinations of aggregate gradations and surfaces for determination of tensile strength in the laboratory. – to compare the impact of curing and cement types on the compressive strength of concrete mixture. The engineer can design concrete mixes in laboratory utilizing two cement types (Type I & Type III). The concrete samples can be cured in three different ways for 24 hours and 7 days (normal curing, water bath, and room temperature). Example 12: Work Zones; Simple Before-and-After Comparisons Area: Work zones Method of Analysis: Simple before-and-after comparisons (exploring the effect of some treat- ment before it is applied versus after it is applied) 1. Research Question/Problem Statement: The crash rate in work zones has been found to be higher than the crash rate on the same roads when a work zone is not present. For this reason, the speed limit in construction zones often is set lower than the prevailing non-work-zone speed limit. The state DOT decides to implement photo-radar speed enforcement in a work zone to determine if this speed-enforcement technique reduces the average speed of free- flowing vehicles in the traffic stream. They measure the speeds of a sample of free-flowing vehicles prior to installing the photo-radar speed-enforcement equipment in a work zone and

48 effective experiment Design and Data analysis in transportation research then measure the speeds of free-flowing vehicles at the same location after implementing the photo-radar system. Question/Issue Use collected data to determine whether a difference exists between results before and after some treatment is applied. For this example, does a photo-radar speed- enforcement system reduce the speed of free-flowing vehicles in a work zone, and, if so, is the reduction statistically significant? 2. Identification and Description of Variables: The variable to be analyzed is the mean speed of vehicles before and after the implementation of a photo-radar speed-enforcement system in a work zone. 3. Data Collection: The speeds of individual free-flowing vehicles are recorded for 30 minutes on a Tuesday between 10:00 a.m. and 10:30 a.m. before installing the photo-radar system. After the system is installed, the speeds of individual free-flowing vehicles are recorded for 30 minutes on a Tuesday between 10:00 a.m. and 10:30 a.m. The before sample contains 120 observations and the after sample contains 100 observations. 4. Specification of Analysis Technique and Data Analysis: A test of the significance of the difference between two means requires a statement of the hypothesis to be tested (Ho) and a statement of the alternate hypothesis (H1). In this example, these hypotheses can be stated as follows: Ho: There is no difference in the mean speed of free-flowing vehicles before and after the photo-radar speed-enforcement system is displayed. H1: There is a difference in the mean speed of free-flowing vehicles before and after the photo-radar speed-enforcement system is displayed. Because these two samples are independent, a simple t-test is appropriate to test the stated hypotheses. This test requires the following procedure: Step 1. Compute the mean speed (x _ ) for the before sample (x _ b) and the after sample (x _ a) using the following equation: x x n n ni i i n i b a= = = = ∑ 1 120 100; and Results: x _ b = 53.1 mph and x _ a = 50.5 mph. Step 2. Compute the variance (S2) for each sample using the following equation: S x x n i i i n 2 2 1 1 = −( ) − − ∑ where na = 100; x _ a= 50.5 mph; nb = 120; and x _ b = 53.1 mph Results: S x x n b b b b 2 2 1 12 06= −( ) − =∑ . and S x x n a a a a 2 2 1 12 97= −( ) − =∑ . . Step 3. Compute the pooled variance of the two samples using the following equation: S x x x x n n p a a b b b a 2 2 2 2 = −( ) + −( ) + − ∑∑ Results: S2p = 12.472 and Sp = 3.532.

examples of effective experiment Design and Data analysis in transportation research 49 Step 4. Compute the t-statistic using the following equation: t x x S n n n n b a p a b a b = − + Result: t = − ( )( ) + = 53 1 50 5 3 532 100 120 100 120 5 43 . . . . . 5. Interpreting the Results: The results of the sample t-test are obtained by comparing the value of the calculated t-statistic (5.43 in this example) with the value of the t-statistic for the level of confidence desired. For a level of confidence of 95%, the t-statistic must be greater than 1.96 to reject the null hypotheses (Ho) that the use of a photo-radar speed-enforcement sys- tem does not change the speed of free-flowing vehicles. (For more information, see NCHRP Project 20-45, Volume 2, Appendix C, Table C-4.) 6. Conclusion and Discussion: The sample problem illustrates the use of a statistical test to determine whether the difference in the value of the variable of interest between the before conditions and the after conditions is statistically significant. The before condition is without photo-radar speed enforcement; and the after condition is with photo-radar speed enforcement. In this sample problem, the computed t-statistic (5.43) is greater than the critical t-statistic (1.96), so the null hypothesis is rejected. This means the change in the speed of free-flowing vehicles when the photo-radar speed-enforcement system is used is statistically significant. The assumption is made that all other factors that would affect the speed of free-flowing vehicles (e.g., traffic mix, weather, or construction activity) are the same in the before-and-after conditions. This test is robust if the normality assumption does not hold completely; however, it should be checked using box plots. For significant departures from normality and variance equality assumptions, non-parametric tests must be conducted. (For more information, see NCHRP Project 20-45, Volume 2, Chapter 6, Section C and also Example 21). The reliability of the results in this example could be improved by using a control group. As the example has been constructed, there is an assumption that the only thing that changed at this site was the use of photo-radar speed enforcement; that is, it is assumed that all observed differences are attributable to the use of the photo-radar. If other factors—even something as simple as a general decrease in vehicle speeds in the area—might have impacted speed changes, the effect of the photo-radar speed enforcement would have to be adjusted for those other factors. Measurements taken at a control site (ideally identical to the experiment site) during the same time periods could be used to detect background changes and then to adjust the photo-radar effects. Such a situation is explored in Example 13. 7. Applications in Other Areas in Transportation: The before-and-after comparison can be used whenever two independent samples of data are (or can be assumed to be) normally distributed with equal variance. Applications of before-and-after comparison in other areas of transportation research may include: • Traffic Operations – to compare the average delay to vehicles approaching a signalized intersection when a fixed time signal is changed to an actuated signal or a traffic-adaptive signal. – to compare the average number of vehicles entering and leaving a driveway when access is changed from full access to right-in, right-out only. • Traffic Safety – to compare the average number of crashes on a section of road before and after the road is resurfaced. – to compare the average number of speeding citations issued per day when a stationary operation is changed to a mobile operation. • Maintenance—to compare the average number of citizen complaints per day when a change is made in the snow plowing policy.

50 effective experiment Design and Data analysis in transportation research Example 13: Traffic Safety; Complex Before-and-After Comparisons and Controls Area: Traffic safety Method of Analysis: Complex before-and-after comparisons using control groups (examining the effect of some treatment or application with consideration of other factors that may also have an effect) 1. Research Question/Problem Statement: A state safety engineer wants to estimate the effec- tiveness of fluorescent orange warning signs as compared to standard orange signs in work zones on freeways and other multilane highways. Drivers can see fluorescent signs from a longer distance than standard signs, especially in low-visibility conditions, and the extra cost of the fluorescent material is not too high. Work-zone safety is a perennial concern, especially on freeways and multilane highways where speeds and traffic volumes are high. Question/Issue How can background effects be separated from the effects of a treatment or application? Compared to standard orange signs, do fluorescent orange warning signs increase safety in work zones on freeways and multilane highways? 2. Identification and Description of Variables: The engineer quickly concludes that there is a need to collect and analyze safety surrogate measures (e.g., traffic conflicts and late lane changes) rather than collision data. It would take a long time and require experimentation at many work zones before a large sample of collision data could be ready for analysis on this question. Surrogate measures relate to collisions, but they are much more numerous and it is easier to collect a large sample of them in a short time. For a study of traffic safety, surrogate measures might include near-collisions (traffic conflicts), vehicle speeds, or locations of lane changes. In this example, the engineer chooses to use the location of the lane-change maneuver made by drivers in a lane to be closed entering a work zone. This particular surrogate safety measure is a measure of effectiveness (MOE). The hypothesis is that the farther downstream at which a driver makes a lane change out of a lane to be closed—when the highway is still below capacity—the safer the work zone. 3. Data Collection: The engineer establishes site selection criteria and begins examining all active work zones on freeways and multilane highways in the state for possible inclusion in the study. The site selection criteria include items such as an active work zone, a cooperative contractor, no interchanges within the approach area, and the desired lane geometry. Seven work zones meet the criteria and are included in the study. The engineer decides to use a before-and-after (sometimes designated B/A or b/a) experiment design with randomly selected control sites. The latter are sites in the same population as the treatment sites; that is, they meet the same selection criteria but are untreated (i.e., standard warning signs are employed, not the fluorescent orange signs). This is a strong experiment design because it minimizes three common types of bias in experiments: history, maturation, and regression to the mean. History bias exists when changes (e.g., new laws or large weather events) happen at about the same time as the treatment in an experiment, so that the engineer or analyst cannot separate the effect of the treatment from the effects of the other events. Maturation bias exists when gradual changes occur throughout an extended experiment period and cannot be separated from the effects of the treatment. Examples of maturation bias might involve changes like the aging of driver populations or new vehicles with more air bags. History and maturation biases are referred to as specification errors and are described in more detail in NCHRP Project 20-45, Volume 2,

examples of effective experiment Design and Data analysis in transportation research 51 Chapter 1, in the section “Quasi-Experiments.” Regression-to-the-mean bias exists when sites with the highest MOE levels in the before time period are treated. If the MOE level falls in the after period, the analyst can never be sure how much of the fall was due to the treatment and how much was due to natural fluctuations in the values of the MOE back toward its usual mean value. A before-and-after study with randomly selected control sites minimizes these biases because their effects are expected to apply just as much to the treatment sites as to the control sites. In this example, the engineer randomly selects four of the seven work zones to receive fluorescent orange signs. The other three randomly selected work zones received standard orange signs and are the control sites. After the signs have been in place for a few weeks (a common tactic in before-and-after studies to allow regular drivers to get used to the change), the engineer collects data at all seven sites. The location of each vehicle’s lane-change maneuver out of the lane to be closed is measured from video tape recorded for several hours at each site. Table 19 shows the lane-change data at the midpoint between the first warning sign and beginning of the taper. Notice that the same number of vehicles is observed in the before-and- after periods for each type of site. 4. Specification of Analysis Technique and Data Analysis: Depending on their format, data from a before-and-after experiment with control sites may be analyzed several ways. The data in the table lend themselves to analysis with a chi-square test to see whether the distributions between the before-and-after conditions are the same at both the treatment and control sites. (For more information about chi-square testing, see NCHRP Project 20-45, Volume 2, Chapter 6, Section E, “Chi-Square Test for Independence.”) To perform the chi-square test on the data for Example 13, the engineer first computes the expected value in each cell. For the cell corresponding to the before time period for control sites, this value is computed as the row total (3361) times the column total (2738) divided by the grand total (6714): 3361 2738 6714 1371 = vehicles The engineer next computes the chi-square value for each cell using the following equation: χi i i i O E E 2 2 = −( ) where Oi is the number of actual observations in cell i and Ei is the expected number of observations in cell i. For example, the chi-square value in the cell corresponding to the before time period for control sites is (1262 - 1371)2 / 1371 = 8.6. The engineer then sums the chi-square values from all four cells to get 29.1. That sum is then compared to the critical chi-square value for the significance level of 0.025 with 1 degree of freedom (degrees of freedom = number of rows - 1 * number of columns - 1), which is shown on a standard chi-square distribution table to be 5.02 (see NCHRP Project 20-45, Volume 2, Appendix C, Table C-2.) A significance level of 0.025 is not uncommon in such experiments (although 0.05 is a general default value), but it is a standard that is difficult but not impossible to meet. Time Period Number of Vehicles Observed in Lane to be Closed at Midpoint Control Treatment Total Before 1262 2099 3361 After 1476 1877 3353 Total 2738 3976 6714 Table 19. Lane-change data for before-and-after comparison using controls.

52 effective experiment Design and Data analysis in transportation research 5. Interpreting the Results: Because the calculated chi-square value is greater than the critical chi-square value, the engineer concludes that there is a statistically significant difference in the number of vehicles in the lane to be closed at the midpoint between the before-and-after time periods for the treatment sites relative to what would be expected based on the control sites. In other words, there is a difference that is due to the treatment. 6. Conclusion and Discussion: The experiment results show that fluorescent orange signs in work zone approaches like those tested would likely have a safety benefit. Although the engi- neer cannot reasonably estimate the number of collisions that would be avoided by using this treatment, the before-and-after study with control using a safety surrogate measure makes it clear that some collisions will be avoided. The strength of the experiment design with randomly selected control sites means that agencies can have confidence in the results. The consequences of an error in an analysis like this that results in the wrong conclusion can be devastating. If the error leads an agency to use a safety measure more than it should, precious safety funds will be wasted that could be put to better use. If the error leads an agency to use the safety measure less often than it should, money will be spent on measures that do not prevent as many collisions. With safety funds in such short supply, solid analyses that lead to effective decisions on countermeasure deployment are of great importance. A before-and-after experiment with control is difficult to arrange in practice. Such an experiment is practically impossible using collision data, because that would mean leaving some higher collision sites untreated during the experiment. Such experiments are more plausible using surrogate measures like the one described in this example. 7. Applications in Other Areas of Transportation Research: Before-and-after experiments with randomly selected control sites are difficult to arrange in transportation safety and other areas of transportation research. The instinct to apply treatments to the worst sites, rather than randomly—as this method requires—is difficult to overcome. Despite the difficulties, such experiments are sometimes performed in: • Traffic Operations—to test traffic control strategies at a number of different intersections. • Pavement Engineering—to compare new pavement designs and maintenance processes to current designs and practice. • Materials—to compare new materials, mixes, or processes to standard mixtures or processes. Example 14: Work Zones; Trend Analysis Area: Work zones Method of Analysis: Trend analysis (examining, describing, and modeling how something changes over time) 1. Research Question/Problem Statement: Measurements conducted over time often reveal patterns of change called trends. A model may be used to predict some future measurement, or the relative success of a different treatment or policy may be assessed. For example, work/ construction zone safety has been a concern for highway officials, engineers, and planners for many years. Is there a pattern of change? Question/Issue Can a linear model represent change over time? In this particular example, is there a trend over time for motor vehicle crashes in work zones? The problem is to predict values of crash frequency at specific points in time. Although the question is simple, the statistical modeling becomes sophisticated very quickly.

examples of effective experiment Design and Data analysis in transportation research 53 2. Identification and Description of Variables: Highway safety, rather the lack of it, is revealed by the total number of fatalities due to motor vehicle crashes. The percentage of those deaths occurring in work zones reveals a pattern over time (Figure 12). The data points for the graph are calculated using the following equation: WZP a b YEAR u= + + where WZP = work zone percentage of total fatalities, YEAR = calendar year, and u = an error term, as used here. 3. Data Collection: The base data are obtained from the Fatality Analysis Reporting System maintained by the National Highway Traffic Safety Administration (NHTSA), as reported at www.workzonesafety.org. The data are state specific as well as for the country as a whole, and cover a period of 26 years from 1982 through 2007. The numbers of fatalities from motor vehicle crashes in and not in construction/maintenance zones (work zones) are used to compute the percentage of fatalities in work zones for each of the 26 years. 4. Specification of Analysis Techniques and Data Analysis: Ordinary least squares (OLS) regression is used to develop the general model specified above. The discussion in this example focuses on the resulting model and the related statistics. (See also examples 15, 16, and 17 for details on calculations. For more information about OLS regression, see NCHRP Project 20-45, Volume 2, Chapter 4, Section B, “Linear Regression.”) Looking at the data in Figure 12 another way, WZP = -91.523 (-8.34) (0.000) + 0.047(YEAR) (8.51) (0.000) R = 0.867 t-values p-values R2 = 0.751 The trend is significant: the line (trend) shows an increase of 0.047% each year. Generally, this trend shows that work-zone fatalities are increasing as a percentage of total fatalities. 5. Interpreting the Results: This experiment is a good fit and generally shows that work-zone fatalities were an increasing problem over the period 1982 through 2007. This is a trend that highway officials, engineers, and planners would like to change. The analyst is therefore interested in anticipating the trajectory of the trend. Here the trend suggests that things are getting worse. Figure 12. Percentage of all motor vehicle fatalities occurring in work zones.

54 effective experiment Design and Data analysis in transportation research How far might authorities let things go—5%? 10%? 25%? Caution must be exercised when interpreting a trend beyond the limits of the available data. Technically the slope, or b-coefficient, is the trend of the relationship. The a-term from the regression, also called the intercept, is the value of WZP when the independent variable equals zero. The intercept for the trend in this example would technically indicate that the percentage of motor vehicle fatalities in work zones in the year zero would be -91.5%. This is absurd on many levels. There could be no motor vehicles in year zero, and what is a negative percentage of the total? The absurdity of the intercept in this example reveals that trends are limited concepts, limited to a relevant time frame. Figure 12 also suggests that the trend, while valid for the 26 years in aggregate, doesn’t work very well for the last 5 years, during which the percentages are consistently falling, not rising. Something seems to have changed around 2002; perhaps the highway officials, engineers, and planners took action to change the trend, in which case, the trend reversal would be considered a policy success. Finally, some underlying assumptions must be considered. For example, there is an implicit assumption that the types of roads with construction zones are similar from year to year. If this assumption is not correct (e.g., if a greater number of high speed roads, where fatalities may be more likely, are worked on in some years than in others), then interpreting the trend may not make much sense. 6. Conclusion and Discussion: The computation of this dependent variable (the percent of motor-vehicle fatalities occurring in work zones, or MZP) is influenced by changes in the number of work-zone fatalities and the number of non-work-zone fatalities. To some extent, both of these are random variables. Accordingly, it is difficult to distinguish a trend or trend reversal from a short series of possibly random movements in the same direction. Statistically, more observations permit greater confidence in non-randomness. It is also possible that a data series might be recorded that contains regular, non-random movements that are unrelated to a trend. Consider the dependent variable above (MZP), but measured using monthly data instead of annual data. Further, imagine looking at such data for a state in the upper Midwest instead of for the nation as a whole. In this new situation, the WZP might fall off or halt altogether each winter (when construction and maintenance work are minimized), only to rise again in the spring (reflecting renewed work-zone activity). This change is not a trend per se, nor is it random. Rather, it is cyclical. 7. Applications in Other Areas of Transportation Research: Applications of trend analysis models in other areas of transportation research include: • Transportation Safety—to identify trends in traffic crashes (e.g., motor vehicle/deer) over time on some part of the roadway system (e.g., freeways). • Public Transportation—to determine the trend in rail passenger trips over time (e.g., in response to increasing gas prices). • Pavement Engineering—to monitor the number of miles of pavement that is below some service-life threshold over time. • Environment—to monitor the hours of truck idling time in rest areas over time. Example 15: Structures/Bridges; Trend Analysis Area: Structures/bridges Method of Analysis: Trend analysis (examining a trend over time) 1. Research Question/Problem Statement: A state agency wants to monitor trends in the condition of bridge superstructures in order to perform long-term needs assessment for bridge rehabilitation or replacement. Bridge condition rating data will be analyzed for bridge

examples of effective experiment Design and Data analysis in transportation research 55 2. Identification and Description of Variables: Bridge inspection generally entails collection of numerous variables including location information, traffic data, structural elements (type and condition), and functional characteristics. Based on the severity of deterioration and the extent of spread through a bridge component, a condition rating is assigned on a dis- crete scale from 0 (failed) to 9 (excellent). Generally a condition rating of 4 or below indicates deficiency in a structural component. The state agency inspects approximately 300 bridges every year (denominator). The number of superstructures that receive a rating of 4 or below each year (number of events, numerator) also is recorded. The agency is concerned with the change in overall rate (calculated per 100) of structurally deficient bridge superstructures. This rate, which is simply the ratio of the numerator to the denominator, is the indicator (dependent variable) to be examined for trend over a time period of 15 years. Notice that the unit of analysis is the time period and not the individual bridge superstructures. 3. Data Collection: Data are collected for bridges scheduled for inspection each year. It is important to note that the bridge condition rating scale is based on subjective categories, and therefore there may be inherent variability among inspectors in their assignments of rates to bridge superstructures. Also, it is assumed that during the time period for which the trend analysis is conducted, no major changes are introduced in the bridge inspection methods. Sample data provided in Table 20 show the rate (per 100), number of bridges per year that received a score of four or below, and total number of bridges inspected per year. 4. Specification of Analysis Technique and Data Analysis: The data set consists of 15 observa- tions, one for each year. Figure 13 shows a scatter plot of the rate (dependent variable) versus time in years. The scatter plot does not indicate the presence of any outliers. The scatter plot shows a seemingly increasing linear trend in the rate of deficient superstructures over time. No need for data transformation or smoothing is apparent from the examination of the scatter plot in Figure 13. To determine whether the apparent linear trend is statistically significant in this data, ordinary least squares (OLS) regression can be employed. Question/Issue Use collected data to determine if the values that some variables have taken show an increasing trend or a decreasing trend over time. In this example, determine if levels of structural deficiency in bridge superstructures have been increasing or decreasing over time, and determine how rapidly the increase or decrease has occurred. No. Year Rate (per 100) Number of Events (Numerator) Number of Bridges Inspected (Denominator) 1 1990 8.33 25 300 2 1991 8.70 26 299 5 1994 10.54 31 294 11 2000 13.55 42 310 15 2004 14.61 45 308 Table 20. Sample bridge inspection data. superstructures that have been inspected over a period of 15 years. The objective of this study is to examine the overall pattern of change in the indicator variable over time.

56 effective experiment Design and Data analysis in transportation research The linear regression model takes the following form: y x ei o i i= + +β β1 where i = 1, 2, . . . , n (n = 15 in this example), y = dependent variable (rate of structurally deficient bridge superstructures), x = independent variable (time), bo = y-intercept (only provides reference point), b1 = slope (change in unit y for a change in unit x), and ei = residual error. The first step is to estimate the bo and b1 in the regression function. The residual errors (e) are assumed to be independently and identically distributed (i.e., they are mutually independent and have the same probability distribution). b1 and bo can be computed using the following equations: ˆ . ˆ β β 1 1 2 1 0 454= −( ) −( ) −( ) = = = = ∑ ∑ x x y y x x i i i n i i n o y x− =β1 8 396. where y _ is the overall mean of the dependent variable and x _ is the overall mean of the independent variable. The prediction equation for rate of structurally deficient bridge superstructures over time can be written using the following equation: ˆ ˆ ˆ . .y x xo= + = +β β1 8 396 0 454 That is, as time increases by a year, the rate of structurally deficient bridge superstructures increases by 0.454 per 100 bridges. The plot of the regression line is shown in Figure 14. Figure 14 indicates some small variability about the regression line. To conduct hypothesis testing for the regression relationship (Ho: b1 = 0), assessment of this variability and the assumption of normality would be required. (For a discussion on assumptions for residual errors, see NCHRP Project 20-45, Volume 2, Chapter 4.) Like analysis of variance (ANOVA, described in examples 8, 9, and 10), statistical inference is initiated by partitioning the total sum of squares (TSS) into the error sum of squares (SSE) Figure 13. Scatter plot of time versus rate. 7.00 9.00 11.00 13.00 15.00 Time in years Ra te p er 1 00 1 3 5 7 9 11 13 15

examples of effective experiment Design and Data analysis in transportation research 57 and the model sum of squares (SSR). That is, TSS = SSE + SSR. The TSS is defined as the sum of the squares of the difference of each observation from the overall mean. In other words, deviation of observation from overall mean (TSS) = deviation of observation from prediction (SSE) + deviation of prediction from overall mean (SSR). For our example, TSS y y SSR x x i i n i = −( ) = = −( ) = = ∑ 2 1 1 2 2 60 892 57 7 . ˆ .β 90 3 102 1i n SSE TSS SSR = ∑ = − = . Regression analysis computations are usually summarized in a table (see Table 21). The mean squared errors (MSR, MSE) are computed by dividing the sums of squares by corresponding model and error degrees of freedom. For the null hypothesis (Ho: b1 = 0) to be true, the expected value of MSR is equal to the expected value of MSE such that F = MSR/MSE should be a random draw from an F-distribution with 1, n - 2 degrees of freedom. From the regression shown in Table 21, F is computed to be 242.143, and the probability of getting a value larger than the F computed is extremely small. Therefore, the null hypothesis is rejected; that is, the slope is significantly different from zero, and the linearly increasing trend is found to be statistically significant. Notice that a slope of zero implies that knowing a value of the independent variable provides no insight on the value of the dependent variable. 5. Interpreting the Results: The linear regression model does not imply any cause-and-effect relationship between the independent and dependent variables. The y-intercept only provides a reference point, and the relationship need not be linear outside the data range. The 95% confidence interval for b1 is computed as [0.391, 0.517]; that is, the analyst is 95% confident that the true mean increase in the rate of structurally deficient bridge superstructures is between Plot of regression line y = 8.396 + 0.454x R2 = 0.949 7.00 9.00 11.00 13.00 15.00 1 3 5 7 9 11 13 15 Time in years Ra te p er 1 00 Figure 14. Plot of regression line. Source Sum of Squares (SS) Degrees of Freedom (df) Mean Square F Significance Regression 57.790 1 57.790 (MSR) 242.143 8.769e-10 Error 3.102 13 0.239 (MSE) Total 60.892 14 Table 21. Analysis of regression table.

58 effective experiment Design and Data analysis in transportation research 0.391% and 0.517% per year. (For a discussion on computing confidence intervals, see NCHRP Project 20-45, Volume 2, Chapter 4.) The coefficient of determination (R2) provides an indication of the model fit. For this example, R2 is calculated using the following equation: R SSE TSS 2 0 949= = . The R2 indicates that the regression model accounts for 94.9% of the total variation in the (hypothetical) data. It should be noted that such a high value of R2 is almost impossible to attain from analysis of real observational data collected over a long time. Also, distributional assumptions must be checked before proceeding with linear regression, as serious violations may indicate the need for data transformation, use of non-linear regression or non-parametric methods, and so on. 6. Conclusion and Discussion: In this example, simple linear regression has been used to deter- mine the trend in the rate of structurally deficient bridge superstructures in a geographic area. In addition to assessing the overall patterns of change, trend analysis may be performed to: • study the levels of indicators of change (or dependent variables) in different time periods to evaluate the impact of technical advances or policy changes; • compare different geographic areas or different populations with perhaps varying degrees of exposure in absolute and relative terms; and • make projections to monitor progress toward an objective. However, given the dynamic nature of trend data, many of these applications require more sophisticated techniques than simple linear regression. An important aspect of examining trends over time is the accuracy of numerator and denominator data. For example, bridge structures may be examined more than once during the analysis time period, and retrofit measures may be taken at some deficient bridges. Also, the age of structures is not accounted for in this analysis. For the purpose of this example, it is assumed that these (and other similar) effects are negligible and do not confound the data. In real-life application, however, if the analysis time period is very long, it becomes extremely important to account for changes in factors that may have affected the dependent variable(s) and their measurement. An example of the latter could be changes in the volume of heavy trucks using the bridge, changes in maintenance policies, or changes in plowing and salting regimes. 7. Applications in Other Areas of Transportation Research: Trend analysis is carried out in many areas of transportation research, such as: • Transportation Planning/Traffic Operations—to determine the need for capital improve- ments by examining traffic growth over time. • Traffic Safety—to study the trends in overall, fatal, and/or injury crash rates over time in a geographic area. • Pavement Engineering—to assess the long-term performance of pavements under varying loads. • Environment—to monitor the emission levels from commercial traffic over time with growth of industrial areas. Example 16: Transportation Planning; Multiple Regression Analysis Area: Transportation planning Method of Analysis: Multiple regression analysis (testing proposed linear models with more than one independent variable when all variables are continuous)

examples of effective experiment Design and Data analysis in transportation research 59 1. Research Question/Problem Statement: Transportation planners and engineers often work on variations of the classic four-step transportation planning process for estimat- ing travel demand. The first step, trip generation, generally involves developing a model that can be used to predict the number of trips originating or ending in a zone, which is a geographical subdivision of a corridor, city, or region (also referred to as a traffic analysis zone or TAZ). The objective is to develop a statistical relationship (a model) that can be used to explain the variation in a dependent variable based on the variation of one or more independent variables. In this example, ordinary least squares (OLS) regres- sion is used to develop a model between trips generated (the dependent variable) and demographic, socio-economic, and employment variables (independent variables) at the household level. Question/Issue Can a linear relationship (model) be developed between a dependent variable and one or more independent variables? In this application, the dependent variable is the number of trips produced by households. Independent variables include persons, workers, and vehicles in a household, household income, and average age of persons in the household. The basic question is whether the relationship between the dependent (Y) and independent (X) variables can be represented by a linear model using two coefficients (a and b), expressed as follows: Y X= +a b i where a = the intercept and b = the slope of the line. If the relationship being examined involves more than one independent variable, the equa- tion will simply have more terms. In addition, in a more formal presentation, the equation will also include an error term, e, added at the end. 2. Identification and Description of Variables: Data for four-step modeling of travel demand or for calibration of any specific model (e.g., trip generation or trip origins) come from a variety of sources, ranging from the U.S. Census to mail or telephone surveys. The data that are collected will depend, in part, on the specific purpose of the modeling effort. Data appropriate for a trip-generation model typically are collected from some sort of household survey. For the dependent variable in a trip-generation model, data must be collected on trip-making characteristics. These characteristics could include something as simple as the total trips made by a household in a day or involve more complicated break- downs by trip purpose (e.g., work-related trips versus shopping trips) and time of day (e.g., trips made during peak and non-peak hours). The basic issue that must be addressed is to determine the purpose of the proposed model: What is to be estimated or predicted? Weekdays and work trips normally are associated with peak congestion and are often the focus of these models. For the independent variable(s), the analyst must first give some thought to what would be the likely causes for household trips to vary. For example, it makes sense intuitively that household size might be pertinent (i.e., it seems reasonable that more persons in the household would lead to a higher number of household trips). Household members could be divided into workers and non-workers, two variables instead of one. Likewise, other socio-economic characteristics, such as income-related variables, might also make sense as candidate variables for the model. Data are collected on a range of candidate variables, and

60 effective experiment Design and Data analysis in transportation research the analysis process is used to sort through these variables to determine which combination leads to the best model. To be used in ordinary regression modeling, variables need to be continuous; that is, measured ratio or interval scale variables. Nominal data may be incorporated through the use of indicator (dummy) variables. (For more information on continuous variables, see NCHRP Project 20-45, Volume 2, Chapter 1; for more information on dummy variables, see NCHRP Project 20-45, Volume 2, Chapter 4). 3. Data Collection: As noted, data for modeling travel demand often come from surveys designed especially for the modeling effort. Data also may be available from centralized sources such as a state DOT or local metropolitan planning organization (MPO). 4. Specification of Analysis Techniques and Data Analysis: In this example, data for 178 house- holds in a small city in the Midwest have been provided by the state DOT. The data are obtained from surveys of about 15,000 households all across the state. This example uses only a tiny portion of the data set (see Table 22). Based on the data, a fairly obvious relationship is initially hypothesized: more persons in a household (PERS) should produce more person- trips (TRIPS). In its simplest form, the regression model has one dependent variable and one independent variable. The underlying assumption is that variation in the independent variable causes the variation in the dependent variable. For example, the dependent variable might be TRIPSi (the count of total trips made on a typical weekday), and the independent variable might be PERS (the total number of persons, or occupants, in the household). Expressing the relation- ship between TRIPS and PERS for the ith household in a sample of households results in the following hypothesized model: TRIPS PERSi i i= + +a b i ε where a and b are coefficients to be determined by ordinary least squares (OLS) regression analysis and ei is the error term. The difference between the value of TRIPS for any household predicted using the devel- oped equation and the actual observed value of TRIPS for that same household is called the residual. The resulting model is an equation for the best fit straight line (for the given data) where a is the intercept and b is the slope of the line. (For more information about fitted regression and measures of fit see NCHRP Project 20-45, Volume 2, Chapter 4). In Table 22, R is the multiple R, the correlation coefficient in the case of the simplest linear regression involving one variable (also called univariate regression). The R2 (coefficient of determination) may be interpreted as the proportion of the variance of the dependent variable explained by the fitted regression model. The adjusted R2 corrects for the number of independent variables in the equation. A “perfect” R2 of 1.0 could be obtained if one included enough independent variables (e.g., one for each observation), but doing so would hardly be useful. Coefficients t-values (statistics) p-values Measures of Fit a = 3.347 4.626 0.000 R = 0.510 b = 2.001 7.515 0.000 R2 = 0.260 Adjusted R2 = 0.255 Table 22. Regression model statistics.

examples of effective experiment Design and Data analysis in transportation research 61 Restating the now-calibrated model, TRIPS PERS= +4 626 7 515. . i The statistical significance of each coefficient estimate is evaluated with the p-values of calculated t-statistics, provided the errors are normally distributed. The p-values (also known as probability values) generally indicate whether the coefficients are significantly different from zero (which they need to be in order for the model to be useful). More formally stated, a p-value is the probability of a Type I error. In this example, the t- and p-values shown in Table 22 indicate that both a and b are sig- nificantly different from zero at a level of significance greater than the 99.9% confidence level. P-values are generally offered as two-tail (two-sided hypothesis testing) test values in results from most computer packages; one-tail (one-sided) values may sometimes be obtained by dividing the printed p-values by two. (For more information about one-sided versus two- sided hypothesis testing, see NCHRP Project 20-45, Volume 2, Chapter 4.) The R2 may be tested with an F-statistic; in this example, the F was calculated as 56.469 (degrees of freedom = 2, 176) (See NCHRP Project 20-45, Volume 2, Chapter 4). This means that the model explains a significant amount of the variation in the dependent variable. A plot of the estimated model (line) and the actual data are shown in Figure 15. A strict interpretation of this model suggests that a household with zero occupants (PERS = 0) will produce 3.347 trips per day. Clearly, this is not feasible because there can’t be a household of zero persons, which illustrates the kind of problem encountered when a model is extrapolated beyond the range of the data used for the calibration. In other words, a formal test of the intercept (the a) is not always meaningful or appropriate. Extension of the Model to Multivariate Regression: When the list of potential inde- pendent variables is considered, the researcher or analyst might determine that more than one cause for variation in the dependent variable may exist. In the current example, the question of whether there is more than one cause for variation in the number of trips can be considered. 0 1 2 3 4 5 6 7 8 9 10 PERS 0 10 20 30 40 TR IP S Figure 15. Plot of the line for the estimated model.

62 effective experiment Design and Data analysis in transportation research The model just discussed for evaluating the effect of one independent variable is called a uni- variate model. Should the final model for this example be multivariate? Before determining the final model, the analyst may want to consider whether a variable or variables exist that further clarify what has already been modeled (e.g., more persons cause more trips). The variable PERS is a crude measure, made up of workers and non-workers. Most households have one or two workers. It can be shown that a measure of the non-workers in the household is more effective in explaining trips than is total persons; so a new variable, persons minus workers (DEP), is calculated. Next, variables may exist that address entirely different causal relationships. It might be hypothesized that as the number of registered motor vehicles available in the household (VEH) increases, the number of trips will increase. It may also be argued that as household income (INC, measured in thousands of dollars) increases, the number of trips will increase. Finally, it may be argued that as the average age of household occupants (AVEAGE) increases, the number of trips will decrease because retired people generally make fewer trips. Each of these statements is based upon a logical argument (hypothesis). Given these arguments, the hypothesized multivariate model takes the following form: TRIPS DEP VEH INC AVEAGE= + + + + +a b c d ei i i i ε The results from fitting the multivariate model are given in Table 23. Results of the analysis of variance (ANOVA) for the overall model are shown in Table 24. 5. Interpreting the Results: It is common for regression packages to provide some values in scientific notation as shown for the p-values in Table 23. The coefficient d, showing the relationship of TRIPS with INC, is read 1.907 E-05, which in turn is read as 1.907  10-5 or 0.000001907. All coefficients are of the expected sign and significantly different from 0 (at the 0.05 level) except for d. However, testing the intercept makes little sense. (The intercept value would be the number of trips for a household with 0 vehicles, 0 income, 0 average age, and 0 depen- dents, a most unlikely household.) The overall model is significant as shown by the F-ratio and its p-value, meaning that the model explains a significant amount of the variation in Coefficients t-values (statistics) p-values Measures of Fit a = 8.564 6.274 3.57E-09* R = 0.589 b = 0.899 2.832 0.005 R2 = 0.347 c = 1.067 3.360 0.001 adjusted R2 = 0.330 d = 1.907E-05* 1.927 0.056 e = -0.098 -4.808 3.68E-06 *See note about scientific notation in Section 5, Interpreting the Results. Table 23. Results from fitting the multivariate model. ANOVA Sum of Squares (SS) Degrees of Freedom (df) F-ratio p-value Regression 1487.5 4 19.952 3.4E-13 Residual 2795.7 150 Table 24. ANOVA results for the overall model.

examples of effective experiment Design and Data analysis in transportation research 63 the dependent variable. This model should reliably explain 33% of the variance of house- hold trip generation. Caution should be exercised when interpreting the significance of the R2 and the overall model because it is not uncommon to have a significant F-statistic when some of the coefficients in the equation are not significant. The analyst may want to consider recalibrating the model without the income variable because the coefficient d was insignificant. 6. Conclusion and Discussion: Regression, particularly OLS regression, relies on several assumptions about the data, the nature of the relationships, and the results. Data are assumed to be interval or ratio scale. Independent variables generally are assumed to be measured without error, so all error is attributed to the model fit. Furthermore, indepen- dent variables should be independent of one another. This is a serious concern because the presence in the model of related independent variables, called multicollinearity, compro- mises the t-tests and confuses the interpretation of coefficients. Tests of this problem are available in most statistical software packages that include regression. Look for Variance- Inflation Factor (VIF) and/or Tolerance tests; most packages will have one or the other, and some will have both. In the example above where PERS is divided into DEP and workers, knowing any two variables allows the calculation of the third. Including all three variables in the model would be a case of extreme multicollinearity and, logically, would make no sense. In this instance, because one variable is a linear combination of the other two, the calculations required (within the analysis program) to calibrate the model would actually fail. If the independent variables are simply highly correlated, the regression coefficients (at a minimum) may not have intuitive meaning. In general, equations or models with highly correlated independent variables are to be avoided; alternative models that examine one variable or the other, but not both, should be analyzed. It is also important to analyze the error distributions. Several assumptions relate to the errors and their distributions (normality, constant variance, uncorrelated, etc.) In transportation plan- ning, spatial variables and associations might become important; they require more elaborate constructs and often different estimation processes (e.g., Bayesian, Maximum Likelihood). (For more information about errors and error distributions, see NCHRP Project 20-45, Volume 2, Chapter 4.) Other logical considerations also exist. For example, for the measurement units of the different variables, does the magnitude of the result of multiplying the coefficient and the measured variable make sense and/or have a reasonable effect on the predicted magnitude of the dependent variable? Perhaps more importantly, do the independent variables make sense? In this example, does it make sense that changes in the number of vehicles in the household would cause an increase or decrease in the number of trips? These are measures of operational significance that go beyond consideration of statistical significance, but are no less important. 7. Applications in Other Areas of Transportation Research: Regression is a very important technique across many areas of transportation research, including: • Transportation Planning – to include the other half of trip generation, e.g., predicting trip destinations as a function of employment levels by various types (factory, commercial), square footage of shopping center space, and so forth. – to investigate the trip distribution stage of the 4-step model (log transformation of the gravity model). • Public Transportation—to predict loss/liability on subsidized freight rail lines (function of segment ton-miles, maintenance budgets and/or standards, operating speeds, etc.) for self-insurance computations. • Pavement Engineering—to model pavement deterioration (or performance) as a function of easily monitored predictor variables.

64 effective experiment Design and Data analysis in transportation research Example 17: Traffic Operations; Regression Analysis Area: Traffic operations Method of Analysis: Regression analysis (developing a model to predict the values that some variable can take as a function of one or more other variables, when not all variables are assumed to be continuous) 1. Research Question/Problem Statement: An engineer is concerned about false capacity at inter- sections being designed in a specified district. False capacity occurs where a lane is dropped just beyond a signalized intersection. Drivers approaching the intersection and knowing that the lane is going to be dropped shortly afterward avoid the lane. However, engineers estimating the capacity and level of service of the intersection during design have no reliable way to estimate the percentage of traffic that will avoid the lane (the lane distribution). Question/Issue Develop a model that can be used to predict the values that a dependent vari- able can take as a function of changes in the values of the independent variables. In this particular instance, how can engineers make a good estimate of the lane distribution of traffic volume in the case of a lane drop just beyond an intersec- tion? Can a linear model be developed that can be used to predict this distribu- tion based on other variables? The basic question is whether a linear relationship exists between the dependent variable (Y; in this case, the lane distribution percentage) and some independent variable(s) (X). The relationship can be expressed using the following equation: Y X= +a b i where a is the intercept and b is the slope of the line (see NCHRP Project 20-45, Volume 2, Chapter 4, Section B). 2. Identification and Description of Variables: The dependent variable of interest in this example is the volume of traffic in each lane on the approach to a signalized intersection with a lane drop just beyond. The traffic volumes by lane are converted into lane utilization factors (fLU), to be consistent with standard highway capacity techniques. The Highway Capacity Manual defines fLU using the following equation: f v v N LU g g = ( )1 where Vg is the flow rate in a lane group in vehicles per hour, Vg1 is the flow rate in the lane with the highest flow rate of any in the group in vehicles per hour, and N is the number of lanes in the lane group. The engineer thinks that lane utilization might be explained by one or more of 15 different factors, including the type of lane drop, the distance from the intersection to the lane drop, the taper length, and the heavy vehicle percentage. All of the variables are continuous except the type of lane drop. The type of lane drop is used to categorize the sites. 3. Data Collection: The engineer locates 46 lane-drop sites in the area and collects data at these sites by means of video recording. The engineer tapes for up to 3 hours at each site. The data are summarized in 15-minute periods, again to be consistent with standard highway capacity practice. For one type of lane-drop geometry, with two through lanes and an exclusive right- turn lane on the approach to the signalized intersection, the engineer ends up with 88 valid

examples of effective experiment Design and Data analysis in transportation research 65 data points (some sites have provided more than one data point), covering 15 minutes each, to use in equation (model) development. 4. Specification of Analysis Technique and Data Analysis: Multiple (or multivariate) regression is a standard statistical technique to develop predictive equations. (More information on this topic is given in NCHRP Project 20-45, Volume 2, Chapter 4, Section B). The engineer performs five steps to develop the predictive equation. Step 1. The engineer examines plots of each of the 15 candidate variables versus fLU to see if there is a relationship and to see what forms the relationships might take. Step 2. The engineer screens all 15 candidate variables for multicollinearity. (Multicollinearity occurs when two variables are related to each other and essentially contribute the same informa- tion to the prediction.) Multicollinearity can lead to models with poor predicting power and other problems. The engineer examines the variables for multicollinearity by • looking at plots of each of the 15 candidate variables against every other candidate variable; • calculating the correlation coefficient for each of the 15 candidate independent variables against every other candidate variable; and • using more sophisticated tests (such as the variance influence factor) that are available in statistical software. Step 3. The engineer reduces the set of candidate variables to eight. Next, the engineer uses statistical software to select variables and estimate the coefficients for each selected variable, assuming that the regression equation has a linear form. To select variables, the engineer employs forward selection (adding variables one at a time until the equation fit ceases to improve significantly) and backward elimination (starting with all candidate variables in the equation and removing them one by one until the equation fit starts to deteriorate). The equation fit is measured by R2 (for more information, see NCHRP Project 20-45, Volume 2, Chapter 4, Section B, under the heading, “Descriptive Measures of Association Between X and Y”), which shows how well the equation fits the data on a scale from 0 to 1, and other factors provided by statistical software. In this case, forward selection and backward elimination result in an equation with five variables: • Drop: Lane drop type, a 0 or 1 depending on the type; • Left: Left turn status, a 0 or 1 depending on the types of left turns allowed; • Length: The distance from the intersection to the lane drop, in feet ÷ 1000; • Volume: The average lane volume, in vehicles per hour per lane ÷ 1000; and • Sign: The number of signs warning of the lane drop. Notice that the first two variables are discrete variables and had to assume a zero-or-one format to work within the regression model. Each of the five variables has a coefficient that is significantly different from zero at the 95% confidence level, as measured by a t-test. (For more information, see NCHRP Project 20-45, Volume 2, Chapter 4, Section B, “How Are t-statistics Interpreted?”) Step 4. Once an initial model has been developed, the engineer plots the residuals for the tentative equation to see whether the assumed linear form is correct. A residual is the differ- ence, for each observation, between the prediction the equation makes for fLU and the actual value of fLU. In this example, a plot of the predicted value versus the residual for each of the 88 data points shows a fan-like shape, which indicates that the linear form is not appropriate. (NCHRP Project 20-45, Volume 2, Chapter 4, Section B, Figure 6 provides examples of residual plots that are and are not desirable.) The engineer experiments with several other model forms, including non-linear equations that involve transformations of variables, before settling on a lognormal form that provides a good R2 value of 0.73 and a desirable shape for the residual plot.

66 effective experiment Design and Data analysis in transportation research Step 5. Finally, the engineer examines the candidate equation for logic and practicality, asking whether the variables make sense, whether the signs of the variables make sense, and whether the variables can be collected easily by design engineers. Satisfied that the answers to these questions are “yes,” the final equation (model) can be expressed as follows: f Drop Left LLU = − − + +exp . . . .0 539 0 218 0 148 0 178i i i ength Volume Sign+ −( )0 627 0 105. .i i 5. Interpreting the Results: The process described in this example results in a useful equation for estimating the lane utilization in a lane to be dropped, thereby avoiding the estimation of false capacity. The equation has five terms and is non-linear, which will make its use a bit challenging. However, the database is large, the equation fits the data well, and the equation is logical, which should boost the confidence of potential users. If potential users apply the equation within the ranges of the data used for the calibration, the equation should provide good predictions. Applying any model outside the range of the data on which it was calibrated increases the likelihood of an inaccurate prediction. 6. Conclusion and Discussion: Regression is a powerful statistical technique that provides models engineers can use to make predictions in the absence of direct observation. Engineers tempted to use regression techniques should notice from this and other examples that the effort is substantial. Engineers using regression techniques should not skip any of the steps described above, as doing so may result in equations that provide poor predictions to users. Analysts considering developing a regression model to help make needed predictions should not be intimidated by the process. Although there are many pitfalls in developing a regression model, analysts considering making the effort should also consider the alternative: how the prediction will be made in the absence of a model. In the absence of a model, predic- tions of important factors like lane utilization would be made using tradition, opinion, or simple heuristics. With guidance from NCHRP Project 20-45 and other texts, and with good software available to make the calculations, credible regression models often can be developed that perform better than the traditional prediction methods. Because regression models developed by transportation engineers are often reused in later studies by others, the stakes are high. The consequences of a model that makes poor pre- dictions can be severe in terms of suboptimal decisions. Lane utilization models often are employed in traffic studies conducted to analyze new development proposals. A model that under-predicts utilization in a lane to be dropped may mean that the development is turned down due to the anticipated traffic impacts or that the developer has to pay for additional and unnecessary traffic mitigation measures. On the other hand, a model that over-predicts utilization in a lane to be dropped may mean that the development is approved with insufficient traffic mitigation measures in place, resulting in traffic delays, collisions, and the need for later intervention by a public agency. 7. Applications in Other Areas of Transportation Research: Regression is used in almost all areas of transportation research, including: • Transportation Planning—to create equations to predict trip generation and mode split. • Traffic Safety—to create equations to predict the number of collisions expected on a particular section of road. • Pavement Engineering/Materials—to predict long-term wear and condition of pavements. Example 18: Transportation Planning; Logit and Related Analysis Area: Transportation planning Method of Analysis: Logit and related analysis (developing predictive models when the dependent variable is dichotomous—e.g., 0 or 1)

examples of effective experiment Design and Data analysis in transportation research 67 2. Identification and Description of Variables: Considering a typical, traditional urban area in the United States, it is reasonable to argue that the likelihood of taking public transit to work (Y) will be a function of income (X). Generally, more income means less likelihood of taking public transit. This can be modeled using the following equation: Y X ui i i= + +β β1 2 where Xi = family income, Y = 0 if the family uses public transit, and Y = 1 if the family doesn’t use public transit. 3. Data Collection: These data normally are obtained from travel surveys conducted at the local level (e.g., by a metropolitan area or specific city), although the agency that collects the data often is a state DOT. 4. Specification of Analysis Techniques and Data Analysis: In this example the dependent variable is dichotomous and is a linear function of an explanatory variable. Consider the equation E(YiXi) = b1 + b2Xi. Notice that if Pi = probability that Y = 1 (household utilizes transit), then (1 - Pi) = probability that Y = 0 (doesn’t utilize transit). This has been called a linear probability model. Note that within this expression, “i” refers to a household. Thus, Y has the distribution shown in Table 25. Any attempt to estimate this relationship with standard (OLS) regression is saddled with many problems (e.g., non-normality of errors, heteroscedasticity, and the possibility that the predicted Y will be outside the range 0 to 1, to say nothing of pretty terrible R2 values). Question/Issue Can a linear model be developed that can be used to predict the probability that one of two choices will be made? In this example, the question is whether a household will use public transit (or not). Rather than being continuous (as in linear regression), the dependent variable is reduced to two categories, a dichotomous variable (e.g., yes or no, 0 or 1). Although the question is simple, the statistical modeling becomes sophisticated very quickly. 1. Research Question/Problem Statement: Transportation planners often utilize variations of the classic four-step transportation planning process for predicting travel demand. Trip generation, trip distribution, mode split, and trip assignment are used to predict traffic flows under a variety of forecasted changes in networks, population, land use, and controls. Mode split, deciding which mode of transportation a traveler will take, requires predicting mutually exclusive outcomes. For example, will a traveler utilize public transit or drive his or her own car? Table 25. Distribution of Y. Values that Y Takes Probability Meaning/Interpretation 1 Pi Household uses transit 0 1 – Pi Household does not use transit 1.0 Total

68 effective experiment Design and Data analysis in transportation research An alternative formulation for estimating Pi, the cumulative logistic distribution, is expressed by the following equation: Pi Xi = + − +( ) 1 1 1 2ε β β This function can be plotted as a lazy Z-curve where on the left, with low values of X (low household income), the probability starts near 1 and ends at 0 (Figure 16). Notice that, even at 0 income, not all households use transit. The curve is said to be asymptotic to 1 and 0. The value of Pi varies between 1 and 0 in relation to income, X. Manipulating the definition of the cumulative logistic distribution from above, 1 11 2+( ) =− +( )ε β β Xi iP P Pi i Xi+( ) =− +( )ε β β1 2 1 P Pi Xi iε β β− +( ) = −1 2 1 ε β β− +( ) = −1 2 1Xi i i P P and ε β β1 2 1 +( ) = − Xi i i P P The final expression is the ratio of the probability of utilizing public transit divided by the probability of not utilizing public transit. It is called the odds ratio. Next, taking the natural log of both sides (and reversing) results in the following equation: L P P Xi i i i= −   = +ln 1 1 2β β L is called the logit, and this is called a logit model. The left side is the natural log of the odds ratio. Unfortunately, this odds ratio is meaningless for individual households where the prob- ability is either 0 or 1 (utilize or not utilize). If the analyst uses standard OLS regression on this Figure 16. Plot of cumulative logistic distribution showing a lazy Z-curve.

examples of effective experiment Design and Data analysis in transportation research 69 equation, with data for individual households, there is a problem because when Pi happens to equal either 0 or 1 (which is all the time!), the odds ratio will, as a result, equal either 0 or infinity (and the logarithm will be undefined) for all observations. However, by using groups of households the problem can be mitigated. Table 26 presents data based on a survey of 701 households, more than half of which use transit (380). The income data are recorded for intervals; here, interval mid-points (Xj) are shown. The number of households in each income category is tallied (Nj), as is the number of households in each income category that utilizes public transit (nj). It is important to note that while there are more than 700 households (i), the number of observations (categories, j) is only 13. Using these data, for each income bracket, the probability of taking transit can be estimated as follows: P n N j j j  = This equation is an expression of relative frequency (i.e., it expresses the proportion in income bracket “j” using transit). An examination of Table 26 shows clearly that there is progression of these relative frequen- cies, with higher income brackets showing lower relative frequencies, just as was hypothesized. We can calculate the odds ratio for each income bracket listed in Table 26 and estimate the following logit function with OLS regression: L n N n N Xj j j j j j= −       = +ln 1 1 2β β The results of this regression are shown in Table 27. The results also can be expressed as an equation: LogOddsRatio X= −1 037 0 00003863. .  5. Interpreting the Results: This model provides a very good fit. The estimates of the coefficients can be inserted in the original cumulative logistic function to directly estimate the probability of using transit for any given X (income level). Indeed, the logistic graph in Figure 16 is produced with the estimated function. Xj ($) Nj (Households) nj (Utilizing Transit) Pj (Defined Above) $6,000 40 30 0.750 $8,000 55 39 0.709 $10,000 65 43 0.662 $13,000 88 58 0.659 $15,000 118 69 0.585 $20,000 81 44 0.543 $25,000 70 33 0.471 $30,000 62 25 0.403 $35,000 40 16 0.400 $40,000 30 11 0.367 $50,000 22 6 0.273 $60,000 18 4 0.222 $75,000 12 2 0.167 Total: 701 380 Table 26. Data examined by groups of households.

70 effective experiment Design and Data analysis in transportation research 6. Conclusion and Discussion: This approach to estimation is not without further problems. For example, the N within each income bracket needs to be sufficiently large that the relative fre- quency (and therefore the resulting odds ratio) is accurately estimated. Many statisticians would say that a minimum of 25 is reasonable. This approach also is limited by the fact that only one independent variable is used (income). Common sense suggests that the right-hand side of the function could logically be expanded to include more than one predictor variable (more Xs). For example, it could be argued that educational level might act, along with income, to account for the probability of using transit. However, combining predictor variables severely impinges on the categories (the j) used in this OLS regression formulation. To illustrate, assume that five educational categories are used in addition to the 13 income brackets (e.g., Grade 8 or less, high school graduate to Grade 9, some college, BA or BS degree, and graduate degree). For such an OLS regression analysis to work, data would be needed for 5 × 13, or 65 categories. Ideally, other travel modes should also be considered. In the example developed here, only transit and not-transit are considered. In some locations it is entirely reasonable to examine private auto versus bus versus bicycle versus subway versus light rail (involving five modes, not just two). This notion of a polychotomous logistic regression is possible. However, five modes cannot be estimated with the OLS regression technique employed above. The logit above is a variant of the binomial distribution and the polychotomous logistic model is a variant of the multi- nomial distribution (see NCHRP Project 20-45, Volume 2, Chapter 5). Estimation of these more advanced models requires maximum likelihood methods (as described in NCHRP Project 20-45, Volume 2, Chapter 5). Other model variants are based upon other cumulative probability distributions. For exam- ple, there is the probit model, in which the normal cumulative density function is used. The probit model is very similar to the logit model, but it is more difficult to estimate. 7. Applications in Other Areas of Transportation Research: Applications of logit and related models abound within transportation studies. In any situation in which human behavior is relegated to discrete choices, the category of models may be applied. Examples in other areas of transportation research include: • Transportation Planning—to model any “choice” issue, such as shopping destination choices. • Traffic Safety—to model dichotomous responses (e.g., did a motorist slow down or not) in response to traffic control devices. • Highway Design—to model public reactions to proposed design solutions (e.g., support or not support proposed road diets, installation of roundabouts, or use of traffic calming techniques). Example 19: Public Transit; Survey Design and Analysis Area: Public transit Method of Analysis: Survey design and analysis (organizing survey data for statistical analysis) Coefficients t-values (statistics) p-values Measures of “Fit” 1 = 1.037 12.156 0.000 R = 0.980 2 = -0.00003863 β β -16.407 0.000 R2 = 0.961 adjusted R2 = 0.957 Table 27. Results of OLS regression.

examples of effective experiment Design and Data analysis in transportation research 71 2. Identification and Description of Variables: Two types of variables are needed for this analysis. The first is data on the characteristics of the riders, such as gender, age, and access to an automobile. These data are discrete variables. The second is data on the riders’ stated responses to proposed changes in the fare or service characteristics. These data also are treated as discrete variables. Although some, like the fare, could theoretically be continuous, they are normally expressed in discrete increments (e.g., $1.00, $1.25, $1.50). 3. Data Collection: These data are normally collected by agencies conducting a survey of the transit users. The initial step in the experiment design is to choose the variables to be collected for each of these two data sets. The second step is to determine how to categorize the data. Both steps are generally based on past experience and common sense. Some of the variables used to describe the characteristics of the transit user are dichotomous, such as gender (male or female) and access to an automobile (yes or no). Other variables, such as age, are grouped into discrete categories within which the transit riding characteristics are similar. For example, one would not expect there to be a difference between the transit trip needs of a 14-year-old student and a 15-year-old student. Thus, the survey responses of these two age groups would be assigned to the same age category. However, experience (and common sense) leads one to differentiate a 19-year-old transit user from a 65-year-old transit user, because their purposes for taking trips and their perspectives on the relative value of the fare and the service components are both likely to be different. Obtaining user responses to changes in the fare or service is generally done in one of two ways. The first is to make a statement and ask the responder to mark one of several choices: strongly agree, agree, neither agree nor disagree, disagree, and strongly disagree. The number of statements used in the survey depends on how many parameter changes are being contemplated. Typical statements include: 1. I would increase the number of trips I make each month if the fare were reduced by $0.xx. 2. I would increase the number of trips I make each month if I could purchase a monthly pass. 3. I would increase the number of trips I make each month if the waiting time at the stop were reduced by 10 minutes. 4. I would increase the number of trips I make each month if express services were available from my origin to my destination. The second format is to propose a change and provide multiple choices for the responder. Typical questions for this format are: 1. If the fare were increased by $0.xx per trip I would: a) not change the number of trips per month b) reduce the non-commute trips c) reduce both the commute and non-commute trips d) switch modes 2. If express service were offered for an additional $0.xx per trip I would: a) not change the number of trips per month on this local service b) make additional trips each month c) shift from the local service to the express service Question/Issue Use and analysis of data collected in a survey. Results from a survey of transit users are used to estimate the change in ridership that would result from a change in the service or fare. 1. Research Question/Problem Statement: The transit director is considering changes to the fare structure and the service characteristics of the transit system. To assist in determining which changes would be most effective or efficient, a survey of the current transit riders is developed.

72 effective experiment Design and Data analysis in transportation research These surveys generally are administered by handing a survey form to people as they enter the transit vehicle and collecting them as people depart the transit vehicle. The surveys also can be administered by mail, telephone, or in a face-to-face interview. In constructing the questions, care should be taken to use terms with which the respondents will be familiar. For example, if the system does not currently offer “express” service, this term will need to be defined in the survey. Other technical terms should be avoided. Similarly, the word “mode” is often used by transportation professionals but is not commonly used by the public at large. The length of a survey is almost always an issue as well. To avoid asking too many questions, each question needs to be reviewed to see if it is really necessary and will produce useful data (as opposed to just being something that would be nice to know). 4. Specification of Analysis Technique and Data Analysis: The results of these surveys often are displayed in tables or in frequency distribution diagrams (see also Example 1 and Example 2). Table 28 lists responses to a sample question posed in the form of a statement. Figure 17 shows the frequency diagram for these data. Similar presentations can be made for any of the groupings included in the first type of variables discussed above. For example, if gender is included as a Type 1 question, the results might appear as shown in Table 29 and Figure 18. Figure 18 shows the frequency diagram for these data. Presentations of the data can be made for any combination of the discrete variable groups included in the survey. For example, to display responses of female users over 65 years old, Strongly Agree Agree Neither Agree nor Disagree Disagree Strongly Disagree Total responses 450 600 300 400 100 Table 28. Table of responses to sample statement, “I would increase the number of trips I make each month if the fare were reduced by $0.xx.” 450 600 300 400 100 0 50 100 150 200 250 300 350 400 450 500 550 600 Strongly agree agree neither agree nor disagree disagree strongly disagree Figure 17. Frequency diagram for total responses to sample statement.

examples of effective experiment Design and Data analysis in transportation research 73 all of the survey forms on which these two characteristics (female and over 65 years old) are checked could be extracted and recorded in a table and shown in a frequency diagram. 5. Interpreting the Results: Survey data can be used to compare the responses to fare or service changes of different groups of transit users. This flexibility can be important in determining which changes would impact various segments of transit users. The information can be used to evaluate various fare and service options being considered and allows the transit agency to design promotions to obtain the greatest increase in ridership. For example, by creating fre- quency diagrams to display the responses to statements 2, 3, and 4 listed in Section 3, the engi- neer can compare the impact of changing the fare versus changing the headway or providing express services in the corridor. Organizing response data according to different characteristics of the user produces con- tingency tables like the one illustrated for males and females. This table format can be used to conduct chi-square analysis to determine if there is any statistically significant difference among the various groups. (Chi-square analysis is described in more detail in Example 4.) 6. Conclusions and Discussion: This example illustrates how to obtain and present quan- titative information using surveys. Although survey results provide reasonably good esti- mates of the relative importance users place on different transit attributes (fare, waiting time, hours of service, etc.), when determining how often they would use the system, the magnitude of users’ responses often is overstated. Experience shows that what users say they would do (their stated preference) generally is different than what they actually do (their revealed preference). Strongly Agree Agree Neither Agree nor Disagree Disagree Strongly Disagree Male 200 275 200 200 70 Female 250 325 100 200 30 Total responses 450 600 300 400 100 Table 29. Contingency table showing responses by gender to sample statement, “I would increase the number of trips I make each month if the fare were reduced by $0.xx.” 200 275 200 200 70 250 325 100 200 30 0 50 100 150 200 250 300 350 Strongly agree agree neither agree nor disagree disagree strongly disagree Male Female Figure 18. Frequency diagram showing responses by gender to sample statement.

74 effective experiment Design and Data analysis in transportation research In this example, 1,050 of the 1,850 respondents (57%) have responded that they would use the bus service more frequently if the fare were decreased by $0.xx. Five hundred respondents (27%) have indicated that they would not use the bus service more frequently, and 300 respondents (16%) have indicated that they are not sure if they would change their bus use frequency. These percentages show the stated preferences of the users. The engineer does not yet know the revealed preferences of the users, but experience suggests that it is unlikely that 57% of the riders would actually increase the number of trips they make. 7. Applications in Other Area in Transportation: Survey design and analysis techniques can be used to collect and present data in many areas of transportation research, including: • Transportation Planning—to assess public response to a proposal to enact a local motor fuel tax to improve road maintenance in a city or county. • Traffic Operations—to assess public response to implementing road diets (e.g., 4-lane to 3-lane conversions) on different corridors in a city. • Highway Design—to assess public response to proposed alternative cross-section designs, such as a boulevard design versus an undivided multilane design in a corridor. Example 20: Traffic Operations; Simulation Area: Traffic operations Method of Analysis: Simulation (using field data to simulate, or model, operations or outcomes) 1. Research Question/Problem Statement: A team of engineers wants to determine whether one or more unconventional intersection designs will produce lower travel times than a conventional design at typical intersections for a given number of lanes. There is no way to collect field data to compare alternative intersection designs at a particular site. Macroscopic traffic operations models like those in the Highway Capacity Manual do a good job of estimating delay at specific points but are unable to provide travel time estimates for unconventional designs that consist of several smaller intersections and road segments. Microscopic simulation models measure the behaviors of individual vehicles as they traverse the highway network. Such simulation models are therefore very flexible in the types of networks and measures that can be examined. The team in this example turns to a simulation model to determine how other intersection designs might work. Question/Issue Developing and using a computer simulation model to examine operations in a computer environment. In this example, a traffic operations simulation model is used to show whether one or more unconventional intersection designs will produce lower travel times than a conventional design at typical intersections for a given number of lanes. 2. Identification and Description of Variables: The engineering team simulates seven different intersections to provide the needed scope for their findings. At each intersection, the team examines three different sets of traffic volumes: volumes from the evening (p.m.) peak hour, a typical midday off-peak hour, and a volume that is 15% greater than the p.m. peak hour to represent future conditions. At each intersection, the team models the current conventional intersection geometry and seven unconventional designs: the quadrant roadway, median U-turn, superstreet, bowtie, jughandle, split intersection, and continuous flow intersection. Traffic simulation models break the roadway network into nodes (intersections) and links (segments between intersections). Therefore, the engineering team has to design each of the

examples of effective experiment Design and Data analysis in transportation research 75 alternatives at each test site in terms of numbers of lanes, lane lengths, and such, and then faithfully translate that geometry into links and nodes that the simulation model can use. For each combination of traffic volume and intersection design, the team uses software to find the optimum signal timing and uses that during the simulation. To avoid bias, the team keeps all other factors (e.g., network size, numbers of lanes, turn lane lengths, truck percentages, average vehicle speeds) constant in all simulation runs. 3. Data Collection: The field data collection necessary in this effort consists of noting the current intersection geometries at the seven test intersections and counting the turning movements in the time periods described above. In many simulation efforts, it is also necessary to collect field data to calibrate and validate the simulation model. Calibration is the process by which simulation output is compared to actual measurements for some key measure(s) such as travel time. If a difference is found between the simulation output and the actual measurement, the simulation inputs are changed until the difference disappears. Validation is a test of the calibrated simulation model, comparing simulation output to a previously unused sample of actual field measurements. In this example, however, the team determines that it is unnecessary to collect calibration and validation data because a recent project has successfully calibrated and validated very similar models of most of these same unconventional designs. The engineer team uses the CORSIM traffic operations simulation model. Well known and widely used, CORSIM models the movement of each vehicle through a specified network in small time increments. CORSIM is a good choice for this example because it was originally designed for problems of this type, has produced appropriate results, has excellent animation and other debugging features, runs quickly in these kinds of cases, and is well-supported by the software developers. The team makes two CORSIM runs with different random number seeds for each combina- tion of volume and design at each intersection, or 48 runs for each intersection altogether. It is necessary to make more than one run (or replication) of each simulation combination with different random number seeds because of the randomness built into simulation models. The experiment design in this case allows the team to reduce the number of replications to two; typical practice in simulations when one is making simple comparisons between two variables is to make at least 5 to 10 replications. Each run lasts 30 simulated minutes. Table 30 shows the simulation data for one of the seven intersections. The lowest travel time produced in each case is bolded. Notice that Table 30 does not show data for the bowtie design. That design became congested (gridlocked) and produced essentially infinite travel times for this intersection. Handling overly congested networks is a difficult problem in many efforts and with several different simulation software packages. The best current advice is for analysts to not push their networks too hard and to scan often for gridlock. 4. Specification of Analysis Technique and Data Analysis: The experiment assembled in this example uses a factorial design. (Factorial design also is discussed in Example 11.) The team analyzes the data from this factorial experiment using analysis of variance (ANOVA). Because Time of Day Total Travel Time, Vehicle-hours, Average of Two Simulation Runs Conventional Quadrant Median U Superstreet Jughandle Split Continuous Midday 67 64 61 74 63 59* 75 P.M. peak 121 95 119 179 139 114 106 Peak + 15% 170 *Lowest total travel time. 135 145 245 164 180 142 Table 30. Simulation results for different designs and time of day.

76 effective experiment Design and Data analysis in transportation research the experimenter has complete control in a simulation, it is common to use efficient designs like factorials and efficient analysis methods like ANOVA to squeeze all possible information out of the effort. Statistical tests comparing the individual mean values of key results by factor are common ways to follow up on ANOVA results. Although ANOVA will reveal which factors make a significant contribution to the overall variance in the dependent variable, means tests will show which levels of a significant factor differ from the other levels. In this example, the team uses Tukey’s means test, which is available as part of the battery of standard tests accom- panying ANOVA in statistical software. (For more information about ANOVA, see NCHRP Project 20-45, Volume 2, Chapter 4, Section A.) 5. Interpreting the Results: For the data shown in Table 30, the ANOVA reveals that the volume and design factors are statistically significant at the 99.99% confidence level. Furthermore, the interaction between the volume and design factors also is statistically significant at the 99.99% level. The means tests on the design factors show that the quadrant roadway is significantly different from (has a lower overall travel time than) the other designs at the 95% level. The next- best designs overall are the median U-turn and the continuous flow intersection; these are not statistically different from each other at the 95% level. The third tier of designs consists of the conventional and the split, which are statistically different from all others at the 95% level but not from each other. Finally, the jughandle and the superstreet designs are statistically different from each other and from all other designs at the 95% level according to the means test. Through the simulation, the team learns that several designs appear to be more efficient than the conventional design, especially at higher volume levels. From the results at all seven intersections, the team sees that the quadrant roadway and median U-turn designs generally lead to the lowest travel times, especially with the higher volume levels. 6. Conclusion and Discussion: Simulation is an effective tool to analyze traffic operations, as at the seven intersections of interest in this example. No other tool would allow such a robust comparison of many different designs and provide the results for travel times in a larger net- work rather than delays at a single spot. The simulation conducted in this example also allows the team to conduct an efficient factorial design, which maximizes the information provided from the effort. Simulation is a useful tool in research for traffic operations because it • affords the ability to conduct randomized experiments, • allows the examination of details that other methods cannot provide, and • allows the analysis of large and complex networks. In practice, simulation also is popular because of the vivid and realistic animation output provided by common software packages. The superb animations allow analysts to spot and treat flaws in the design or model and provide agencies an effective tool by which to share designs with politicians and the public. Although simulation results can sometimes be surprising, more often they confirm what the analysts already suspect based on simpler analyses. In the example described here, the analysts suspected that the quadrant roadway and median U-turn designs would perform well because these designs had performed well in prior Highway Capacity Manual calculations. In many studies, simulations provide rich detail and vivid animation but no big surprises. 7. Applications in Other Areas of Transportation Research: Simulations are critical analysis methods in several areas of transportation research. Besides traffic operations, simulations are used in research related to: • Maintenance—to model the lifetime performance of traffic signs. • Traffic Safety – to examine vehicle performance and driver behaviors or performance. – to predict the number of collisions from a new roadway design (potentially, given the recent development of the FHWA SSAM program).

examples of effective experiment Design and Data analysis in transportation research 77 Example 21: Traffic Safety; Non-parametric Methods Area: Traffic safety Method of Analysis: Non-parametric methods (methods used when data do not follow assumed or conventional distributions, such as when comparing median values) 1. Research Question/Problem Statement: A city traffic engineer has been receiving many citizen complaints about the perceived lack of safety at unsignalized midblock crosswalks. Apparently, some motorists seem surprised by pedestrians in the crosswalks and do not yield to the pedestrians. The engineer believes that larger and brighter warning signs may be an inexpensive way to enhance safety at these locations. Question/Issue Determine whether some treatment has an effect when data to be tested do not follow known distributions. In this example, a nonparametric method is used to determine whether larger and brighter warning signs improve pedestrian safety at unsignalized midblock crosswalks. The null hypothesis and alternative hypothesis are stated as follows: Ho: There is no difference in the median values of the number of conflicts before and after a treatment. Ha: There is a difference in the median values. 2. Identification and Description of Variables: The engineer would like to collect collision data at crosswalks with improved signs, but it would take a long time at a large sample of crosswalks to collect a reasonable sample size of collisions to answer the question. Instead, the engineer collects data for conflicts, which are near-collisions when one or both of the involved entities brakes or swerves within 2 seconds of a collision to avoid the collision. Research literature has shown that conflicts are related to collisions, and because conflicts are much more numerous than collisions, it is much quicker to collect a good sample size. Conflict data are not nearly as widely used as collision data, however, and the underlying distribution of conflict data is not clear. Thus, the use of non-parametric methods seems appropriate. 3. Data Collection: The engineer identifies seven test crosswalks in the city based on large pedes- trian volumes and the presence of convenient vantage points for observing conflicts. The engi- neering staff collects data on traffic conflicts for 2 full days at each of the seven crosswalks with standard warning signs. The engineer then has larger and brighter warning signs installed at the seven sites. After waiting at least 1 month at each site after sign installation, the staff again collects traffic conflicts for 2 full days, making sure that weather, light, and as many other conditions as possible are similar between the before-and-after data collection periods at each site. 4. Specification of Analysis Technique and Data Analysis: A nonparametric statistical test is an efficient way to analyze data when the underlying distribution is unclear (as in this example using conflict data) and when the sample size is small (as in this example with its small number of sites). Several such tests, such as the sign test and the Wilcoxon signed-rank (Wilcoxon rank-sum) test are plausible in this example. (For more information about nonparametric tests, see NCHRP Project 20-45, Volume 2, Chapter 6, Section D, “Hypothesis About Population Medians for Independent Samples.” ) The decision is made to use the Wilcoxon signed-rank test because it is a more powerful test for paired numerical measurements than other tests, and this example uses paired (before-and-after) measurements. The sign test is a popular nonparametric test for paired data but loses information contained in numerical measurements by reducing the data to a series of positive or negative signs.

78 effective experiment Design and Data analysis in transportation research Having decided on the Wilcoxon signed-rank test, the engineer arranges the data (see Table 31). The third row of the table is the difference between the frequencies of the two conflict measurements at each site. The last row shows the rank order of the sites from lowest to highest based on the absolute value of the difference. Site 3 has the least difference (35 - 33 = 2) while Site 7 has the greatest difference (54 - 61 = -16). The Wilcoxon signed-rank test ranks the differences from low to high in terms of absolute values. In this case, that would be 2, 3, 7, 7, 12, 15, and 16. The test statistic, x, is the sum of the ranks that have positive differences. In this example, x = 1 + 2 + 3.5 + 3.5 + 6 = 16. Notice that all but the sixth and seventh ranked sites had positive differences. Notice also that the tied differences were assigned ranks equal to the average of the ranks they would have received if they were just slightly different from each other. The engineer then consults a table for the Wilcoxon signed-rank test to get a critical value against which to compare. (Such a table appears in NCHRP Project 20-45, Volume 2, Appendix C, Table C-8.) The standard table for a sample size of seven shows that the critical value for a one-tailed test (testing whether there is an improvement) with a confidence level of 95% is x = 24. 5. Interpreting the Results: Because the calculated value (x = 16) is less than the critical value (x = 24), the engineer concludes that there is not a statistically significant difference between the number of conflicts recorded with standard signs and the number of conflicts recorded with larger and brighter signs. 6. Conclusion and Discussion: Nonparametric tests do not require the engineer to make restric- tive assumptions about an underlying distribution and are therefore good choices in cases like this, in which the sample size is small and the data collected do not have a familiar underlying distribution. Many nonparametric tests are available, so analysts should do some reading and searching before settling on the best one for any particular case. Once a nonparametric test is determined, it is usually easy to apply. This example also illustrates one of the potential pitfalls of statistical testing. The engineer’s conclusion is that there is not a statistically significant difference between the number of conflicts recorded with standard signs and the number of conflicts recorded with larger and brighter signs. That conclusion does not necessarily mean that larger and brighter signs are a bad idea at sites similar to those tested. Notice that in this experiment, larger and brighter signs produced lower conflict frequencies at five of the seven sites, and the average number of conflicts per site was lower with the larger and brighter signs. Given that signs are relatively inexpensive, they may be a good idea at sites like those tested. A statistical test can provide useful information, especially about the quality of the experiment, but analysts must be careful not to interpret the results of a statistical test too strictly. In this example, the greatest danger to the validity of the test result lies not in the statistical test but in the underlying before-and-after test setup. For the results to be valid, it is necessary that the only important change that affects conflicts at the test sites during data collection be Site 1 Site 2 Site 3 Site 4 Site 5 Site 6 Site 7 Standard signs 170 39 35 32 32 19 45 Larger and brighter signs 155 26 33 29 25 31 61 Difference 15 7 2 3 7 -12 -16 Rank of absolute difference 6 73.5 1 2 3.5 5 Table 31. Number of conflicts recorded during each (equal) time period at each site.

examples of effective experiment Design and Data analysis in transportation research 79 the new signs. The engineer has kept the duration short between the before-and-after data collection periods, which helps minimize the chances of other important changes. However, if there is any reason to suspect other important changes, these test results should be viewed skeptically and a more sophisticated test strategy should be employed. 7. Applications in Other Areas of Transportation Research: Nonparametric tests are helpful when researchers are working with small sample sizes or sample data wherein the underlying distribution is unknown. Examples of other areas of transportation research in which non- parametric tests may be applied include: • Transportation Planning, Public Transportation—to analyze data from surveys and questionnaires when the scale of the response calls into question the underlying distribution. Such data are often analyzed in transportation planning and public transportation. • Traffic Operations—to analyze small samples of speed or volume data. • Structures, Pavements—to analyze quality ratings of pavements, bridges, and other trans- portation assets. Such ratings also use scales. Resources The examples used in this report have included references to the following resources. Researchers are encouraged to consult these resources for more information about statistical procedures. Freund, R. J. and W. J. Wilson (2003). Statistical Methods. 2d ed. Burlington, MA: Academic Press. See page 256 for a discussion of Tukey’s procedure. Kutner, M. et al. (2005). Applied Linear Statistical Models. 5th ed. Boston: McGraw-Hill. See page 746 for a discussion of Tukey’s procedure. NCHRP CD-22: Scientific Approaches to Transportation Research, Vol. 1 and 2. 2002. Transpor- tation Research Board of the National Academies, Washington, D.C. This two-volume electronic manual developed under NCHRP Project 20-45 provides a comprehensive source of information on the conduct of research. The manual includes state-of-the-art techniques for problem state- ment development; literature searching; development of the research work plan; execution of the experiment; data collection, management, quality control, and reporting of results; and evaluation of the effectiveness of the research, as well as the requirements for the systematic, pro- fessional, and ethical conduct of transportation research. For readers’ convenience, the references to NCHRP Project 20-45 from the various examples contained in this report are summarized here by topic and location in NCHRP CD-22. More information about NCHRP CD-22 is available at http://www.trb.org/Main/Blurbs/152122.aspx. • Analysis of Variance (one-way ANOVA and two-way ANOVA): See Volume 2, Chapter 4, Section A, Analysis of Variance Methodology (pp. 113, 119–31). • Assumptions for residual errors: See Volume 2, Chapter 4. • Box plots; Q-Q plots: See Volume 2, Chapter 6, Section C. • Chi-square test: See Volume 2, Chapter 6, Sections E (Chi-Square Test for Independence) and F. • Chi-square values: See Volume 2, Appendix C, Table C-2. • Computations on unbalanced designs and multi-factorial designs: See Volume 2, Chapter 4, Section A, Analysis of Variance Methodology (pp. 119–31). • Confidence intervals: See Volume 2, Chapter 4. • Correlation coefficient: See Volume 2, Appendix A, Glossary, Correlation Coefficient. • Critical F-value: See Volume 2, Appendix C, Table C-5. • Desirable and undesirable residual plots (scatter plots): See Volume 2, Chapter 4, Section B, Figure 6.

80 effective experiment Design and Data analysis in transportation research • Equation fit: See Volume 2, Chapter 4, Glossary, Descriptive Measures of Association Between X and Y. • Error distributions (normality, constant variance, uncorrelated, etc.): See Volume 2, Chapter 4 (pp. 146–55). • Experiment design and data collection: See Volume 2, Chapter 1. • Fcrit and F-distribution table: See Volume 2, Appendix C, Table C-5. • F-test (or F-test): See Volume 2, Chapter 4, Section A, Compute the F-ratio Test Statistic (p. 124). • Formulation of formal hypotheses for testing: See Volume 1, Chapter 2, Hypothesis; Volume 2, Appendix A, Glossary. • History and maturation biases (specification errors): See Volume 2, Chapter 1, Quasi- Experiments. • Indicator (dummy) variables: See Volume 2, Chapter 4 (pp. 142–45). • Intercept and slope: See Volume 2, Chapter 4 (pp. 140–42). • Maximum likelihood methods: See Volume 2, Chapter 5 (pp. 208–11). • Mean and standard deviation formulas: See Volume 2, Chapter 6, Table C, Frequency Distribu- tions, Variance, Standard Deviation, Histograms, and Boxplots. • Measured ratio or interval scale: See Volume 2, Chapter 1 (p. 83). • Multinomial distribution and polychotomous logistical model: See Volume 2, Chapter 5 (pp. 211–18). • Multiple (multivariate) regression: See Volume 2, Chapter 4, Section B. • Non-parametric tests: See Volume 2, Chapter 6, Section D. • Normal distribution: See Volume 2, Appendix A, Glossary, Normal Distribution. • One- and two-sided hypothesis testing (one- and two-tail test values): See Volume 2, Chapter 4 (pp. 161 and 164–5). • Ordinary least squares (OLS) regression: See Volume 2, Chapter 4, Section B, Linear Regression. • Sample size and confidence: See Volume 2, Chapter 1, Sample Size Determination. • Sample size determination based on statistical power requirements: See Volume 2, Chapter 1, Sample Size Determination (p. 94). • Sign test and the Wilcoxon signed-rank (Wilcoxon rank-sum) test: See Volume 2, Chapter 6, Section D, and Appendix C, Table C-8, Hypothesis About Population Medians for Independent Samples. • Split samples: See Volume 2, Chapter 4, Section A, Analysis of Variance Methodology (pp. 119–31). • Standard chi-square distribution table: See Volume 2, Appendix C, Table C-2. • Standard normal values: See Volume 2, Appendix C, Table C-1. • tcrit values: See Volume 2, Appendix C, Table C-4. • t-statistic: See Volume 2, Appendix A, Glossary. • t-statistic using equation for equal variance: See Volume 2, Appendix C, Table C-4. • t-test: See Volume 2, Chapter 4, Section B, How are t-statistics Interpreted? • Tabularized values of t-statistic: See Volume 2, Appendix C, Table C-4. • Tukey’s test, Bonferroni’s test, Scheffe’s test: See Volume 2, Chapter 4, Section A, Analysis of Variance Methodology (pp. 119–31). • Types of data and implications for selection of analysis techniques: See Volume 2, Chapter 1, Identification of Empirical Setting.

Abbreviations and acronyms used without definitions in TRB publications: AAAE American Association of Airport Executives AASHO American Association of State Highway Officials AASHTO American Association of State Highway and Transportation Officials ACI–NA Airports Council International–North America ACRP Airport Cooperative Research Program ADA Americans with Disabilities Act APTA American Public Transportation Association ASCE American Society of Civil Engineers ASME American Society of Mechanical Engineers ASTM American Society for Testing and Materials ATA American Trucking Associations CTAA Community Transportation Association of America CTBSSP Commercial Truck and Bus Safety Synthesis Program DHS Department of Homeland Security DOE Department of Energy EPA Environmental Protection Agency FAA Federal Aviation Administration FHWA Federal Highway Administration FMCSA Federal Motor Carrier Safety Administration FRA Federal Railroad Administration FTA Federal Transit Administration HMCRP Hazardous Materials Cooperative Research Program IEEE Institute of Electrical and Electronics Engineers ISTEA Intermodal Surface Transportation Efficiency Act of 1991 ITE Institute of Transportation Engineers NASA National Aeronautics and Space Administration NASAO National Association of State Aviation Officials NCFRP National Cooperative Freight Research Program NCHRP National Cooperative Highway Research Program NHTSA National Highway Traffic Safety Administration NTSB National Transportation Safety Board PHMSA Pipeline and Hazardous Materials Safety Administration RITA Research and Innovative Technology Administration SAE Society of Automotive Engineers SAFETEA-LU Safe, Accountable, Flexible, Efficient Transportation Equity Act: A Legacy for Users (2005) TCRP Transit Cooperative Research Program TEA-21 Transportation Equity Act for the 21st Century (1998) TRB Transportation Research Board TSA Transportation Security Administration U.S.DOT United States Department of Transportation

TRB’s National Cooperative Highway Research Program (NCHRP) Report 727: Effective Experiment Design and Data Analysis in Transportation Research describes the factors that may be considered in designing experiments and presents 21 typical transportation examples illustrating the experiment design process, including selection of appropriate statistical tests.

The report is a companion to NCHRP CD-22, Scientific Approaches to Transportation Research, Volumes 1 and 2 , which present detailed information on statistical methods.

READ FREE ONLINE

Welcome to OpenBook!

You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

Do you want to take a quick tour of the OpenBook's features?

Show this book's table of contents , where you can jump to any chapter by name.

...or use these buttons to go back to the previous chapter or skip to the next one.

Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

To search the entire text of this book, type in your search term here and press Enter .

Share a link to this book page on your preferred social network or via email.

View our suggested citation for this chapter.

Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

Get Email Updates

Do you enjoy reading reports from the Academies online for free ? Sign up for email notifications and we'll let you know about new publications in your areas of interest when they're released.

Data Analysis

Statistical and Computational Methods for Scientists and Engineers

• © 2014
• Latest edition
• Siegmund Brandt 0

Department of Physics, University of Siegen, Siegen, Germany

You can also search for this author in PubMed   Google Scholar

• Provides rigorous mathematical treatment of practical statistical methods for data analysis
• Serves as a graduate textbook and reference guide for those interested in the fundamentals of data analysis
• Useful for all fields of science and engineering requiring an understanding of statistical methods applied to experimental data
• Includes example programs and solutions to programming problems which are written in the modern computer language Java
• Modernizes the content in the previous edition and shortens the length of the text
• Includes supplementary material: sn.pub/extras

6.58m Accesses

35 Citations

35 Altmetric

This is a preview of subscription content, log in via an institution to check access.

Access this book

• Available as EPUB and PDF
• Read on any device
• Instant download
• Own it forever
• Compact, lightweight edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info
• Durable hardcover edition

Tax calculation will be finalised at checkout

Other ways to access

Licence this eBook for your library

Institutional subscriptions

About this book

The fourth edition of this successful textbook presents a comprehensive introduction to statistical and numerical methods for the evaluation of empirical and experimental data. Equal weight is given to statistical theory and practical problems. The concise mathematical treatment of the subject matter is illustrated by many examples and for the present edition a library of Java programs has been developed. It comprises methods of numerical data analysis and graphical representation as well as many example programs and solutions to programming problems.

The book is conceived both as an introduction and as a work of reference. In particular it addresses itself to students, scientists and practitioners in science and engineering as a help in the analysis of their data in laboratory courses, in working for bachelor or master degrees, in thesis work, and in research and professional work.

Similar content being viewed by others

Using MATLAB® for Statistical Analysis

On analytical, computational and historical developments of statistics and its applications.

Six textbook mistakes in data analysis

Analysis of variance.

• Computer-generated Random Numbers
• Data Analysis Textbook
• Data Analysis using Java
• Error Propagation
• Fundamentals of Data Analysis
• Graduate-Level Data Analysis
• Introduction to Statistical Methods
• Java Programs for Data Analysis
• Least Squares
• Linear Regression
• Matrix Algebra
• Matrix Calculation
• Monte Carlo Methods
• Polynomial Regression
• Singular-Value Analysis
• Statistical Distributions
• Statistical Error

Testing Statistical Hypotheses

Time series analysis, table of contents (13 chapters), front matter, introduction.

Siegmund Brandt

Probabilities

Random variables: distributions, computer generated random numbers: the monte carlo method, some important distributions and theorems, the method of maximum likelihood, the method of least squares, function minimization, linear and polynomial regression, back matter, authors and affiliations, about the author.

Siegmund Brandt is Emeritus Professor of Physics at the University of Siegen. With his group he worked on experiments in elementary-particle physics at the research centers DESY in Hamburg and CERN in Geneva in which the analysis of the experimental data plays an important role. He is author or coauthor of textbooks which have appeared in ten languages.

Bibliographic Information

Book Title : Data Analysis

Book Subtitle : Statistical and Computational Methods for Scientists and Engineers

Authors : Siegmund Brandt

DOI : https://doi.org/10.1007/978-3-319-03762-2

Publisher : Springer Cham

eBook Packages : Physics and Astronomy , Physics and Astronomy (R0)

Copyright Information : Springer Nature Switzerland AG 2014

Hardcover ISBN : 978-3-319-03761-5 Published: 26 February 2014

Softcover ISBN : 978-3-319-34779-0 Published: 30 April 2017

eBook ISBN : 978-3-319-03762-2 Published: 14 February 2014

Edition Number : 4

Number of Pages : XX, 523

Number of Illustrations : 134 b/w illustrations

Topics : Mathematical Methods in Physics , Mathematical and Computational Engineering , Statistics for Engineering, Physics, Computer Science, Chemistry and Earth Sciences , Math. Applications in Chemistry , Numerical and Computational Physics, Simulation

• Publish with us

Policies and ethics

• Find a journal
• Track your research

• OU Homepage
• OU Social Media
• The University of Oklahoma

Quantitative Research Methods

Statistical Analysis References

• Online Statistics Education: An Interactive Multimedia Course of Study This open and free introductory statistics textbook covers topics typical for a college-level non-math majors statistics course. Topics include distributions, probability, research design, estimation, hypothesis testing, power and effect size, comparison of means, regression, analysis of variance (ANOVA), transformations, chi square, and non-parametric (distribution-free) tests). It is available as a pdf, online, or as an epub. An Instructor's Manual and PowerPoint slides are also available upon request from the project leader at Rice University.
• Introductory Statistics A free and open introductory statistics textbook for non-math majors. "They have sought to present only the core concepts and use a wide-ranging set of exercises for each concept to drive comprehension. [...] a smaller and less intimidating textbook that trades some extended and unnecessary topics for a better-focused presentation of the central material." It covers descriptive statistics, probability, distributions, discrete and continuous random variables, estimation, hypothesis testing, comparison of means, correlation and regression, chi square, and F-tests.
• Introductory Statistics with Randomization and Simulation "We hope readers will take away three ideas from this book in addition to forming a foundation of statistical thinking and methods. (1) Statistics is an applied field with a wide range of practical applications. (2) You don't have to be a math guru to learn from interesting, real data. (3) Data are messy, and statistical tools are imperfect. However, when you understand the strengths and weaknesses of these tools, you can use them to learn interesting things about the world." This free and open introductory statistics textbook for non-math majors discusses data and data collection, foundations for inference with randomization and simulations (then leading into standard parametric statistics), inference with categorical and numerical data, and linear, multiple logistic regression. An introduction to probability is included as an appendix.

• Statistics LibreTexts Bookshelf Curates multiple openly available statistics textbooks.

• Encyclopedia of Statistical Sciences "Reference tool covering statistics, probability theory, biostatistics, quality control, and economics with emphasis in applications of statistical methods in sociology, engineering, computer science, biomedicine, psychology, survey methodology, and other client disciplines." A good source for topics less often covered in the general textbooks.
• The Concise Encyclopedia of Statistics "More than 500 entries include definitions, history, mathematical details, limitations, examples, references, and further readings. All entries include cross-references as well as the key citations. The back matter includes a timeline of statistical inventions." Another good resource for topics not included in the general texts listed previously.

Meta-analysis

• Meta-Analysis: quantitative methods for research synthesis by Fredric Marc Wolf Call Number: Online access ISBN: 0585216975 Publication Date: 1986 "Meta-Analysis shows how to apply statistical methods to achieve a literature review of a common research domain. It demonstrates the use of combined tests and measures of effect size to synthesise quantitatively the results of independent studies for both group differences and correlations."
• Meta-Analysis: cumulating research findings across studies by John E. Hunter, Frank L. Schmidt, and Gregg B. Jackson Call Number: Online access ISBN: 0803918631 Publication Date: 1982 "Meta-analysis is a way of synthesizing previous research on a subject in order to assess what has already been learned, and even to derive new conclusions from the mass of already researched data. "

• Integrating Omics Data by George Tseng; Debashis Ghosh; Xianghong Jasmine Zhou Call Number: Online access ISBN: 9781107706484 Publication Date: 2015 "As the technologies have become mature and the price affordable, omics data are rapidly generated, and the problem of information integration and modeling of multi-lab and/or multi-omics data is becoming a growing one in the bioinformatics field. This book provides comprehensive coverage of these topics and will have a long-lasting impact on this evolving subject. Each chapter, written by a leader in the field, introduces state-of-the-art methods to handle information integration, experimental data, and database problems of omics data."

• Meta-analysis in social research by Gene V. Glass, Barry McGaw, and Mary Lee Smith Call Number: Online access Publication Date: 1981 Covers the problems of research review and integration; meta-analysis of research; finding studies; describing, classifying and coding research studies; measuring study findings; techniques of analysis; and an evaluation of meta-analysis.

Ordination and Principal Components Analysis (PCA)

• The Ordination Web Page "designed to address some of the most frequently asked questions about ordination. It is my intention to gear this page towards the student and the practitioner rather than the ordination specialist"
• Nature Methods: Principal Components Analysis "Principal component analysis (PCA) simplifies the complexity in high-dimensional data while retaining trends and patterns. It does this by transforming the data into fewer dimensions, which act as summaries of features. "
• A One-Stop Shop for Principal Components Analysis "The rationale for this method, the math under the hood, some best practices, and potential drawbacks to the method."

• NIST/SEMATECH e-Handbook of Statistical Methods: Principal Components "A Multivariate Analysis problem could start out with a substantial number of correlated variables. Principal Component Analysis is a dimension-reduction tool that can be used advantageously in such situations. Principal component analysis aims at reducing a large set of variables to a small set that still contains most of the information in the large set. "

Bayesian Statistics

Experimental Design for Social Sciences

Qualitative Research Methods

• Qualitative Research Methods: A Data Collector's Field Guide "This how-to guide covers the mechanics of data collection for applied qualitative research. It is appropriate for novice and experienced researchers alike. It can be used as both a training tool and a daily reference manual for field team members. The question and answer format and modular design make it easy for readers to find information on a particular topic quickly."

Experimental Design for Ecology and Evolutionary Biology

• Privacy Policy

Home » Experimental Design – Types, Methods, Guide

Experimental Design – Types, Methods, Guide

Table of Contents

Experimental Design

Experimental design is a process of planning and conducting scientific experiments to investigate a hypothesis or research question. It involves carefully designing an experiment that can test the hypothesis, and controlling for other variables that may influence the results.

Experimental design typically includes identifying the variables that will be manipulated or measured, defining the sample or population to be studied, selecting an appropriate method of sampling, choosing a method for data collection and analysis, and determining the appropriate statistical tests to use.

Types of Experimental Design

Here are the different types of experimental design:

Completely Randomized Design

In this design, participants are randomly assigned to one of two or more groups, and each group is exposed to a different treatment or condition.

Randomized Block Design

This design involves dividing participants into blocks based on a specific characteristic, such as age or gender, and then randomly assigning participants within each block to one of two or more treatment groups.

Factorial Design

In a factorial design, participants are randomly assigned to one of several groups, each of which receives a different combination of two or more independent variables.

Repeated Measures Design

In this design, each participant is exposed to all of the different treatments or conditions, either in a random order or in a predetermined order.

Crossover Design

This design involves randomly assigning participants to one of two or more treatment groups, with each group receiving one treatment during the first phase of the study and then switching to a different treatment during the second phase.

Split-plot Design

In this design, the researcher manipulates one or more variables at different levels and uses a randomized block design to control for other variables.

Nested Design

This design involves grouping participants within larger units, such as schools or households, and then randomly assigning these units to different treatment groups.

Laboratory Experiment

Laboratory experiments are conducted under controlled conditions, which allows for greater precision and accuracy. However, because laboratory conditions are not always representative of real-world conditions, the results of these experiments may not be generalizable to the population at large.

Field Experiment

Field experiments are conducted in naturalistic settings and allow for more realistic observations. However, because field experiments are not as controlled as laboratory experiments, they may be subject to more sources of error.

Experimental Design Methods

Experimental design methods refer to the techniques and procedures used to design and conduct experiments in scientific research. Here are some common experimental design methods:

Randomization

This involves randomly assigning participants to different groups or treatments to ensure that any observed differences between groups are due to the treatment and not to other factors.

Control Group

The use of a control group is an important experimental design method that involves having a group of participants that do not receive the treatment or intervention being studied. The control group is used as a baseline to compare the effects of the treatment group.

Blinding involves keeping participants, researchers, or both unaware of which treatment group participants are in, in order to reduce the risk of bias in the results.

Counterbalancing

This involves systematically varying the order in which participants receive treatments or interventions in order to control for order effects.

Replication

Replication involves conducting the same experiment with different samples or under different conditions to increase the reliability and validity of the results.

This experimental design method involves manipulating multiple independent variables simultaneously to investigate their combined effects on the dependent variable.

This involves dividing participants into subgroups or blocks based on specific characteristics, such as age or gender, in order to reduce the risk of confounding variables.

Data Collection Method

Experimental design data collection methods are techniques and procedures used to collect data in experimental research. Here are some common experimental design data collection methods:

Direct Observation

This method involves observing and recording the behavior or phenomenon of interest in real time. It may involve the use of structured or unstructured observation, and may be conducted in a laboratory or naturalistic setting.

Self-report Measures

Self-report measures involve asking participants to report their thoughts, feelings, or behaviors using questionnaires, surveys, or interviews. These measures may be administered in person or online.

Behavioral Measures

Behavioral measures involve measuring participants’ behavior directly, such as through reaction time tasks or performance tests. These measures may be administered using specialized equipment or software.

Physiological Measures

Physiological measures involve measuring participants’ physiological responses, such as heart rate, blood pressure, or brain activity, using specialized equipment. These measures may be invasive or non-invasive, and may be administered in a laboratory or clinical setting.

Archival Data

Archival data involves using existing records or data, such as medical records, administrative records, or historical documents, as a source of information. These data may be collected from public or private sources.

Computerized Measures

Computerized measures involve using software or computer programs to collect data on participants’ behavior or responses. These measures may include reaction time tasks, cognitive tests, or other types of computer-based assessments.

Video Recording

Video recording involves recording participants’ behavior or interactions using cameras or other recording equipment. This method can be used to capture detailed information about participants’ behavior or to analyze social interactions.

Data Analysis Method

Experimental design data analysis methods refer to the statistical techniques and procedures used to analyze data collected in experimental research. Here are some common experimental design data analysis methods:

Descriptive Statistics

Descriptive statistics are used to summarize and describe the data collected in the study. This includes measures such as mean, median, mode, range, and standard deviation.

Inferential Statistics

Inferential statistics are used to make inferences or generalizations about a larger population based on the data collected in the study. This includes hypothesis testing and estimation.

Analysis of Variance (ANOVA)

ANOVA is a statistical technique used to compare means across two or more groups in order to determine whether there are significant differences between the groups. There are several types of ANOVA, including one-way ANOVA, two-way ANOVA, and repeated measures ANOVA.

Regression Analysis

Regression analysis is used to model the relationship between two or more variables in order to determine the strength and direction of the relationship. There are several types of regression analysis, including linear regression, logistic regression, and multiple regression.

Factor Analysis

Factor analysis is used to identify underlying factors or dimensions in a set of variables. This can be used to reduce the complexity of the data and identify patterns in the data.

Structural Equation Modeling (SEM)

SEM is a statistical technique used to model complex relationships between variables. It can be used to test complex theories and models of causality.

Cluster Analysis

Cluster analysis is used to group similar cases or observations together based on similarities or differences in their characteristics.

Time Series Analysis

Time series analysis is used to analyze data collected over time in order to identify trends, patterns, or changes in the data.

Multilevel Modeling

Multilevel modeling is used to analyze data that is nested within multiple levels, such as students nested within schools or employees nested within companies.

Applications of Experimental Design 

Experimental design is a versatile research methodology that can be applied in many fields. Here are some applications of experimental design:

• Medical Research: Experimental design is commonly used to test new treatments or medications for various medical conditions. This includes clinical trials to evaluate the safety and effectiveness of new drugs or medical devices.
• Agriculture : Experimental design is used to test new crop varieties, fertilizers, and other agricultural practices. This includes randomized field trials to evaluate the effects of different treatments on crop yield, quality, and pest resistance.
• Environmental science: Experimental design is used to study the effects of environmental factors, such as pollution or climate change, on ecosystems and wildlife. This includes controlled experiments to study the effects of pollutants on plant growth or animal behavior.
• Psychology : Experimental design is used to study human behavior and cognitive processes. This includes experiments to test the effects of different interventions, such as therapy or medication, on mental health outcomes.
• Engineering : Experimental design is used to test new materials, designs, and manufacturing processes in engineering applications. This includes laboratory experiments to test the strength and durability of new materials, or field experiments to test the performance of new technologies.
• Education : Experimental design is used to evaluate the effectiveness of teaching methods, educational interventions, and programs. This includes randomized controlled trials to compare different teaching methods or evaluate the impact of educational programs on student outcomes.
• Marketing : Experimental design is used to test the effectiveness of marketing campaigns, pricing strategies, and product designs. This includes experiments to test the impact of different marketing messages or pricing schemes on consumer behavior.

Examples of Experimental Design 

Here are some examples of experimental design in different fields:

• Example in Medical research : A study that investigates the effectiveness of a new drug treatment for a particular condition. Patients are randomly assigned to either a treatment group or a control group, with the treatment group receiving the new drug and the control group receiving a placebo. The outcomes, such as improvement in symptoms or side effects, are measured and compared between the two groups.
• Example in Education research: A study that examines the impact of a new teaching method on student learning outcomes. Students are randomly assigned to either a group that receives the new teaching method or a group that receives the traditional teaching method. Student achievement is measured before and after the intervention, and the results are compared between the two groups.
• Example in Environmental science: A study that tests the effectiveness of a new method for reducing pollution in a river. Two sections of the river are selected, with one section treated with the new method and the other section left untreated. The water quality is measured before and after the intervention, and the results are compared between the two sections.
• Example in Marketing research: A study that investigates the impact of a new advertising campaign on consumer behavior. Participants are randomly assigned to either a group that is exposed to the new campaign or a group that is not. Their behavior, such as purchasing or product awareness, is measured and compared between the two groups.
• Example in Social psychology: A study that examines the effect of a new social intervention on reducing prejudice towards a marginalized group. Participants are randomly assigned to either a group that receives the intervention or a control group that does not. Their attitudes and behavior towards the marginalized group are measured before and after the intervention, and the results are compared between the two groups.

When to use Experimental Research Design 

Experimental research design should be used when a researcher wants to establish a cause-and-effect relationship between variables. It is particularly useful when studying the impact of an intervention or treatment on a particular outcome.

Here are some situations where experimental research design may be appropriate:

• When studying the effects of a new drug or medical treatment: Experimental research design is commonly used in medical research to test the effectiveness and safety of new drugs or medical treatments. By randomly assigning patients to treatment and control groups, researchers can determine whether the treatment is effective in improving health outcomes.
• When evaluating the effectiveness of an educational intervention: An experimental research design can be used to evaluate the impact of a new teaching method or educational program on student learning outcomes. By randomly assigning students to treatment and control groups, researchers can determine whether the intervention is effective in improving academic performance.
• When testing the effectiveness of a marketing campaign: An experimental research design can be used to test the effectiveness of different marketing messages or strategies. By randomly assigning participants to treatment and control groups, researchers can determine whether the marketing campaign is effective in changing consumer behavior.
• When studying the effects of an environmental intervention: Experimental research design can be used to study the impact of environmental interventions, such as pollution reduction programs or conservation efforts. By randomly assigning locations or areas to treatment and control groups, researchers can determine whether the intervention is effective in improving environmental outcomes.
• When testing the effects of a new technology: An experimental research design can be used to test the effectiveness and safety of new technologies or engineering designs. By randomly assigning participants or locations to treatment and control groups, researchers can determine whether the new technology is effective in achieving its intended purpose.

How to Conduct Experimental Research

Here are the steps to conduct Experimental Research:

• Identify a Research Question : Start by identifying a research question that you want to answer through the experiment. The question should be clear, specific, and testable.
• Develop a Hypothesis: Based on your research question, develop a hypothesis that predicts the relationship between the independent and dependent variables. The hypothesis should be clear and testable.
• Design the Experiment : Determine the type of experimental design you will use, such as a between-subjects design or a within-subjects design. Also, decide on the experimental conditions, such as the number of independent variables, the levels of the independent variable, and the dependent variable to be measured.
• Select Participants: Select the participants who will take part in the experiment. They should be representative of the population you are interested in studying.
• Randomly Assign Participants to Groups: If you are using a between-subjects design, randomly assign participants to groups to control for individual differences.
• Conduct the Experiment : Conduct the experiment by manipulating the independent variable(s) and measuring the dependent variable(s) across the different conditions.
• Analyze the Data: Analyze the data using appropriate statistical methods to determine if there is a significant effect of the independent variable(s) on the dependent variable(s).
• Draw Conclusions: Based on the data analysis, draw conclusions about the relationship between the independent and dependent variables. If the results support the hypothesis, then it is accepted. If the results do not support the hypothesis, then it is rejected.
• Communicate the Results: Finally, communicate the results of the experiment through a research report or presentation. Include the purpose of the study, the methods used, the results obtained, and the conclusions drawn.

Purpose of Experimental Design 

The purpose of experimental design is to control and manipulate one or more independent variables to determine their effect on a dependent variable. Experimental design allows researchers to systematically investigate causal relationships between variables, and to establish cause-and-effect relationships between the independent and dependent variables. Through experimental design, researchers can test hypotheses and make inferences about the population from which the sample was drawn.

Experimental design provides a structured approach to designing and conducting experiments, ensuring that the results are reliable and valid. By carefully controlling for extraneous variables that may affect the outcome of the study, experimental design allows researchers to isolate the effect of the independent variable(s) on the dependent variable(s), and to minimize the influence of other factors that may confound the results.

Experimental design also allows researchers to generalize their findings to the larger population from which the sample was drawn. By randomly selecting participants and using statistical techniques to analyze the data, researchers can make inferences about the larger population with a high degree of confidence.

Overall, the purpose of experimental design is to provide a rigorous, systematic, and scientific method for testing hypotheses and establishing cause-and-effect relationships between variables. Experimental design is a powerful tool for advancing scientific knowledge and informing evidence-based practice in various fields, including psychology, biology, medicine, engineering, and social sciences.

Advantages of Experimental Design 

Experimental design offers several advantages in research. Here are some of the main advantages:

• Control over extraneous variables: Experimental design allows researchers to control for extraneous variables that may affect the outcome of the study. By manipulating the independent variable and holding all other variables constant, researchers can isolate the effect of the independent variable on the dependent variable.
• Establishing causality: Experimental design allows researchers to establish causality by manipulating the independent variable and observing its effect on the dependent variable. This allows researchers to determine whether changes in the independent variable cause changes in the dependent variable.
• Replication : Experimental design allows researchers to replicate their experiments to ensure that the findings are consistent and reliable. Replication is important for establishing the validity and generalizability of the findings.
• Random assignment: Experimental design often involves randomly assigning participants to conditions. This helps to ensure that individual differences between participants are evenly distributed across conditions, which increases the internal validity of the study.
• Precision : Experimental design allows researchers to measure variables with precision, which can increase the accuracy and reliability of the data.
• Generalizability : If the study is well-designed, experimental design can increase the generalizability of the findings. By controlling for extraneous variables and using random assignment, researchers can increase the likelihood that the findings will apply to other populations and contexts.

Limitations of Experimental Design

Experimental design has some limitations that researchers should be aware of. Here are some of the main limitations:

• Artificiality : Experimental design often involves creating artificial situations that may not reflect real-world situations. This can limit the external validity of the findings, or the extent to which the findings can be generalized to real-world settings.
• Ethical concerns: Some experimental designs may raise ethical concerns, particularly if they involve manipulating variables that could cause harm to participants or if they involve deception.
• Participant bias : Participants in experimental studies may modify their behavior in response to the experiment, which can lead to participant bias.
• Limited generalizability: The conditions of the experiment may not reflect the complexities of real-world situations. As a result, the findings may not be applicable to all populations and contexts.
• Cost and time : Experimental design can be expensive and time-consuming, particularly if the experiment requires specialized equipment or if the sample size is large.
• Researcher bias : Researchers may unintentionally bias the results of the experiment if they have expectations or preferences for certain outcomes.
• Lack of feasibility : Experimental design may not be feasible in some cases, particularly if the research question involves variables that cannot be manipulated or controlled.

About the author

Muhammad Hassan

Researcher, Academic Writer, Web developer

You may also like

Questionnaire – Definition, Types, and Examples

Case Study – Methods, Examples and Guide

Observational Research – Methods and Guide

Quantitative Research – Methods, Types and...

Qualitative Research Methods

Explanatory Research – Types, Methods, Guide

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Published: 15 May 2024

A large-scale machine learning analysis of inorganic nanoparticles in preclinical cancer research

• Bárbara B. Mendes   ORCID: orcid.org/0000-0001-8630-1119 1   na1 ,
• Zilu Zhang   ORCID: orcid.org/0009-0000-2180-5957 2   na1 ,
• João Conniot 1 ,
• Diana P. Sousa   ORCID: orcid.org/0000-0003-3474-5417 1 ,
• João M. J. M. Ravasco 1 ,
• Lauren A. Onweller   ORCID: orcid.org/0009-0004-0865-4495 2 ,
• Andżelika Lorenc   ORCID: orcid.org/0000-0002-1474-7864 3 , 4 ,
• Tiago Rodrigues   ORCID: orcid.org/0000-0002-1581-5654 3 ,
• Daniel Reker   ORCID: orcid.org/0000-0003-4789-7380 2 , 5 &
• João Conde   ORCID: orcid.org/0000-0001-8422-6792 1  

Nature Nanotechnology ( 2024 ) Cite this article

1091 Accesses

14 Altmetric

Metrics details

• Nanoparticles
• Nanotechnology in cancer

Owing to their distinct physical and chemical properties, inorganic nanoparticles (NPs) have shown promising results in preclinical cancer therapy, but designing and engineering them for effective therapeutic purposes remains a challenge. Although a comprehensive database of inorganic NP research is not currently available, it is crucial for developing effective cancer therapies. In this context, machine learning (ML) has emerged as a transformative tool, but its adaptation to nanomedicine is hindered by inexistent or small datasets. Here we assembled a large database of inorganic NPs, comprising experimental datasets from 745 preclinical studies in cancer nanomedicine. Using descriptive statistics and explainable ML models we mined this database to gain knowledge of inorganic NP design patterns and inform future NP research for cancer treatment. Our analyses suggest that NP shape and therapy type are prominent features in determining in vivo efficacy, measured as a percentage of tumour reduction. Moreover, our database provides a large-scale open-access resource for discriminative ML that the broader nanotechnology community can utilize. Our work blueprints data mining for translational cancer research and offers evidence for standardizing NP reporting to accelerate and de-risk inorganic NP-based drug delivery, which may help to improve patient outcomes in clinical settings.

This is a preview of subscription content, access via your institution

Access options

Access Nature and 54 other Nature Portfolio journals

Get Nature+, our best-value online-access subscription

24,99 € / 30 days

cancel any time

Subscribe to this journal

Receive 12 print issues and online access

251,40 € per year

only 20,95 € per issue

Buy this article

• Purchase on Springer Link
• Instant access to full article PDF

Prices may be subject to local taxes which are calculated during checkout

Similar content being viewed by others

Computationally guided high-throughput design of self-assembling drug nanoparticles

In silico modelling of cancer nanomedicine, across scales and transport barriers

Evolutionary computational platform for the automatic discovery of nanocarriers for cancer treatment

Data availability.

The full database is publicly available via GitHub at https://github.com/RekerLab/NanoAnalysis . Source data are provided with this paper.

Code availability

The code to create our machine learning models and perform the analysis described here is publicly available via GitHub at https://github.com/RekerLab/NanoAnalysis .

Mendes, B. B., Sousa, D. P., Conniot, J. & Conde, J. Nanomedicine-based strategies to target and modulate the tumor microenvironment. Trends Cancer 7 , 847–862 (2021).

Article   CAS   PubMed   Google Scholar  

Bobo, D., Robinson, K. J., Islam, J., Thurecht, K. J. & Corrie, S. R. Nanoparticle-based medicines: a review of FDA-approved materials and clinical trials to date. Pharm. Res. 33 , 2373–2387 (2016).

Anselmo, A. C. & Mitragotri, S. Nanoparticles in the clinic: an update. Bioeng. Transl. Med. 4 , e10143 (2019).

Article   PubMed   PubMed Central   Google Scholar  

Anselmo, A. C. & Mitragotri, S. Nanoparticles in the clinic: an update post COVID-19 vaccines. Bioeng. Transl. Med. 6 , e10246 (2021).

Article   CAS   PubMed   PubMed Central   Google Scholar  

Mendes, B. B. et al. Nanodelivery of nucleic acids. Nat. Rev. Methods Primers 2 , 24 (2022).

van der Meel, R. et al. Smart cancer nanomedicine. Nat. Nanotechnol. 14 , 1007–1017 (2019).

Janjua, T. I., Cao, Y., Yu, C. & Popat, A. Clinical translation of silica nanoparticles. Nat. Rev. Mater. 6 , 1072–1074 (2021).

Das, C. G. A., Kumar, V. G., Dhas, T. S., Karthick, V. & Kumar, C. M. V. Nanomaterials in anticancer applications and their mechanism of action - a review. Nanomedicine 47 , 102613 (2023).

Gavas, S., Quazi, S. & Karpiński, T. M. Nanoparticles for cancer therapy: current progress and challenges. Nanoscale Res. Lett. 16 , 173 (2021).

Faria, M., Björnmalm, M., Crampin, E. J. & Caruso, F. A few clarifications on MIRIBEL. Nat. Nanotechnol. 15 , 2–3 (2020).

Faria, M. et al. Minimum information reporting in bio–nano experimental literature. Nat. Nanotechnol. 13 , 777–785 (2018).

Lorenc, A. et al. Machine learning for next-generation nanotechnology in healthcare. Matter 4 , 3078–3080 (2021).

Article   CAS   Google Scholar  

Mitchell, M. J. et al. Engineering precision nanoparticles for drug delivery. Nat. Rev. Drug Discov. 20 , 101–124 (2021).

Boehnke, N. et al. Massively parallel pooled screening reveals genomic determinants of nanoparticle delivery. Science 377 , eabm5551 (2023).

Article   Google Scholar  

Brockow, K. et al. Experience with polyethylene glycol allergy-guided risk management for COVID-19 vaccine anaphylaxis. Allergy 77 , 2200–2210 (2022).

Sellaturay, P., Nasser, S., Islam, S., Gurugama, P. & Ewan, P. W. Polyethylene glycol (PEG) is a cause of anaphylaxis to the Pfizer/BioNTech mRNA COVID-19 vaccine. Clin. Exp. Allergy 51 , 861–863 (2021).

Stone, C. A. Jr. et al. Immediate hypersensitivity to polyethylene glycols and polysorbates: more common than we have recognized. J. Allergy Clin. Immunol. Pract. 7 , 1533–1540.e8 (2019).

Article   PubMed   Google Scholar  

Chenthamara, D. et al. Therapeutic efficacy of nanoparticles and routes of administration. Biomater. Res. 23 , 20 (2019).

Sung, H. et al. Global cancer statistics 2020: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J. Clin. 71 , 209–249 (2021).

Global Burden of Disease 2019 Cancer Collaboration. Cancer incidence, mortality, years of life lost, years lived with disability, and disability-adjusted life years for 29 cancer groups from 2010 to 2019: a systematic analysis for the Global Burden of Disease Study 2019. JAMA Oncol. 8 , 420–444 (2022).

Alvarez, E. M. et al. The global burden of adolescent and young adult cancer in 2019: a systematic analysis for the Global Burden of Disease Study 2019. Lancet Oncol. 23 , 27–52 (2022).

Chen, Y., Chen, H. & Shi, J. In vivo bio-safety evaluations and diagnostic/therapeutic applications of chemically designed mesoporous silica nanoparticles. Adv. Mater. 25 , 3144–3176 (2013).

Iscaro, A., Howard, F. N. & Muthana, M. Nanoparticles: properties and applications in cancer immunotherapy. Curr. Pharm. Des. 25 , 1962–1979 (2019).

Zhou, H. et al. Biodegradable inorganic nanoparticles for cancer theranostics: insights into the degradation behavior. Bioconjug. Chem. 31 , 315–331 (2020).

Zhang, Y. et al. Prolonged local in vivo delivery of stimuli-responsive nanogels that rapidly release doxorubicin in triple-negative breast cancer cells. Adv. Healthc. Mater. 9 , 1901101 (2020).

Conde, J., Oliva, N., Zhang, Y. & Artzi, N. Local triple-combination therapy results in tumour regression and prevents recurrence in a colon cancer model. Nat. Mater. 15 , 1128–1138 (2016).

Kwong, B., Gai, S. A., Elkhader, J., Wittrup, K. D. & Irvine, D. J. Localized immunotherapy via liposome-anchored anti-CD137 + IL-2 prevents lethal toxicity and elicits local and systemic antitumor immunity. Cancer Res. 73 , 1547–1558 (2013).

Li, W. et al. Hyaluronic acid ion-pairing nanoparticles for targeted tumor therapy. J. Control. Release 225 , 170–182 (2016).

Lei, C. et al. Local release of highly loaded antibodies from functionalized nanoporous support for cancer immunotherapy. J. Am. Chem. Soc. 132 , 6906–6907 (2010).

Fransen, M. F., van der Sluis, T. C., Ossendorp, F., Arens, R. & Melief, C. J. M. Controlled local delivery of CTLA-4 blocking antibody induces CD8 + T-cell-dependent tumor eradication and decreases risk of toxic side effects. Clin. Cancer Res. 19 , 5381–5389 (2013).

Ishihara, J. et al. Matrix-binding checkpoint immunotherapies enhance antitumor efficacy and reduce adverse events. Sci. Transl. Med. 9 , eaan0401 (2017).

Errington, T. M., Denis, A., Perfito, N., Iorns, E. & Nosek, B. A. Challenges for assessing replicability in preclinical cancer biology. eLife 10 , e67995 (2021).

Wilhelm, S. et al. Analysis of nanoparticle delivery to tumours. Nat. Rev. Mater. 1 , 16014 (2016).

Cheng, Y.-H., He, C., Riviere, J. E., Monteiro-Riviere, N. A. & Lin, Z. Meta-analysis of nanoparticle delivery to tumors using a physiologically based pharmacokinetic modeling and simulation approach. ACS Nano 14 , 3075–3095 (2020).

Zhong, R. et al. Hydrogels for RNA delivery. Nat. Mater. https://doi.org/10.1038/s41563-023-01472-w (2023).

Lasagna-Reeves, C. et al. Bioaccumulation and toxicity of gold nanoparticles after repeated administration in mice. Biochem. Biophys. Res. Commun. 393 , 649–655 (2010).

Hatakeyama, H., Akita, H. & Harashima, H. A multifunctional envelope type nano device (MEND) for gene delivery to tumours based on the EPR effect: a strategy for overcoming the PEG dilemma. Adv. Drug Deliv. Rev. 63 , 152–160 (2011).

Harris, J. M., Martin, N. E. & Modi, M. Pegylation. Clin. Pharmacokinet. 40 , 539–551 (2001).

Suk, J. S., Xu, Q., Kim, N., Hanes, J. & Ensign, L. M. PEGylation as a strategy for improving nanoparticle-based drug and gene delivery. Adv. Drug Deliv. Rev. 99 , 28–51 (2016).

Zhang, M. et al. Influencing factors and strategies of enhancing nanoparticles into tumors in vivo. Acta Pharm. Sin. B 11 , 2265–2285 (2021).

Nguyen, L. N. M. et al. The exit of nanoparticles from solid tumours. Nat. Mater. 22 , 1261–1272 (2023).

Setyawati, M. I. et al. Titanium dioxide nanomaterials cause endothelial cell leakiness by disrupting the homophilic interaction of VE–cadherin. Nat. Commun. 4 , 1673 (2013).

Shamay, Y. et al. Quantitative self-assembly prediction yields targeted nanomedicines. Nat. Mater. 17 , 361–368 (2018).

Reker, D. et al. Computationally guided high-throughput design of self-assembling drug nanoparticles. Nat. Nanotechnol. 16 , 725–733 (2021).

Bannigan, P. et al. Machine learning models to accelerate the design of polymeric long-acting injectables. Nat. Commun. 14 , 35 (2023).

Lundberg, S. M. et al. From local explanations to global understanding with explainable AI for trees. Nat. Mach. Intell. 2 , 56–67 (2020).

Caballero, D. et al. Precision biomaterials in cancer theranostics and modelling. Biomaterials 280 , 121299 (2022).

Zhao, Y. et al. A comparison between sphere and rod nanoparticles regarding their in vivo biological behavior and pharmacokinetics. Sci. Rep. 7 , 4131 (2017).

Kolhar, P. et al. Using shape effects to target antibody-coated nanoparticles to lung and brain endothelium. Proc. Natl Acad. Sci. USA 110 , 10753–10758 (2013).

Zhang, M., Kim, H. S., Jin, T. & Moon, W. K. Near-infrared photothermal therapy using EGFR-targeted gold nanoparticles increases autophagic cell death in breast cancer. J. Photochem. Photobiol. B 170 , 58–64 (2017).

Jo, Y. et al. Chemoresistance of cancer cells: requirements of tumor microenvironment-mimicking in vitro models in anti-cancer drug development. Theranostics 8 , 5259–5275 (2018).

Guo, B. et al. Molecular engineering of conjugated polymers for biocompatible organic nanoparticles with highly efficient photoacoustic and photothermal performance in cancer theranostics. ACS Nano 11 , 10124–10134 (2017).

Li, Z. et al. Small gold nanorods laden macrophages for enhanced tumor coverage in photothermal therapy. Biomaterials 74 , 144–154 (2016).

Das, P., Delost, M. D., Qureshi, M. H., Smith, D. T. & Njardarson, J. T. A survey of the structures of US FDA approved combination drugs. J. Med. Chem. 62 , 4265–4311 (2019).

Fernandes Neto, J. M. et al. Multiple low dose therapy as an effective strategy to treat EGFR inhibitor-resistant NSCLC tumours. Nat. Commun. 11 , 3157 (2020).

Kim, M. H. et al. The effect of VEGF on the myogenic differentiation of adipose tissue derived stem cells within thermosensitive hydrogel matrices. Biomaterials 31 , 1213–1218 (2010).

Pedregosa, F. et al. Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12 , 2825–2830 (2011).

Google Scholar  

Chen, T. & Guestrin, C. XGBoost: a scalable tree boosting system. in Proc. of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining 785–794 (Association for Computing Machinery, 2016).

Lundberg, S. M. & Lee, S.-I. A unified approach to interpreting model predictions. in Advances in Neural Information Processing Systems (eds Guyon, I. et al.) Vol. 30 (Curran Associates, Inc., 2017).

Download references

Acknowledgements

This project received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (ERC-StG-2019-848325), the Duke Science and Technology Initiative and the National Institutes of Health NIGMS grant R35GM151255. We acknowledge Fundação para a Ciência e a Tecnologia (FCT) for financial support in the framework of the PhD grant 2020.06638.BD (D.P.S.), the Duke Department of Biomedical Engineering for support through a BME Fellowship (Z.Z.), the National Science Foundation (NSF) for support through the Graduate Research Fellowship DGE2129754 (L.A.O.) and the ERASMUS+ programme (A.L.).

Author information

These authors contributed equally: Bárbara B. Mendes, Zilu Zhang.

Authors and Affiliations

ToxOmics, NOVA Medical School, Faculdade de Ciências Médicas (NMS|FCM), Universidade NOVA de Lisboa, Lisbon, Portugal

Bárbara B. Mendes, João Conniot, Diana P. Sousa, João M. J. M. Ravasco & João Conde

Department of Biomedical Engineering, Duke University, Durham, NC, USA

Zilu Zhang, Lauren A. Onweller & Daniel Reker

Instituto de Investigação do Medicamento (iMed), Faculdade de Farmácia, Universidade de Lisboa, Lisbon, Portugal

Andżelika Lorenc & Tiago Rodrigues

Department of Biopharmacy, Ludwik Rydygier Collegium Medicum in Bydgoszcz, Nicolaus Copernicus University in Toruń, Bydgoszcz, Poland

Andżelika Lorenc

Duke Cancer Institute, Duke University School of Medicine, Durham, NC, USA

Daniel Reker

You can also search for this author in PubMed   Google Scholar

Contributions

J. Conde conceived the idea and concept of the study. D.R. conceived the ML platform. T.R. conceived the data curation. B.B.M., J. Conniot, D.P.S., J.M.J.M.R. and J. Conde collected all of the data from the published manuscripts, organized the dataset and calculated the correlations. Z.Z., L.A.O. and A.L. conducted the data analysis, text mining and designed, implemented and evaluated the ML models. J. Conde, D.R. and T.R provided guidance and supervised the work. All authors contributed to the writing and editing of the paper, and all authors approved the final version of the paper.

Corresponding authors

Correspondence to Tiago Rodrigues , Daniel Reker or João Conde .

Ethics declarations

Competing interests.

J. Conde and T.R. are co-founders and shareholders of TargTex SA Targeted Therapeutics for Glioblastoma Multiforme. J. Conde is a member of the Global Burden of Disease (GBD) consortium from the Institute for Health Metrics and Evaluation (IHME), University of Washington, USA, and member of the Scientific Advisory Board of Vector Bioscience, Cambridge. T.R. acts as a consultant to the pharmaceutical, biotechnology and technology industry and is a full member of the Acceleration Consortium, University of Toronto. D.R. acts as a consultant to the pharmaceutical and biotechnology industry, as a scientific mentor for Start2 and serves on the scientific advisory board of Areteia Therapeutics. The other authors declare no competing interests.

Peer review

Peer review information.

Nature Nanotechnology thanks Natalie Boehnke, Karolina Jagiełło and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary information.

Supplementary Results and Discussion, Figs. 1– 9, Tables 1–8 and Refs. 1–11.

Source data

Source data fig. 2.

Alluvial plot source data for Fig. 2c.

Source Data Fig. 4

Raw data for Fig. 4d, alluvial plot source data for Fig. 4e and statistical source data for Fig. 4f.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Cite this article.

Mendes, B.B., Zhang, Z., Conniot, J. et al. A large-scale machine learning analysis of inorganic nanoparticles in preclinical cancer research. Nat. Nanotechnol. (2024). https://doi.org/10.1038/s41565-024-01673-7

Download citation

Received : 03 May 2023

Accepted : 10 April 2024

Published : 15 May 2024

DOI : https://doi.org/10.1038/s41565-024-01673-7

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

• Open access
• Published: 13 May 2024

Identification and verification of a novel signature that combines cuproptosis-related genes with ferroptosis-related genes in osteoarthritis using bioinformatics analysis and experimental validation

• Baoqiang He 1 , 2   na1 ,
• Yehui Liao 1   na1 ,
• Minghao Tian 1 ,
• Chao Tang 1 ,
• Qiang Tang 1 ,
• Wenyang Zhou 1 ,
• Yebo Leng 1 , 3 &
• Dejun Zhong 1 , 2  

Arthritis Research & Therapy volume  26 , Article number:  100 ( 2024 ) Cite this article

207 Accesses

Metrics details

Exploring the pathogenesis of osteoarthritis (OA) is important for its prevention, diagnosis, and treatment. Therefore, we aimed to construct novel signature genes (c-FRGs) combining cuproptosis-related genes (CRGs) with ferroptosis-related genes (FRGs) to explore the pathogenesis of OA and aid in its treatment.

Materials and methods

Differentially expressed c-FRGs (c-FDEGs) were obtained using R software. Enrichment analysis was performed and a protein–protein interaction (PPI) network was constructed based on these c-FDEGs. Then, seven hub genes were screened. Three machine learning methods and verification experiments were used to identify four signature biomarkers from c-FDEGs, after which gene set enrichment analysis, gene set variation analysis, single-sample gene set enrichment analysis, immune function analysis, drug prediction, and ceRNA network analysis were performed based on these signature biomarkers. Subsequently, a disease model of OA was constructed using these biomarkers and validated on the GSE82107 dataset. Finally, we analyzed the distribution of the expression of these c-FDEGs in various cell populations.

A total of 63 FRGs were found to be closely associated with 11 CRGs, and 40 c-FDEGs were identified. Bioenrichment analysis showed that they were mainly associated with inflammation, external cellular stimulation, and autophagy. CDKN1A, FZD7, GABARAPL2, and SLC39A14 were identified as OA signature biomarkers, and their corresponding miRNAs and lncRNAs were predicted. Finally, scRNA-seq data analysis showed that the differentially expressed c-FRGs had significantly different expression distributions across the cell populations.

Four genes, namely CDKN1A, FZD7, GABARAPL2, and SLC39A14, are excellent biomarkers and prospective therapeutic targets for OA.

Introduction

As a degenerative disease that is difficult to reverse, osteoarthritis (OA) is often accompanied by joint pain, stiffness, joint swelling, restricted movement, and joint deformity, all of which seriously affect daily life activities. The structural changes in OA mainly involve the articular cartilage, subchondral bone, ligaments, capsule, synovium, and periarticular muscles [ 1 ]. The prevalence of OA is steadily rising due to the aging population and the obesity epidemic [ 1 ], and it has placed a significant burden on society [ 2 ]. Currently, the main treatments for OA remain nonsteroidal anti-inflammatory drugs (NSAIDs), pain medications, and joint replacement surgery. However, these treatments cannot reduce the incidence of the early stages of the disease [ 3 ], prevent further cartilage degeneration, or promote cartilage regeneration [ 4 ]. Therefore, further understanding of the pathophysiological mechanisms of OA could aid in the development of additional approaches for more effective diagnosis and treatment.

Ferroptosis is a specific type of programmed cell death driven by iron-dependent lipid peroxidation characterized by an abnormal accumulation of lipid reactive oxygen species (ROS) [ 5 , 6 ]. This programmed cell death was first reported and named by Dixon in 2012 [ 7 ]. Many studies have demonstrated that ferroptosis and the development of OA are closely related [ 8 , 9 , 10 , 11 ], and ferroptosis-related genes (FRGs) can help in the diagnosis of OA, as well as in predicting the immune status of patients with OA [ 12 , 13 ].

Copper is an indispensable trace element involved in a wide range of biological reactions. A small study reported elevated plasma and synovial copper concentrations in patients with OA compared with healthy controls [ 14 ], and another study also found that elevated levels of copper were associated with an increased risk of OA [ 15 ]. When the oxidizing capacity of copper ions in the body exceeds the antioxidant capacity of the body, joints can be destroyed [ 16 ]. Cuproptosis is a novel form of programmed cell death during which copper binds directly to the fatty acylated components of the tricarboxylic acid (TCA) cycle, thereby leading to an increase in toxic proteins and ultimately to cell death [ 17 ]. Ferroptosis is an iron-dependent programmed cell death caused by lipid peroxidation and the massive accumulation of reactive oxygen radicals[ 7 ]. Furthermore, copper and iron are closely related; copper is essential for iron absorption, meaning that copper deficiency or overload can impair the balance of iron metabolism [ 18 ]. When the balance of iron metabolism is disturbed, lipid peroxidation and oxidative stress may be induced, which in turn leads to ferroptosis and alters the expression of FRGs [ 19 , 20 , 21 ]. However, it has not yet been reported whether new signature genes (c-FRGs) combining cuproptosis-related genes (CRGs) with FRGs are beneficial for the diagnosis and treatment of OA.

In this study, we explored and analyzed the immune characteristics and biological functions of c-FRGs in patients with OA. In addition, we screened key ferroptosis-related biomarkers associated with cuproptosis in OA, constructed ceRNA networks, and predicted potential drugs for OA treatment. Our results suggest that c-FRGs may play an important role in the pathophysiological process of OA and provide new directions and ideas for OA research.

Data collection

The US National Center for Biotechnology Information (NCBI) gene expression omnibus (GEO) is the world's largest international public repository of high-throughput molecular information. Using “osteoarthritis” as a search term, the GEO database ( https://www.ncbi.nlm.nih.gov/geo/ ) was searched for appropriate datasets, and four datasets that met the study requirements were downloaded. These four datasets were GSE55235, GSE169077, GSE55457, and GSE55584, and the chip type was Affymetrix Human Genome U133a. We eventually obtained 25 normal human synovial samples and 32 OA synovial samples from the four datasets as samples for the follow-up study. To assess the accuracy of the analysis, the GSE82107 dataset was used as validation sets. In addition, the FRGs and CRGs were obtained from the published literature [ 6 ] and the FerrDb website ( http://www.zhounan.org/ferrdb/ ).

Extraction of c-FRGs and obtaining differentially expressed c-FRGs

Inter-batch differences between the four groups (GSE55235, GSE169077, GSE55457, and GSE55584) were eliminated using “affy” packet merging and the “sva” packet. We performed a Pearson correlation analysis of CRGs with FRGs to obtain particular FRGs (c-FRGs) that were highly correlated with CRGs (|r| > 0.5, adj. p value < 0.05). Differentially expressed genes (DEGs) and differentially expressed c-FRGs (c-FDEGs) were obtained using the “limma” package ( p value < 0.05).

Function enrichment analysis and protein–protein interaction (PPI) networks

To acquire disease-related biological functions and signaling pathways, Gene Ontology (GO) enrichment analysis and Kyoto Encyclopedia of Genes and Genomes (KEGG) pathway enrichment analysis of c-FDEGs were performed. GO enrichment analysis was used to describe the molecular functions (MF), cellular components (CC), and biological processes (BP) involved in the target genes ( p -value < 0.05). KEGG analysis was used to systematically analyze gene functions and to link genomic information and functional information ( p -value < 0.05). The results of the gene set enrichment analysis (GSEA), GO enrichment analysis, and KEGG pathway enrichment analysis of the c-FDEGs were visualized using the “ClusterProfiler” package in R. GSEA was based on the gene set (h. all. v7. 5. 1. symbols. gmt), which was downloaded from MSigDB ( https://www.gsea-msigdb.org/gsea/msigdb/index.jsp ). The STRING database is used for searching interactions between known proteins and for predicting interactions between proteins and is one of the most data-rich and widely used databases for studying protein interactions. Protein interaction analysis was performed on all c-FDEGs through the STRING website ( https://string-db.org/ ) and visualized using Cytoscape software. The degree values of the c-FDEGs were calculated using the cytoHubba plugin, and the top seven genes were used as hub genes.

Acquisition and validation of biomarkers

In this research, we used three machine learning algorithms: support vector machine recursive feature elimination (SVM-RFE), least absolute shrinkage and selection operator (LASSO) regression analysis, and random forest analysis (RF). First, we used the “e1071” R package for SVM-RFE analysis. Subsequently, the “glmnet” package was used to perform LASSO regression analysis. In addition, RF was conducted adopting the “randomForest” package, and genes with importance > 1 were retained. The crossover genes obtained by these three methods were regarded as prospective biomarkers for OA.

Construction and validation of disease model (nomogram)

In addition, a nomogram based on characteristic biomarkers was structured using the “rms” R package. Receiver operating characteristic (ROC) analysis was performed on the biomarkers and the obtained models, and the area under the curve (AUC) values were calculated with the “pROC” package to assess the diagnostic efficacy of the potential biomarkers. In addition, the four biomarkers and the obtained disease nomogram were validated on the GSE82107 validation set.

Collection of clinical samples

Synovial tissue collection and all experimental procedures were approved by the Institutional Review Board of the Affiliated Hospital of Southwest Medical University (KY2023293) in accordance with the guidelines of the Chinese Health Sciences Administration, and written informed consent was obtained from the donors. Synovial tissue from the suprapatellar bursa was collected as OA synovial samples and normal control samples, respectively, from patients who met the American College of Rheumatology criteria for the diagnosis of primary symptomatic knee OA (n=6; men: 3, women: 3; age: 55-70 years) and from patients who underwent trauma-related lower extremity amputation but did not have osteoarthritis or rheumatoid arthritis (n=6; men: 4, women: 2; age: 50-67 years). All samples were collected within two hours of arthroplasty or lower limb amputation and were divided into two portions for subsequent immunofluorescence staining and western blot experiments, respectively.

Immunofluorescence staining

Mid-sagittal sections (4-μm thick) of paraffin-embedded clinical synovial specimens were incubated for 1 hour at room temperature, after which the slides were closed with 10% bovine serum (Solarbio, Beijing, China) for 1 hour at room temperature and then incubated with primary antibodies for 16 hours at 4°C. The fluorescent dye was incubated for 1 hour at room temperature, and the slides were subsequently sealed with DAPI Sealer (Thermo Fisher Scientific, Waltham, MA, USA).

Western blot analysis

Protein lysates were extracted from synovial tissue samples and lysed with RIPA buffer to extract the total protein. After conducting a BCA protein assay (Beyotime, Shanghai, China), 5 × sample buffer (Servicebio, Wuhan, China) was added to the protein lysates. Equal amounts of lysates were then separated through SDS-PAGE and transferred to a 0.22-um PVDF microporous membrane (Merck Millipore, Burlington, MA, USA). Next, the membrane was sealed with 5% skimmed milk and incubated with the primary antibody for 16 hours at 4°C, after which the membrane was incubated with the secondary antibody for 60 minutes at room temperature. Target protein bands were visualized using FDbio-Dura ECL (Merck Millipore, Burlington, MA, USA). The antibodies used for immunofluorescence and western blot in this study were as follows: rabbit anti-FZD7 (Cat. #: DF8657, 1:1,000; AFFBIOTECH, USA), rabbit anti-SLC39A14 (ZIP14) (Cat. #: 26540-1-AP, 1:1,000, Proteintech, Rosemont, IL, USA), rabbit anti-CDKN1A (p21) (Cat. #: 2947T, 1:1,000, Cell Signaling Technology, Danvers, MA, USA), rabbit anti-GABARAPL2 (Cat. #: 14256T, 1:1,000, Cell Signaling Technology), anti-GAPDH (Cat. #: 60004 -1-Ig, 1:1,000, Proteintech, USA), and species-matched HRP-conjugated secondary antibody (Cat. #: SA00001-1, 1:1,000; Proteintech, USA).

ssGSEA, GSEA, and GSVA for differentially expressed c-FRGs

The gene set (h.all.v2022.1.Hs.symbols.gmt), a collection of 50 symbolic gene sets for humans, was downloaded from MSigDB ( https://www.gsea-msigdb.org/gsea/msigdb/index.jsp ). The 50 symbolic human gene set scores were calculated for each sample using single-sample GSEA (ssGSEA), and differential scores were obtained for the non-OA and OA groups. The “corrplot” package was used to perform correlation analysis between biomarkers and ssGSEA gene sets. Next, GSEA and gene set variation analysis (GSVA) were performed for the four biomarkers, the seven hub genes, and the remaining 29 differentially expressed c-FRGs.

Prediction of therapeutic drugs

The gene–drug interaction database (DGIDB, http://www.dgidb.org ) [ 22 ] can help researchers annotate known pharmacogenetic interactions and potential drug accessibility–related genes. In this research, we used DGIdb to filter potential drugs targeted to biomarkers so as to identify new therapeutic targets. The obtained drug prediction results were also imported into Cytoscape (v3.9.1) software for visualization.

Construction of ceRNA network

The miRanda, TargetScan, and miRDB databases are authoritative databases used for predicting miRNA–target gene regulatory relationships, and spongeScan is a web tool designed for sequence-based complementary detection of miRNA-binding elements in lncRNA sequences. Biomarkers of common mRNA–miRNA interactions were identified in miRanda ( http://www.microrna.org/microrna/home.do ), TargetScan ( http://www.targetscan.org ), and miRDB ( https://mirdb.org ). miRNA–lncRNA interactions were obtained from Spongescan ( http://spongescan.rc.ufl.edu ). These interactions were imported into Cytoscape to construct the ceRNA network.

Immune infiltration analysis

To better understand the changes that occur in the immune system of patients with OA, the “CIBERSORT” R package was used to describe the basic expression of 22 immune cell subtypes. Next, we analyzed the correlation between potential biomarkers, hub genes, and the 22 immune cell types.

scRNA‑seq analysis

The OA synovial scRNA-seq data (GSE152805) from three patients were obtained from the GEO database and analyzed using the "Seurat" software package. To ensure high quality of the data, we removed low-quality cells (cells with <200 or >10,000 detected genes, >10% of mitochondrial genes, or <300 or >30,000 expressed genes) and low-expressed genes (any gene expressed in less than three cells). We used the "NormalizeData" function to normalize the gene expression of the included cells and performed principal component analysis (PCA) using the top 2000 highly variable genes to extract the top 12 principal components (PCs), which were retained for further analysis using the "FindVariableFeatures" function. To perform unsupervised and unbiased clustering of cell subpopulations, the "FindNeighbors," "FindClusters" (resolution = 0.6), and "RunUMAP" functions were applied. Each cell cluster was manually annotated according to the cell-specific marker genes. These marker genes were obtained from previously published literature[ 23 , 24 ] and from the CellMarker website ( http://xteam.xbio.top/CellMarker/ ). Finally, we used CellChat (1.6.1) for the inference and analysis of cell–cell communication.

Figure 1 describes the entire flow of the study.

A graphical flowchart of the study design

Extracting c-FRGs and obtaining differentially expressed c-FRGs

After merging the GSE55235, GSE169077, GSE55457, and GSE55584 datasets (Table 1 ), the newly produced gene expression matrices were subjected to normalization and presented as bidimensional PCA plots prior to and after processing (Fig. 2 a and b), indicating that the final sample data obtained were plausible. A total of 63 FRGs were found to be closely associated with 11 CRGs (Fig. 2 e, Supplementary Table 1 ). A total of 4167 DEGs were determined and identified (Fig. 2 c). There were a total of 40 c-FDEGs, including 13 upregulated genes and 27 downregulated genes (Fig. 2 d, Supplementary Table 2 ). The correlations between the 40 c-FDEGs are shown in Supplementary Figure 1 . The expression patterns of the 40 c-FDEGs are visualized in the heatmap (Fig. 2 f).

Extraction of particular ferroptosis-related genes (c-FRGs) and obtainment of differentially expressed c-FRGs (c-FDEGs). a, b Two-dimensional PCA cluster plot of GSE55235, GSE169077, GSE55457, and GSE55584 datasets before and after normalization. c Volcano plot of DEGs. Red spots represent upregulated genes and green spots represent downregulated genes. d Overall expression landscape of c-FRGs in osteoarthritis (OA). * P < 0.05; ** P < 0.01; *** P < 0. 001. OA represents the OA group and Normal represents the normal control group. e Extraction of c-FDEGs. f  Heatmap of c-FDEGs. The redder the color, the higher the expression; conversely, the bluer the color, the lower the expression

Function enrichment analysis

Understanding the signaling pathways, biological processes, and interrelationships involved in c-FDEGs is of great importance in revealing the pathogenesis of OA. GO enrichment analysis showed that c-FDEGs were significantly enriched in the regulation of the inflammatory response (BP), the positive regulation of cellular catabolic process (BP), the autophagosome membrane (CC), the recycling endosome (CC), and NF-κB binding (MF) (Fig. 3 a, Supplementary Table 3 ). KEGG pathway analysis showed that these c-FDEGs were mainly involved in the IL-17 signaling pathway, NOD-like receptor signaling pathway, HIF-1 signaling pathway, and TNF signaling pathway (Fig. 3 b, Supplementary Table 4 ). GSEA suggested that the development of OA may be associated with hypoxia, MYC targets v2, the P53 pathway, the inflammatory response, TNFα signaling via NF-κB, the interferon-α response, and peroxisome (Fig. 3 c and d).

Functional analyses: ( a ) Gene Ontology (GO) enrichment analysis showed that the 40 c-FDEGs were significantly enriched in the regulation of the inflammatory response, the positive regulation of cellular catabolic process, the autophagosome membrane, the recycling endosome, and NF-κB binding. b Kyoto Encyclopedia of Genes and Genomes (KEGG) pathway analysis showed that these c-FDEGs were mainly involved in the IL-17 signaling pathway, NOD-like receptor signaling pathway, HIF-1 signaling pathway, and TNF signaling pathway. c Gene set enrichment analysis (GSEA) in the normal control group and (d) GSEA in the OA group based on the core set of 50 human genes suggested that the development of OA may be associated with hypoxia, MYC targets v2, the P53 pathway, the inflammatory response, TNFα signaling via NF-κB, the interferon-α response, and peroxisome

Building PPI networks

The String database is a database that can be used to retrieve interactions between known and predicted proteins. To explore the interactions between each c-FDEG, all of the abovementioned 40 c-FDEGs were imported into the STRING database. The PPI network of c-FDEGs after deleting isolated c-FDEGs and adding the six related CRGs (without CDKN2A) is shown in Fig. 4 a. The cytoHubba plugin in Cytoscape software was used to calculate the degree values (degrees) of the top seven genes (IL6, IL1B, RELA, PTGS2, EGFR, CDKN2A, and SOCS1) as the PPI network’s hub genes (Fig. 4 b).

Protein–protein interaction (PPI) network and core gene screening. a PPI network constructed from 40 c-FDEGs; red triangles represent c-FDEGs, green triangles represent CRGs that are closely related to them, and the correlation between c-FDEGs and CRGs is indicated by dashed lines. b The top seven core gene interaction networks calculated using the cytoHubba plugin: the darker the color, the more powerful the critical degree

Machine learning algorithm–based biomarker screening for patients with OA

In this study, 40 c-FDEGs were further analyzed for potential biomarkers associated with OA using multiple machine learning methods. SVM-RFE analysis showed that the model containing 24 genes had the best accuracy (Fig. 5 a). LASSO regression analysis showed that the model was able to accurately predict OA when λ was equal to 12. Thus, the LASSO regression model generated 12 candidate genes (Fig. 5 b). We retained the candidate biomarkers with RF results importance > 1 (Fig. 5 c). Lastly, the results of these three methods were integrated, and CDKN1A, FZD7, GABARAPL2, and SLC39A14 were identified as the final potential biomarkers for OA (Fig. 5 d).

Machine learning-based potential biomarker screening. a SVM-RFE model with the optimal error rate when the number of signature genes was 58. b LASSO regression model. c Random forest model and the top 20 genes in terms of importance. d The final biomarkers screened using three machine learning algorithms

Experimental validation of four biomarkers

To validate the results of the bioinformatics analysis, we collected OA samples (n=6) and normal group samples (n=6), respectively, and performed western blot analysis and immunofluorescence staining (Fig. 6 ). Both results were consistent with the bioinformatics analysis, i.e., higher expression of FZD7 and GABARAPL2 and lower expression of CDKN1A (p21) and SLC39A14 (ZIP14) in the OA group compared with the normal group.

Experimental validation of four biomarkers. a Representative immunofluorescence staining images of the four biomarker proteins (p21, FZD7, GABARAPL2, and ZIP14) in the normal and OA groups, with nuclei stained blue with 4’,6-diamidino-2-phenylindole. Scale bar = 25 µm. b Semi-quantitative analysis of mean fluorescence intensity of the four biomarker proteins in the normal and OA groups ( n = 6). (c, d) Representative western blotting and statistical comparisons of the four biomarker proteins in the normal and OA groups ( n = 6). * p < 0.05, ** p < 0.01, all by independent samples t-test

To better capture the function of the four biomarkers in OA, GSEA, GSVA, and ssGSEA were conducted on each of the above biomarkers (Fig. 7 ). The ssGSEA showed that the OA group was significantly enriched in Notch signaling, interferon alpha (IFN-α) response, the Wnt/β-catenin pathway, bile acid metabolism, and peroxisome, while the non-OA group was mainly enriched in TNFα signaling via NF-κB, hypoxia, MYC targets v2, the P53 pathway, the inflammatory response, PI3K AKT mTOR signaling, and IL6 JAK STAT3 signaling (Fig. 7 i). Correlation analysis showed that CDKN1A and SLC39A14 were significantly positively correlated with the gene sets of hypoxia, TNF-α signaling via NF-κB, the P53 pathway, and mTORC1 signaling. Meanwhile, GABARAPL2 and FZD7 showed significant negative correlations with the gene sets of TNF-α signaling via NF-κB, PI3K AKT mTOR signaling, and mTORC1 signaling (Fig. 7 j). The single-gene GSEA results for the seven hub genes are shown in Supplementary Figure 2 (a–g). The remaining 29 differentially expressed c-FRGs are shown in Supplementary Figure 3 .

GSEA, GSVA, and ssGSEA results of four potential biomarkers. a–d Single-gene GSEA-KEGG pathway analysis of four potential biomarkers. We show the top six pathways with the smallest p -value. e–h High- and low-expression groups based on the expression levels of each potential biomarker combined with gene set variation analysis (GSVA). Red means the pathway is significantly upregulated, green means the pathway is significantly downregulated, and gray means the pathway is not statistically significant. i ssGSEA of OA and normal controls based on the h.all.v7.5.1.symbols.gmt gene set. * P < 0.05; ** P < 0.01; *** P < 0. 001. Treat represents the OA group, and control represents the normal group. (j) Correlation of four biomarkers with 50 human symbolic gene sets from the h.all.v7.5.1.symbols.gmt gene set

Using the above four biomarkers, a disease nomogram was constructed. The AUC values of the individual genes CDKN1A, FZD7, GABARAPL2, and SLC39A4 were 0.931, 0.879, 0.989, and 0.850, respectively, all of which were greater than 0.85 (Fig. 8 a), further indicating that the above genes had good diagnostic ability (Fig. 8 b). The AUC value of this model was 0. 996, which was significantly greater than the AUC value of individual biomarkers, indicating that this model had good diagnostic value (Fig. 8 c and d). To verify whether the above model is diagnostically meaningful, validation was performed on the GSE8207 dataset. The results showed that the AUC values of the four biomarkers were all greater than 0.7, and the AUC value of the model was 1 for the validation set (Fig. 8 f). These results indicate that CDKN1A, FZD7, GABARAPL2, and SLC39A4 are effective disease biomarkers for OA and that the model has high diagnostic efficacy.

Validation of four biomarkers. a ROC analysis of the four biomarkers. b ROC analysis of the disease model constructed from the four biomarkers. c, d Nomograms based on the disease model: we obtained the corresponding scores for each genetic variable, drew a vertical line above the “points” axis, summed the scores of all predictor variables, found the final value on the “total score” axis, and then drew a straight line on the “probability” axis to determine the patient’s risk of osteoarthritis. e, f Validation of the disease model and four biomarkers on the GSE82107 validation dataset

Construction of drug prediction network and lncRNA–miRNA–mRNA network

The corresponding drug prediction network was constructed using the database based on the four biomarkers (Supplementary Figure 4 a). The predicted drugs were celecoxib, paclitaxel, carboplatin, acetaminophen, vantictumab, and nortriptyline. Based on the competitive endogenous RNA hypothesis, an lncRNA–miRNA–mRNA competitive endogenous RNA (ceRNA) network was constructed to explore the function of lncRNA as an miRNA sponge in OA. We obtained 150 target miRNAs based on these biomarkers. Then, 48 lncRNAs were obtained based on these miRNA predictions. The four biomarkers with predicted miRNAs and lncRNAs were introduced into Cytoscape, and constituted a ceRNA network containing 48 lncRNA nodes, 150 miRNA nodes, 4 hub gene nodes, and 198 edges (Supplementary Figure 4 b).

The immune microenvironment plays an important role in the progression of OA. Therefore, with the help of CIBERSORT, we summarized the differences in immune infiltration by immune cell subpopulations between OA samples and non-OA tissues (Fig 9 a). The OA samples contained a higher proportion of memory B cells, M0 macrophages, M2 macrophages, and resting mast cells than the control group, as well as a lower proportion of resting CD4 memory T cells and activated mast cells. Correlation analysis showed that activated mast cells showed positive correlations with PTGS2, IL6, and IL1B, and the correlation between activated mast cells and PTGS2 was the highest (0. 686) (Fig. 9 b). There were positive correlations between IL1B, PTGS2, and M1 macrophages, resting CD4 memory T cells and PTGS2, and regulatory T cells (Tregs) and RELA. There were significant negative correlations between follicular helper T cells and RELA, as well as between plasma cells and SLC39A14 (Fig. 9 c and d).

Results of immune infiltration by CIBERSORTx. a Bar plot showing the composition of 22 types of immune cells. b Box plot presenting the difference of immune infiltration of 22 types of immune cells. Treat represents the OA group, and Control represents the normal group. c Heatmap showing the correlation between seven hub genes and 22 types of immune cells in osteoarthritis. d Correlation between the four biomarkers and 22 types of immune cells in osteoarthritis

Single‑cell analysis

The scRNA-seq data from three OA synovial samples were obtained from the GSE152805 dataset. After initial quality control, we finally retained 10,194 cells for cell annotation (Supplementary Figure 5 ). The top 2000 highly variable genes were selected for further analysis (Supplementary Figure 5 b). We used the "RunPCA" function to reduce the dimensionality and obtained 14 clusters (Supplementary Figures 6 d and e); the first five DEGs of each cluster are shown in Supplementary Table 5 . Later, we performed cellular annotation using marker genes and annotated seven cell populations: fibroblasts (77.7%), macrophages (8.8%), dendritic cells (DCs) (3.6%), endothelial cells (ECs) (3.5%), smooth muscle cells (SMCs) (3.4%), T cells (1.8%), and mast cells (1.2%) (Fig. 10 a). Next, we performed differential gene expression analysis on these seven cell populations to verify the accuracy of the cell annotation (Fig. 10 b). Figures 10 c and d show the distribution and expression of seven hub genes and four biomarker genes in different cell populations. We found that 11 c-FRGs were significantly different in macrophages, DCs, mast cells, and NK cells. For example, IL1B, PTGS2, and SLC39A4 were significantly highly expressed in some cells, whereas they were significantly less expressed, or even absent, in other cells. We used CellChat to identify differentially overexpressed ligands and receptors for each cell population. In total, 254 significant ligand–receptor pairs were detected, which were further classified into 62 signaling pathways (Table 2 ). We found that the immune cells interacted weakly with each other; however, the non-immune cells had extensive communication interactions with other cells and were involved in various paracrine and autocrine signaling interactions (Fig. 10 e to g).

Analysis of single-cell RNA sequencing data from three OA synovial samples. a UMAP plot of scRNA-seq showing unsupervised clusters colored according to putative cell types among a total of 10,194 cells in OA synovial samples. The percentages of total acquired cells were as follows: 77.7% fibroblasts, 8.8% macrophages, 3.6% dendritic cells (DCs), 3.5% endothelial cells (ECs), 3.4% smooth muscle cells (SMCs), 1.8% T cells, and 1.2% mast cells. b Heatmap depicting the expression levels of the top five marker genes among seven detected cell clusters. c, d UMAP plots and violin plots showing the expression of the selected seven hub c-FRGs and four potential biomarkers for each cell type. e Interaction net count plot of OA synovial cells. The thicker the line, the greater the number of interactions. f Interaction weight plot of synovial cells. The thicker the line, the stronger the interaction weights/strength between the two cell types. g Detailed network of cell–cell interactions among seven cell subsets

Copper is an irreplaceable trace metal element that participates in a variety of biological processes. When copper ions accumulate in excess, they eventually lead to cell death, and this new form of programmed cell death is known as cuproptosis [ 17 ]. A recent report has demonstrated that copper levels are significantly higher in the serum and synovial tissue of patients with OA than in controls [ 14 ]. Evidence from several studies suggests that the development of OA is closely related to ferroptosis in articular cartilage and synovium [ 25 , 26 , 27 , 28 , 29 ], and that OA can be treated to some extent by modulation of ferroptosis [ 29 , 30 ]. Additionally, previous studies have reported that copper and iron levels are closely correlated with each other in patients with OA [ 14 , 15 , 31 ].

In this study, we identified transcriptional alterations and expression of c-FRGs based on the GSE55235, GSE169077, GSE55457, and GSE55584 datasets. Forty c-FDEGs were identified in 63 c-FRGs. GO enrichment analysis showed that these 40 c-FDEGs were mainly associated with the inflammatory response, cellular response to external stimulus, and autophagy. The KEGG enrichment analysis showed that these genes were highly enriched mainly in the IL-17 signaling pathway, NOD-like receptor signaling pathway, HIF-1 signaling pathway, and TNFα signaling pathway. For both OA and non-OA groups, GSEA and ssGSEA showed that OA was mainly associated with the enrichments in Notch signaling, adipogenesis, xenobiotic metabolism, fatty acid metabolism, peroxisome, TNFα signaling via NF-κB, the inflammatory response, PI3K AKT mTOR signaling, and IL6 JAK STAT3 signaling. This indicates that the mechanism of OA development is closely related to fatty acid metabolism, the inflammatory response, immune regulation, and cell adhesion.

We analyzed the PPI results using the cytoHubba plugin in Cytoscape, revealing seven key c-FDEGs, including IL6, IL1B, RELA, PTGS2, EGFR, CDKN2A, and SOCS1. GSEA and GSVA of the seven genes revealed that IL6, IL1B, RELA, PTGS2, SOCS1, and EGFR were closely associated with inflammation, immune regulation, extracellular matrix, and cell adhesion pathways in OA, which is consistent with previous findings [ 32 , 33 ]. Interestingly, we also found that they were closely associated with lipid metabolism and fatty acid metabolism in OA. Considering that increased iron accumulation, free radical production, fatty acid supply, and increased lipid peroxidation are key to the induction of ferroptosis [ 5 , 6 , 7 ], it is possible that they affect the development of OA by regulating lipid metabolism and fatty acid metabolism, which affects ferroptosis; however, this needs to be further investigated.

Notably, CDKN2A acts as both a cuproptosis-related gene and a ferroptosis-related gene simultaneously. CDKN2A is often considered an important gene in cellular senescence and aging [ 34 ], and it is used as a molecular marker of cellular senescence [ 35 ]. Our study showed that CDKN2A expression was higher in patients with OA, suggesting that CDKN2A may contribute to the development of OA by affecting cellular senescence and thereby promoting the development of OA.

This is the first study to use the new signature genes combining CRGs with FRGs to reveal the pathogenesis of OA and aid in its treatment. We executed three machine learning algorithms using the 40 c-FDEGs mentioned above and eventually identified four biomarkers: CDKN1A, FZD7, GABARAPL2, and SLC39A14.

Frizzled7 (FZD7) is known to be a receptor of the Wnt pathway. Fzl receptors are usually classified as belonging to the G protein receptor family and are rich in cysteine, which can directly interact with Wnt proteins and thus activate downstream responses [ 36 , 37 , 38 ]. Numerous studies have shown that excessive upregulation or downregulation of Wnt signaling pathways in OA may lead to cartilage damage and ultimately accelerate the progression of OA. Therefore, it is necessary and important to maintain a balance in the biological activity of Wnt-related pathways [ 39 , 40 , 41 ]. In the present study, FZD7 was significantly increased in the OA group compared with the non-OA group. Therefore, we speculate that an excess of FZD7 may lead to the abnormal activation of Wnt-related pathways and ultimately accelerate the development of OA.

ZIP14 (SLC39A14) is a metal transporter [ 42 ] that affects the metabolic balance of zinc, manganese, iron, copper, and other metals [ 43 ]. For example, ZIP14 can transport non-transferrin-bound iron (NTBI) [ 44 ] and ZIP14 can transport cadmium and manganese through metal/bicarbonate symbiotic activity [ 45 ]. It has been shown that OA is closely related to the metabolic balance of metals such as iron, copper, and manganese [ 14 , 15 , 31 , 46 , 47 , 48 ]. In this study, we found that ZIP14 was greatly reduced in the OA group compared with the non-OA group. Furthermore, scRNA-seq analysis showed that the distribution of SLC39A14 in OA patients varied significantly among cell populations, with low or even no expression in some cells, which is likely to disrupt the metal metabolic balance in the joints and eventually cause the accumulation of metals such as iron and copper. Therefore, SLC39A14 (ZIP14) may be a very important therapeutic target for OA treatment in the future.

ssGSEA showed that CDKN1A significantly positively correlated with TNF-α signaling via NF-κB, the TGF-β signaling pathway, hypoxia, the P53 pathway, apoptosis, mTORC1 signaling, and other gene sets, suggesting that CDKN1A may affect OA by regulating inflammation, apoptosis, and hypoxia. Although both the CDKN1A and GABARAPL2 genes have been reported previously [ 49 , 50 , 51 , 52 ], their relationship with ferroptosis and cuproptosis in OA is not yet known. This suggests that these genes may be targets not only for immunotherapy, inflammation, and autophagy but also for the treatment of cuproptosis and ferroptosis in OA. Notably, we found that melphalan, paclitaxel, vinblastine, and vantictumab may serve as potential drugs for the treatment of OA. Previous studies have reported that they act therapeutically by regulating CDKN1A or FZD7 [ 53 , 54 , 55 ], thus affecting processes such as the cell cycle, cell proliferation, and apoptosis, which also validates our prediction. We then constructed a disease model of OA based on these four biomarkers that could significantly improve our ability to recognize OA at an early stage. Thus, our findings suggest that CDKN1A, FZD7, GABARAPL2, and SLC39A14 are excellent disease biomarkers and potential therapeutic targets for OA, and the disease model constructed based on them has good diagnostic efficacy.

Recently, an increasing number of studies have shown that immune cell infiltration is essential for OA onset and development and cartilage repair [ 56 , 57 , 58 ]. Our study showed a close relationship between the seven hub genes and immune cells. Notably, there were significant positive correlations of PTGS2, IL6, and IL1B with M1 macrophages and activated mast cells. Previous studies have demonstrated that the activation of macrophages and mast cells may significantly accelerate the progression of OA [ 58 , 59 , 60 ]. Therefore, we speculate that PTGS2, IL6, and IL1B may influence the onset and progression of OA by regulating these cells. Interestingly, scRNA-seq analysis further revealed that PTGS2 was significantly highly expressed in mast cells, leading us to speculate that PTGC2 may influence the progression of OA by regulating the activation of mast cells and thus the progression of OA. Surprisingly, we found weak interactions between immune cells in the synovial tissue of patients with OA, whereas there were complex communication networks between immune and non-immune cells (fibroblasts, SMCs, and ECs). These hypotheses and questions require more studies to reveal intricate interrelationships between these c-FRGs, immune cells, and OA.

In addition, we found that C10orf91 could regulate CDKN1A and SLC39A14 by regulating hsa-miR-149-3p, hsa-miR-423-5p, hsa-miR-31-5p, and hsa-miR-30b-3p. Both hsa-miR-513a-3p and has-miR-548c-3p can regulate both CDKN1A and GABARAPL2; however, no related study has been reported yet, so this needs to be further investigated and validated in the future.

This study was conducted mainly using bioinformatics analysis, and despite the combination of scRNA-seq analysis and the use of powerful machine learning algorithms, such as RF and SVM-RFE, there are still some limitations to our study. First, the small sample size of the analysis may have led to inaccuracies in the determination of hub genes, CIBERSORT analysis, and single-cell analysis. Second, although the disease model nomogram was well validated, the data was obtained retrospectively from public databases, meaning that inherent selection bias may have affected their accuracy. In addition, while our data can show the correlation between OA and immune cells, they cannot reveal causality. Extensive prospective studies, as well as complementary in vivo and in vitro experimental studies, are necessary to validate the accuracy of potential therapeutic targets and biomarkers.

Conclusions

Our study showed that four genes—CDKN1A, FZD7, GABARAPL2, and SLC39A14—are good disease biomarkers and potential therapeutic targets for OA. Our study provides a theoretical basis and research direction for understanding the role of c-FRGs in the pathophysiological process and for potential therapeutic intervention in OA.

Availability of data and materials

The datasets used or analysed during the current study are available from the corresponding author on reasonable request.

Abbreviations

• Osteoarthritis

Nonsteroidal anti-inflammatory drugs

Reactive oxygen species

Ferroptosis-related genes

Tricarboxylic acid

Cuproptosis-related genes

The new signature genes combining cuproptosis-related genes (CRGs) with ferroptosis-related genes (FRGs)

National Center for Biotechnology Information

Gene expression omnibus

Differentially expressed genes

Differentially expressed c-FRGs

Gene Ontology

Kyoto Encyclopedia of Genes and Genomes

Gene set enrichment analysis

Support vector machine recursive feature elimination

Random forest analysis

Least absolute shrinkage and selection operator

Receiver operating characteristic

Hunter DJ, Bierma-Zeinstra S. Osteoarthritis. Lancet. 2019;393(10182):1745–59.

Article   CAS   PubMed   Google Scholar  

Hunter DJ, Schofield D, Callander E. The individual and socioeconomic impact of osteoarthritis. Nat Rev Rheumatol. 2014;10(7):437–41.

Article   PubMed   Google Scholar  

Markenson JA. ACP Journal Club. Review: glucosamine and chondroitin, alone or in combination, do not clinically improve knee or hip pain in osteoarthritis. Ann Intern Med. 2011;154(6):Jc3-4.

Martel-Pelletier J, Barr AJ, Cicuttini FM, Conaghan PG, Cooper C, Goldring MB, Goldring SR, Jones G, Teichtahl AJ, Pelletier JP. Osteoarthritis Nat Rev Dis Primers. 2016;2:16072.

Jiang X, Stockwell BR, Conrad M. Ferroptosis: mechanisms, biology and role in disease. Nat rev mol cell bio. 2021;22(4):266–82.

Article   Google Scholar  

Stockwell BR, Friedmann Angeli JP, Bayir H, Bush AI, Conrad M, Dixon SJ, Fulda S, Gascón S, Hatzios SK, Kagan VE, et al. Ferroptosis: A Regulated Cell Death Nexus Linking Metabolism, Redox Biology, and Disease. Cell. 2017;171(2):273–85.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Dixon SJ, Lemberg KM, Lamprecht MR, Skouta R, Zaitsev EM, Gleason CE, Patel DN, Bauer AJ, Cantley AM, Yang WS, et al. Ferroptosis: an iron-dependent form of nonapoptotic cell death. Cell. 2012;149(5):1060–72.

Mobasheri A, Rayman MP, Gualillo O, Sellam J, van der Kraan P, Fearon U. The role of metabolism in the pathogenesis of osteoarthritis. Nat rev rheumatol. 2017;13(5):302–11.

Robinson WH, Lepus CM, Wang Q, Raghu H, Mao R, Lindstrom TM, Sokolove J. Low-grade inflammation as a key mediator of the pathogenesis of osteoarthritis. Nat rev rheumatol. 2016;12(10):580–92.

Wang S, Li W, Zhang P, Wang Z, Ma X, Liu C, Vasilev K, Zhang L, Zhou X, Liu L, et al. Mechanical overloading induces GPX4-regulated chondrocyte ferroptosis in osteoarthritis via Piezo1 channel facilitated calcium influx. J adv res. 2022;41((null)):63–75.

Yao X, Sun K, Yu S, Luo J, Guo J, Lin J, Wang G, Guo Z, Ye Y, Guo F. Chondrocyte ferroptosis contribute to the progression of osteoarthritis. J orthop transl. 2021;27((null)):33–43.

Google Scholar  

Liu H, Deng Z, Yu B, Liu H, Yang Z, Zeng A, Fu M. Identification of SLC3A2 as a Potential Therapeutic Target of Osteoarthritis Involved in Ferroptosis by Integrating Bioinformatics, Clinical Factors and Experiments. Cells. 2022;11(21):3430.

Xia L, Gong N. Identification and verification of ferroptosis-related genes in the synovial tissue of osteoarthritis using bioinformatics analysis. Front Mol Biosci. 2022;9:992044.

Yazar M, Sarban S, Kocyigit A, Isikan UE. Synovial fluid and plasma selenium, copper, zinc, and iron concentrations in patients with rheumatoid arthritis and osteoarthritis. Biol trace elem res. 2005;106(2):123–32.

Zhou J, Liu C, Sun Y, Francis M, Ryu MS, Grider A, Ye K. Genetically predicted circulating levels of copper and zinc are associated with osteoarthritis but not with rheumatoid arthritis. Osteoarthr cartilage. 2021;29(7):1029–35.

Article   CAS   Google Scholar  

Tiku ML, Narla H, Jain M, Yalamanchili P. Glucosamine prevents in vitro collagen degradation in chondrocytes by inhibiting advanced lipoxidation reactions and protein oxidation. Arthritis res ther. 2007;9(4):R76.

Article   PubMed   PubMed Central   Google Scholar  

Tsvetkov P, Coy S, Petrova B, Dreishpoon M, Verma A, Abdusamad M, Rossen J, Joesch-Cohen L, Humeidi R, Spangler RD, et al. Copper induces cell death by targeting lipoylated TCA cycle proteins. Science. 2022;375(6586):1254–61.

Myint ZW, Oo TH, Thein KZ, Tun AM, Saeed H. Copper deficiency anemia: review article. Ann hematol. 2018;97(9):1527–34.

Jiang X, Stockwell BR, Conrad M. Ferroptosis: mechanisms, biology and role in disease. Nat Rev Mol Cell Biol. 2021;22(4):266–82.

Dixon SJ, Olzmann JA: The cell biology of ferroptosis. Nat Rev Mol Cell Biol 2024.  https://doi.org/10.1038/s41580-024-00703-5 .

Stockwell BR. Ferroptosis turns 10: Emerging mechanisms, physiological functions, and therapeutic applications. Cell. 2022;185(14):2401–21.

Freshour SL, Kiwala S, Cotto KC, Coffman AC, McMichael JF, Song JF, Griffith JJ, Griffith M, Griffith OL, Wagner AH. Integration of the Drug-Gene Interaction Database (DGIdb 4.0) with open crowdsource efforts. Nucleic Acids Res. 2021;49(D1):D1144-d1151.

Chou CH, Jain V, Gibson J, Attarian DE, Haraden CA, Yohn CB, Laberge RM, Gregory S, Kraus VB. Synovial cell cross-talk with cartilage plays a major role in the pathogenesis of osteoarthritis. Sci Rep. 2020;10(1):10868.

Huang ZY, Luo ZY, Cai YR, Chou CH, Yao ML, Pei FX, Kraus VB, Zhou ZK. Single cell transcriptomics in human osteoarthritis synovium and in silico deconvoluted bulk RNA sequencing. Osteoarthritis Cartilage. 2022;30(3):475–80.

Lv Z, Han J, Li J, Guo H, Fei Y, Sun Z, Dong J, Wang M, Fan C, Li W, et al. Single cell RNA-seq analysis identifies ferroptotic chondrocyte cluster and reveals TRPV1 as an anti-ferroptotic target in osteoarthritis. EBioMedicine. 2022;84:104258.

Miao Y, Chen Y, Xue F, Liu K, Zhu B, Gao J, Yin J, Zhang C, Li G. Contribution of ferroptosis and GPX4’s dual functions to osteoarthritis progression. EBioMedicine. 2022;76:103847.

Sun K, Guo Z, Hou L, Xu J, Du T, Xu T, Guo F. Iron homeostasis in arthropathies: From pathogenesis to therapeutic potential. Ageing res rev. 2021;72:101481.

Zhang S, Xu J, Si H, Wu Y, Zhou S, Shen B. The Role Played by Ferroptosis in Osteoarthritis: Evidence Based on Iron Dyshomeostasis and Lipid Peroxidation. Antioxidants (Basel). 2022;11(9):1668.

Zhou X, Zheng Y, Sun W, Zhang Z, Liu J, Yang W, Yuan W, Yi Y, Wang J, Liu J. D-mannose alleviates osteoarthritis progression by inhibiting chondrocyte ferroptosis in a HIF-2α-dependent manner. Cell proliferat. 2021;54(11):e13134.

Guo Z, Lin J, Sun K, Guo J, Yao X, Wang G, Hou L, Xu J, Guo J, Guo F. Deferoxamine Alleviates Osteoarthritis by Inhibiting Chondrocyte Ferroptosis and Activating the Nrf2 Pathway. Front Pharmacol. 2022;13:791376.

Lin R, Deng C, Li X, Liu Y, Zhang M, Qin C, Yao Q, Wang L, Wu C. Copper-incorporated bioactive glass-ceramics inducing anti-inflammatory phenotype and regeneration of cartilage/bone interface. Theranostics. 2019;9(21):6300–13.

Chen YH, Hsieh SC, Chen WY, Li KJ, Wu CH, Wu PC, Tsai CY, Yu CL. Spontaneous resolution of acute gouty arthritis is associated with rapid induction of the anti-inflammatory factors TGFβ1, IL-10 and soluble TNF receptors and the intracellular cytokine negative regulators CIS and SOCS3. Ann rheum dis. 2011;70(9):1655–63.

Kobayashi H, Chang SH, Mori D, Itoh S, Hirata M, Hosaka Y, Taniguchi Y, Okada K, Mori Y, Yano F, et al. Biphasic regulation of chondrocytes by Rela through induction of anti-apoptotic and catabolic target genes. Nat Commun. 2016;7:13336.

Melzer D, Pilling LC, Ferrucci L. The genetics of human ageing. Nat rev genet. 2020;21(2):88–101.

Lye JJ, Latorre E, Lee BP, Bandinelli S, Holley JE, Gutowski NJ, Ferrucci L, Harries LW. Astrocyte senescence may drive alterations in GFAPα, CDKN2A p14(ARF), and TAU3 transcript expression and contribute to cognitive decline. Geroscience. 2019;41(5):561–73.

Bhanot P, Brink M, Samos CH, Hsieh JC, Wang Y, Macke JP, Andrew D, Nathans J, Nusse R. A new member of the frizzled family from Drosophila functions as a Wingless receptor. Nature. 1996;382(6588):225–30.

Yang-Snyder J, Miller JR, Brown JD, Lai CJ, Moon RT. A frizzled homolog functions in a vertebrate Wnt signaling pathway. Curr Biol. 1996;6(10):1302–6.

Foord SM, Bonner TI, Neubig RR, Rosser EM, Pin JP, Davenport AP, Spedding M, Harmar AJ. International Union of Pharmacology. XLVI. G protein-coupled receptor list. Pharmacol Rev. 2005;57(2):279–88.

Nalesso G, Thomas BL, Sherwood JC, Yu J, Addimanda O, Eldridge SE, Thorup AS, Dale L, Schett G, Zwerina J, et al. WNT16 antagonises excessive canonical WNT activation and protects cartilage in osteoarthritis. Ann Rheum Dis. 2017;76(1):218–26.

Tong W, Zeng Y, Chow DHK, Yeung W, Xu J, Deng Y, Chen S, Zhao H, Zhang X, Ho KK, et al. Wnt16 attenuates osteoarthritis progression through a PCP/JNK-mTORC1-PTHrP cascade. Ann Rheum Dis. 2019;78(4):551–61.

Lories RJ, Corr M, Lane NE. To Wnt or not to Wnt: the bone and joint health dilemma. Nat Rev Rheumatol. 2013;9(6):328–39.

Aydemir TB, Cousins RJ. The Multiple Faces of the Metal Transporter ZIP14 (SLC39A14). J nutr. 2018;148(2):174–84.

Pinilla-Tenas JJ, Sparkman BK, Shawki A, Illing AC, Mitchell CJ, Zhao N, Liuzzi JP, Cousins RJ, Knutson MD, Mackenzie B. Zip14 is a complex broad-scope metal-ion transporter whose functional properties support roles in the cellular uptake of zinc and nontransferrin-bound iron. Am J Physiol Cell Physiol. 2011;301(4):C862-871.

Liuzzi JP, Aydemir F, Nam H, Knutson MD, Cousins RJ. Zip14 (Slc39a14) mediates non-transferrin-bound iron uptake into cells. Proc Natl Acad Sci U S A. 2006;103(37):13612–7.

Girijashanker K, He L, Soleimani M, Reed JM, Li H, Liu Z, Wang B, Dalton TP, Nebert DW. Slc39a14 gene encodes ZIP14, a metal/bicarbonate symporter: similarities to the ZIP8 transporter. Mol Pharmacol. 2008;73(5):1413–23.

He H, Wang Y, Yang Z, Ding X, Yang T, Lei G, Li H, Xie D. Association between serum zinc and copper concentrations and copper/zinc ratio with the prevalence of knee chondrocalcinosis: a cross-sectional study. BMC Musculoskelet Disord. 2020;21(1):97.

Li G, Cheng T, Yu X. The Impact of Trace Elements on Osteoarthritis. Front Med (Lausanne). 2021;8:771297.

Roczniak W, Brodziak-Dopierała B, Cipora E, Jakóbik-Kolon A, Kluczka J, Babuśka-Roczniak M. Factors that Affect the Content of Cadmium, Nickel, Copper and Zinc in Tissues of the Knee Joint. Biol trace elem res. 2017;178(2):201–9.

Bertram KL, Narendran N, Tailor P, Jablonski C, Leonard C, Irvine E, Hess R, Masson AO, Abubacker S, Rinker K, et al. 17-DMAG regulates p21 expression Irvine  to induce chondrogenesis in vitro and in vivo. Dis Model Mech. 2018;11((10):dmm033662.

Ding Z, Lu W, Dai C, Huang W, Liu F, Shan W, Cheng C, Xu J, Yin Z, He W. The CRD of Frizzled 7 exhibits chondroprotective effects in osteoarthritis via inhibition of the canonical Wnt3a/β-catenin signaling pathway. Int immunopharmacol. 2020;82:106367.

Faust HJ, Zhang H, Han J, Wolf MT, Jeon OH, Sadtler K, Peña AN, Chung L, Maestas DR, Tam AJ, et al. IL-17 and immunologically induced senescence regulate response to injury in osteoarthritis. J clin invest. 2020;130(10):5493–507.

Xing D, Wang B, Xu Y, Tao K, Lin J. Overexpression of microRNA-1 controls the development of osteoarthritis via targeting FZD7 of Wnt/β-catenin signaling. Osteoarthr cartilage. 2016;24:S181-s182.

Flanagan DJ, Barker N, Costanzo NSD, Mason EA, Gurney A, Meniel VS, Koushyar S, Austin CR, Ernst M, Pearson HB, et al. Frizzled-7 Is Required for Wnt Signaling in Gastric Tumors with and Without Apc Mutations. Cancer Res. 2019;79(5):970–81.

Cho HJ, Mei AHC, Tung K, Han J, Perumal D, Keats JJ, Auclair D, Chari A, Jagannath S, Parekh S. MAGE-A3 Promotes Chemotherapy Resistance and Proliferation in Multiple Myeloma through Regulation of BIM and p21Cip1. Blood. 2018;132:4464.

Wang Z, Li Y, Wu D, Yu S, Wang Y, Leung Chan F. Nuclear receptor HNF4α performs a tumor suppressor function in prostate cancer via its induction of p21-driven cellular senescence. Oncogene. 2020;39(7):1572–89.

Sakkas LI, Platsoucas CD. The role of T cells in the pathogenesis of osteoarthritis. Arthritis Rheum. 2007;56(2):409–24.

Li M, Yin H, Yan Z, Li H, Wu J, Wang Y, Wei F, Tian G, Ning C, Li H, et al. The immune microenvironment in cartilage injury and repair. Acta biomater. 2022;140:23–42.

Wang Q, Lepus CM, Raghu H, Reber LL, Tsai MM, Wong HH, von Kaeppler E, Lingampalli N, Bloom MS, Hu N, et al. IgE-mediated mast cell activation promotes inflammation and cartilage destruction in osteoarthritis. Elife. 2019;8:e39905.

Liu B, Zhang M, Zhao J, Zheng M, Yang H. Imbalance of M1/M2 macrophages is linked to severity level of knee osteoarthritis. Exp Ther Med. 2018;16(6):5009–14.

CAS   PubMed   PubMed Central   Google Scholar  

Lee H, Kashiwakura J, Matsuda A, Watanabe Y, Sakamoto-Sasaki T, Matsumoto K, Hashimoto N, Saito S, Ohmori K, Nagaoka M, et al. Activation of human synovial mast cells from rheumatoid arthritis or osteoarthritis patients in response to aggregated IgG through Fcγ receptor I and Fcγ receptor II. Arthritis rheum-us. 2013;65(1):109–19.

Download references

Acknowledgments

This study was a re-analysis based on published data from the GEO database. We would like to thank the GEO database for sharing the data.

This study was supported by Sichuan Medical Association (No. S17075, Q22008, Q21005), the Sichuan Science and Technology Program(No. 24NSFSC2177), the Science and Technology Strategic Cooperation Project between the People's Government of Luzhou City and Southwest Medical University (No. 2020LZXNYDJ22), the Doctoral Research Initiation Fund of Affiliated Hospital of Southwest Medical University (No. 22155), and Sichuan Student Innovation and Entrepreneurship Training Program Project (No. S202010632174).

Author information

Baoqiang He and Yehui Liao are contributed equally.

Authors and Affiliations

Department of Orthopedics, The Affiliated Hospital of Southwest Medical University, No. 25 Taping Street, Lu Zhou City, China

Baoqiang He, Yehui Liao, Minghao Tian, Chao Tang, Qiang Tang, Fei Ma, Wenyang Zhou, Yebo Leng & Dejun Zhong

Southwest Medical University, Lu Zhou City, China

Baoqiang He & Dejun Zhong

Meishan Tianfu New Area People’s Hospital, Meishan City, China

You can also search for this author in PubMed   Google Scholar

Contributions

HBQ, LYB, LYH and ZDJ designed the study. Data analysis was performed by HBQ, TC, TQ and MF. HBQ, TMH and ZWY carried out the experiments. HBQ, LYB, and ZDJ wrote the first draft. ZDJ critically revised the manuscript. All authors read and approved the final manuscript.

Corresponding authors

Correspondence to Yebo Leng or Dejun Zhong .

Ethics declarations

Ethics approval and consent to participate.

Synovial tissue collection and all experimental procedures were approved by the Institutional Review Board of the Affiliated Hospital of Southwest Medical University (KY2023293) in accordance with the guidelines of the Chinese Health Sciences Administration, and written informed consent was obtained from the donors.

Consent for publication

All authors agree to publish.

Competing interests

The authors declare no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary material 1., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

He, B., Liao, Y., Tian, M. et al. Identification and verification of a novel signature that combines cuproptosis-related genes with ferroptosis-related genes in osteoarthritis using bioinformatics analysis and experimental validation. Arthritis Res Ther 26 , 100 (2024). https://doi.org/10.1186/s13075-024-03328-3

Download citation

Received : 22 January 2024

Accepted : 23 April 2024

Published : 13 May 2024

DOI : https://doi.org/10.1186/s13075-024-03328-3

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Cuproptosis
• Ferroptosis
• Machine learning
• Bioinformatics

Arthritis Research & Therapy

ISSN: 1478-6362

• General enquiries: [email protected]

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Indian J Anaesth
• v.60(9); 2016 Sep

Basic statistical tools in research and data analysis

Zulfiqar ali.

Department of Anaesthesiology, Division of Neuroanaesthesiology, Sheri Kashmir Institute of Medical Sciences, Soura, Srinagar, Jammu and Kashmir, India

S Bala Bhaskar

1 Department of Anaesthesiology and Critical Care, Vijayanagar Institute of Medical Sciences, Bellary, Karnataka, India

Statistical methods involved in carrying out a study include planning, designing, collecting data, analysing, drawing meaningful interpretation and reporting of the research findings. The statistical analysis gives meaning to the meaningless numbers, thereby breathing life into a lifeless data. The results and inferences are precise only if proper statistical tests are used. This article will try to acquaint the reader with the basic research tools that are utilised while conducting various studies. The article covers a brief outline of the variables, an understanding of quantitative and qualitative variables and the measures of central tendency. An idea of the sample size estimation, power analysis and the statistical errors is given. Finally, there is a summary of parametric and non-parametric tests used for data analysis.

INTRODUCTION

Statistics is a branch of science that deals with the collection, organisation, analysis of data and drawing of inferences from the samples to the whole population.[ 1 ] This requires a proper design of the study, an appropriate selection of the study sample and choice of a suitable statistical test. An adequate knowledge of statistics is necessary for proper designing of an epidemiological study or a clinical trial. Improper statistical methods may result in erroneous conclusions which may lead to unethical practice.[ 2 ]

Variable is a characteristic that varies from one individual member of population to another individual.[ 3 ] Variables such as height and weight are measured by some type of scale, convey quantitative information and are called as quantitative variables. Sex and eye colour give qualitative information and are called as qualitative variables[ 3 ] [ Figure 1 ].

Classification of variables

Quantitative variables

Quantitative or numerical data are subdivided into discrete and continuous measurements. Discrete numerical data are recorded as a whole number such as 0, 1, 2, 3,… (integer), whereas continuous data can assume any value. Observations that can be counted constitute the discrete data and observations that can be measured constitute the continuous data. Examples of discrete data are number of episodes of respiratory arrests or the number of re-intubations in an intensive care unit. Similarly, examples of continuous data are the serial serum glucose levels, partial pressure of oxygen in arterial blood and the oesophageal temperature.

A hierarchical scale of increasing precision can be used for observing and recording the data which is based on categorical, ordinal, interval and ratio scales [ Figure 1 ].

Categorical or nominal variables are unordered. The data are merely classified into categories and cannot be arranged in any particular order. If only two categories exist (as in gender male and female), it is called as a dichotomous (or binary) data. The various causes of re-intubation in an intensive care unit due to upper airway obstruction, impaired clearance of secretions, hypoxemia, hypercapnia, pulmonary oedema and neurological impairment are examples of categorical variables.

Ordinal variables have a clear ordering between the variables. However, the ordered data may not have equal intervals. Examples are the American Society of Anesthesiologists status or Richmond agitation-sedation scale.

Interval variables are similar to an ordinal variable, except that the intervals between the values of the interval variable are equally spaced. A good example of an interval scale is the Fahrenheit degree scale used to measure temperature. With the Fahrenheit scale, the difference between 70° and 75° is equal to the difference between 80° and 85°: The units of measurement are equal throughout the full range of the scale.

Ratio scales are similar to interval scales, in that equal differences between scale values have equal quantitative meaning. However, ratio scales also have a true zero point, which gives them an additional property. For example, the system of centimetres is an example of a ratio scale. There is a true zero point and the value of 0 cm means a complete absence of length. The thyromental distance of 6 cm in an adult may be twice that of a child in whom it may be 3 cm.

STATISTICS: DESCRIPTIVE AND INFERENTIAL STATISTICS

Descriptive statistics[ 4 ] try to describe the relationship between variables in a sample or population. Descriptive statistics provide a summary of data in the form of mean, median and mode. Inferential statistics[ 4 ] use a random sample of data taken from a population to describe and make inferences about the whole population. It is valuable when it is not possible to examine each member of an entire population. The examples if descriptive and inferential statistics are illustrated in Table 1 .

Example of descriptive and inferential statistics

Descriptive statistics

The extent to which the observations cluster around a central location is described by the central tendency and the spread towards the extremes is described by the degree of dispersion.

Measures of central tendency

The measures of central tendency are mean, median and mode.[ 6 ] Mean (or the arithmetic average) is the sum of all the scores divided by the number of scores. Mean may be influenced profoundly by the extreme variables. For example, the average stay of organophosphorus poisoning patients in ICU may be influenced by a single patient who stays in ICU for around 5 months because of septicaemia. The extreme values are called outliers. The formula for the mean is

where x = each observation and n = number of observations. Median[ 6 ] is defined as the middle of a distribution in a ranked data (with half of the variables in the sample above and half below the median value) while mode is the most frequently occurring variable in a distribution. Range defines the spread, or variability, of a sample.[ 7 ] It is described by the minimum and maximum values of the variables. If we rank the data and after ranking, group the observations into percentiles, we can get better information of the pattern of spread of the variables. In percentiles, we rank the observations into 100 equal parts. We can then describe 25%, 50%, 75% or any other percentile amount. The median is the 50 th percentile. The interquartile range will be the observations in the middle 50% of the observations about the median (25 th -75 th percentile). Variance[ 7 ] is a measure of how spread out is the distribution. It gives an indication of how close an individual observation clusters about the mean value. The variance of a population is defined by the following formula:

where σ 2 is the population variance, X is the population mean, X i is the i th element from the population and N is the number of elements in the population. The variance of a sample is defined by slightly different formula:

where s 2 is the sample variance, x is the sample mean, x i is the i th element from the sample and n is the number of elements in the sample. The formula for the variance of a population has the value ‘ n ’ as the denominator. The expression ‘ n −1’ is known as the degrees of freedom and is one less than the number of parameters. Each observation is free to vary, except the last one which must be a defined value. The variance is measured in squared units. To make the interpretation of the data simple and to retain the basic unit of observation, the square root of variance is used. The square root of the variance is the standard deviation (SD).[ 8 ] The SD of a population is defined by the following formula:

where σ is the population SD, X is the population mean, X i is the i th element from the population and N is the number of elements in the population. The SD of a sample is defined by slightly different formula:

where s is the sample SD, x is the sample mean, x i is the i th element from the sample and n is the number of elements in the sample. An example for calculation of variation and SD is illustrated in Table 2 .

Example of mean, variance, standard deviation

Normal distribution or Gaussian distribution

Most of the biological variables usually cluster around a central value, with symmetrical positive and negative deviations about this point.[ 1 ] The standard normal distribution curve is a symmetrical bell-shaped. In a normal distribution curve, about 68% of the scores are within 1 SD of the mean. Around 95% of the scores are within 2 SDs of the mean and 99% within 3 SDs of the mean [ Figure 2 ].

Normal distribution curve

Skewed distribution

It is a distribution with an asymmetry of the variables about its mean. In a negatively skewed distribution [ Figure 3 ], the mass of the distribution is concentrated on the right of Figure 1 . In a positively skewed distribution [ Figure 3 ], the mass of the distribution is concentrated on the left of the figure leading to a longer right tail.

Curves showing negatively skewed and positively skewed distribution

Inferential statistics

In inferential statistics, data are analysed from a sample to make inferences in the larger collection of the population. The purpose is to answer or test the hypotheses. A hypothesis (plural hypotheses) is a proposed explanation for a phenomenon. Hypothesis tests are thus procedures for making rational decisions about the reality of observed effects.

Probability is the measure of the likelihood that an event will occur. Probability is quantified as a number between 0 and 1 (where 0 indicates impossibility and 1 indicates certainty).

In inferential statistics, the term ‘null hypothesis’ ( H 0 ‘ H-naught ,’ ‘ H-null ’) denotes that there is no relationship (difference) between the population variables in question.[ 9 ]

Alternative hypothesis ( H 1 and H a ) denotes that a statement between the variables is expected to be true.[ 9 ]

The P value (or the calculated probability) is the probability of the event occurring by chance if the null hypothesis is true. The P value is a numerical between 0 and 1 and is interpreted by researchers in deciding whether to reject or retain the null hypothesis [ Table 3 ].

P values with interpretation

If P value is less than the arbitrarily chosen value (known as α or the significance level), the null hypothesis (H0) is rejected [ Table 4 ]. However, if null hypotheses (H0) is incorrectly rejected, this is known as a Type I error.[ 11 ] Further details regarding alpha error, beta error and sample size calculation and factors influencing them are dealt with in another section of this issue by Das S et al .[ 12 ]

Illustration for null hypothesis

PARAMETRIC AND NON-PARAMETRIC TESTS

Numerical data (quantitative variables) that are normally distributed are analysed with parametric tests.[ 13 ]

Two most basic prerequisites for parametric statistical analysis are:

• The assumption of normality which specifies that the means of the sample group are normally distributed
• The assumption of equal variance which specifies that the variances of the samples and of their corresponding population are equal.

However, if the distribution of the sample is skewed towards one side or the distribution is unknown due to the small sample size, non-parametric[ 14 ] statistical techniques are used. Non-parametric tests are used to analyse ordinal and categorical data.

Parametric tests

The parametric tests assume that the data are on a quantitative (numerical) scale, with a normal distribution of the underlying population. The samples have the same variance (homogeneity of variances). The samples are randomly drawn from the population, and the observations within a group are independent of each other. The commonly used parametric tests are the Student's t -test, analysis of variance (ANOVA) and repeated measures ANOVA.

Student's t -test

Student's t -test is used to test the null hypothesis that there is no difference between the means of the two groups. It is used in three circumstances:

where X = sample mean, u = population mean and SE = standard error of mean

where X 1 − X 2 is the difference between the means of the two groups and SE denotes the standard error of the difference.

• To test if the population means estimated by two dependent samples differ significantly (the paired t -test). A usual setting for paired t -test is when measurements are made on the same subjects before and after a treatment.

The formula for paired t -test is:

where d is the mean difference and SE denotes the standard error of this difference.

The group variances can be compared using the F -test. The F -test is the ratio of variances (var l/var 2). If F differs significantly from 1.0, then it is concluded that the group variances differ significantly.

Analysis of variance

The Student's t -test cannot be used for comparison of three or more groups. The purpose of ANOVA is to test if there is any significant difference between the means of two or more groups.

In ANOVA, we study two variances – (a) between-group variability and (b) within-group variability. The within-group variability (error variance) is the variation that cannot be accounted for in the study design. It is based on random differences present in our samples.

However, the between-group (or effect variance) is the result of our treatment. These two estimates of variances are compared using the F-test.

A simplified formula for the F statistic is:

where MS b is the mean squares between the groups and MS w is the mean squares within groups.

Repeated measures analysis of variance

As with ANOVA, repeated measures ANOVA analyses the equality of means of three or more groups. However, a repeated measure ANOVA is used when all variables of a sample are measured under different conditions or at different points in time.

As the variables are measured from a sample at different points of time, the measurement of the dependent variable is repeated. Using a standard ANOVA in this case is not appropriate because it fails to model the correlation between the repeated measures: The data violate the ANOVA assumption of independence. Hence, in the measurement of repeated dependent variables, repeated measures ANOVA should be used.

Non-parametric tests

When the assumptions of normality are not met, and the sample means are not normally, distributed parametric tests can lead to erroneous results. Non-parametric tests (distribution-free test) are used in such situation as they do not require the normality assumption.[ 15 ] Non-parametric tests may fail to detect a significant difference when compared with a parametric test. That is, they usually have less power.

As is done for the parametric tests, the test statistic is compared with known values for the sampling distribution of that statistic and the null hypothesis is accepted or rejected. The types of non-parametric analysis techniques and the corresponding parametric analysis techniques are delineated in Table 5 .

Analogue of parametric and non-parametric tests

Median test for one sample: The sign test and Wilcoxon's signed rank test

The sign test and Wilcoxon's signed rank test are used for median tests of one sample. These tests examine whether one instance of sample data is greater or smaller than the median reference value.

This test examines the hypothesis about the median θ0 of a population. It tests the null hypothesis H0 = θ0. When the observed value (Xi) is greater than the reference value (θ0), it is marked as+. If the observed value is smaller than the reference value, it is marked as − sign. If the observed value is equal to the reference value (θ0), it is eliminated from the sample.

If the null hypothesis is true, there will be an equal number of + signs and − signs.

The sign test ignores the actual values of the data and only uses + or − signs. Therefore, it is useful when it is difficult to measure the values.

Wilcoxon's signed rank test

There is a major limitation of sign test as we lose the quantitative information of the given data and merely use the + or – signs. Wilcoxon's signed rank test not only examines the observed values in comparison with θ0 but also takes into consideration the relative sizes, adding more statistical power to the test. As in the sign test, if there is an observed value that is equal to the reference value θ0, this observed value is eliminated from the sample.

Wilcoxon's rank sum test ranks all data points in order, calculates the rank sum of each sample and compares the difference in the rank sums.

Mann-Whitney test

It is used to test the null hypothesis that two samples have the same median or, alternatively, whether observations in one sample tend to be larger than observations in the other.

Mann–Whitney test compares all data (xi) belonging to the X group and all data (yi) belonging to the Y group and calculates the probability of xi being greater than yi: P (xi > yi). The null hypothesis states that P (xi > yi) = P (xi < yi) =1/2 while the alternative hypothesis states that P (xi > yi) ≠1/2.

Kolmogorov-Smirnov test

The two-sample Kolmogorov-Smirnov (KS) test was designed as a generic method to test whether two random samples are drawn from the same distribution. The null hypothesis of the KS test is that both distributions are identical. The statistic of the KS test is a distance between the two empirical distributions, computed as the maximum absolute difference between their cumulative curves.

Kruskal-Wallis test

The Kruskal–Wallis test is a non-parametric test to analyse the variance.[ 14 ] It analyses if there is any difference in the median values of three or more independent samples. The data values are ranked in an increasing order, and the rank sums calculated followed by calculation of the test statistic.

Jonckheere test

In contrast to Kruskal–Wallis test, in Jonckheere test, there is an a priori ordering that gives it a more statistical power than the Kruskal–Wallis test.[ 14 ]

Friedman test

The Friedman test is a non-parametric test for testing the difference between several related samples. The Friedman test is an alternative for repeated measures ANOVAs which is used when the same parameter has been measured under different conditions on the same subjects.[ 13 ]

Tests to analyse the categorical data

Chi-square test, Fischer's exact test and McNemar's test are used to analyse the categorical or nominal variables. The Chi-square test compares the frequencies and tests whether the observed data differ significantly from that of the expected data if there were no differences between groups (i.e., the null hypothesis). It is calculated by the sum of the squared difference between observed ( O ) and the expected ( E ) data (or the deviation, d ) divided by the expected data by the following formula:

A Yates correction factor is used when the sample size is small. Fischer's exact test is used to determine if there are non-random associations between two categorical variables. It does not assume random sampling, and instead of referring a calculated statistic to a sampling distribution, it calculates an exact probability. McNemar's test is used for paired nominal data. It is applied to 2 × 2 table with paired-dependent samples. It is used to determine whether the row and column frequencies are equal (that is, whether there is ‘marginal homogeneity’). The null hypothesis is that the paired proportions are equal. The Mantel-Haenszel Chi-square test is a multivariate test as it analyses multiple grouping variables. It stratifies according to the nominated confounding variables and identifies any that affects the primary outcome variable. If the outcome variable is dichotomous, then logistic regression is used.

SOFTWARES AVAILABLE FOR STATISTICS, SAMPLE SIZE CALCULATION AND POWER ANALYSIS

Numerous statistical software systems are available currently. The commonly used software systems are Statistical Package for the Social Sciences (SPSS – manufactured by IBM corporation), Statistical Analysis System ((SAS – developed by SAS Institute North Carolina, United States of America), R (designed by Ross Ihaka and Robert Gentleman from R core team), Minitab (developed by Minitab Inc), Stata (developed by StataCorp) and the MS Excel (developed by Microsoft).

There are a number of web resources which are related to statistical power analyses. A few are:

• StatPages.net – provides links to a number of online power calculators
• G-Power – provides a downloadable power analysis program that runs under DOS
• Power analysis for ANOVA designs an interactive site that calculates power or sample size needed to attain a given power for one effect in a factorial ANOVA design
• SPSS makes a program called SamplePower. It gives an output of a complete report on the computer screen which can be cut and paste into another document.

It is important that a researcher knows the concepts of the basic statistical methods used for conduct of a research study. This will help to conduct an appropriately well-designed study leading to valid and reliable results. Inappropriate use of statistical techniques may lead to faulty conclusions, inducing errors and undermining the significance of the article. Bad statistics may lead to bad research, and bad research may lead to unethical practice. Hence, an adequate knowledge of statistics and the appropriate use of statistical tests are important. An appropriate knowledge about the basic statistical methods will go a long way in improving the research designs and producing quality medical research which can be utilised for formulating the evidence-based guidelines.

Financial support and sponsorship

Conflicts of interest.

There are no conflicts of interest.

• Open access
• Published: 17 May 2024

Ferroptosis is a protective factor for the prognosis of cancer patients: a systematic review and meta-analysis

• Shen Li 1   na1 ,
• Kai Tao 2   na1 ,
• Hong Yun 1   na1 ,
• Jiaqing Yang 1 , 2 ,
• Yuanling Meng 3 ,
• Fan Zhang 4 &
• Xuelei Ma 1  

BMC Cancer volume  24 , Article number:  604 ( 2024 ) Cite this article

115 Accesses

1 Altmetric

Metrics details

Cancer is a leading global cause of death. Conventional cancer treatments like surgery, radiation, and chemotherapy have associated side effects. Ferroptosis, a nonapoptotic and iron-dependent cell death, has been identified and differs from other cell death types. Research has shown that ferroptosis can promote and inhibit tumor growth, which may have prognostic value. Given the unclear role of ferroptosis in cancer biology, this meta-analysis aims to investigate its impact on cancer prognosis.

This systematic review and meta-analysis conducted searches on PubMed, Embase, and the Cochrane Library databases. Eight retrospective studies were included to compare the impact of ferroptosis inhibition and promotion on cancer patient prognosis. The primary endpoints were overall survival (OS) and progression-free survival (PFS). Studies lacking clear descriptions of hazard ratios (HR) and 95% confidence intervals for OS and PFS were excluded. Random-effects meta-analysis and meta-regression were performed on the included study data to assess prognosis differences between the experimental and control groups. Meta-analysis results included HR and 95% confidence intervals.

This study has been registered with PROSPERO, CRD 42023463720 on September 27, 2023.

A total of 2,446 articles were screened, resulting in the inclusion of 5 articles with 938 eligible subjects. Eight studies were included in the meta-analysis after bias exclusion. The meta-analysis, after bias exclusion, demonstrated that promoting ferroptosis could increase cancer patients’ overall survival (HR 0.31, 95% CI 0.21–0.44) and progression-free survival (HR 0.26, 95% CI 0.16–0.44) compared to ferroptosis inhibition. The results showed moderate heterogeneity, suggesting that biological activities promoting cancer cell ferroptosis are beneficial for cancer patient’s prognosis.

Conclusions

This systematic review and meta-analysis demonstrated that the promotion of ferroptosis yields substantial benefits for cancer prognosis. These findings underscore the untapped potential of ferroptosis as an innovative anti-tumor therapeutic strategy, capable of addressing challenges related to drug resistance, limited therapeutic efficacy, and unfavorable prognosis in cancer treatment.

Registration

CRD42023463720.

Peer Review reports

Cancer has progressively become the world’s leading cause of mortality, imposing substantial disease burdens. According to GLOBOCAN 2020, the global cancer burden will reach 28.4 million cases in 2040 [ 1 ]. And approximately one in every five men and one in every six women will develop cancer, with one in eight men and one in ten women succumbing to cancer before reaching 75 years of age [ 2 ]. It is estimated that over half of all cancer-related deaths (57.3%) and nearly half of all new cancer cases (48.4%) are concentrated in Asia [ 2 ]. Presently, common treatments for cancer encompass surgery, radiation, and chemotherapy [ 3 , 4 ]. However, these approaches may harm normal cells and result in significant side effects, including hepatotoxicity, ototoxicity, cardiotoxicity, nausea, vomiting, and more [ 5 , 6 ]. Despite advancements in therapy, cancer remains the second leading global cause of death, following ischemic heart disease, and is projected to become the leading cause by 2060 [ 7 ].

In 2012, a nonapoptotic, iron-dependent form of cell death initiated by the oncogenic Ras-selective lethal small molecule erastin was termed “ferroptosis” [ 8 ]. Ferroptosis exhibits distinct morphological characteristics compared to other regulated cell death forms. Notably, ferroptosis lacks the hallmark signs of apoptosis, such as chromatin condensation and apoptotic bodies, instead manifesting as shrunken mitochondria, reduced mitochondrial cristae, and an accumulation of lipid peroxides [ 8 , 9 , 10 ]. Its underlying mechanism also differs from other regulated cell death processes. Ferroptosis is inhibited by the system xc-—GSH—GPX4 pathway and is induced by the accumulation of phospholipid hydroperoxides, rather than the involvement of cell death executioner proteins such as caspases and mixed lineage kinase domain-like protein, among others [ 9 , 11 ].

An increasing body of research has explored the role of ferroptosis in tumors, suggesting its dual role in tumor promotion and inhibition. Various experimental agents, including erastin, RSL3, and drugs such as sorafenib, sulfasalazine, statins, and artemisinin, along with ionizing radiation and cytokines like IFN-γ and TGF-β1, can induce ferroptosis and inhibit tumors [ 12 ]. However, emerging evidence hints at ferroptosis potentially promoting tumor growth by triggering inflammation-associated immunosuppression within the tumor microenvironment [ 12 , 13 ]. Numerous studies have also indicated the prognostic value of ferroptosis [ 14 , 15 , 16 , 17 , 18 ].

Given the unclear role of ferroptosis in cancer biology, we conducted this meta-analysis to investigate its impact on cancer prognosis.

Search strategy and selection criteria

This systematic review and meta-analysis were conducted following PRISMA guidelines. PubMed, EMBASE, and the Cochrane Library were systematically searched from their inception until February 27, 2024, with no language restrictions. The search strategy included the following terms: (ferroptosis or oxytosis) AND (Neoplasm or Tumor or Tumors or Neoplasia or Cancer or Cancers or Malignant Neoplasm or Malignancy or Malignant Neoplasms or Neoplasms, Malignant or Benign Neoplasms or Neoplasm, Benign or Malignancies or Neoplasm, Malignant or Benign Neoplasm or Neoplasms, Benign or Neoplasias) AND (prognosis or Prognoses or Prognostic Factors or Prognostic Factor or Factor, Prognostic or Factors, Prognostic) as free text.

The objective of this study is to investigate and elucidate the impact of ferroptosis on cancer patients’ prognosis. We will compare the differences in prognosis between cancer patients with genes that promote ferroptosis and those with genes that inhibit it. The primary endpoints of the study include HRs and 95% confidence intervals for OS and PFS. It is important to note that the upregulation and downregulation of ferroptosis-related genes are not used as criteria for grouping; rather, the experimental and control groups are divided based on the ultimate impact of genes on ferroptosis. This meta-analysis was limited to studies conducted in humans. Participant data from cohort studies were extracted and analyzed. The collected information included the first author, study period, country of study, study size, ferroptosis-related gene, the effect of genes on ferroptosis, type of cancer, HR, and 95% confidence intervals for OS and PFS.

Both exclusion and inclusion criteria were pre-specified. Studies demonstrating a relationship between prognosis and ferroptosis in cancer patients were selected. Inclusion criteria were as follows: (1) Articles were limited to those involving human samples only. (2) All cancer patients had been diagnosed by pathological evidence. (3) Expression of ferroptosis-related genes had been assessed through immunohistochemistry from tumor specimens, conducted according to standard protocols. (4) All patients had been subject to follow-up, and results had been reported. Exclusion criteria encompassed: (1) Duplicate articles. (2) Article types other than original research, such as reviews, meta-analyses, letters, or editorial comments. (3) Studies involving cellular or animal-based research. (4) Patients with multiple primary cancers. The literature search, study selection, and data extraction were independently performed by Shen Li and Kai Tao, with any discrepancies reviewed and resolved by another author, Xuelei Ma, through consensus.

Data analysis

We employed Stata 14 software to calculate statistics. The specific analysis method is as follows: (1) We collected and analyzed the HR for OS and PFS reported in the included studies. The results were visualized using forest plots to illustrate the differences in prognosis between cancer patients whose genes promote ferroptosis and those whose genes inhibit it, thereby demonstrating the impact of ferroptosis on the prognosis of cancer patients. (2) Heterogeneity test was conducted by I 2 statistic to assess the heterogeneity of the results. Low heterogeneity was defined as an I 2 value less than or equal to 25%, moderate heterogeneity as between 25 and 75%, and high heterogeneity as exceeding 75%. (3) To evaluate potential publication bias, we employed funnel plots and conducted Egger tests. A p -value greater than 0.05 in Egger test indicates no significant bias. (4) Sensitivity analysis was conducted to examine any studies with significant influence on the overall results. (5) Meta-regression was conducted to assess the potential influence of covariates on the outcome [ 19 , 20 ]. We subjected the included covariates to regression testing, including country of study, ferroptosis-related gene, the effect of genes on ferroptosis, and type of cancer, to explore possible sources of heterogeneity and reduce potential bias. This study has been registered with PROSPERO, CRD 42023463720.

Bias analysis and quality assessment

Three researchers (LS, YJQ and TK) independently conducted a bias risk assessment following the Cochrane Bias Assessment Handbook. Considering that all included studies were retrospective articles, this study employed the Cochrane bias risk tool, which comprises five domains, to evaluate the risk of bias in each included study: (1) selection bias, (2) measurement bias, (3) data integrity bias, (4) outcome selection bias, and (5) other biases. Each researcher independently assessed the risk as low, high, or unclear for each domain. In cases of any uncertainty, Dr. Xuelei Ma made the final judgment. Based on the risk of bias, the quality of evidence was categorized as very low, low, moderate, or high. The quality assessment of this study adheres to the GRADE system.

We identified a total of 2,446 articles through literature searches, with 6 articles from the Cochrane Library and 2,440 from other databases, including PubMed and Embase. We excluded 962 duplicate articles. Among the remaining literature, we excluded 1,477 articles after abstract screening as they did not align with our research objectives. Subsequently, we conducted full-text reviews and eligibility assessments on the remaining 7 articles. Ultimately, we included 5 articles in our analysis. The review process was conducted independently by LS, TK and MYL, with a third reviewer, Xuelei Ma, reassessing articles with uncertain eligibility. The process is illustrated in Fig.  1 .

Study selection

Among the five clinical articles, all studies were conducted in Asia, with 2 studies in China (40%) and 3 in Japan (60%). The research covered various cancer types, including gastric cancer and esophageal cancer of the digestive system, epithelial ovarian cancer of the female reproductive system, and osteosarcoma originating from undifferentiated bone fibrous tissue. In terms of age reporting, the median age of patients with epithelial ovarian cancer was 52 years, while osteosarcoma patients had an average age of 30.2 years, which is consistent with the characteristics of these two diseases. Three out of the five articles included two studies each, resulting in a total of 8 studies. Glutathione peroxidase 4 (GPX4) was the most studied gene (4/8, 50%) related to regulating ferroptosis. Like most other genes, GPX4 plays a role in inhibiting ferroptosis by suppressing lipid peroxidation. In contrast, heme oxygenase 1 (HMOX1), through catalyzing the degradation of heme into divalent iron ions, biliverdin, and CO, can promote ferroptosis by increasing the labile iron pool (LIP) (1/8, 12.5%). It’s worth noting that, as shown in Table  1 , only 3 studies (3/8, 37.5%) reported cut-off values, while the rest did not report them. We will discuss the importance of this missing data in the Discussion section.

Main outcome

A total of 8 studies reported HRs and 95% confidence intervals for OS. The forest plot indicates that the ferroptosis-promoting group had better OS compared to the ferroptosis-inhibiting group (HR 0.43, 95% CI 0.22–0.83). Data analysis reports substantial heterogeneity (I 2  = 87.8%, 95% CI 45.6%-94.7%) (Fig.  2 ). After conducting sensitivity analysis, we found that the study by Song et al. might introduce significant bias. After excluding this study and reanalyzing the data, the results showed that the ferroptosis-promoting group had better OS compared to the ferroptosis-inhibiting group (HR 0.31, 95% CI 0.21–0.44), with decreased heterogeneity (I 2  = 58.1%, 95% CI 0%-82.7%), indicating moderate heterogeneity (Fig.  3 ).

Forest plot of the pooled overall survival between the ferroptosis-promoting group and the ferroptosis-inhibiting group

Forest plot of the pooled overall survival between the ferroptosis-promoting group and the ferroptosis-inhibiting group after excluding one study with a large bias

Six studies reported HRs and 95% CIs for PFS. The analysis results suggest that the ferroptosis-promoting group had better PFS compared to the ferroptosis-inhibiting group (HR 0.47, 95% CI 0.17–1.30), although it was not statistically significant. Data analysis reports high heterogeneity (I 2  = 93.2%, 95% CI 43.5%-97.5%) (Fig.  4 ). After conducting sensitivity analysis, similar to the OS results, we found that the study by Song et al. might introduce significant bias. After excluding this study, the results showed that the ferroptosis-promoting group had significantly better PFS prognosis compared to the ferroptosis-inhibiting group (HR 0.26, 95% CI 0.16–0.44), with moderate heterogeneity (I 2  = 69.7%, 95% CI 0%-89.6%), and the results were statistically significant (Fig.  5 ).

Forest plot of the pooled progression-free survival between the ferroptosis-promoting group and the ferroptosis-inhibiting group

Forest plot of the pooled progression-free survival between the ferroptosis-promoting group and the ferroptosis-inhibiting group after excluding one study with a large bias

Separate meta-regression analyses for OS and PFS results revealed that covariates such as country of study, ferroptosis-related gene, the effect of genes on ferroptosis, and type of cancer had no influence on the results.

Risk of bias in studies

All included studies underwent a risk of bias assessment following the guidelines recommended by the Cochrane Handbook, which includes five bias domains. We classified 2 studies as having low bias risk (2/8, 25%), indicating low bias risk across all domains. Five studies exhibited some lower risk (5/8, 62.5%), suggesting mild uncertainty in at least one domain but no definite high risk. One study had a high risk (1/8, 12.5%), indicating high bias risk in more than one domain. No studies presented a higher risk overall. The reasons for non-low bias risk were predominantly due to incomplete outcome data (9/14, 64%). In multiple lower risk studies, the reason for uncertain bias in other domains was the lack of reported cut-off values. We believe that different cut-off values can introduce a certain degree of bias into study results, which may affect the interpretation of the results of the study Moreover, we excluded a study of Song, which have introduced a large bias because its results were not reported clearly and correctly with low credibility. We conducted a thorough review of their experimental procedures and relevant sensitivity analysis, concluding that it could affect the overall bias risk of the study. After excluding the study by Song et al., the Egger tests for OS and PFS had p -values of 0.20 and 0.205, respectively, indicating no significant publication bias.

To the best of our knowledge, this systematic review represents the pioneering effort to explore the correlation between ferroptosis and cancer prognosis. Through a comprehensive meta-analysis, we aimed to determine whether ferroptosis influences cancer prognosis and its potential applicability as a therapeutic target. The hallmarks of tumorigenesis encompass the evasion of regulatory cell death, unbridled proliferation, and cellular immortality [ 26 , 27 ]. The resistance exhibited by cancer cells poses a formidable challenge in cancer treatment, as conventional chemotherapy agents often fall short in inducing effective cell death [ 28 ]. Ferroptosis emerges as a promising strategy to overcome this resistance [ 27 ]. Nevertheless, ferroptosis assumes a dual role in the context of anti-tumor immunity. CD8 + T cells, for instance, can secrete Interferon-γ to promote ferroptosis in cancer cells, while ferroptotic cancer cells can reciprocally enhance the maturation of dendritic cells and macrophage efficiency [ 13 ]. However, it’s worth noting that some T helper cell subsets and CD8 + T cells can themselves undergo ferroptosis, thereby tempering the overall impact of ferroptosis on anti-tumor immunity [ 13 ].

In our study, we have uncovered that the promotion of ferroptosis in cancer cells serves as a protective factor for cancer patient prognosis. In our analysis of OS, involving eight studies, the results indicate that patients in the group where ferroptosis is promoted exhibit improved overall survival rates compared to the group where it is inhibited (HR 0.43, 95% CI 0.22–0.83). Following a sensitivity analysis, we observed certain biases in the study conducted by Song et al. Upon a thorough review of the research, we discovered that this study found ZFP36 can express in both tumor and para-carcinoma tissues, and the expression of ZFP36 was higher in para-carcinoma tissues Elevated ZFP36 expression inhibits ferroptosis, consequently leading to fewer instances of ferroptosis in the tumor-adjacent tissue, resulting in better patient prognoses. However, in the other studies included, ferroptosis-regulating genes were all found to be overexpressed or suppressed in tumor tissue instead of tumor-adjacent tissue. Meanwhile, the low accuracy of results from the study of Song et al. can introduce bias to our study. So we exclude this particular article to assure the quality of our results. Upon its exclusion, patients in the group where ferroptosis is promoted demonstrated better overall survival rates (HR 0.31, 95% CI 0.21–0.44), with reduced study heterogeneity and a higher p -value in the Egger test. For this intriguing study, we look forward to future research that directly investigates the role of ZFP36 in tumor tissue and whether it presents contrasting effects on patient prognosis. In our study on PFS, after sensitivity analysis, forest plots indicated that patients in the group where ferroptosis is promoted exhibit improved overall survival rates compared to the group where it is inhibited (HR 0.26, 95% CI 0.16–0.44). The heterogeneity could have raised from the absence of the cut-off values, different countries, the differences of ferroptosis-related genes, the type of cancers and the effect of genes on ferroptosis. After conducting meta-regression, we did not identify covariates including country, ferroptosis-related genes, type of cancer and the effect of genes influencing the results. Considering that 5 of the 8 studies we included did not report the cut-off value, we could not include this in meta-regression, which can lead to potential heterogeneity.

As the pioneering meta-analysis investigating the impact of ferroptosis on cancer patient prognosis, we are pleased to find that it serves as a protective factor for cancer patient prognoses. Ferroptosis, as a novel biological behavior distinct from apoptosis, holds promise as a potential approach in cancer treatment. Currently, we have identified numerous key genes in the ferroptosis pathways, and if ferroptosis proves to be an effective cancer treatment modality, targeting these genes would hold significant clinical relevance. These potential targets included down-regulation of GPX4, ZFP36, SLC7A11, FSP1 expression and up-regulation of HMOX1 expression. Moving forward, there is a promising potential to translate these interventions targeting specific factors into practical applications in clinical therapy. This holds great promise as an exciting new avenue in the realm of cancer bio-therapy.

Despite our rigorous article selection, feature extraction, and analysis, this study has certain limitations. Firstly, we require more clinical research, whether retrospective or randomized controlled studies, to substantiate the favorable impact of promoting ferroptosis in cancer cells on the prognosis of cancer patients, including both OS and PFS, both of which are pivotal for patients’ quality of life. Secondly, the cut-off value is a critical parameter; regrettably, many of the articles we included did not report this metric, making it challenging to assess the extent of bias in prognosis results due to cut-off value variations. We also hope that future related meta-analyses will delve further into the influence of cut-off values.

This meta-analysis, by comparing the promotion and inhibition of ferroptosis in cancer patients, reveals that fostering ferroptosis in cancer cells is a protective factor for cancer patient prognosis. Ferroptosis-related genes hold the potential to become novel biomarkers for targeted therapy, and promoting ferroptosis in cancer cells could represent a new and effective approach to cancer treatment.

Availability of data and materials

To ensure transparency and reproducibility of the study, all data generated or analyzed during this study are included in this published article and its supplementary information files. The datasets used and analyzed during the current study are available from the corresponding author on reasonable request. Please note that data sharing is intended for academic research purposes only and not for other purposes.

Sung H, Ferlay J, Siegel RL, Laversanne M, Soerjomataram I, Jemal A, et al. Global cancer statistics 2020: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin. 2021;71(3):209–49.

Article   PubMed   Google Scholar  

Bray F, Ferlay J, Soerjomataram I, Siegel RL, Torre LA, Jemal A. Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin. 2018;68(6):394–424.

Wang Y, Jiang H, Fu L, Guan L, Yang J, Ren J, et al. Prognostic value and immunological role of PD-L1 gene in pan-cancer. BMC Cancer. 2024;24(1):20.

Article   PubMed   PubMed Central   Google Scholar  

Zaimy MA, Saffarzadeh N, Mohammadi A, Pourghadamyari H, Izadi P, Sarli A, et al. New methods in the diagnosis of cancer and gene therapy of cancer based on nanoparticles. Cancer Gene Ther. 2017;24(6):233–43.

Article   CAS   PubMed   Google Scholar  

Oun R, Moussa YE, Wheate NJ. The side effects of platinum-based chemotherapy drugs: a review for chemists. Dalton Trans. 2018;47(19):6645–53.

Wang K, Tepper JE. Radiation therapy-associated toxicity: etiology, management, and prevention. CA Cancer J Clin. 2021;71(5):437–54.

Mattiuzzi C, Lippi G. Current cancer epidemiology. Journal of epidemiology and global health. 2019;9(4):217–22.

Dixon SJ, Lemberg KM, Lamprecht MR, Skouta R, Zaitsev EM, Gleason CE, et al. Ferroptosis: an iron-dependent form of nonapoptotic cell death. Cell. 2012;149(5):1060–72.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Jiang X, Stockwell BR, Conrad M. Ferroptosis: mechanisms, biology and role in disease. Nat Rev Mol Cell Biol. 2021;22(4):266–82.

Stockwell BR, Friedmann Angeli JP, Bayir H, Bush AI, Conrad M, Dixon SJ, et al. Ferroptosis: a regulated cell death nexus linking metabolism, redox biology, and disease. Cell. 2017;171(2):273–85.

Galluzzi L, Vitale I, Aaronson SA, Abrams JM, Adam D, Agostinis P, et al. Molecular mechanisms of cell death: recommendations of the nomenclature committee on cell death 2018. Cell Death Differ. 2018;25(3):486–541.

Chen X, Kang R, Kroemer G, Tang D. Broadening horizons: the role of ferroptosis in cancer. Nat Rev Clin Oncol. 2021;18(5):280–96.

Lei G, Zhuang L, Gan B. Targeting ferroptosis as a vulnerability in cancer. Nat Rev Cancer. 2022;22(7):381–96.

Chen H, He Y, Pan T, Zeng R, Li Y, Chen S, et al. Ferroptosis-related gene signature: a new method for personalized risk assessment in patients with diffuse large B-cell lymphoma. Pharmgenomics Pers Med. 2021;14:609–19.

PubMed   PubMed Central   Google Scholar  

Hsieh PL, Chao SC, Chu PM, Yu CC. Regulation of ferroptosis by non-coding RNAs in head and neck cancers. Int J Mol Sci. 2022;23(6):3142.

Liu Y, Duan C, Dai R, Zeng Y. Ferroptosis-mediated crosstalk in the tumor microenvironment implicated in cancer progression and therapy. Front Cell Dev Biol. 2021;9:739392.

Yu S, Jia J, Zheng J, Zhou Y, Jia D, Wang J. Recent progress of ferroptosis in lung diseases. Front Cell Dev Biol. 2021;9:789517.

Zeng H, You C, Zhao L, Wang J, Ye X, Yang T, et al. Ferroptosis-associated classifier and indicator for prognostic prediction in cutaneous melanoma. J Oncology. 2021;2021:3658196.

Article   Google Scholar  

Wang Y, Fu L, Lu T, Zhang G, Zhang J, Zhao Y, et al. Clinicopathological and prognostic significance of long non-coding RNA MIAT in human cancers: a review and meta-analysis. Front Genet. 2021;12:729768.

Wang Y, Jiang X, Zhang D, Zhao Y, Han X, Zhu L, et al. LncRNA DUXAP8 as a prognostic biomarker for various cancers: a meta-analysis and bioinformatics analysis. Front Genet. 2022;13:907774.

Shishido Y, Amisaki M, Matsumi Y, Yakura H, Nakayama Y, Miyauchi W, et al. Antitumor effect of 5-aminolevulinic acid through ferroptosis in esophageal squamous cell carcinoma. Ann Surg Oncol. 2021;28(7):3996–4006.

Song P, Xie Z, Chen C, Chen L, Wang X, Wang F, et al. Identification of a novel iron zinc finger protein 36 (ZFP36) for predicting the overall survival of osteosarcoma based on the Gene Expression Omnibus (GEO) database. Annals of translational medicine. 2021;9(20):1552.

Sugezawa K, Morimoto M, Yamamoto M, Matsumi Y, Nakayama Y, Hara K, et al. GPX4 regulates tumor cell proliferation via suppressing ferroptosis and exhibits prognostic significance in gastric cancer. Anticancer Res. 2022;42(12):5719–29.

Wu X, Shen S, Qin J, Fei W, Fan F, Gu J, et al. High co-expression of SLC7A11 and GPX4 as a predictor of platinum resistance and poor prognosis in patients with epithelial ovarian cancer. BJOG. 2022;129 Suppl 2(Suppl 2):40–9.

Miyauchi W, Shishido Y, Matsumi Y, Matsunaga T, Makinoya M, Shimizu S, et al. Simultaneous regulation of ferroptosis suppressor protein 1 and glutathione peroxidase 4 as a new therapeutic strategy of ferroptosis for esophageal squamous cell carcinoma. Esophagus. 2023;20(3):492–501.

Wang Y, Lin K, Xu T, Wang L, Fu L, Zhang G, et al. Development and validation of prognostic model based on the analysis of autophagy-related genes in colon cancer. Aging. 2021;13(14):19028–47.

Zhao L, Zhou X, Xie F, Zhang L, Yan H, Huang J, et al. Ferroptosis in cancer and cancer immunotherapy. Cancer communications (London, England). 2022;42(2):88–116.

Xu T, Ding W, Ji X, Ao X, Liu Y, Yu W, et al. Molecular mechanisms of ferroptosis and its role in cancer therapy. J Cell Mol Med. 2019;23(8):4900–12.

Download references

Acknowledgements

Not applicable.

No funding.

Author information

Shen Li, Kai Tao and Hong Yun contributed equally to this work.

Authors and Affiliations

Department of Biotherapy, West China Hospital and State Key Laboratory of Biotherapy, Sichuan University, Chengdu, Sichuan, China

Shen Li, Hong Yun, Jiaqing Yang & Xuelei Ma

West China School of Medicine, West China Hospital, Sichuan University, Chengdu, Sichuan, China

Kai Tao & Jiaqing Yang

West China School of Stomatology, Sichuan University, Chengdu, Sichuan, China

Yuanling Meng

Health Management Center, General Practice Medical Center, West China Hospital, Sichuan University, Chengdu, Sichuan, China

You can also search for this author in PubMed   Google Scholar

Contributions

Study Concept and design: MXL, LS, ZF; Search Strategy: TK, LS, YH; Selection Criteria: TK, LS, YJQ, YH; Quality Assessment: LS, YJQ, MXL and TK; Drafting of the Manuscript: LS, TK, YH. All authors read and approved the final manuscript.

Corresponding authors

Correspondence to Fan Zhang or Xuelei Ma .

Ethics declarations

Ethics approval and consent to participate, consent for publication, competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary material 1., supplementary material 2., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Li, S., Tao, K., Yun, H. et al. Ferroptosis is a protective factor for the prognosis of cancer patients: a systematic review and meta-analysis. BMC Cancer 24 , 604 (2024). https://doi.org/10.1186/s12885-024-12369-5

Download citation

Received : 07 November 2023

Accepted : 10 May 2024

Published : 17 May 2024

DOI : https://doi.org/10.1186/s12885-024-12369-5

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Ferroptosis

ISSN: 1471-2407

• Submission enquiries: [email protected]
• General enquiries: [email protected]

• Open access
• Published: 14 May 2024

PLAG1 interacts with GPX4 to conquer vulnerability to sorafenib induced ferroptosis through a PVT1/miR-195-5p axis-dependent manner in hepatocellular carcinoma

• Jiarui Li 1 ,
• Yilan Li 1 ,
• Denghui Wang 1 ,
• Rui Liao 1 &
• Zhongjun Wu 1  

Journal of Experimental & Clinical Cancer Research volume  43 , Article number:  143 ( 2024 ) Cite this article

351 Accesses

1 Altmetric

Metrics details

Sorafenib is a standard first-line treatment for advanced hepatocellular carcinoma (HCC), yet its effectiveness is often constrained. Emerging studies reveal that sorafenib triggers ferroptosis, an iron-dependent regulated cell death (RCD) mechanism characterized by lipid peroxidation. Our findings isolate the principal target responsible for ferroptosis in HCC cells and outline an approach to potentially augment sorafenib's therapeutic impact on HCC.

We investigated the gene expression alterations following sgRNA-mediated knockdown induced by erastin and sorafenib in HCC cells using CRISPR screening-based bioinformatics analysis. Gene set enrichment analysis (GSEA) and the "GDCRNATools" package facilitated the correlation studies. We employed tissue microarrays and cDNA microarrays for validation. Ubiquitination assay, Chromatin immunoprecipitation (ChIP) assay, RNA immunoprecipitation (RIP) assay, and dual-luciferase reporter assay were utilized to delineate the specific mechanisms underlying ferroptosis in HCC cells.

Our study has revealed that pleiomorphic adenoma gene 1 (PLAG1), a gene implicated in pleomorphic adenoma, confers resistance to ferroptosis in HCC cells treated with sorafenib. Sorafenib leads to the opposite trend of protein and mRNA levels of PLAG1, which is not caused by affecting the stability or ubiquitination of PLAG1 protein, but by the regulation of PLAG1 at the transcriptional level by its upstream competitive endogenous long non-coding RNA (lncRNA) plasmacytoma variant translocation 1 (PVT1). Data from 139 HCC patients showed a significant positive correlation between PLAG1 and GPX4 levels in tumor samples, and PLAG1 is instrumental in redox homeostasis by driving the expression of glutathione peroxidase 4 (GPX4), the enzyme that reduces lipid peroxides (LPOs), which further leads to ferroptosis inhibition.

Conclusions

Ferroptosis is a promising target for cancer therapy, especially for patients resistant to standard chemotherapy or immunotherapy. Our findings indicate that PLAG1 holds therapeutic promise and may enhance the efficacy of sorafenib in treating HCC.

Introduction

HCC is the second most common cause of cancer-related deaths globally, largely due to late diagnosis and its links to lifestyle factors such as alcohol consumption, and conditions like metabolic syndrome and viral infections [ 1 ]. One major issue with HCC treatment is the low efficacy rate of sorafenib, an FDA-approved drug [ 2 ], which shows favorable responses in only 30% of HCC patients. Furthermore, resistance to sorafenib often develops within a year [ 3 ]. Understanding the molecular mechanisms behind sorafenib resistance and identifying new targets are critical for advancing cancer therapeutics and increasing patient survival rates.

Sorafenib, an oral multi-kinase inhibitor, has been shown benefits against various tumors, notably in liver cancer. Its efficacy results from its ability to inhibit multiple kinases, leading to halted tumor cell growth, restricted angiogenesis, and induced apoptosis [ 4 , 5 ]. However, acquired or intrinsic resistance to apoptosis can limit the effectiveness of sorafenib-induced cell death. Interestingly, sorafenib also triggers ferroptosis, a newly recognized form of RCD, in hepatoma cells [ 6 , 7 ]. This action isn't based on its multi-kinase inhibitory function and enhances the drug's anti-cancer effect [ 8 ]. As opposed to regular apoptosis and necrosis, ferroptosis is caused by cellular contraction and increased mitochondrial membrane density, consequently to the intracellular buildup of iron-dependent lipoperoxidation. Therefore, encouraging ferroptosis via sorafenib represents a potential new direction in liver cancer treatment enhancement.

PLAG1, a gene located on chromosome 8q12, consists of 5 exons and has been linked to cancer-related activities such as proliferation, migration, and invasion [ 9 , 10 ]. The gene bears crucial similarities with PLAGL2, both structurally and functionally. In liver cancer, PLAGL2 enhances the activation of hypoxia-inducible factor 1 (HIF1). This factor promotes the transcription of the SLC7A11 subunit of the cystine/glutamate antiporter system Xc(-), effectively preventing the onset of ferroptosis [ 11 , 12 ]. PLAG1 contains seven typical C2H2 zinc finger structures, highlighting its potential transcription factor role, particularly through its COOH-terminal domain's trans-activation ability [ 13 ]. However, the role of PLAG1 in the regulation of sorafenib-induced ferroptosis in liver cancer remains a subject for further investigation.

The enzyme GPX4 is crucial for the clearance of LPOs [ 14 ]. This process relies on the system Xc(-), which transports cystine into the cell, an essential component for glutathione (GSH) synthesis during cellular scavenging activities [ 15 ]. GPX4 uses glutathione to reduce LPOs into phospholipid molecules, making its role vital in boosting the cell's antioxidant capacity, which helps protect liver cancer cells from ferroptosis. Consequently, focusing on manipulating GPX4 could offer a potential strategy in enhancing tumor therapy, particularly in overcoming sorafenib resistance.

Our study indicates the impact of PLAG1 on ferroptosis inhibition in the case of sorafenib treatment, is determined by the regulation of the PVT1/miR-195-5p axis. Additionally, a positive correlation between PLAG1 and GPX4 expression in HCC patients and a crosstalk between PLAG1 and GPX4 in ferroptosis inhibition are due to PLAG1's ability to amplify GPX4 gene transcription by binding to its promoter region. In conclusion, our research clarifies the role of PLAG1 in ferroptosis and its potential as a responsive therapeutic target for HCC patients resistant to sorafenib treatment.

Materials and methods

Cell lines and culture.

The liver cancer cell lines (HCCLM9, HCCLM3, SK-hep1, MHCC97H, and MHCC97L) and the normal hepatocyte MIHA were obtained from Shanghai Institutes of Biological Sciences (Shanghai, China). They were cultured in 90% DMEM (ZQXZ-bio, Shanghai, China) supplemented with 10% fetal bovine serum (SORFA, Beijing, China) at a temperature of 37 °C with 5% CO 2 .

Lentivirus production

Hanbio Biotechnology (Shanghai, China) provided lentivirus vectors that include shRNA to silence PLAG1 (targeting GGAGCACCUUAAAUCUCAUTT), lentiviruses that overexpress PLAG1 (LV-PLAG1), shRNA to silence PVT1 (targeting GGCCTCGTGTCTATTAAAT), and a negative control lentivirus. Genechem (Shanghai, China) sold lentiviruses called LV-PVT1, which overexpressed PVT1, as well as control lentiviruses. The MHCC97H (MOI = 5) or MHCC97L (MOI = 30) cell lines were seeded in six-well plates with 150,000 cells per well, followed by lentiviral infection on the following day. To establish stably infected cell lines, 2 μg/ ml puromycin was applied after infection 72 h for at least two weeks.

Cell transfection

Tsingke Biotechnology (Beijing, China) provided the expression vector pcDNA3.1( +) (PVT1) and the empty plasmid pcDNA3.1( +) (Vector). The PVT1 siRNA (which targets GGCCTCGTGTCTATTAAAT) was acquired from GenePharma (Suzhou, China). RiboBio (Guangzhou, China) sold the hsa-miR-195-5p mimic (miR10000461–1–5) and hsa-miR-195-5p inhibitor (miR20000461–1–5) that were acquired. The cell transfection was carried out in accordance with the guidelines provided by the manufacturers.

cDNA Microarray and quantitative real‑time polymerase chain reaction (qRT‑PCR)

We purchased the cDNA microarray (MecDNA-HLivH087Su02 and MicDNA-HLivH087Su02) from Shanghai Outdo Biotech Co. The FastPure Cell/Tissue Total RNA Isolation Kit V2 (Vazyme, Nanjing, China) was utilized for the purification of total RNA. The PrimeScript™ RT reagent Kit (Takara, Japan) was used to convert 1 μg total RNA into cDNA through reverse transcription. For the examination of miRNA expression, we utilized the Hairpin-it™ miRNA Quantitation Kit (GenePharma, Suzhou, China) along with reverse transcription using specific stem-loop primers for miR-20a-5p, miR-195-5p, miR-17-5p, miR-106b-5p, and miR-93-5p. qRT-PCR was conducted using SYBR Green (Selleck, USA). The parameters for the two-stage amplification process were as follows: initial denaturation at a temperature of 95 °C for a duration of 2 min, followed by 40 cycles of denaturation at 95 °C for 15 s and annealing at 60 °C for 30 s. GAPDH and U6 expression served as the internal reference for mRNA/lncRNA and miRNA, respectively. Supplementary Table  1 displays the specific primers for RNAs.

Immunoblotting

The cells were gathered and broken down in M-PER (R) Mammalian Protein Extraction Reagent (Thermo Fisher Scientific, USA) while being kept on ice for a duration of 30 min. The cell lysates underwent resolution through SDS-PAGE and were subsequently transferred to PVDF membranes (Cytiva, USA). The PVDF filters were treated with QuickBlock™ Western blocking solution (Beyotime, China) for 1 h while gently shaking. Subsequently, the filters were incubated with the specified antibodies at 4 °C overnight. Goat secondary antibodies conjugated with HRP (1:1000, Beyotime, China) were employed. After being washed with TBST, the membranes underwent a thorough cleansing, followed by incubation of the secondary antibody at room temperature for 2 h. To examine the protein bands, we employed a chemiluminescent western blot detection method.

Tissue microarray and immunohistochemistry staining

AiFang Biological (Changsha, China) generated TMA using 139 paraffin-embedded samples. Immunohistochemistry (IHC) Kit (Absin, Shanghai, China) was employed to identify the correlation between PLAG1 and GPX4 expression in hepatocellular carcinoma tissues. For IHC staining, the antibodies employed were anti-PLAG1 (1:100, Novus Biologicals, USA) and anti-GPX4 (1:50, Selleck, USA). Two independent pathologists blinded to the clinical data evaluated the PLAG1 and GPX4 IHC staining using a histological score (H-score) approach. IHC analysis was conducted following the previously mentioned protocol [ 16 ]. Briefly, the staining intensity was classified into the following categories: 0 (-), 1 ( +), 2 (+ +), and 3 (+ + +). The mean proportion of cells with positive staining was assessed using the following scoring system: 0 (< 5%); 1 (5–25%); 2 (26–50%); 3 (51–75%) and 4 (76–100%). The overall scores were calculated by multiplying the scores for staining intensity with the scores for staining proportion, which ranged from 0 to 12. To divide them into low and high expression subgroups, the median H-scores of PLAG1 and GPX4 were utilized as thresholds.

Antibodies and molecular compounds

The antibodies utilized in this investigation for western blot and immunohistochemistry analysis were as follows: PLAG1 (H00005324-M02, Novus Biologicals, 1:500 for WB, 1:50 for IHC), GPX4 (A5569, Selleck, 1:1000 for WB), AGO2 (#2897, Cell Signaling Technology, 1:1000 for WB), GAPDH (60,004–1-lg, Proteintech, 1:50,000 for WB), 4-HNE (MHN-020P, JalCA, 1:4 for IHC), Ubiquitin(AF1705, Beyotime, 1:1000 for WB). Beyotime and Selleck Chemicals were the sources of the secondary antibody conjugates labeled with Horseradish peroxidase (HRP) and Sorafenib (#S7397), respectively. Cycloheximide (CHX, HY-12320) and MG132 (HY-13259) were purchased from Medchemexpress.

Assessment of cellular viability

The CCK8 (Absin, Shanghai, China) was utilized to assess cellular viability. In short, cells were seeded onto 96-well plates at a density of 5,000 cells per well. Once the cells reached 60% confluence, they were treated with either sorafenib (10 μM) or dimethyl sulfoxide (DMSO) for a duration of 24 h. Following this, 10 μL of CCK-8 solution was introduced into each well and incubated for 2 h. Subsequently, the absorbance at 450 nm was determined using a microplate reader.

Investigation of ferroptosis

(1) To determine the Malondialdehyde (MDA) levels, 5 × 10 6 cell lysates were analyzed using a Cell Malondialdehyde Assay Kit (A003-4–1, Jiancheng Bio) following the provided protocol. MDA reacts with thiobarbituric acid (TBA) to produce a crimson MDA-TBA compound, exhibiting a peak absorbance at 532 nm.

(2) The GSH test involved seeding corresponding cells in 6 cm plates at a density of 3 × 10 6 per plate. These cells were then treated with either sorafenib (10 μM) or DMSO for a duration of 48 h. Afterward, the cells were washed and collected using PBS. The lysates were analyzed with a commercially available Reduced GSH Assay Kit (A006-2–1, Jiancheng Bio). Yellow product with maximum light absorption at wavelength 412 nm, and its absorbance is proportional to GSH content.

(3) To identify lipid peroxidation products, we utilized LPOs analysis with Liperfluo (Dojindo, Japan). In short, cells were placed on a glass bottom dish with a diameter of 35 mm (5 × 10 4 cells/dish). After 24 h, the culture medium was exchanged with sorafenib (10 μM) or DMSO and incubated for another 24 h. Following that, 10 µM Liperfluo probes were applied to label LPOs at a working concentration, and incubated for 30 min. Finally, the cells were observed and photographed using confocal microscopy.

(4) To measure intracellular Fe 2+ , a total of 1 × 10 6 cells was rinsed three times with cold PBS (Procell, Wuhan), gathered, suspended, and then exposed to 1 μM FerroOrange probes (Dojindo, Japan) for 30 min at 37 °C. Subsequently, flow cytometry analysis was conducted using Ex 561 nm/Em 570–620 nm.

(5) The Lipid Reactive Oxygen Species (ROS) assay involved incubating 1 × 10 6 cells with 5 μM BODIPY-C11 (Glpbio, USA) at 37 °C for 30 min. The cells were collected and placed in DMEM without serum, then analyzed using flow cytometry or confocal microscopy (with an excitation wavelength of 488 nm and an emission wavelength range of 510–555 nm).

Chromatin immunoprecipitation (ChIP) and truncation assay

An Enzymatic ChIP Kit (Cell Signaling Technology, USA) was utilized for conducting a ChIP assay. JASPAR (https //jaspar.genereg.net) provided the top seven potential sequences between PLAG1 and the GPX4 promoter (-2000 bp ~  + 99 bp). We truncated the GPX4 promoter region and designed primers for each segment that contains the binding site, based on the anticipated distribution of sites in the GPX4 promoter region. Formaldehyde was added to cross-link MHCC97H cells, and glycine was added to terminate the process. Sonication was used to produce fragments ranging from 200 to 900 bp in length from lysates. Specific DNA–protein complexes were immunoprecipitated using an incubating antibody (anti-PLAG1 and lgG) (Novus Biologicals, USA). In addition, MHCC97H cells were plated onto 10 cm dishes and co-transfected with either the PLAG1 overexpression vector or the normal control (NC) groups. The binding of the GPX4 core promoter region (-250 bp ~  + 120 bp) to RNA polymerase II (Pol II) was verified using the same method. The quantification and analysis of purified DNA was performed using qRT-PCR. Supplementary Table  2 displayed the details of the primers.

Construction of competing endogenous RNA (ceRNA) network and the luciferase reporter assay

From TCGA-LIHC, we detected the distinct expression of miRNAs, mRNAs, and lncRNAs. To build the ceRNA network of lncRNA-miRNA-mRNA, we utilized the GDCRNATools R package [ 17 ]. Cytoscape v3.6.0 [ 18 ] was utilized to generate the ceRNA network. We utilized three different target prediction algorithms, namely miRcode [ 19 ], starBase [ 20 ], and spongeScan [ 21 ], to explore the potential miRNAs associated with crosstalk with PVT1. Upon identifying miR-195-5p as the focus of our study, we proceeded to discover and forecast the probable binding locations of miR-195-5p in the 3′-UTR regions of PVT1 and PLAG1. The luciferase reporter vectors PVT1-wild type (WT) and PVT1-mutant (MUT) were created using PmirGLO from Promega, an American company. The introduction of mutations in the potential binding sites of miR-195-5p was carried out using the QuickMutation™ Site-Directed Mutagenesis Kit from Beyotime, a company based in China.24-well plates were used to seed HEK293T cells, which were then co-transfected with PVT1-WT or MUT vectors along with miR-195-5p mimic or miR-NC.Using the identical approach, the confirmation of the connection between miR-195-5p and the 3′-UTR portions of PLAG1 was established.The luciferase activity was evaluated using a luciferase reporter assay kit (GenePharma, China) following the instructions provided by the manufacturers.

RNA immunoprecipitation (RIP)

RIP is a technique used to isolate RNA molecules. RNA immunoprecipitation kit (Geneseed, Guangzhou, China) was used to detect the interaction of PVT1 and miR-195-5p, as well as PLAG1 and miR-195-5p, following the instructions provided by the manufacturer. In brief, around 2 × 10 7 MHCC97H cells were transfected with miR-195-5p mimics or miR-NC using the riboFECT CP Transfection Kit (RiboBio, Guangzhou, China). After being treated, the cells were suspended and broken down in RIP lysis solution. The cell lysates were mixed with RIP immunoprecipitation buffer that had magnetic beads attached to anti-Ago2 antibody (Cell Signaling Technology, USA) and control IgG antibody (Cell Signaling Technology, USA). This mixture was then incubated overnight at a temperature of 4 °C. Following incubation with Proteinase K, the RNA that was immunoprecipitated was eluted, isolated, and measured using qRT-PCR and western blotting.

Fluorescence in Situ Hybridization (FISH)

FISH is a technique that can be used to detect and locate specific DNA/RNA sequences in cells. The location of PVT1 in MHCC97H and MHCC97L cells was observed using the FISH assay. Probes labeled with Cy5 for PVT1 were synthesized by Servicebio (Wuhan, China). HCC cells were grown on coverslips and treated with a FISH probe in hybridization buffer (Servicebio, Wuhan, China) for 16 h at 37 °C. The cell nuclei were then stained with DAPI (4′6-diamidino-2-phenylindole). The Olympus BX51 fluorescence microscope (Tokyo, Japan) was used to capture these images. Supplementary Table  3 contained the displayed probe sequences.

Transmission electron microscopy

The protocol for the transmission electron microscope was executed according to the previously stated description [ 22 ]. In short, MHCC97H cells were cultured on a 60 mm dish with a density of 6 × 10 5 cells per dish and incubated for 24 h prior to treatment. MHCC97H cells were fixed with 2.5% glutaraldehyde at 4 °C for 24 h and treated with sorafenib (10 μM) or DMSO for 24 h. Subsequently, the cells were exposed to 1% osmium tetraoxide for 2 h after drug treatment. Dehydration of the cells lasted for 15 min before embedding them in resin. Using a JEM-1400 electron microscope (JEOL Ltd., Japan), representative images were acquired after slicing and double-staining the samples with uranyl acetate and lead citrate.

Bioinformatics

Data on gene expression and patient information for hepatocellular carcinoma were acquired from the TCGA website (https//cancergenome.nih.gov/). Using the DESeq2 package of R software (version 4.2.1) [ 23 ], we determined the variation in counts within the dataset GSE182185 and GSE109211 obtained from the GEO website (https// www.ncbi.nlm.nih.gov/ ). The difference analysis results were then visualized using the ggplot2, where the horizontal axis represented the position after log (FC) was sorted and the vertical axis represented log (FC). The Hi-C and ChIP-seq data were downloaded from datasets GSE184796 and GSE151287. Initially, the unprocessed sequencing data undergoes quality control procedures, which involve evaluating the quality of the reads and eliminating reads with low quality. Afterwards, the sequencing data is aligned to the reference genome using bwa (v0.7.17) in order to identify their genomic locations [ 24 ]. From the mapped reads, a contact matrix or a bipartite graph is constructed, where the matrix elements or graph edges represent the frequency of interactions between genomic loci. Normalization techniques are then applied to correct for biases arising from sequencing depth and chromosome length. Subsequently, topologically associated domains (TADs) can be identified using juicer software ( http://aidenlab.org/juicer/ ), partitioning the genome into regions with similar intra-chromosomal interaction patterns. The juicer (v3.0.0) can be employed to visualize the contact matrix, TADs, and other structural features [ 25 ].

Animal studies

GemPharmatech (Jiangsu, China) supplied BALB/c nude mice that were 4 weeks old and female. (1) To establish the subcutaneous mouse model, around 5 × 10 6 MHCC97H/sh-NC cells or MHCC97H/sh-PLAG1 cells are surgically inserted under the skin on the right side. Upon reaching a tumor volume of 50 mm 3 , mice harboring MHCC97H/sh-NC cells or MHCC97H/sh-PLAG1 cells were randomly allocated into two groups. One group received vehicle treatment (0.9% NaCl i.g., once daily), while the other group received sorafenib treatment (10 mg/kg i.g., once daily) for a duration of two weeks. Measurements of tumors were taken every second day. The formula for calculating tumor volume is V = (L × W 2 ) / 2, where V denotes the volume of the tumor, L represents the tumor's length, and W represents its width. Following a 14-day period of treatment, the mice were euthanized and the tumors were excised.

(2) To establish the orthotopic mouse model, initially, around 5 × 10 6 MHCC97H/sh-NC cells or MHCC97H/sh-PLAG1 cells were subcutaneously injected into the right flank. Afterwards, we proceeded to dissect the tumors and divide them into 1 mm 3 cubes under sterile conditions on day 14. After segmenting the tumors, they were implanted into the liver parenchyma of female BALB/c nude mice that were 4 weeks old, specifically in the right lobe. On the seventh day, mice carrying MHCC97H/sh-NC cells or MHCC97H/sh-PLAG1 cells were randomly separated into two groups. One group received treatment with a solution of 0.9% NaCl (i.g., once daily), while the other group was administered sorafenib at a dose of 10 mg/kg (i.g., once daily). After 21 days of implantation, the mice were sacrificed and the tumors were dissected. For subsequent IHC experiments, fixation with 4% paraformaldehyde was employed. The tumor volumes were then determined using the identical formula as previously.

Statistical analysis

At least three independent experiments were conducted and the data were presented as the mean plus or minus the standard deviation (± SD). The data from the experiments were examined using either the two-tailed Student's t-test or ANOVA in GraphPad Prism 8. We utilized the Pearson χ 2 test for comparing qualitative variables. P value < 0.05 were considered statistically significant.

The treatment with sorafenib in HCC leads to a decrease in the oncoprotein PLAG1

Our bioinformatics analysis, employing CRISPR-based screening, revealed a marked decrease in sgRNA targeting the oncogenic protein PLAG1 in HCC cells following treatment with ferroptosis inducers erastin and sorafenib. This suggests that PLAG1 potentially contributes to ferroptosis resistance (Fig.  1 A). Kaplan–Meier analysis of overall survival rates concerning 374 patients from the TCGA-LIHC cohort highlighted an inverse relationship between PLAG1 levels and survival rates, indicating its contribution to HCC progression (Fig.  1 B). A more localized study using qRT-PCR demonstrated an elevation in PLAG1 mRNA levels in 87 tumor tissues compared to normal tissues, consistent with data from the TCGA-LIHC RNA sequencing and cDNA-microarray analysis (Fig.  1 C-E). This result was further confirmed by paired samples from HCC patients (Fig.  1 F). Western blotting and qRT-PCR showed that PLAG1 was upregulated in MHCC97L, HCCLM3, HCCLM9, and MHCC97H cells, with the lowest expression in MHCC97L cells and the highest expression in MHCC97H cells compared with normal hepatocyte MIHC (Fig.  1 G and Supplementary Fig.  1 ). Therefore, we selected MHCC97L and MHCC97H for follow-up studies. Fascinatingly, sorafenib dosage demonstrated a proportional suppression of the PLAG1 level in HCC cell lines MHCC97H and MHCC97L suggesting its potential to combat PLAG1 expressions in HCC progression (Fig.  1 H). These findings emphasize the significant role of PLAG1 in the progression of HCC, and its potential as a therapeutic target.

The administration of sorafenib in HCC patients results in a decrease in the oncoprotein PLAG1. ( A ) Sorting charts depicting alterations in reanalyzing of CRISPR screen results, with genes related to ferroptosis exhibiting significant depletion, as indicated by colored dots. Notably, the treatment with erastin and sorafenib leads to a considerable reduction in PLAG1 levels in hepatoma cells. ( B ) Kaplan–Meier graphs illustrating the negative correlation between PLAG1 mRNA levels and the survival outcomes of 374 HCC patients obtained from the TCGA database. A comparative analysis was conducted to assess the expression of PLAG1 in HCC and healthy tissues. Data from the TCGA database ( C ) and cDNA microarray ( D ) were utilized for this purpose. ( E ) The qRT-PCR technique was employed to measure the levels of PLAG1 mRNA expression in 21-paired tumor and peritumoral tissues obtained from HCC patients. The obtained data were presented as − ΔΔCt values. To assess the expression levels of PLAG1 protein, a western blot analysis was conducted on four hepatocellular carcinoma (HCC) tissues and their corresponding peritumoral tissues ( F ), as well as on MIHA, SK-Hep1, MHCC97L, HCCLM3, HCCLM9, and MHCC97H cell lines ( G ). The results showed that the expression of PLAG1 was lower in MHCC97L cells and higher in MHCC97H cells. Therefore, MHCC97L cells and MHCC97H cells were used for subsequent experiments. Subsequently, the protein levels of PLAG1 in MHCC97H and MHCC97L cell lines were determined using western blot analysis, following treatment with DMSO or sorafenib (SF) for 24 h in a dose-dependent manner ( H ). Statistical analysis revealed significant differences with ** P  < 0.01 and *** P  < 0.001

PLAG1 plays a role in inhibiting ferroptosis in HCC cells treated with sorafenib

Evidence increasingly suggests ferroptosis's role in sorafenib-induced cell death in HCC [ 6 , 26 , 27 ]. This study aimed to ascertain PLAG1's role in ferroptosis inhibition. To this end, HCC cells with PLAG1 knockdown and overexpression were respectively generated in cell MHCC97H and MHCC97L, then treated with sorafenib. Western blot and qRT-PCR analysis confirmed that sorafenib reduced protein and mRNA levels without altering PLAG1 knockdown and overexpression efficiency(Fig.  2 A and B).

A key feature of ferroptosis is LPOs accumulation, often triggered by the depletion of GSH [ 28 , 29 ]. We evaluated the viability of MHCC97H and MHCC97L cells, alongside GSH levels, to determine if PLAG1 could be a potential target for sorafenib-induced ferroptosis. Interestingly, PLAG1 knockdown significantly amplified sorafenib's inhibition of cell viability and depletion of GSH, in contrast to PLAG1 up-regulation (Fig.  2 C and D ). The levels of ferroptosis indicators including LPOs, Lipid ROS, and intracellular iron were assessed using specific probes. Results showed that sorafenib enhanced ferroptosis in MHCC97H cells, seen in elevated levels of intracellular iron, Liperfluo, and Lipid ROS. However, it suppressed ferroptosis in MHCC97L cells, with reduced levels of these markers (Fig.  2 E-H).

PLAG1 inhibits sorafenib-induced ferroptosis. ( A ) Western blot and qRT-PCR were used to detect the relative expression of PLAG1 in MHCC97H and MHCC97L cells transfected with the specified constructs. Transfected cells were exposed to sorafenib (10 µM) for 24 h. Cell viability was assessed using a CCK-8 kit ( C ). The concentration of GSH in the cells was measured using a glutathione assay kit ( D ). Intracellular Fe 2+ levels were determined using the fluorescent probe FerroOrange, while lipid peroxide levels were measured using the fluorescent probe Liperfluo ( E , F ). Scale bar: 100 μm. ( G , H ) Lipid ROS accumulation was analyzed by flow cytometry with BODIPY-C11 staining. ( I ) The transmission electron microscopy was used to observe the structure of mitochondria and showed that knockdown of PLAG1 could promote the increase of mitochondrial electron density and mitochondrial shrinkage. Scale bar: 2μm. The data displayed indicate the mean ± SD obtained from three independent trials. # P  < 0.05; ### P  < 0.001 compared with the control group. * P  < 0.05; ** P  < 0.01; *** P  < 0.001 compared with the sorafenib-induced control group

Ferroptosis fundamentally occurs when a balance between oxidative damage and antioxidant defenses is lost, leading to rampant lipid peroxidation and compromising mitochondrial membrane integrity [ 30 , 31 ]. This destabilization escalates to mitochondrial crest rupture, triggering ferroptosis. Transmission electron microscopy revealed that PLAG1 inhibition increases mitochondrial membrane density, decreases mitochondrial volume, and induces crest membrane disintegration on sorafenib treatment (Fig.  2 I). Taken together, these results suggest that PLAG1 may serve as an attractive strategic target to boost ferroptosis in sorafenib's presence.

PLAG1 controls the occurrence of sorafenib-induced ferroptosis through a PVT1-dependent mechanism with no change in protein stability

Upon verification of the efficacy of PLAG1 knockdown and overexpression, it was observed that the protein and RNA levels of PLAG1 induced by sorafenib exhibited contrasting expression patterns. To elucidate this phenomenon, we initially investigated the impact of sorafenib on the stability of PLAG1 protein through CHX and MG132 experiments (Fig.  3 A and B ). The results showed that sorafenib treatment did not lead to a decrease in the protein stability of PLAG1. Furthermore, an investigation was conducted on the impact of sorafenib on PLAG1 ubiquitination, which resulted in the observation of no substantial changes in the ubiquitination status of PLAG1 after sorafenib administration((Fig.  3 C).

PLAG1 regulates lipid ROS, Fe 2+ and MDA in a PVT1-dependent manner in HCC cells treated with sorafenib. The dependence of Sorafenib-induced PLAG1 expression on PVT1 was investigated in this study. ( A - B ) CHX and MG132 experiments showed that sorafenib induction had no effect on the protein stability of PLAG1. At the same time, PLAG1 ubiquitination experiments showed that the sorafenib experiment did not change the level of PLAG1 ubiquitination ( C ), and we speculated that post-transcriptional regulation may affect the difference in the expression trend of PLAG1 protein and mRNA. ( D ) The construction of the ceRNETs was illustrated in the flowchart. In this network, lncRNA is visually represented by the color green, mRNA by the color light purple, and miRNA by the color pink. Also we observed that PVT1 acts as a ceRNA for PLAG1. ( E ) The subcellular localization of PVT1 was detected in the cytosol using the FISH experiment, with a scale bar of 50 μm. MHCC97H and MHCC97L cells were transfected with indicator constructs and subsequently treated with either DMSO or sorafenib (10 μM) for a duration of 24 h. The mRNA levels of PVT1 were quantified using qRT-PCR ( F ), while the protein levels of PLAG1 were assessed through western blot analysis ( G ). qRT-PCR results showed that PVT1 was able to reverse the effect of sorafenib on PLAG1 at the mRNA level ( H ). Additionally, Spearman's correlation analysis was employed to determine the relationship between PVT1 mRNA levels and PLAG1 mRNA levels in patients with HCC ( I ). Enhancing the downregulation of PLAG1 promoted sorafenib-induced ferroptosis, while this effect was counteracted by upregulating PVT1. Genetically modified MHCC97H and MHCC97L cells were treated with sorafenib (10 μM) for a duration of 24 h. The viability of the cells was assessed using the CCK-8 kit ( J ), the formation of lipids was measured through the MDA assay ( K ), the concentration of GSH in the cells was determined using a glutathione assay kit ( L ), the accumulation of lipid ROS was analyzed by flow cytometry using BODIPY-C11 probes ( M ), and the concentration of intracellular Fe 2+ was analyzed using FerroOrange probes ( N ). Downregulation of PVT1 rescued the inhibition of ferroptotic events caused by overexpression of PLAG1, which included reduction in cell viability ( O ), MDA levels ( P ), GSH depletion ( Q ), lipid ROS ( R ), and intracellular Fe 2+ ( S ). The data displayed indicate mean ± SD obtained from three independent experiments. * P  < 0.05; ** P  < 0.01; *** P  < 0.001

ceRNAs are transcripts that interact with each other at the post-transcriptional level by competing for shared miRNAs [ 32 ]. These miRNAs, typically 21 ~ 23 nucleotides in length, guide the AGO protein to bind to target transcripts through base pairing, ultimately leading to either degradation or translational inhibition [ 33 , 34 ]. Competitive endogenous RNA networks (ceRNETs) facilitate the interaction between protein-encoding mRNAs and non-coding RNAs, thereby potentially influencing post-transcriptional gene expression regulation in various biological states. Utilizing data from the TCGA database, we developed ceRNETs specific to hepatocellular carcinoma, revealing PVT1 as a potential ceRNA for PLAG1 through identification of central network nodes (Fig.  3 D). In addition, our FISH analysis confirmed that PVT1 is mainly localized in the cytoplasm and the expression of PVT1 is upregulated in a concentration-dependent manner by sorafenib (Fig.  3 E and and Supplementary Fig.  2 ).

The role of PVT1 in the pathogenesis of digestive system tumors, including promoting proliferation, metastasis, and autophagy of hepatoma cells, is well-established [ 35 , 36 , 37 ]. However, the impact of PVT1 on PLAG1 regulation remains unclear. We aimed to address this by using the MHCC97H/sh-PVT1 and MHCC97L/PVT1 cell lines in our study. Our qRT-PCR results indicated that sorafenib did not significantly affect PVT1 knockdown or overexpression efficacy, nor did it affect PVT1's positive regulation of PLAG1 as confirmed by western blotting (Fig.  3 F and G). Further, we also found that PVT1 was able to reverse the effect of sorafenib on PLAG1 at the transcriptional level (Fig.  3 H), and a positive correlation was identified between PVT1 and PLAG1 at the transcriptional level using Spearman's correlation analysis (F i g.  3 I, P  < 0.001, r  = 0.500). These suggests that PVT1 may be an upstream factor regulating the upregulation of PLAG1 mRNA expression induced by sorafenib. And that's not all, we also found overexpression of PVT1 partially offset sorafenib-induced cell death due to PLAG1 deficiency in the MHCC97H cell line in a rescue experiment (Fig.  3 J). Correspondingly, PVT1 overexpression mitigated sorafenib-induced MDA, lipid ROS, and Fe 2+ production, whilst boosting GSH levels (Fig.  3 K-N). Conversely, PVT1 inhibition increased ferroptotic processes, such as sorafenib-induced cell death and enhanced MDA, lipid ROS, and Fe 2+ production, whilst reducing GSH levels (Fig.  3 O-S). Overall, this indicates that PLAG1 suppresses sorafenib-induced ferroptosis through a PVT1-dependent mechanism.

The interaction between miR-195-5p and PVT1 or PLAG1 takes place

PVT1 has already been demonstrated to play a significant role in tumor growth via the ceRNA mechanism [ 38 , 39 ]. By analyzing the constructed ceRNA network, we identified the specific miRNAs linked with PVT1. We observed miR-195-5p increased with PVT1 suppression in combination with sorafenib treatment, contrary to a drop in miR-195-5p levels when PVT1 was overexpressed (Fig.  4 A and B ). Further analysis of RNA-seq data from the TCGA database showed a negative correlation between miR-195-5p and PVT1 levels in 374 individuals with HCC (Fig.  4 C, P  < 0.001, r  =  − 0.565). In addition to data analysis of TCGA-LIHC, we also discovered diminished miR-195-5p expression in tumor tissues compared to peri-tumor tissues through cDNA microarray (Fig.  4 D, E and Supplementary Fig.  3 A). The finding suggested a possible regulation by PVT1 of miR-195-5p controlling PLAG1.

Both PVT1 and PLAG1 are interacted with miR-195-5p. The qRT-PCR examination of miR-20a-5p, miR-195-5p, miR-17-5p, miR-106b-5p, and miR-93-5p was conducted on MHCC97H cells ( A ) and MHCC97L cells ( B ) that were transfected with the indicator constructs and subsequently subjected to treatment with DMSO or sorafenib (10 μM) for a duration of 24 h. ( C ) Spearman's correlation investigation was performed to analyze the relationship between miR-195-5p expression and PVT1 expression using data from TCGA-LIHC. ( D ) Comparison of miR-195-5p expression levels in HCC and normal tissues using cDNA microarray analysis. ( E ) The levels of miR-195-5p mRNA expression in 21-paired tumor and peritumoral tissues obtained from patients with HCC were measured using the qRT-PCR method. The data were presented as − ΔΔCt values. ( F ) The impact of miR-195-5p on the protein level of PLAG1 was evaluated using western blot analysis, with the use of DMSO or sorafenib (10 μM) for a duration of 24 h. ( G ) Furthermore, a correlation analysis was conducted to investigate the association between the expression of miR-195-5p and PLAG1 in patients diagnosed with HCC. ( H ) Western blot analysis provided evidence of a reciprocal regulatory effect between PVT1 and miR-195-5p on the expression of PLAG1. The binding sites of miR-195-5p with PVT1 ( I ) or PLAG1 ( J ) were predicted using a public database. HEK293T cells were transfected with WT vectors or MUT vectors, along with miR-195-5p mimics and miR-NC, and luciferase activity was measured. ( K ) The expression of AGO2 protein was detected in MHCC97H cells using western blot analysis, while PVT1 ( L ) and PLAG1 ( M ) were detected using qRT-PCR. The data displayed indicate mean ± SD obtained from three independent experiments. * P  < 0.05; ** P  < 0.01; *** P  < 0.001; n.s., not significant

Our subsequent findings revealed that the introduction of miR-195-5p mimics led to a decrease in PLAG1 protein levels in MHCC97H cells, while miR-195-5p inhibitors increased PLAG1 expression in MHCC97L cells. This regulatory pattern remained consistent during sorafenib treatment (Fig.  4 F). Spearman’s correlation analysis showed a strong negative correlation between miR-195-5p and PLAG1 (Fig. 4 G, P  < 0.001, r  =  − 0.515). Further, we found that PVT1 partially mitigates the suppressive effect of miR-195-5p on PLAG1 abundance and could partially reverse the enhancing effect of PVT1 on PLAG1 expression in HCC cells (Fig.  4 H). These findings indicated that PLAG1 could be a target of the PVT1/miR-195-5p axis.

We then explored a direct interaction between miR-195-5p and PVT1 or PLAG1 messenger RNA. Interestingly, analysis indicated that miR-195-5p could bind to the 3' UTR region of both PVT1 and PLAG1 (Fig. 4I, J and Supplementary Fig.  3 B and C). This information uncovers miR-195-5p as a downstream target of PVT1. The AGO2 protein, an integral component of the RNA-induced silencing complex, plays a pivotal role in facilitating the degradation of target mRNA or impeding its protein synthesis within the miRNA/siRNA pathway [ 40 ]. To ascertain the competition between PVT1 and PLAG1 in the binding of miR-195-5p within HCC cells, the anti-AGO2 RIP assay was employed to demonstrate AGO2's efficient capture by anti-AGO2 antibodies in MHCC97H cells transfected with miR-195-5p mimics (Fig.  4 K), and suggested competition between PVT1 and PLAG1 for miR-195-5p binding within HCC cells. Increased levels of PVT1 and PLAG1 were then detected in the MHCC97H cell group transfected with miR-195-5p mimics compared to the control group, revealing that PVT1 may serve as a molecular sponge for miR-195-5p to regulate PLAG1 (Fig.  4 L and M).

MiR-195-5p reverses the impact of PLAG1 on ferroptosi.

In rescue experiments using miR-195-5p inhibitors and mimics, we found that miR-195-5p inhibitor use led to a decrease in sorafenib-induced cell death in MHCC97H/sh-PLAG1 cells and also inhibited the accumulation of MDA and lipid ROS triggered by PLAG1 depletion (Fig.  5 A, B and D). Additionally, miR-195-5p suppression intensified the drop in GSH levels following PLAG1 knockdown (Fig.  5 C). A contrast was observed with the upregulation of PLAG1 in MHCC97L cells which countered the sorafenib-induced ferroptotic events suppressed by miR-195-5p overexpression (Fig.  5 E-H). Such data indicate that PLAG1's influence on sorafenib-induced ferroptosis is regulated by miR-195-5p levels.

The inhibition of ferroptosis is observed through the regulation of PLAG1 by miR-195-5p. The inhibition of PLAG1 resulted in an augmentation of sorafenib-induced ferroptosis, which was counteracted by the miR-195-5p inhibitor. Cell viability was assessed using the CCK-8 kit ( A ) following sorafenib treatment in MHCC97H cells transfected with the construct. Lipid formation was measured using the MDA assay ( B ), GSH concentration was measured using the glutathione assay kit (C), and lipid ROS accumulation was measured using BODIPY-C11 probes ( D ). In MHCC97L cells treated with sorafenib (10 μM) for 24 h, the overexpression of PLAG1 suppressed ferroptotic events, such as lipid ROS (E), cell viability (F), MDA levels (G), and GSH depletion (H), while the introduction of miR-195-5p mimics rescued these effects. The data presented represent the mean ± SD obtained from three independent experiments. Statistical significance was determined using * P  < 0.05, ** P  < 0.01, and *** P  < 0.001

GPX4 functions as the downstream effector of PLAG1 to influence ferroptosis inhibition

The GSEA enrichment analysis demonstrated that the co-expression of PLAG1 with differentially expressed genes is associated with a regulatory role in the oxidation–reduction reaction (Supplementary Fig.  4 A). Our study investigated the role of PLAG1 in enhancing GPX4 expression in HCC, given GPX4's regulatory function in cellular antioxidant responses. Initially, we examined the correlation between PLAG1 and GPX4 in tissues obtained from HCC patients. In order to establish a correlation between PLAG1 and GPX4, IHC analysis was performed on a total of 139 samples of tumor and peritumoral tissues obtained from the HCC patient cohort. The distribution of PLAG1 and GPX4 was observed in both the cytoplasm and nucleus of HCC cells and peritumoral hepatocytes. The expression patterns of PLAG1 and GPX4 in the tumor tissues of the four groups were depicted in Fig.  6 A, with a score of four or higher indicating positive or high expression, in conjunction with the intensity and diversity of IHC staining. According to the IHC scoring, the levels of PLAG1 in tumor tissues were significantly higher compared to those in peritumoral tissues (Fig.  6 B). Interestingly, 52% of tumor tissue samples with decreased PLAG1 levels also showed reduced GPX4 levels, whereas 79% of samples with increased PLAG1 levels showed elevated GPX4 expression (Fig.  6 C). This finding strongly suggests a direct association between the two in HCC tissues and is backed by the TCGA database analysis which depicted a positive correlation between PLAG1 mRNA and GPX4 mRNA (Supplementary Fig.  4 B). Further experiments in liver cancer cells validated this, showing a significant decrease in GPX4 levels in MHCC97H/shPLAG1 cells when compared to the control group. Conversely, an upsurge in these levels was observed in MHCC97L/PLAG1 cells (Fig.  6 D and E). It appears that the rise in GPX4 protein in relation to PLAG1 expression could be due to transcriptional regulatory mechanisms.

GPX4 functions as the downstream target of PLAG1. The expression of PLAG1 shows a positive correlation with the expression of GPX4 in HCC tissues. (A) Representative figures of the expression patterns of PLAG1 and GPX4 in the indicated groups. Scale bar: 100 μm. (B) IHC score of PLAG1 in 139 HCC samples and peritumoral samples were compared. (C) The percentages of tumor tissues with high or low levels of GPX4 expression in individuals with high or low levels of PLAG1 expression among a total of 139 HCC samples. In MHCC97H treated with either DMSO or sorafenib (10 µM) for 24 h, the protein and mRNA expression levels of GPX4 were reduced by the knockdown of PLAG1. Conversely, the overexpression of PLAG1 in MHCC97L led to an increase in both the mRNA and protein expression levels of GPX4 (D, E). The intracellular Fe 2+ was detected by the FerroOrange probe, scale bar = 50 μm (F, G). And lipid ROS accumulation was analyzed by C11-BODIPY 581/591 probe, scale bar = 50 μm (H, I). The data presented represent the mean ± SD obtained from three independent experiments. Statistical significance was determined using * P  < 0.05, ** P  < 0.01, and *** P  < 0.001

Lipid ROS and Fe 2+ are two important downstream indicators of GPX4 inhibition of ferroptosis. We further assessed their level in sorafenib-treated MHCC97H/sh-PLAG1 and MHCC97L/PLAG1 through FerroOrange and C11 BODIPY 581/591 probe. We found that overexpression of GPX4 could reverse the upregulation of Fe 2+ and lipid ROS promoted by PLAG1 knockdown, while GPX4 interference rescued the downregulation of Fe 2+ and lipid ROS caused by overexpression of PLAG1 (F i g.  6 F-I). In short, our results verified the crosstalk between PLAG1 and GPX4 in ferroptosis inhibition.

The mapping of chromatin state identifies the presence of PLAG1 in the promoter region of GPX4

We undertook a reanalysis of ChIP-seq data to understand the mechanism through which PLAG1 maintains GPX4 gene expression and discovered that around 25% selected by PLAG1 were within 3 kb of the transcription start site (TSS) (Fig.  7 A and Supplementary Fig.  5 A-C). H3K4me3 is mainly enriched in the promoter region near the TSS, while most H3K4me1 and H3K27ac modifications are enriched in the enhancer region [ 41 ]. All identified TSS had the H3K4me3 marker, associated with active promoters, but lacked active enhancer markers H3K27ac and H3K4me1(Fig.  7 A and B). However, regions marked with H3K4me1 didn't have PLAG1 protein and H3K4me3 ( Fig.  7 B ) . Through a comprehensive analysis of the enrichment patterns of PLAG1, H3K4me3, H3K4me1, and H3K27ac in the 3 kb region flanking the TSS of GPX4, it is hypothesized that the transcription factor PLAG1 interacts with the GPX4 promoter to enhance GPX4 transcription.

The mapping of chromatin state identifies the presence of PLAG1 in the promoter region of GPX4. ( A ) The distribution of PLAG1 peaks and the heatmaps displaying PLAG1, H3K27ac, H3K4me1, and H3K4me3 at regulatory elements in close proximity to HepG2 cells are depicted on the left and right sides, respectively. ( B ) The signals of PLAG1, H3K27me3, H3K27ac, and H3K27me1 at the peaks of PLAG1, H3K27me1, and H3K27ac are illustrated on the left, middle, and right sides, respectively. ( C ) Additionally, gene traces for PLAG1, H3K4me3, H3K27Ac, and H3K4me1 ChIP-seq presence at the promoter of GPX4 in HepG2 cells are shown. The ChIP-seq coverage is expressed as reads per million per base pair (rpm/bp). ( D ) The metagene analysis illustrates the global presence of PLAG1, H3K4me3, H3K27ac, and H3K4me1 at the promoters encompassing the transcription start site (TSS) and enhancer regions. ( E ) The presence of PLAG1, H3K4me3, H3K27ac ChIP-seq data, and ATAC-seq data in the vicinity of the GPX4 locus was examined in HepG2 cells, revealing a prominent Hi-C profile. Chromatin looping in HepG2 cells brings the distal GPX4 region closer to its target promoter, as indicated by the arrows. ( F ) The binding motif of PLAG1 and the top seven binding sites in the GPX4 promoter were identified using JASPAR online tools. ( G ) A diagram is presented to illustrate the truncation pattern based on the spatial distribution of the binding sites. ( H ) The results obtained from DNA gel electrophoresis indicate that the chromatin fragments treated with micrococcal nuclease were predominantly observed within the range of 200 to 500 bp. ( I ) In order to investigate the binding of PLAG1 to the P2 truncator of the GPX4 promoter region (-2000 bp ~  + 99 bp) in MHCC97H, ChIP assays and qRT-PCR were conducted. The levels of truncated genomic DNA containing the predicted binding sequence in MHCC97H cells were visualized using DNA gel electrophoresis. ( J ) In order to determine the strength of RNA polymerase II (Pol II) binding to the central region of the GPX4 promoter (-250 bp to 120 bp) in MHCC97H, ChIP assays and qRT-PCR were performed. The presence of gDNA containing the core region of the GPX4 promoter in MHCC97H cells was visualized using DNA gel electrophoresis. ( K ) To investigate the specific binding sequence between PLAG1 and the P2 truncator of the GPX4 promoter, mutant groups were constructed based on the predicted results of the JASPAR database and the luciferase reporter assay was utilized to identify changes in luciferase intensity relative to controls in the case of PLAG1 overexpression. The data presented represent the mean ± SD obtained from three independent experiments. Statistical significance was determined using * P  < 0.05, ** P  < 0.01, and *** P  < 0.001

In eukaryotic cells, the formation of chromatin complexes can result in tightly folded structures, allowing for potential interactions between distant regions on the DNA sequence [ 42 ]. This interaction increases the likelihood of enhancers binding to transcription factors and RNA Pol II. Figure  7 C and D revealed the co-occupation of PLAG1 and H3K4me3 in the promoter region of GPX4. Furthermore, the Hi-C profile suggested that the enhancers of GPX4 may interact with the promoter via a "loop" mechanism, aiding transcription activation (Fig.  7 E).

The Jaspar database was employed to study the potential direct interaction between PLAG1 and GPX4 promoter elements, revealing PLAG1's key binding patterns' ability to interact with the GPX4 promoter (Fig.  7 F and G ). Crosslinked chromatin fragments were treated with nucleic acid micrococcal nuclease and sonication techniques resulting in chromatin fragments from 200 to 500 bp in size (Fig.  7 H). The ChIP assay strongly associates PLAG1 with the P2 region (-1300 to -499 bp) of GPX4 promoter (F i g.  7 I). Additionally, a significant enrichment was observed in the Pol II group within the GPX4's central promoter region, suggesting GPX4 transcription activation (Fig.  7 J). When co-transfected with P2 region reporter plasmids in MHCC97H/PLAG1 cells, there was a significant increase of luciferase activity in the WT and MUT2 groups (Fig.  7 K), suggesting the region between -933 bp and -919 bp as PLAG1's confirmed binding site. This evidence firmly links PLAG1's role in GPX4 gene expression.

The effectiveness of sorafenib is enhanced by inhibiting PLAG1 in vivo

The effectiveness of Sorafenib in inhibiting tumor growth was assessed in subcutaneous and orthotopic mouse models, in conjunction with PLAG1 level reduction. Our results showed that sorafenib had a higher efficacy in reducing tumor volume in MHCC97H/sh-PLAG1 cells than the control group (Fig.  8 A and B). The mRNA levels of PLAG1 were analyzed in 21 sorafenib responders and 46 sorafenib non-responders with HCC from the GSE109211 dataset. The findings revealed that the mRNA levels of PLAG1 were significantly elevated in sorafenib responders compared to sorafenib non-responders (Fig.  8 C). This observation aligns with our prior in vitro investigations, wherein treatment of HCC cells with sorafenib led to an increase in PLAG1 mRNA expression.

The effectiveness of sorafenib in vivo is enhanced by inhibiting PLAG1. ( A ) MHCC97H cells were cultured and subsequently injected subcutaneously into 4-week-old BALB/c nude mice. Once the mice's tumor volume reached 50 mm 3 , sorafenib was administered intragastrically at a dosage of 10 mg/kg per day. The tumor size was measured every other day until the 14th day, at which point the mice were sacrificed. ( B ) BALB/c nude mice were implanted with 5 mm 3 tumor tissue derived from either MHCC97H/sh-NC or MHCC97H/sh-PLAG1 cells in situ. After one week of implantation, the respective groups were administered sorafenib (10 mg/kg, once daily) via intragastric administration. After 21 days of implantation, the mice were euthanized, and their livers were dissected to measure tumor volumes. ( C ) External data from GSE109211 examined the response of PLAG1 mRNA levels to sorafenib. ( D ) The provided representative IHC images of PLAG1, GPX4, and 4-HNE in tumor tissues from the specified orthotopic models were shown. The scale bar represents 100 μm. ( E ) The statistical results of GPX4 and 4-HNE were presented, indicating mean ± SD obtained from three independent trials. * P  < 0.05; ** P  < 0.01; *** P  < 0.001; n.s., not significant

Additionally, reduced GPX4 expression levels were observed in tumors derived from MHCC97H/sh-PLAG1 cells compared to the control. Moreover, sorafenib also significantly reduced GPX4 signals in tumor tissues compared to vehicle-treated groups (Fig.  8 D and E). To investigate potential ferroptosis, we stained the experimental tumor tissue with 4-HNE, a known ferroptosis indicator. The result showed increased 4-HNE levels in tumors from MHCC97H/sh-PLAG1 cells treated with sorafenib, in comparison to the control group (Fig.  8 D and E). These findings suggest that PLAG1 inhibition can amplify the effectiveness of sorafenib in treating animal tumors via GPX4 downregulation and ferroptosis promotion.

Addressing the high mortality rate and poor prognosis associated with liver cancer poses considerable challenges, primarily due to limited treatment options for advanced disease. It underlines the urgent need for identifying new therapeutic targets. Our study indicates PLAG1 as a potential therapeutic target, given its increased expression levels in liver cancer tissues compared to normal ones. Utilizing the TCGA database, we found a correlation between high PLAG1 expression and reduced survival rates among liver cancer patients. While the drug sorafenib has revolutionized the treatment of advanced liver cancer, its efficacy is hampered by tumor heterogeneity and acquired drug resistance. Interestingly, our study observed that PLAG1 expression decreases with increasing dosages of sorafenib. Additionally, PLAG1 overexpression hinders the therapeutic effects of sorafenib on hepatoma cells. These results highlight the critical role of PLAG1 in sorafenib's therapeutic efficacy.

Ferroptosis is an iron-dependent form of cell death that has been linked to various metabolic and homeostasis disorders [ 30 ]. Recent studies show its role in the progression of several diseases including stroke, autoimmune disorders, and cancers such as renal, leukemia, pulmonary, and liver carcinoma [ 43 ]. This discovery presents a promising strategy in cancer research, encouraging the demise of cancer cells through ferroptosis. One particular observation is that sorafenib, a known cancer drug, can trigger ferroptosis in liver cancer treatment by blocking cystine intake which reduces GSH levels and causes accumulation of lipid ROS—ultimately inducing ferroptosis [ 44 ]. Previous investigations have provided evidence that sorafenib possesses the capability to induce the onset of ferroptosis in the treatment of liver cancer [ 6 , 7 ], thus providing the possibility of confirming the inverse regulatory relationship between PLAG1 and ferroptosis. In this investigation, it was ascertained that the suppression of PLAG1, an oncoprotein, enhances the effects of sorafenib. More specifically, MHCC97H/sh-PLAG1 cells exhibited increased lipid peroxidation, decreased intracellular GSH, increased cellular contractions, and an elevation in mitochondrial membrane density. In addition, we also found that PLAG1 exhibited higher knockdown/overexpression efficiency in the sorafenib-induced group compared with the DMSO group. This not only makes up for the shortcomings of the relatively low knockdown/overexpression efficiency of the DMSO group, but also provides favorable support for the later study of sorafenib related ferroptotic events. In short, this suggests that PLAG1 plays a significant role in regulating cellular death and could provide insight into new cancer treatment strategies.

Ferroptosis has been closely linked to non-coding RNA (ncRNA) and cancer [ 45 ]. These ncRNAs are implicated in the fundamental regulatory processes of ferroptosis, encompassing mitochondria-associated proteins, iron metabolism, glutathione metabolism, and lipid peroxidation [ 46 , 47 ]. Recent research has increasingly demonstrated the significant involvement of ncRNAs in the biological functionality of ceRNETs in liver cancer, influencing the binding affinity of miRNAs to target RNAs through the sharing of miRNA response elements [ 32 ]. PVT1, a major oncogenic long non-coding RNA in the gastrointestinal tract, is known to regulate the progression and drug resistance of hepatocellular carcinoma. It modulates downstream genes and biological pathways, impacting various human diseases, including cancer. In particular, PVT1 has been found to control TFR1 and p53 through its function as a miR-214 sponge, suppressing ferroptosis in cases of cerebral ischemia–reperfusion injury [ 48 ]. This was a first-time discovery linking PVT1 to ferroptosis. Later, research by He et al. [ 49 ] found that PVT1 interacts directly with miR-214-3p, preventing it from inhibiting GPX4. However, our understanding of PVT1's role in ferroptosis remains incomplete given the divergent phenotypic outcomes observed. Our study revealed that PVT1 counteracts the suppressive effect of sorafenib on PLAG1, and it enhances PLAG1 expression through its interaction with miR-195-5p. This underlines PLAG1's central role in the regulatory network.

GPX4 is a potential biomarker for ferroptosis, playing a key role in handling LPO levels [ 50 , 51 ]. Its inactivation can imbalance oxidative processes, disturb membrane structures, and trigger ferroptosis, making GPX4 an attractive therapeutic target. Our GSEA enrichment analysis on the TCGA database indicates a strong connection between PLAG1 co-expression genes and redox reactions, suggesting GPX4 could be an influential mediator of PLAG1 in liver cancer. Simultaneously, our observations revealed that sorafenib induced a decrease in GPX4 protein levels, diverging from its RNA expression pattern. Consistent with findings by He et al. [ 49 ], our rescue experiments demonstrated that PVT1 can counteract the up-regulation of GPX4 mRNA levels (Supplementary Fig.  6 ), suggesting a regulatory influence of PVT1 on both PLAG1 and GPX4 at the upstream level.

Previous studies have observed decreased methylation in the GPX4 promoter region and increased levels of H3K4me3 and H3K27ac upstream of GPX4 in various tumor cells [ 52 ], indicating a potential link between heightened GPX4 expression, decreased methylation, and increased histone acetylation in tumors. Our follow-up study found that PLAG1 interacts with the GPX4 promoter and enhances its transcriptional activity in the MHCC97H cell line. Using a truncation assay and analyzing dual-luciferase reporter gene results, we identified region P2/ − 1300 bp ~  − 499 bp as the main area for PLAG1 and GPX4 promoter interaction based on its highest predicted binding score in the JASPRA database.

In summary, our research has clarified the role of PLAG1 in sorafenib-induced ferroptosis. We discovered that PLAG1 downregulation enhances ferroptosis through the regulation of the PVT1/miR-195-5p axis. Additionally, the transcription factor PLAG1, stimulated by upstream factors, interacts with and promotes the transcription of target GPX4, affecting cell death induced by sorafenib treatment in both in vivo and in vitro models (Fig.  9 ). These findings highlight that PLAG1 interacts with GPX4 to conquer vulnerability to sorafenib induced ferroptosis through a PVT1/miR-195-5p axis-dependent manner and reinforce the potential of PLAG1 as a therapeutic target for sorafenib in the treatment of HCC.

Schematic diagram of the present study. In HCC cells, PVT1 acts as a sponge for miR-195-5p, thereby upregulating the expression of PLAG1. Consequently, the increased levels of PLAG1 enhance the expression of GPX4, leading to the inhibition of the ferroptosis signaling pathway

Availability of data and materials

RNA-seq data were deposited into the GEO database ( https://www.ncbi.nlm.nih.gov/geo ) under accession number GSE182185 and GSE109211. The Hi-C and ChIP-seq data were also downloaded from GEO datasets GSE184796 and GSE151287. TCGA-LIHC cohort are available at the following URL: https://portal.gdc.cancer.gov/ .

Abbreviations

• Hepatocellular carcinoma

Regulated cell death

Pleiomorphic adenoma gene 1

Glutathione peroxidase 4

Lipid peroxides

Plasmacytoma variant translocation 1

Hypoxia-inducible factor 1

Glutathione

Quantitative real‑time polymerase chain reaction

Immunohistochemistry

Histological score

Horseradish peroxidase

Dimethyl sulfoxide

Malondialdehyde

Thiobarbituric acid

Reactive oxygen species

Normal control

Polymerase II

Chromatin immunoprecipitation

Competing endogenous RNA

Mutant type

RNA immunoprecipitation

Fluorescence in situ hybridization

Topologically associated domains

Transcription start site

Sung H, Ferlay J, Siegel RL, Laversanne M, Soerjomataram I, Jemal A, et al. Global Cancer Statistics 2020: GLOBOCAN Estimates of Incidence and Mortality Worldwide for 36 Cancers in 185 Countries. CA Cancer J Clin. 2021;71:209–49.

Article   PubMed   Google Scholar  

Dhanasekaran R, Nault JC, Roberts LR, Zucman-Rossi J. Genomic Medicine and Implications for Hepatocellular Carcinoma Prevention and Therapy. Gastroenterology. 2019;156:492–509.

Article   CAS   PubMed   Google Scholar  

Cheng Z, Wei-Qi J, Jin D. New insights on sorafenib resistance in liver cancer with correlation of individualized therapy. Biochim Biophys Acta Rev Cancer. 2020;1874: 188382.

Vogel A, Saborowski A. Current strategies for the treatment of intermediate and advanced hepatocellular carcinoma. Cancer Treat Rev. 2020;82: 101946.

Forner A, Llovet JM, Bruix J. Hepatocellular carcinoma. Lancet. 2012;379:1245–55.

Li Y, Yang W, Zheng Y, Dai W, Ji J, Wu L, et al. Targeting fatty acid synthase modulates sensitivity of hepatocellular carcinoma to sorafenib via ferroptosis. J Exp Clin Cancer Res. 2023;42:6.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Lu Y, Chan YT, Tan HY, Zhang C, Guo W, Xu Y, et al. Epigenetic regulation of ferroptosis via ETS1/miR-23a-3p/ACSL4 axis mediates sorafenib resistance in human hepatocellular carcinoma. J Exp Clin Cancer Res. 2022;41:3.

Louandre C, Ezzoukhry Z, Godin C, Barbare JC, Maziere JC, Chauffert B, et al. Iron-dependent cell death of hepatocellular carcinoma cells exposed to sorafenib. Int J Cancer. 2013;133:1732–42.

Zheng Y, Xu L, Hassan M, Zhou X, Zhou Q, Rakheja D, et al. Bayesian Modeling Identifies PLAG1 as a Key Regulator of Proliferation and Survival in Rhabdomyosarcoma Cells. Mol Cancer Res. 2020;18:364–74.

Huang W, Li BR, Feng H. PLAG1 silencing promotes cell chemosensitivity in ovarian cancer via the IGF2 signaling pathway. Int J Mol Med. 2020;45:703–14.

CAS   PubMed   PubMed Central   Google Scholar  

Wang L, Sun L, Liu R, Mo H, Niu Y, Chen T, et al. Long non-coding RNA MAPKAPK5-AS1/PLAGL2/HIF-1alpha signaling loop promotes hepatocellular carcinoma progression. J Exp Clin Cancer Res. 2021;40:72.

Yuan S, Wei C, Liu G, Zhang L, Li J, Li L, et al. Sorafenib attenuates liver fibrosis by triggering hepatic stellate cell ferroptosis via HIF-1alpha/SLC7A11 pathway. Cell Prolif. 2022;55: e13158.

Van Dyck F, Declercq J, Braem CV, Van de Ven WJ. PLAG1, the prototype of the PLAG gene family: versatility in tumour development (review). Int J Oncol. 2007;30:765–74.

PubMed   Google Scholar  

Liu Y, Wan Y, Jiang Y, Zhang L, Cheng W. GPX4: The hub of lipid oxidation, ferroptosis, disease and treatment. Biochim Biophys Acta Rev Cancer. 2023;1878: 188890.

Bannai S, Kitamura E. Transport interaction of L-cystine and L-glutamate in human diploid fibroblasts in culture. J Biol Chem. 1980;255:2372–6.

Ju HQ, Lu YX, Wu QN, Liu J, Zeng ZL, Mo HY, et al. Disrupting G6PD-mediated Redox homeostasis enhances chemosensitivity in colorectal cancer. Oncogene. 2017;36:6282–92.

Li R, Qu H, Wang S, Wei J, Zhang L, Ma R, et al. GDCRNATools: an R/Bioconductor package for integrative analysis of lncRNA, miRNA and mRNA data in GDC. Bioinformatics. 2018;34:2515–7.

Shannon P, Markiel A, Ozier O, Baliga NS, Wang JT, Ramage D, et al. Cytoscape: a software environment for integrated models of biomolecular interaction networks. Genome Res. 2003;13:2498–504.

Jeggari A, Marks DS, Larsson E. miRcode: a map of putative microRNA target sites in the long non-coding transcriptome. Bioinformatics. 2012;28:2062–3.

Yang JH, Li JH, Shao P, Zhou H, Chen YQ, Qu LH. starBase: a database for exploring microRNA-mRNA interaction maps from Argonaute CLIP-Seq and Degradome-Seq data. Nucleic Acids Res. 2011;39:D202–9.

Furio-Tari P, Tarazona S, Gabaldon T, Enright AJ, Conesa A. spongeScan: A web for detecting microRNA binding elements in lncRNA sequences. Nucleic Acids Res. 2016;44:W176–80.

Cao J, Chen X, Jiang L, Lu B, Yuan M, Zhu D, et al. DJ-1 suppresses ferroptosis through preserving the activity of S-adenosyl homocysteine hydrolase. Nat Commun. 2020;11:1251.

Love MI, Huber W, Anders S. Moderated estimation of fold change and dispersion for RNA-seq data with DESeq2. Genome Biol. 2014;15:550.

Article   PubMed   PubMed Central   Google Scholar  

Li H, Durbin R. Fast and accurate short read alignment with Burrows-Wheeler transform. Bioinformatics. 2009;25:1754–60.

Durand NC, Shamim MS, Machol I, Rao SS, Huntley MH, Lander ES, et al. Juicer Provides a One-Click System for Analyzing Loop-Resolution Hi-C Experiments. Cell Syst. 2016;3:95–8.

Zhang L, Li XM, Shi XH, Ye K, Fu XL, Wang X, et al. Sorafenib triggers ferroptosis via inhibition of HBXIP/SCD axis in hepatocellular carcinoma. Acta Pharmacol Sin. 2023;44:622–34.

Xu FL, Wu XH, Chen C, Wang K, Huang LY, Xia J, et al. SLC27A5 promotes sorafenib-induced ferroptosis in hepatocellular carcinoma by downregulating glutathione reductase. Cell Death Dis. 2023;14:22.

Yang WS, Stockwell BR. Ferroptosis: Death by Lipid Peroxidation. Trends Cell Biol. 2016;26:165–76.

Yang WS, SriRamaratnam R, Welsch ME, Shimada K, Skouta R, Viswanathan VS, et al. Regulation of ferroptotic cancer cell death by GPX4. Cell. 2014;156:317–31.

Tang D, Chen X, Kang R, Kroemer G. Ferroptosis: molecular mechanisms and health implications. Cell Res. 2021;31:107–25.

Stockwell BR. Ferroptosis turns 10: Emerging mechanisms, physiological functions, and therapeutic applications. Cell. 2022;185:2401–21.

Salmena L, Poliseno L, Tay Y, Kats L, Pandolfi PP. A ceRNA hypothesis: the Rosetta Stone of a hidden RNA language? Cell. 2011;146:353–8.

Meister G. Argonaute proteins: functional insights and emerging roles. Nat Rev Genet. 2013;14:447–59.

Bartel DP. MicroRNAs: target recognition and regulatory functions. Cell. 2009;136:215–33.

Zhou DD, Liu XF, Lu CW, Pant OP, Liu XD. Long non-coding RNA PVT1: Emerging biomarker in digestive system cancer. Cell Prolif. 2017;50(6):12398.

Article   Google Scholar  

Yang L, Peng X, Jin H, Liu J. Long non-coding RNA PVT1 promotes autophagy as ceRNA to target ATG3 by sponging microRNA-365 in hepatocellular carcinoma. Gene. 2019;697:94–102.

Xiong X, Yuan J, Zhang N, Zheng Y, Liu J, Yang M. Silencing of lncRNA PVT1 by miR-214 inhibits the oncogenic GDF15 signaling and suppresses hepatocarcinogenesis. Biochem Biophys Res Commun. 2020;521:478–84.

Xie G, Zheng X, Zheng Z, Wu R, Yao Z, Huang W, et al. The ceRNA PVT1 inhibits proliferation of ccRCC cells by sponging miR-328-3p to elevate FAM193B expression. Aging (Albany NY). 2021;13:21712–28.

Chen M, Zhang R, Lu L, Du J, Chen C, Ding K, et al. LncRNA PVT1 accelerates malignant phenotypes of bladder cancer cells by modulating miR-194-5p/BCLAF1 axis as a ceRNA. Aging (Albany NY). 2020;12:22291–312.

Rivas FV, Tolia NH, Song JJ, Aragon JP, Liu J, Hannon GJ, et al. Purified Argonaute2 and an siRNA form recombinant human RISC. Nat Struct Mol Biol. 2005;12:340–9.

Kimura H. Histone modifications for human epigenome analysis. J Hum Genet. 2013;58:439–45.

Bulger M, Groudine M. Functional and mechanistic diversity of distal transcription enhancers. Cell. 2011;144:327–39.

Stockwell BR, Friedmann Angeli JP, Bayir H, Bush AI, Conrad M, Dixon SJ, et al. Ferroptosis: A Regulated Cell Death Nexus Linking Metabolism, Redox Biology, and Disease. Cell. 2017;171:273–85.

Dixon SJ, Patel DN, Welsch M, Skouta R, Lee ED, Hayano M, et al. Pharmacological inhibition of cystine-glutamate exchange induces endoplasmic reticulum stress and ferroptosis. Elife. 2014;3: e02523.

Dai E, Han L, Liu J, Xie Y, Kroemer G, Klionsky DJ, et al. Autophagy-dependent ferroptosis drives tumor-associated macrophage polarization via release and uptake of oncogenic KRAS protein. Autophagy. 2020;16:2069–83.

Jiang L, Kon N, Li T, Wang SJ, Su T, Hibshoosh H, et al. Ferroptosis as a p53-mediated activity during tumour suppression. Nature. 2015;520:57–62.

Xie Y, Zhu S, Song X, Sun X, Fan Y, Liu J, et al. The Tumor Suppressor p53 Limits Ferroptosis by Blocking DPP4 Activity. Cell Rep. 2017;20:1692–704.

Lu J, Xu F, Lu H. LncRNA PVT1 regulates ferroptosis through miR-214-mediated TFR1 and p53. Life Sci. 2020;260: 118305.

He GN, Bao NR, Wang S, Xi M, Zhang TH, Chen FS. Ketamine Induces Ferroptosis of Liver Cancer Cells by Targeting lncRNA PVT1/miR-214-3p/GPX4. Drug Des Devel Ther. 2021;15:3965–78.

Chen X, Li J, Kang R, Klionsky DJ, Tang D. Ferroptosis: machinery and regulation. Autophagy. 2021;17:2054–81.

Ursini F, Maiorino M. Lipid peroxidation and ferroptosis: The role of GSH and GPx4. Free Radic Biol Med. 2020;152:175–85.

Zhang X, Sui S, Wang L, Li H, Zhang L, Xu S, et al. Inhibition of tumor propellant glutathione peroxidase 4 induces ferroptosis in cancer cells and enhances anticancer effect of cisplatin. J Cell Physiol. 2020;235:3425–37.

Download references

This study was supported by the National Natural Science Foundation of China (Nos. 82170666).

Author information

Authors and affiliations.

Department of Hepatobiliary Surgery, the First Affiliated Hospital of Chongqing Medical University, Chongqing, 400010, China

Jiarui Li, Yilan Li, Denghui Wang, Rui Liao & Zhongjun Wu

You can also search for this author in PubMed   Google Scholar

Contributions

JL conceived the experiments, analyzed the data and wrote the manuscript. YL and DW performed the experiments. RL revised the manuscript. ZW supervised this work.

Corresponding author

Correspondence to Zhongjun Wu .

Ethics declarations

Ethics approval and consent to participate.

The research was approved by the Medical Ethic Committee of Chongqing Medical University.

Competing interests

The authors report no conflicts of interest in this work.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary material 1., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Li, J., Li, Y., Wang, D. et al. PLAG1 interacts with GPX4 to conquer vulnerability to sorafenib induced ferroptosis through a PVT1/miR-195-5p axis-dependent manner in hepatocellular carcinoma. J Exp Clin Cancer Res 43 , 143 (2024). https://doi.org/10.1186/s13046-024-03061-4

Download citation

Received : 18 April 2024

Accepted : 29 April 2024

Published : 14 May 2024

DOI : https://doi.org/10.1186/s13046-024-03061-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Ferroptosis

Journal of Experimental & Clinical Cancer Research

ISSN: 1756-9966

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

Help | Advanced Search

Computer Science > Computation and Language

Title: optimization techniques for sentiment analysis based on llm (gpt-3).

Abstract: With the rapid development of natural language processing (NLP) technology, large-scale pre-trained language models such as GPT-3 have become a popular research object in NLP field. This paper aims to explore sentiment analysis optimization techniques based on large pre-trained language models such as GPT-3 to improve model performance and effect and further promote the development of natural language processing (NLP). By introducing the importance of sentiment analysis and the limitations of traditional methods, GPT-3 and Fine-tuning techniques are introduced in this paper, and their applications in sentiment analysis are explained in detail. The experimental results show that the Fine-tuning technique can optimize GPT-3 model and obtain good performance in sentiment analysis task. This study provides an important reference for future sentiment analysis using large-scale language models.

Submission history

Access paper:.

• Other Formats

References & Citations

• Google Scholar
• Semantic Scholar

BibTeX formatted citation

Bibliographic and Citation Tools

Code, data and media associated with this article, recommenders and search tools.

• Institution

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs .