hypothesis feasibility study

• Correspondence
• Open access
• Published: 16 July 2010

What is a pilot or feasibility study? A review of current practice and editorial policy

• Mubashir Arain 1 ,
• Michael J Campbell 1 ,
• Cindy L Cooper 1 &
• Gillian A Lancaster 2  

BMC Medical Research Methodology volume  10 , Article number:  67 ( 2010 ) Cite this article

188k Accesses

788 Citations

30 Altmetric

Metrics details

In 2004, a review of pilot studies published in seven major medical journals during 2000-01 recommended that the statistical analysis of such studies should be either mainly descriptive or focus on sample size estimation, while results from hypothesis testing must be interpreted with caution. We revisited these journals to see whether the subsequent recommendations have changed the practice of reporting pilot studies. We also conducted a survey to identify the methodological components in registered research studies which are described as 'pilot' or 'feasibility' studies. We extended this survey to grant-awarding bodies and editors of medical journals to discover their policies regarding the function and reporting of pilot studies.

Papers from 2007-08 in seven medical journals were screened to retrieve published pilot studies. Reports of registered and completed studies on the UK Clinical Research Network (UKCRN) Portfolio database were retrieved and scrutinized. Guidance on the conduct and reporting of pilot studies was retrieved from the websites of three grant giving bodies and seven journal editors were canvassed.

54 pilot or feasibility studies published in 2007-8 were found, of which 26 (48%) were pilot studies of interventions and the remainder feasibility studies. The majority incorporated hypothesis-testing (81%), a control arm (69%) and a randomization procedure (62%). Most (81%) pointed towards the need for further research. Only 8 out of 90 pilot studies identified by the earlier review led to subsequent main studies. Twelve studies which were interventional pilot/feasibility studies and which included testing of some component of the research process were identified through the UKCRN Portfolio database. There was no clear distinction in use of the terms 'pilot' and 'feasibility'. Five journal editors replied to our entreaty. In general they were loathe to publish studies described as 'pilot'.

Pilot studies are still poorly reported, with inappropriate emphasis on hypothesis-testing. Authors should be aware of the different requirements of pilot studies, feasibility studies and main studies and report them appropriately. Authors should be explicit as to the purpose of a pilot study. The definitions of feasibility and pilot studies vary and we make proposals here to clarify terminology.

Peer Review reports

A brief definition is that a pilot study is a 'small study for helping to design a further confirmatory study'[ 1 ]. A very useful discussion of exactly what is a pilot study has been given by Thabane et al. [ 2 ] Such kinds of study may have various purposes such as testing study procedures, validity of tools, estimation of the recruitment rate, and estimation of parameters such as the variance of the outcome variable to calculate sample size etc. In pharmacological trials they may be referred to as 'proof of concept' or Phase I or Phase II studies. It has become apparent to us when reviewing research proposals that small studies with all the trappings of a major study, such as randomization and hypothesis testing may be labeled a 'pilot' because they do not have the power to test clinically meaningful hypotheses. The authors of such studies perhaps hope that reviewers will regard a 'pilot' more favourably than a small clinical trial. This lead us to ask when it is legitimate to label a study as a 'pilot' or 'feasibility' study, and what features should be included in these types of studies.

Lancaster et al [ 3 ] conducted a review of seven major medical journals in 2000-1 to produce evidence regarding the components of pilot studies for randomized controlled trials. Their search included both 'pilot' and 'feasibility' studies as keywords. They reported certain recommendations: having clear objectives in a pilot study, inappropriateness of mixing pilot data with main research study, using mainly descriptive statistics obtained and caution regarding the use of hypothesis testing for conclusions. Arnold et al [ 1 ] recently reviewed pilot studies particularly related to critical care medicine by searching the literature from 1997 to 2007. They provided narrative descriptions of some pilot papers particularly those describing critical care medicine procedures. They pointed out that few pilot trials later evolved into subsequent published major trials. They made useful distinctions between: pilot work which is any background research to inform a future study, a pilot study which has specific hypotheses, objectives and methodology and a pilot trial which is a stand-alone pilot study and includes a randomization procedure. They excluded feasibility studies from their consideration.

Thabane et al [ 2 ] gave a checklist of what they think should be included in a pilot study. They included 'feasibility' or 'vanguard' studies but did not distinguish them from pilot studies. They provided a good discussion on how to interpret a pilot study. They stress that not only the outcome or surrogate outcome for the subsequent main study should be described but also that a pilot study should have feasibility outcomes which should be clearly defined and described. Their article was opinion based and not supported by a review of current practice.

The objective of this paper is to provide writers and reviewers of research proposals with evidence from a variety of sources for which components they should expect, and which are unnecessary or unhelpful, in a study which is labeled as a pilot or feasibility study. To do this we repeated Lancaster et al's [ 3 ] review for current papers see if there has been any change in how pilot studies were reported since their study. As many pilot studies are never published we also identified pilot studies which were registered with the UK Clinical Research Network (UKCRN) Portfolio Database. This aims to be a "complete picture of the clinical research which is currently taking place across the UK". All studies included have to have been peer reviewed through a formal independent process. We examined the websites of some grant giving bodies to find their definition of a pilot study and their funding policy toward them. Finally we contacted editors of leading medical journals to discover their policy of accepting studies described as 'pilot' or 'feasibility'.

Literature survey

MEDLINE, Web of Science and university library data bases were searched for the years 2007-8 using the same key words "Pilot" or "Feasibility" as used by Lancaster et al. [ 3 ]. We reviewed the same four general medicine journals: the British Medical Journal (BMJ), Lancet, the New England Journal of Medicine (NEJM) and the Journal of American Medical Association (JAMA) and the same three specialist journals: British Journal of Surgery (BJS), British Journal of Cancer (BJC), British Journal of Obstetrics and Gynecology (BJOG). We excluded review papers. The full text of the relevant papers was obtained. GL reviewed 20 papers and classified them into groups as described in her original paper [ 3 ]. Subsequently MA, in discussion with MC, designed a data extraction form to classify the papers. We changed one category from GL's original paper. We separated the category 'Phase I/II trials' from the 'Piloting new treatment, technique, combination of treatments' category. We then classified the remaining paper into the categories described in Table 1 . The total number of research papers by journal was obtained by searching journal article with abstracts (excluding reviews) using Pubmed. We searched citations to see whether the pilot studies identified by Lancaster et al [ 3 ] eventually led to main trials.

Portfolio database review

The (UKCRN) Portfolio Database was searched for the terms 'feasibility' or 'pilot' in the title or research summary. Duplicate cases and studies classified as 'observational' were omitted. From the remaining studies those classified as 'closed' were selected to exclude studies which may not have started or progressed. Data were extracted directly from the research summary of the database or where that was insufficient the principle investigator was contacted for related publications or study protocols.

Editor and funding agency survey

We wrote to the seven medical journal editors of the same journals used by Lancaster et al. [ 3 ], (BMJ, Lancet, NEJM, JAMA. BJS, BJC and BJOG) and looked at the policies of three funding agencies (British Medical Research Council, Research for Patient Benefit and NETSCC (National Institute for Health Research Trials and Studies Coordinating Centre). We wished to explore whether there was any specified policy of the journal for publishing pilot trials and how the editors defined a pilot study. We also wished to see if there was funding for pilot studies.

Initially 77 papers were found in the target journals for 2007-8 but 23 were review papers or commentaries or indirectly referred to the word "pilot" or "feasibility" and were not actually pilot studies leaving a total of 54 papers. Table 1 shows the results by journal and by type of study and also shows the numbers reported by Lancaster et al. [ 3 ] for 2000-01 in the same medical journals. There was a decrease in the proportion of pilot studies published over the period of time, however the difference was not statistically significant (2.0% vs 1.6%; X 2 = 1.6, P = 0.2). It is noticeable that the Phase I or Phase II studies are largely confined to the cancer journals.

Lancaster et al [ 3 ] found that 50% of pilot studies reported the intention of further work yet we identified only 8 (8.8%) which were followed up by a major study. Of these 2 (25%) were published in the same journal as the pilot.

Twenty-six of the studies found in 2007-8 were described as pilot or feasibility studies for randomized clinical trials (RCTs) including Phase II studies. Table 2 gives the numbers of studies which describe specific components of RCTs. Sample size calculations were performed and reported in 9 (36%) of the studies. Hypothesis testing and performing inferential statistics to report significant results was observed in 21 (81%) of pilot studies. The processes of blinding was observed in only 5 (20%) although the randomization procedure was applied or tested in 16 (62%) studies. Similarly a control group was assigned in most of the studies (n = 18; 69%). As many as 21 (81%) of pilot studies suggested the need for further investigation of the tested drug or procedure and did not report conclusive results on the basis of their pilot data. The median number of participants was 76, inter-quartile range (42, 216).

Of the 54 studies in 2007-8, a total of 20 were described as 'pilot' and 34 were described as 'feasibility' studies. Table 3 contrasts those which were identified by the keyword 'pilot' with those identified by 'feasibility'. Those using 'pilot' were more likely to have a pre-study sample size estimate, to use randomization and to use a control group. In the 'pilot' group 16(80%) suggested further study, in contrast to 15 (44%) in the 'feasibility' group.

A total of 34 studies were identified using the term 'feasibility' or 'pilot' in the title or research summary which were prospective interventional studies and were closed, i.e. not currently running and available for analysis. Only 12 studies were interventional pilot/feasibility studies which included testing of some component of the research process. Of these 5 were referred to as 'feasibility', 6 as 'pilot' and 1 as both 'feasibility' and 'pilot' (Table 4 ).

The methodological components tested within these studies were: estimation of sample size; number of subjects eligible; resources (e.g. cost), time scale; population-related (e.g. exclusion criteria), randomisation process/acceptability; data collection systems/forms; outcome measures; follow-up (response rates, adherence); overall design; whole trial feasibility. In addition to one or more of these, some studies also looked at clinical outcomes including: feasibility/acceptability of intervention; dose, efficacy and safety of intervention.

The results are shown in Table 4 . Pilot studies alone included estimation of sample size for a future bigger study and tested a greater number of components in each study. The majority of the pilots and the feasibility studies ran the whole study 'in miniature' as it would be in the full study, with or without randomization.

As an example of a pilot study consider 'CHOICES: A pilot patient preference randomised controlled trial of admission to a Women's Crisis House compared with psychiatric hospital admissions' http://www.iop.kcl.ac.uk/projects/default.aspx?id=10290 . This study looked at multiple components of a potential bigger study. It aimed to determine the proportion of women unwilling to be randomised, the feasibility of a patient preference RCT design, the outcome and cost measures to determine which outcome measures to use, the recruitment and drop out rates; and to estimate the levels of outcome variability to calculate sample sizes for the main study. It also intended to develop a user focused and designed instrument which is the outcome from the study. The sample size was 70.

The editors of five (out of seven) medical journals responded to our request for information regarding publishing policy for pilot studies. Four of the journals did not have a specified policy about publishing pilot studies and mostly reported that pilot trials cannot be published if the standard is lower than a full clinical trial requirement. The Lancet has started creating space for preliminary phase I trials and set a different standard for preliminary studies. Most of the other journals do not encourage the publication of pilot studies because they consider them less rigorous than main studies. Nevertheless some editors accepted pilot studies for publication by compromising only on the requirement for a pre-study sample size calculation. All other methodological issued were considered as important as for the full trials, such as trial registration, randomization, hypothesis testing, statistical analysis and reporting according to the CONSORT guidelines.

All three funding bodies made a point to note that pilot and feasibility studies would be considered for funding. Thabane et al [ 2 ] provided a list of websites which define pilot or feasibility studies. We considered the NETSCC definition to be most helpful and to most closely mirror what investigators are doing and it is given below.

NETSCC definition of pilot and feasibility studies http://www.netscc.ac.uk/glossary/

Feasibility Studies

Feasibility Studies are pieces of research done before a main study. They are used to estimate important parameters that are needed to design the main study. For instance:

standard deviation of the outcome measure, which is needed in some cases to estimate sample size,

willingness of participants to be randomised,

willingness of clinicians to recruit participants,

number of eligible patients,

characteristics of the proposed outcome measure and in some cases feasibility studies might involve designing a suitable outcome measure,

follow-up rates, response rates to questionnaires, adherence/compliance rates, ICCs in cluster trials, etc.

Feasibility studies for randomised controlled trials may not themselves be randomised. Crucially, feasibility studies do not evaluate the outcome of interest; that is left to the main study.

If a feasibility study is a small randomised controlled trial, it need not have a primary outcome and the usual sort of power calculation is not normally undertaken. Instead the sample size should be adequate to estimate the critical parameters (e.g. recruitment rate) to the necessary degree of precision.

Pilot studies

A Pilot Study is a version of the main study that is run in miniature to test whether the components of the main study can all work together. It is focused on the processes of the main study, for example to ensure recruitment, randomisation, treatment, and follow-up assessments all run smoothly. It will therefore resemble the main study in many respects. In some cases this will be the first phase of the substantive study and data from the pilot phase may contribute to the final analysis; this can be referred to as an internal pilot. Alternatively at the end of the pilot study the data may be analysed and set aside, a so-called external pilot.

In our repeat of Lancaster et al's study [ 3 ] we found that the reporting of pilot studies was still poor. It is generally accepted that small, underpowered clinical trials are unethical [ 4 ]. Thus it is not an excuse to label such a study as a pilot and hope to make it ethical. We have shown that pilot studies have different objectives to RCTs and these should be clearly described. Participants in such studies should be informed that they are in a pilot study and that there may not be a further larger study.

It is helpful to make a more formal distinction between a 'pilot' and a 'feasibility' study. We found that studies labeled 'feasibility' were conducted with more flexible methodology compared to those labeled 'pilot'. For example the term 'feasibility' has been used for large scale studies such as a screening programme applied at a population level to determine the initial feasibility of the programme. On the other hand 'pilot' studies were reported with more rigorous methodological components like sample size estimation, randomization and control group selection than studies labeled 'feasibility'. We found the NETSCC definition to be the most helpful since it distinguishes between these types of study.

In addition it was observed that most of the pilot studies report their results as inconclusive, with the intention of conducting a further, larger study. In contrast, several of the feasibility studies did not admit such an intention. On the basis of their intention one would have expected about 45 of the studies identified by Lancaster et al in 2000/1 to have been followed by a bigger study whereas we only found 8. This would reflect the opinion of most of the journal editors and experts who responded to our survey, who felt that pilot studies rarely act as a precursor for a bigger study. The main reason given was that if the pilot shows significant results then researchers may not find it necessary to conduct the main trial. In addition if the results are unfavorable or the authors find an unfeasible procedure, the main study is less likely to be considered useful. Our limited review of funding bodies was encouraging. Certainly when reviewing grant applications, we have found it helpful to have the results of a pilot study included in the bid. We think that authors of pilots studies should be explicit as to their purpose, e.g. to test a new procedure in preparation for a clinical trial. We also think that authors of proposals for pilot studies should be more explicit as to the criteria which lead to further studies being abandoned, and that this should be an important part of the proposal.

In the Portfolio Database review, only pilot studies cited an intention to estimate sample size calculations for future studies and the majority of pilot studies were full studies run with smaller sample sizes to test out a number of methodological components and clinical outcomes simultaneously. In comparison the feasibility studies tended to focus on fewer methodological components within individual studies. For example, the 6 pilot studies reported the intention to evaluate a total of 17 methodological components whereas in the 5 feasibility studies a total of only 6 methodological components were specifically identified as being under investigation (Table 4 ). However, both pilot and feasibility studies included trials run as complete studies, including randomization, but with sample sizes smaller than would be intended in the full study and the distinction between the two terms was not clear-cut.

Another reason for conducting a pilot study is to provide information to enable a sample size calculation in a subsequent main study. However since pilot studies tend to be small, the results should be interpreted with caution [ 5 ]. Only a small proportion of published pilot studies reported pre-study sample size calculations. Most journal editors reported that a sample size calculation is not a mandatory criterion for publishing pilot studies and suggested that it should not be done.

Some authors suggest that analysis of pilot studies should mainly be descriptive,[ 3 , 6 ] as hypothesis testing requires a powered sample size which is usually not available in pilot studies. In addition, inferential statistics and testing hypothesis for effectiveness require a control arm which may not be present in all pilot studies. However most of the pilot interventional studies in this review contained a control group and the authors performed and reported hypothesis testing for one or more variables. Some tested the effectiveness of an intervention and others just performed statistical testing to discover any important associations in the study variables. Observed practice is not necessarily good practice and we concur with Thabane et al [ 2 ] that any testing of an intervention needs to be reported cautiously.

The views of the journal editors, albeit from a small sample, were not particularly encouraging and reflected the experience of Lancaster et al [ 3 ]. Pilot studies, by their nature, will not produce 'significant' (i.e P < 0.05) results. We believe that publishing the results of well conducted pilot or feasibility studies is important for research, irrespective of outcome.. There is an increasing awareness that publishing only 'significant' results can lead to considerably error [ 7 ]. The journals we considered were all established, paper journals and perhaps the newer electronic journals will be more willing to consider the publication of the results from these types of studies.

We may expect that trials will increasingly be used to evaluate 'complex interventions'[ 8 , 9 ]. The MRC guidelines [ 8 ] explicitly suggest that preliminary studies, including pilots, be used prior to any major trial which seeks to evaluate a package of interventions (such as an educational course), rather than a single intervention (such as a drug). Thus it is likely that reviewers will be increasingly asked to pronounce on these and will require guidance as to how to review them.

Conclusions

We conclude that pilot studies are still poorly reported, with inappropriate emphasis on hypothesis-testing. We believe authors should be aware of the different requirements of pilot studies and feasibility studies and report them appropriately. We found that in practice the definitions of feasibility and pilot studies are not distinct and vary between health research funding bodies and we suggest use of the NETSCC definition to clarify terminology.

Arnold DM, Burns KE, Adhikari NK, Kho ME, Meade MO, Cook DJ: McMaster Critical Care Interest Group. The design and interpretation of pilot trials in clinical research in critical care. Crit Care Med. 2009, 37 (Suppl 1): S69-74. 10.1097/CCM.0b013e3181920e33.

Article   PubMed   Google Scholar  

Thabane L, Ma J, Chu R, Cheng J, Ismaila A, Rios LP, Robson R, Thabane M, Goldsmith CH: A tutorial on pilot studies: The what, why and How. BMC Medical Research Methodology. 2010, 10: 1-10.1186/1471-2288-10-1.

Article   PubMed   PubMed Central   Google Scholar  

Lancaster GA, Dodd S, Williamson PR: Design and analysis of pilot studies: recommendations for good practice. J Eval Clin Pract. 2004, 10: 307-12. 10.1111/j..2002.384.doc.x.

Halpern SD, Karlawish JH, Berlin JA: The continuing unethical conduct of underpowered clinical trials. JAMA. 2002, 288: 358-62. 10.1001/jama.288.3.358.

Kraemer HC, Mintz J, Noda A, Tinklenberg J, Yesavage JA: Caution regarding the use of pilot studies to guide power calculations for study proposals. Arch Gen Psychiatry. 2006, 63: 484-9. 10.1001/archpsyc.63.5.484.

Grimes DA, Schulz KF: Descriptive studies: what they can and cannot do. Lancet. 2002, 359: 145-9. 10.1016/S0140-6736(02)07373-7.

Ioannidis JPA: Why Most Published Research Findings Are False. PLoS Med. 2005, 2 (8): e124-10.1371/journal.pmed.0020124.

Lancaster GA, Campbell MJ, Eldridge S, Farrin A, Marchant M, Muller S, Perera R, Peters TJ, Prevost AT, Rait G: Trials in Primary Care: issues in the design of complex intervention. Statistical Methods in Medical Research. 2010, to appear

Google Scholar  

Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M: Developing and evaluating complex interventions: new guidance. 2008, Medical Research Council, [ http://www.mrc.ac.uk/Utilities/Documentrecord/index.htm?d=MRC004871 ]

Pre-publication history

The pre-publication history for this paper can be accessed here: http://www.biomedcentral.com/1471-2288/10/67/prepub

Download references

Author information

Authors and affiliations.

Health Services Research, ScHARR, University of Sheffield, Regent Court, Regent St Sheffield, S1 4DA, UK

Mubashir Arain, Michael J Campbell & Cindy L Cooper

Department of Mathematics and Statistics, University of Lancaster, LA1 4YF, UK

Gillian A Lancaster

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Michael J Campbell .

Additional information

Competing interests.

The authors declare that they have no competing interests.

Authors' contributions

MA reviewed the papers of 2000/1 and those of 2007/8 under the supervision of MC and helped to draft the manuscript. MC conceived of the study, and participated in its design and coordination and drafted the manuscript. CC conducted the portfolio database study and commented on the manuscript. GA conducted the original study, reviewed 20 papers and commented on the manuscript. All authors read and approved the final manuscript.

Rights and permissions

Open Access This article is published under license to BioMed Central Ltd. This is an Open Access article is distributed under the terms of the Creative Commons Attribution License ( https://creativecommons.org/licenses/by/2.0 ), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and permissions

About this article

Cite this article.

Arain, M., Campbell, M.J., Cooper, C.L. et al. What is a pilot or feasibility study? A review of current practice and editorial policy. BMC Med Res Methodol 10 , 67 (2010). https://doi.org/10.1186/1471-2288-10-67

Download citation

Received : 20 May 2010

Accepted : 16 July 2010

Published : 16 July 2010

DOI : https://doi.org/10.1186/1471-2288-10-67

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Pilot Study
• Feasibility Study
• Journal Editor
• Methodological Component

BMC Medical Research Methodology

ISSN: 1471-2288

• General enquiries: [email protected]

Pilot Study in Research: Definition & Examples

Julia Simkus

Editor at Simply Psychology

BA (Hons) Psychology, Princeton University

Julia Simkus is a graduate of Princeton University with a Bachelor of Arts in Psychology. She is currently studying for a Master's Degree in Counseling for Mental Health and Wellness in September 2023. Julia's research has been published in peer reviewed journals.

Learn about our Editorial Process

Saul Mcleod, PhD

Editor-in-Chief for Simply Psychology

BSc (Hons) Psychology, MRes, PhD, University of Manchester

Saul Mcleod, PhD., is a qualified psychology teacher with over 18 years of experience in further and higher education. He has been published in peer-reviewed journals, including the Journal of Clinical Psychology.

Olivia Guy-Evans, MSc

Associate Editor for Simply Psychology

BSc (Hons) Psychology, MSc Psychology of Education

Olivia Guy-Evans is a writer and associate editor for Simply Psychology. She has previously worked in healthcare and educational sectors.

On This Page:

A pilot study, also known as a feasibility study, is a small-scale preliminary study conducted before the main research to check the feasibility or improve the research design.

Pilot studies can be very important before conducting a full-scale research project, helping design the research methods and protocol.

How Does it Work?

Pilot studies are a fundamental stage of the research process. They can help identify design issues and evaluate a study’s feasibility, practicality, resources, time, and cost before the main research is conducted.

It involves selecting a few people and trying out the study on them. It is possible to save time and, in some cases, money by identifying any flaws in the procedures designed by the researcher.

A pilot study can help the researcher spot any ambiguities (i.e., unusual things), confusion in the information given to participants, or problems with the task devised.

Sometimes the task is too hard, and the researcher may get a floor effect because none of the participants can score at all or can complete the task – all performances are low.

The opposite effect is a ceiling effect, when the task is so easy that all achieve virtually full marks or top performances and are “hitting the ceiling.”

This enables researchers to predict an appropriate sample size, budget accordingly, and improve the study design before performing a full-scale project.

Pilot studies also provide researchers with preliminary data to gain insight into the potential results of their proposed experiment.

However, pilot studies should not be used to test hypotheses since the appropriate power and sample size are not calculated. Rather, pilot studies should be used to assess the feasibility of participant recruitment or study design.

By conducting a pilot study, researchers will be better prepared to face the challenges that might arise in the larger study. They will be more confident with the instruments they will use for data collection.

Multiple pilot studies may be needed in some studies, and qualitative and/or quantitative methods may be used.

To avoid bias, pilot studies are usually carried out on individuals who are as similar as possible to the target population but not on those who will be a part of the final sample.

Feedback from participants in the pilot study can be used to improve the experience for participants in the main study. This might include reducing the burden on participants, improving instructions, or identifying potential ethical issues.

Experiment Pilot Study

In a pilot study with an experimental design , you would want to ensure that your measures of these variables are reliable and valid.

You would also want to check that you can effectively manipulate your independent variables and that you can control for potential confounding variables.

A pilot study allows the research team to gain experience and training, which can be particularly beneficial if new experimental techniques or procedures are used.

Questionnaire Pilot Study

It is important to conduct a questionnaire pilot study for the following reasons:

• Check that respondents understand the terminology used in the questionnaire.
• Check that emotive questions are not used, as they make people defensive and could invalidate their answers.
• Check that leading questions have not been used as they could bias the respondent’s answer.
• Ensure that the questionnaire can be completed in a reasonable amount of time. If it’s too long, respondents may lose interest or not have enough time to complete it, which could affect the response rate and the data quality.

By identifying and addressing issues in the pilot study, researchers can reduce errors and risks in the main study. This increases the reliability and validity of the main study’s results.

Assessing the practicality and feasibility of the main study

Testing the efficacy of research instruments

Identifying and addressing any weaknesses or logistical problems

Collecting preliminary data

Estimating the time and costs required for the project

Determining what resources are needed for the study

Identifying the necessity to modify procedures that do not elicit useful data

Adding credibility and dependability to the study

Pretesting the interview format

Enabling researchers to develop consistent practices and familiarize themselves with the procedures in the protocol

Addressing safety issues and management problems

Limitations

Require extra costs, time, and resources.

Do not guarantee the success of the main study.

Contamination (ie: if data from the pilot study or pilot participants are included in the main study results).

Funding bodies may be reluctant to fund a further study if the pilot study results are published.

Do not have the power to assess treatment effects due to small sample size.

• Viscocanalostomy: A Pilot Study (Carassa, Bettin, Fiori, & Brancato, 1998)
• WHO International Pilot Study of Schizophrenia (Sartorius, Shapiro, Kimura, & Barrett, 1972)
• Stephen LaBerge of Stanford University ran a series of experiments in the 80s that investigated lucid dreaming. In 1985, he performed a pilot study that demonstrated that time perception is the same as during wakefulness. Specifically, he had participants go into a state of lucid dreaming and count out ten seconds, signaling the start and end with pre-determined eye movements measured with the EOG.
• Negative Word-of-Mouth by Dissatisfied Consumers: A Pilot Study (Richins, 1983)
• A pilot study and randomized controlled trial of the mindful self‐compassion program (Neff & Germer, 2013)
• Pilot study of secondary prevention of posttraumatic stress disorder with propranolol (Pitman et al., 2002)
• In unstructured observations, the researcher records all relevant behavior without a system. There may be too much to record, and the behaviors recorded may not necessarily be the most important, so the approach is usually used as a pilot study to see what type of behaviors would be recorded.
• Perspectives of the use of smartphones in travel behavior studies: Findings from a literature review and a pilot study (Gadziński, 2018)

Further Information

• Lancaster, G. A., Dodd, S., & Williamson, P. R. (2004). Design and analysis of pilot studies: recommendations for good practice. Journal of evaluation in clinical practice, 10 (2), 307-312.
• Thabane, L., Ma, J., Chu, R., Cheng, J., Ismaila, A., Rios, L. P., … & Goldsmith, C. H. (2010). A tutorial on pilot studies: the what, why and how. BMC Medical Research Methodology, 10 (1), 1-10.
• Moore, C. G., Carter, R. E., Nietert, P. J., & Stewart, P. W. (2011). Recommendations for planning pilot studies in clinical and translational research. Clinical and translational science, 4 (5), 332-337.

Carassa, R. G., Bettin, P., Fiori, M., & Brancato, R. (1998). Viscocanalostomy: a pilot study. European journal of ophthalmology, 8 (2), 57-61.

Gadziński, J. (2018). Perspectives of the use of smartphones in travel behaviour studies: Findings from a literature review and a pilot study. Transportation Research Part C: Emerging Technologies, 88 , 74-86.

In J. (2017). Introduction of a pilot study. Korean Journal of Anesthesiology, 70 (6), 601–605. https://doi.org/10.4097/kjae.2017.70.6.601

LaBerge, S., LaMarca, K., & Baird, B. (2018). Pre-sleep treatment with galantamine stimulates lucid dreaming: A double-blind, placebo-controlled, crossover study. PLoS One, 13 (8), e0201246.

Leon, A. C., Davis, L. L., & Kraemer, H. C. (2011). The role and interpretation of pilot studies in clinical research. Journal of psychiatric research, 45 (5), 626–629. https://doi.org/10.1016/j.jpsychires.2010.10.008

Malmqvist, J., Hellberg, K., Möllås, G., Rose, R., & Shevlin, M. (2019). Conducting the Pilot Study: A Neglected Part of the Research Process? Methodological Findings Supporting the Importance of Piloting in Qualitative Research Studies. International Journal of Qualitative Methods. https://doi.org/10.1177/1609406919878341

Neff, K. D., & Germer, C. K. (2013). A pilot study and randomized controlled trial of the mindful self‐compassion program. Journal of Clinical Psychology, 69 (1), 28-44.

Pitman, R. K., Sanders, K. M., Zusman, R. M., Healy, A. R., Cheema, F., Lasko, N. B., … & Orr, S. P. (2002). Pilot study of secondary prevention of posttraumatic stress disorder with propranolol. Biological psychiatry, 51 (2), 189-192.

Richins, M. L. (1983). Negative word-of-mouth by dissatisfied consumers: A pilot study. Journal of Marketing, 47 (1), 68-78.

Sartorius, N., Shapiro, R., Kimura, M., & Barrett, K. (1972). WHO International Pilot Study of Schizophrenia1. Psychological medicine, 2 (4), 422-425.

Teijlingen, E. R; V. Hundley (2001). The importance of pilot studies, Social research UPDATE, (35)

Related Articles

Research Methodology

Qualitative Data Coding

What Is a Focus Group?

Cross-Cultural Research Methodology In Psychology

What Is Internal Validity In Research?

Research Methodology , Statistics

What Is Face Validity In Research? Importance & How To Measure

Criterion Validity: Definition & Examples

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here .

Loading metrics

Open Access

Peer-reviewed

Research Article

Defining Feasibility and Pilot Studies in Preparation for Randomised Controlled Trials: Development of a Conceptual Framework

* E-mail: [email protected]

Affiliation Centre for Primary Care and Public Health, Queen Mary University of London, London, United Kingdom

Affiliation Department of Mathematics and Statistics, Lancaster University, Lancaster, Lancashire, United Kingdom

Affiliation School of Health and Related Research, University of Sheffield, Sheffield, South Yorkshire, United Kingdom

Affiliation Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, Ontario, Canada

Affiliation Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, Oxfordshire, United Kingdom

Affiliation Centre of Academic Primary Care, University of Aberdeen, Aberdeen, Scotland, United Kingdom

• Sandra M. Eldridge, 
• Gillian A. Lancaster, 
• Michael J. Campbell, 
• Lehana Thabane, 
• Sally Hopewell, 
• Claire L. Coleman, 
• Christine M. Bond

• Published: March 15, 2016
• https://doi.org/10.1371/journal.pone.0150205
• Reader Comments

We describe a framework for defining pilot and feasibility studies focusing on studies conducted in preparation for a randomised controlled trial. To develop the framework, we undertook a Delphi survey; ran an open meeting at a trial methodology conference; conducted a review of definitions outside the health research context; consulted experts at an international consensus meeting; and reviewed 27 empirical pilot or feasibility studies. We initially adopted mutually exclusive definitions of pilot and feasibility studies. However, some Delphi survey respondents and the majority of open meeting attendees disagreed with the idea of mutually exclusive definitions. Their viewpoint was supported by definitions outside the health research context, the use of the terms ‘pilot’ and ‘feasibility’ in the literature, and participants at the international consensus meeting. In our framework, pilot studies are a subset of feasibility studies, rather than the two being mutually exclusive. A feasibility study asks whether something can be done, should we proceed with it, and if so, how. A pilot study asks the same questions but also has a specific design feature: in a pilot study a future study, or part of a future study, is conducted on a smaller scale. We suggest that to facilitate their identification, these studies should be clearly identified using the terms ‘feasibility’ or ‘pilot’ as appropriate. This should include feasibility studies that are largely qualitative; we found these difficult to identify in electronic searches because researchers rarely used the term ‘feasibility’ in the title or abstract of such studies. Investigators should also report appropriate objectives and methods related to feasibility; and give clear confirmation that their study is in preparation for a future randomised controlled trial designed to assess the effect of an intervention.

Citation: Eldridge SM, Lancaster GA, Campbell MJ, Thabane L, Hopewell S, Coleman CL, et al. (2016) Defining Feasibility and Pilot Studies in Preparation for Randomised Controlled Trials: Development of a Conceptual Framework. PLoS ONE 11(3): e0150205. https://doi.org/10.1371/journal.pone.0150205

Editor: Chiara Lazzeri, Azienda Ospedaliero-Universitaria Careggi, ITALY

Received: August 13, 2015; Accepted: February 10, 2016; Published: March 15, 2016

Copyright: © 2016 Eldridge et al. This is an open access article distributed under the terms of the Creative Commons Attribution License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: Due to a requirement by the ethics committee that the authors specified when the data will be destroyed, the authors are not able to give unlimited access to the Delphi study quantitative data. These data are available from Professor Sandra Eldridge. Data will be available upon request to all interested researchers. Qualitative data from the Delphi study are not available because the authors do not have consent from participants for wider distribution of this more sensitive data.

Funding: The authors received small grants from Queen Mary University of London (£7495), University of Sheffield (£8000), NIHR RDS London (£2000), NIHR RDS South East (£2400), Chief Scientist Office Scotland (£1000). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Competing interests: All authors have completed the ICMJE disclosure form at http://www.icmje.org/coi_disclosure.pdf and declare support from the following organisations that might have an interest in the submitted work – Queen Mary University of London, Sheffield University, NIHR, Chief Scientist Office Scotland; financial relationships with NIHR, MRC, EC FP7, Canadian Institute for Health Research, Wiley, who might have an interest in the submitted work in the previous three years. No other relationships or activities have influenced the submitted work. This does not alter the authors' adherence to PLOS ONE policies on sharing data and materials.

Introduction

There is a large and growing number of studies in the literature that authors describe as feasibility or pilot studies. In this paper we focus on feasibility and pilot studies conducted in preparation for a future definitive randomised controlled trial (RCT) that aims to assess the effect of an intervention. We are primarily concerned with stand-alone studies that are completed before the start of such a definitive RCT, and do not specifically cover internal pilot studies which are designed as the early stage of a definitive RCT; work on the conduct of internal pilot studies is currently being carried out by the UK MRC Network of Hubs for Trials Methodology Research. One motivating factor for the work reported in this paper was the inconsistent use of terms. For example, in the context of RCTs ‘pilot study’ is sometimes used to refer to a study addressing feasibility in preparation for a larger RCT, but at other times it is used to refer to a small scale, often opportunistic, RCT which assesses efficacy or effectiveness.

A second, related, motivating factor was the lack of agreement in the research community about the use of the terms ‘pilot’ and ‘feasibility’ in relation to studies conducted in preparation for a future definitive RCT. In a seminal paper in 2004 reviewing the literature in relation to pilot and feasibility studies conducted in preparation for an RCT [ 1 ], Lancaster et al reported that they could find no formal guidance as to what constituted a pilot study. In the updated UK Medical Research Council (MRC) guidance on designing and evaluating complex interventions published four years later, feasibility and pilot studies are explicitly recommended, particularly in relation to identifying problems that might occur in an ensuing RCT of a complex intervention [ 2 ]. However, while the guidance suggests possible aims of such studies, for example, testing procedures for their acceptability, estimating the likely rates of recruitment and retention of subjects, and the calculation of appropriate sample sizes, no explicit definitions of a ‘pilot study’ or ‘feasibility study’ are provided. In 2010, Thabane and colleagues presented a number of definitions of pilot studies taken from various health related websites [ 3 ]. While these definitions vary, most have in common the idea of conducting a study in advance of a larger, more comprehensive, investigation. Thabane et al also considered the relationship between pilot and feasibility, suggesting that feasibility should be the main emphasis of a pilot study and that ‘a pilot study is synonymous with a feasibility study intended to guide the planning of a large scale investigation’. However, at about the same time, the UK National Institute for Health Research (NIHR) developed definitions of pilot and feasibility studies that are mutually exclusive, suggesting that feasibility studies occurred slightly earlier in the research process and that pilot studies are ‘a version of the main study that is run in miniature to test whether the components of the main study can all work together’. Arain et al . felt that the NIHR definitions were helpful, and showed that studies identified using the keyword ‘feasibility’ had different characteristics from those identified as ‘pilot’ studies [ 4 ]. The NIHR wording for pilot studies has been changed more recently to ‘a smaller version of the main study used to test whether the components of the main study can all work together’ ( Fig 1 ). Nevertheless, it still contrasts with the MRC framework guidance that explicitly states: ‘A pilot study need not be a “scale model” of the planned main-stage evaluation, but should address the main uncertainties that have been identified in the development work’ [ 2 ]. These various, sometimes conflicting, approaches to the interpretation of the terms ‘pilot’ and ‘feasibility’ exemplify differences in current usage and opinion in the research community.

• PPT PowerPoint slide
• PNG larger image
• TIFF original image

https://doi.org/10.1371/journal.pone.0150205.g001

While lack of agreement about definitions may not necessarily affect research quality, it can become problematic when trying to develop guidance for research conduct because of the need for clarity over what the guidance applies to and therefore what it should contain. Previous research has identified weaknesses in the reporting and conduct of pilot and feasibility studies [ 1 , 3 , 4 , 7 ], particularly in relation to studies conducted in preparation for a future definitive RCT assessing the effect of an intervention or therapy. While undertaking research to develop guidance to address some of the weaknesses in reporting these studies, we became convinced by the current interest in this area, the lack of clarity, and the differences of opinion in the research community, that a re-evaluation of the definitions of pilot and feasibility studies was needed. This paper describes the process and results of this re-evaluation and suggests a conceptual framework within which researchers can operate when designing and reporting pilot/feasibility studies. Since our work on reporting guidelines focused specifically on pilot and feasibility studies in preparation for an RCT assessing the effect of some intervention or therapy, we restrict our re-evaluation to these types of pilot and feasibility studies.

The process of developing and validating the conceptual framework for defining pilot and feasibility studies was, to a large extent, integral to the development of our reporting guidelines, the core components of which were a large Delphi study and an international expert consensus meeting focused on developing an extension of the 2010 CONSORT statement for RCTs [ 8 ] to randomised pilot studies. The reporting guidelines, Delphi study and consensus meeting are therefore referred to in this paper. However, the reporting guidelines will be reported separately; this paper focuses on our conceptual framework.

Developing a conceptual framework—Delphi study

Following research team discussion of our previous experience with, and research on, pilot and feasibility studies we initially produced mutually exclusive definitions of pilot and feasibility studies based on, but not identical to, the definitions used by the NIHR. We drew up two draft reporting checklists based on the 2010 CONSORT statement [ 8 ], one for what we had defined as feasibility studies and one for what we had defined as pilot studies. We constructed a Delphi survey, administered on-line by Clinvivo [ 9 ], to obtain consensus on checklist items for inclusion in a reporting guideline, and views on the definitions. Following user-testing of a draft version of the survey with a purposive sample of researchers active in the field of trials and pilot studies, and a workshop at the 2013 Society for Clinical Trials Conference in Boston, we further refined the definitions, checklists, survey introduction and added additional questions.

The first round of the main Delphi survey included: a description and explanation of our definitions of pilot and feasibility studies including examples (Figs 2 and 3 ); questions about participants’ characteristics; 67 proposed items for the two checklists and questions about overall appropriateness of the guidelines for feasibility or pilot studies; and four questions related to the definitions of feasibility and pilot studies: How appropriate do you think our definition for a pilot study conducted in preparation for an RCT is ? How appropriate do you think our definition for a feasibility study conducted in preparation for an RCT is ? How appropriate is the way we have distinguished between two different types of study conducted in preparation for an RCT ? How appropriate are the labels ‘pilot’ and ‘feasibility’ for the two types of study we have distinguished ? Participants were asked to rate their answers to the four questions on a nine-point scale from ‘not at all appropriate’ to ‘completely appropriate’. There was also a space for open comments about the definitions. The second round included results from the first round and again asked for further comments about the definitions.

https://doi.org/10.1371/journal.pone.0150205.g002

https://doi.org/10.1371/journal.pone.0150205.g003

Participants for the main survey were identified as likely users of the checklist including trialists, methodologists, statisticians, funders and journal editors. Three hundred and seventy potential participants were approached by email from the project team or directly from Clinvivo. These were individuals identified based on personal networks, authors of relevant studies in the literature, members of the Canadian Institute of Health Research, Biostatistics section of Statistics Society of Canada, and the American Statistical Society. The International Society for Clinical Biostatistics and the Society for Clinical Trials kindly forwarded our email to their entire membership. There was a link within the email to the on-line questionnaire. Each round lasted three weeks and participants were sent one reminder a week before the closure of each survey. The survey took place between August and October 2013. Ethical approval was granted by the ScHARR research ethics committee at the University of Sheffield.

Developing a conceptual framework—Open meeting and research team meetings

The results of the Delphi survey pertaining to the definitions of feasibility and pilot studies were presented to an open meeting at the 2 nd UK MRC Trials Methodology Conference in Edinburgh in November 2013 [ 13 ]. Attendees chose their preferred proposition from four propositions regarding the definitions, based variously on our original definitions, the NIHR and MRC views of pilot and feasibility studies and different views expressed in the Delphi survey. At a subsequent two-day research team meeting we collated the findings from the Delphi survey and the open meeting, and considered definitions of piloting and feasiblity outside the health research context found from on-line searches using the terms ‘pilot definition’, ‘feasiblity definition’, ‘pilot study definition’ and ‘feasibility study definition’ in Google. We expected all searches to give a very large number of hits and examined the first two pages of hits only from each search. From this, we developed a conceptual framework reflecting consensus about the definitions, types and roles of feasibility and pilot studies conducted in preparation for an RCT evaluating the effect of an intervention or therapy. To ensure we incorporated the views of all researchers likely to be conducting pilot/feasiblity studies, two qualitative researchers joined the second day of the meeting which focused on agreeing this framework. Throughout this process we continually referred back to examples that we had identified to check that our emerging definitions were workable.

Validating the conceptual framework—systematic review

To validate the proposed conceptual framework, we identified a selection of recently reported studies that fitted our definition of pilot and feasibility studies, and tested a number of hypotheses in relation to these studies. We expected that approximately 30 reports would be sufficient to test the hypotheses. We conducted a systematic review to identify studies that authors described as pilot or feasibility studies, by searching Medline via PubMed for studies that had the words ‘pilot’ or ‘feasibility’ in the title. To increase the likelihood that the studies would be those conducted in preparation for a randomised controlled trial of the effect of a therapy or intervention we limited our search to those that contained the word ‘trial’ in the title or abstract. For full details of the search strategy see S1 Fig .

To focus on current practice, we selected the 150 most recent studies from those identified by the electronic search. We did not exclude protocols since we were primarily interested in identifying the way researchers characterised their study and any possible future study and the relationship between them; we expected investigators to describe these aspects of their studies in a similar way in protocols and reports of findings. Two research team members independently reviewed study abstracts to assess whether each study fitted our working definition of a pilot or feasibility study in preparation for an RCT evaluating the effect of an intervention or therapy. Where reviewers disagreed, studies were classed as ‘possible inclusions’ and disagreements resolved by discussion with referral to the full text of the paper as necessary. Given the difficulty of interpreting some reports and to ensure that all research team members agreed on inclusion, the whole team then reviewed relevant extracted sections of the papers provisionally agreed for inclusion. We recognised that abstracts of some studies might not include appropriate information, and therefore that our initial abstract review could have excluded some relevant studies; we explored the extent of this potential omission of studies by reviewing the full texts of a random sample of 30 studies from the original 150. Since our prime goal was to identify a manageable number of relevant studies in order to test our hypotheses rather than identify all possible relevant studies we did not include any additional studies as a result of this exploratory study.

We postulated that the following hypotheses would support our conceptual framework:

• The words ‘pilot’ and ‘feasibility’ are both used in the literature to describe studies undertaken in preparation for an RCT evaluating the effect of an intervention or therapy
• It is possible to identify a subset of studies within the literature that are RCTs conducted in preparation for a larger RCT which evaluates the effect of an intervention or therapy. Authors do not use the term ‘pilot trial’ consistently in relation to these studies.
• Within the literature it is not possible to apply unique mutually exclusive definitions of pilot and feasibility studies in preparation for an RCT evaluating the effect of an intervention or therapy that are consistent with the way authors describe their studies.
• Amongst feasibility studies in preparation for an RCT which evaluates the effect of an intervention or therapy it is possible to identify some studies that are not pilot studies as defined within our conceptual framework, but are studies that acquire information about the feasibility of applying an intervention in a future study.

In order to explore these hypotheses, we categorised included studies into three groups that tallied with our framework (see results for details): randomised pilot studies, non-randomised pilot studies, feasibility studies that are not pilot studies. We also extracted data on objectives, and the phrases that indicated that the studies were conducted in preparation for a subsequent RCT.

Validating the conceptual framework—Consensus meeting

We also took an explanation and visual representation of our framework to an international consensus meeting primarily designed to reach consensus on an extension of the 2010 CONSORT statement to randomised pilot studies. There were 19 invited participants with known expertise, experience, or interest in pilot and feasibility studies, including representatives of CONSORT, funders, journal editors, and those who had been involved in writing the NIHR definitions of pilot and feasibility studies and the MRC guidance on designing and evaluating complex interventions. Thus this was an ideal forum in which to discuss the framework also. This project was not concerned with any specific disease, and was methodological in design; no patients or public were involved.

Ninety-three individuals, including chief investigators, statisticians, trial managers, clinicians, research assistants and a funder, participated in the first round of the Delphi survey and 79 in the second round. Over 70% of participants in the first round felt that our definitions, the way we had distinguished between pilot and feasibility studies, and the labels ‘pilot’ and ‘feasibility’ were appropriate. However, these four items had some of the lowest appropriateness ratings in the survey and there were a large number of comments both in direct response to our four survey items related to appropriateness of definitions, and in open comment boxes elsewhere in the survey. Some of these comments are presented in Fig 4 . Some participants commented favourably on the definitions we had drawn up (quote 1) but others were confused by them (quote 2). Several compared our definitions to the NIHR definitions pointing out the differences (quote 3) and suggesting this might make it particularly difficult for the research community to understand our definitions (quote 4). Some expressed their own views about the definitions (quote 5); largely these tallied with the NIHR definitions. Others noted that both the concept of feasibility and the word itself were often used in relation to studies which investigators referred to as pilot studies (quote 6). Others questioned whether it was practically and/or theoretically possible to make a distinction between pilot and feasibility studies (quote 6, quote 7), suggesting that the two terms are not mutually exclusive and that feasibility was more of an umbrella term for studies conducted prior to the main trial. Some participants felt that, using our definitions, feasibility studies would be less structured and more variable and therefore their quality would be less appropriately assessed via a checklist (quote 8). These responses regarding definitions mirrored what we had found in the user-testing of the Delphi survey, the Society for Clinical Trials workshop, and differences of opinion already apparent in the literature. In the second round of the survey there were few comments about definitions.

https://doi.org/10.1371/journal.pone.0150205.g004

There was a wide range of participants in the open meeting, including senior quantitative and qualitative methodologists, and a funding body representative. The four propositions we devised to cover different views about definitions of pilot and feasibility studies are shown in Fig 5 . Fourteen out of the fifteen attendees who voted on these propositions preferred propositions 3 or 4, based on comments from the Delphi survey and the MRC guidance on designing and evaluating complex interventions respectively. Neither of these propositions implied mutually exclusive definitions of pilot and feasibility studies.

https://doi.org/10.1371/journal.pone.0150205.g005

Definitions of feasibility outside the health research context focus on the likelihood of being able to do something. For example, the Oxford on-line dictionary defines feasibility as: ‘The state or degree of being easily or conveniently done’ [ 14 ] and a feasibility study as: ‘An assessment of the practicality of a proposed plan or method’ [ 15 ]. Some definitions also suggest that a feasibility study should help with decision making, for example [ 16 ]: ‘The feasibility study is an evaluation and analysis of the potential of a proposed project. It is based on extensive investigation and research to support the process of decision making’. Outside the health research context the word ‘pilot’ has several different meanings but definitions of pilot studies usually focus on an experiment, project or development undertaken in advance of a future wider experiment, project or development. For example the Oxford on-line dictionary describes a pilot study as: ‘Done as an experiment or test before being introduced more widely’ [ 17 ]. Several definitions carry with them ideas that the purpose of a pilot study is also to facilitate decision making, for example ‘a small-scale experiment or set of observations undertaken to decide how and whether to launch a full-scale project’ [ 18 ] and some definitions specifically mention feasibility, for example: ‘a small scale preliminary study conducted in order to evaluate feasibility’ [ 19 ].

In keeping with these definitions not directly related to the health research context, we agreed that feasiblity is a concept encapsulating ideas about whether it is possible to do something and that a feasibility study asks whether something can be done , should we proceed with it , and if so , how . While piloting is also concerned with whether something can be done and whether and how we should proceed with it, it has a further dimension; piloting is implementing something, or part of something, in a way you intend to do it in future to see whether it can be done in practice. We therefore agreed that a pilot study is a study in which a future study or part of a future study , is conducted on a smaller scale to ask the question whether something can be done , should we proceed with it , and if so , how . The corollary of these definitions is that all pilot studies are feasibility studies but not all feasibility studies are pilot studies. Within the context of RCTs, the focus of our research, the ‘something’ in the definitions can be replaced with ‘a future RCT evaluating the effect of an intervention or therapy’. Studies that address the question of whether the RCT can be done, should we proceed with it and if so how, can then be classed as feasibility or pilot studies. Some of these studies may, of course, have other objectives but if they are mainly focusing on feasiblity of the future RCT we would include them as feasiblity studies. All three studies used as examples in our Delphi survey [ 10 – 12 ] satisfy the definition of a feasiblity study. However, a study by Piot et al , that we encountered while developing the Delphi study, does not. This study is described as a pilot trial in the abstract but the authors present only data on effectiveness and although they state that their results require confirmation in a larger study it is not clear that their pilot study was conducted in preparation for such a larger study [ 20 ]. On the other hand, Palmer et al ‘performed a feasibility study to determine whether patient and surgeon opinion was permissive for a Randomised Controlled Trial (RCT) comparing operative with non-operative treatment for FAI [femoroacetabular impingement]’ [ 12 ]. Heazell et al describe the aim of their randomised study as ‘to address whether a randomised controlled trial (RCT) of the management of RFM [reduced fetal movement] was feasible’ [ 10 ]. Their study was piloting many of the aspects they hoped to implement in a larger trial of RFM, thus making this also a pilot study, whereas the study conducted by Palmer et al , which comprised a questionnare to clinicians and seeking patient opinion, is not a pilot study but is a feasibility study.

Within our framework, some important studies conducted in advance of a future RCT to evaluate the effect of a therapy or intervention are not feasibility studies. For example, a systematic review, usually an essential pre-requisite for such an RCT, normally addresses whether the future RCT is necessary or desirable , not whether it is feasible . To reflect this, we developed a comprehensive diagrammatical representation of our framework for studies conducted in preparation for an RCT which, for completeness, includes, on the left hand side, early studies that are not pilot and feasibility studies, such as systematic reviews and, along the bottom, details of existing or planned reporting guidelines for different types of study ( S2 Fig ).

Validating the conceptual framework—Systematic review

From the 150 most recent studies identified by our electronic search, we identified 27 eligible reports ( Fig 6 ). In keeping with our working definition of a pilot or feasibility study, to be included the reports had to show evidence that investigators were addressing at least some feasibility objectives and that the study was in preparation for a future RCT evaluating the effect of an intervention. Ideally we would have stipulated that the primary objective of the study should be a feasibility objective but, given the nature of the reporting of most of these studies, we felt this would be too restrictive.

https://doi.org/10.1371/journal.pone.0150205.g006

The 27 studies are reported in Table 1 and results relating to terminology that authors used summarised in Table 2 . Results in Table 2 support our first hypothesis that the words ‘pilot’ and ‘feasibility’ are both used in the literature to describe studies undertaken in preparation for a randomised controlled trial of effectiveness; 63% (17/27) used both terms somewhere in the title or abstract. The table also supports our second hypothesis that amongst the subset of feasibility studies in preparation for an RCT that are themselves RCTs, authors do not use the term ‘pilot trial’ consistently in relation to these studies; of the 18 randomised studies only eight contained the words ‘pilot’ and ‘trial’ in the title. Our third hypothesis, namely that it is not possible to apply unique mutually exclusive definitions of pilot and feasibility studies in preparation for an RCT that are consistent with the way authors describe their studies, is supported by the characteristics of studies presented in Table 1 and summarised in Table 2 . We could find no design or other features (such as randomisation or presence of a control group) that distinguished between those that investigators called feasibility studies and those that they called pilot studies. However, the fourth hypothesis, that amongst studies in preparation for an RCT evaluating the effect of an intervention or therapy it is possible to identify some studies that explore the feasibility of a certain intervention or acquire related information about the feasibility of applying an intervention in a future study but are not pilot studies, was not supported; we identified no such studies amongst those reported in Table 1 . Nevertheless, we had identified two prior to carrying out the review [ 10 , 15 ].

https://doi.org/10.1371/journal.pone.0150205.t001

https://doi.org/10.1371/journal.pone.0150205.t002

Out of our exploratory sample of 30 study reports for which we reviewed full texts rather than only titles and abstracts, we identified 10 that could be classed as pilot or feasibility studies using our framework. We had already identified four of these in our sample reported in Table 1 , but had failed to identify the other six. As expected, this was because key information to identify them as pilot or feasiblity studies such as the fact that they were in preparation for a larger RCT, or that the main objectives were to do with feasiblity were not included in the abstract. Thus our assumption that an initial screen using only abstracts resulted in the omission of some pilot and feasiblity studies was correct.

International consensus meeting participants agreed with the general tenets of our conceptual framework including the ideas that all pilot studies are feasibility studies but that some feasibility studies are not pilot studies. They suggested that any definitive diagrammatic representation should more strongly reflect non-linearity in the ordering of feasibility studies. As a result of their input we produced a new, simplified, diagrammatical representation of the framework ( Fig 7 ) which focuses on the key elements represented inside an oval shape on our original diagram, omits the wider context outside this shape, and highlights some features, including the non-linearity, more clearly.

https://doi.org/10.1371/journal.pone.0150205.g007

The finalised framework

Fig 7 represents the framework. The figure indicates that where there is uncertainty about future RCT feasibility, a feasibility study is appropriate. Feasibility is thus an overarching concept within which we distinguish between three distinct types of study. Randomised pilot studies are those studies in which the future RCT, or parts of it, including the randomisation of participants, is conducted on a smaller scale (piloted) to see if it can be done. Thus randomised pilot studies can include studies that for the most part reflect the design of a future definitive trial but, if necessary due to remaining uncertainty, may involve trying out alternative strategies, for example, collecting an outcome variable via telephone for some participants and on-line for others. Within the framework randomised pilot studies could also legitimately be called randomised feasibility studies. Two-thirds of the studies presented in Table 1 are of this type.

Non-randomised pilot studies are similar to randomised pilot studies; they are studies in which all or part of the intervention to be evaluated and other processes to be undertaken in a future trial is/are carried out (piloted) but without randomisation of participants. These could also legitimately be called by the umbrella term, feasibility study. These studies cover a wide range from those that are very similar to randomised pilot studies except that the intervention and control groups have not been randomised, to those in which only the intervention, and no other trial processes, are piloted. One-third of studies presented in Table 1 are of this type.

Feasibility studies that are not pilot studies are those in which investigators attempt to answer a question about whether some element of the future trial can be done but do not implement the intervention to be evaluated or other processes to be undertaken in a future trial, though they may be addressing intervention development in some way. Such studies are rarer than the other types of feasibility study and, in fact, none of the studies in Table 1 were of this type. Nevertheless, we include these studies within the framework because they do exist; the Palmer study [ 15 ] in which surgeons and patients were asked about the feasibility of randomisation is one such example. Other examples might be interviews to ascertain the acceptability of an intervention, or questionnaires to assess the types of outcomes participants might think important. Within the framework these studies can be called feasibility studies but cannot be called pilot studies since no part of the future randomised controlled trial is being conducted on a smaller scale.

Investigators may conduct a number of studies to assess feasibility of an RCT to test the effect of any intervention or therapy. While it may be most common to carry out what we have referred to as feasibility studies that are not pilot studies before non-randomised pilot studies , and non-randomised pilot studies prior to randomised pilot studies , the process of feasibility work is not necessarily linear and such studies can in fact be conducted in any order. For completeness the diagram indicates the location of internal pilot studies.

There are diverse views about the definitions of pilot and feasibility studies within the research community. We reached consensus over a conceptual framework for the definitions of these studies in which feasibility is an overarching concept for studies assessing whether a future study, project or development can be done. For studies conducted in preparation for a RCT assessing the effect of a therapy or intervention, three distinct types of study come under the umbrella of feasibility studies: randomised pilot studies, non-randomised pilot studies, feasibility studies that are not pilot studies. Thus pilot studies are a subset of feasibility studies. A review of the literature confirmed that it is not possible to apply mutually exclusive definitions of pilot and feasibility studies in preparation for such an RCT that are consistent with the way authors describe their studies. For example Lee et al [ 31 ], Boogerd et al [ 22 ] and Wolf et al [ 38 ] all describe randomised studies exploring the feasibility of introducing new systems (brain computer interface memory training game, on-line interactive treatment environment, bed-exit alarm respectively) but Lee et al describe their study as a ‘A Randomized Control Pilot Study’, with the word ‘feasibility’ used in the abstract and text, while the study by Boogerd et al . is titled ‘Teaming up: feasibility of an online treatment environment for adolescents with type 1 diabetes’, and Wolf at al describe their study as a pilot study without using the word ‘feasibility’.

Our re-evaluation of the definitions of pilot and feasibility studies was conducted over a period of time with input via a variety of media by multi-disciplinary and international researchers, publishers, editors and funders. It was to some extent a by-product of our work developing reporting guidelines for such studies. Nevertheless, we were able to gather a wide range of expert views, and the iterative nature of the development of our thinking has been an important part of obtaining consensus. Other parallel developments, including the recent establishment of the new Pilot and Feasibility Studies journal [ 48 ], suggest that our work is, indeed, timely. We encountered several difficulties in reviewing empirical study reports. Firstly, it was sometimes hard to assess whether studies were planned in preparation for an RCT or whether the authors were conducting a small study and simply commenting on the fact that a larger RCT would be useful. Secondly, objectives were sometimes unclear, and/or effectiveness objectives were often emphasised in spite of recommendations that pilot and feasibility studies should not be focusing on effectiveness [ 1 , 4 ]. In identifying relevant studies we erred on the side of inclusiveness, acknowledging that getting these studies published is not easy and that there are, as yet, no definitive reporting guidelines for investigators to follow. Lastly, our electronic search was unable to identify any feasibility studies that were not pilot studies according to our definitions. Subsequent discussion with qualitative researchers suggested that this is because such studies are often not described as feasibility studies in the title or abstract.

Our framework is compatible with the UK MRC guidance on complex interventions which suggests a ‘feasibility and piloting’ phase as part of the work to design and evaluate such interventions without any explicit distinction between pilot and feasibility studies. In addition, although our framework has a different underlying principle from that adopted by UK NIHR, the NIHR definition of a pilot study is not far from the subset of studies we have described as randomised pilot studies. Although there appears to be increasing interest in pilot and feasibility studies, as far as we are aware no other funding bodies specifically address the nature of such studies. The National Institute for Health in the USA does, however, routinely require published pilot studies before considering funding applications for certain streams, and the Canadian Institutes of Health Research routinely have calls for pilot or feasibility studies in different clinical areas to gather evidence necessary to determine the viability of new research directions determined by their strategic funding plans. These approaches highlight the need for clarity regarding what constitutes a pilot study.

There are several previous reviews of empirical pilot and feasibility studies [ 1 , 4 , 7 ]. In the most recent, reviewing studies published between 2000 and 2009 [ 7 ], the authors identified a large number of studies, described similar difficulty in identifying whether a larger study was actually being planned, and similar lack of consistency in the way the terms ‘pilot’ and ‘feasibility’ are used. Nevertheless, in methodological work, many researchers have adopted fairly rigid definitions of pilot and feasibility studies. For example, Bugge et al in developing the ADEPT framework refer to the NIHR definitions and suggest that feasibility studies ask questions about ‘whether the study can be done’ while pilot trials are ‘(a miniature version of the main trial), which aim to test aspects of study design and processes for the implementation of a larger main trial in the future’ [ 49 ]. Although not explicitly stated, the text seems to suggest that pilot and feasibility studies are mutually exclusive. Our work indicates that this is neither necessary nor desirable. There is, however, general agreement in the literature about the purpose of pilot and feasibility studies. For example, pilot trials are ‘to provide sufficient assurance to enable a larger definitive trial to be undertaken’ [ 50 ], and pilot studies are ‘designed to test the performance characteristics and capabilities of study designs, measures, procedures, recruitment criteria, and operational strategies that are under consideration for use in a subsequent, often larger, study’ [ 51 ], and ‘play a pivotal role in the planning of large-scale and often expensive investigations’ [ 52 ]. Within our framework we define all studies aiming to assess whether a future RCT is do-able as ‘feasibility studies’. Some might argue that the focus of their study in preparation for a future RCT is acceptability rather than feasibility, and indeed, in other frameworks, such as the RE-AIM framework [ 53 ], feasibility and acceptability are seen as two different concepts. However, it is perfectly possible to explore the acceptability of an intervention, of a data collection process or of randomisation in order to determine the feasibility of a putative larger RCT. Thus the use of the term ‘feasibility study’ for a study in preparation for a future RCT is not incompatible with the exploration of issues other than feasibility within the study itself.

There are numerous previous studies in which the investigators review the literature and seek the counsel of experts to develop definitions and clarify terminology. Most of these relate to clinical or physiological definitions [ 54 – 56 ]. A few explorations of definitions relate to concepts such as quality of life [ 57 ]. Implicit in much of this work is that from time to time definitions need rethinking as knowledge and practice moves on. From an etymological point of view this makes sense. In fact, the use of the word ‘pilot’ to mean something that is a prototype of something else only appears to emerge in the middle of the twentieth century and the first use of the word in relation to research design that we could find was in 1947—a pilot survey [ 58 ]. Thus we do not have to look very far back to see changes in the use of one of the words we have been dealing with in developing our conceptual framework. We hope what we are proposing here is helpful in the early twenty-first century to clarify the use of the words ‘pilot’ and ‘feasibility’ in a health research context.

We suggest that researchers view feasibility as an overarching concept, with all studies done in preparation for a main study open to being called feasibility studies, and with pilot studies as a subset of feasibility studies. All such studies should be labelled ‘pilot’ and/or ‘feasibility’ as appropriate, preferably in the title of a report, but if not certainly in the abstract. This recommendation applies to all studies that contribute to an assessment of the feasibility of an RCT evaluating the effect of an intervention. Using either of the terms in the title will be most helpful for those conducting future electronic searches. However, we recognise that for qualitative studies, authors may find it convenient to use the terms in the abstract rather than the title. Authors also need to describe objectives and methods well, reporting clearly if their study is in preparation for a future RCT to evaluate the effect of an intervention or therapy.

Though the focus of this work was on the definitions of pilot and feasibility studies and extensive recommendations for the conduct of these studies is outside its scope, we suggest that in choosing what type of feasibility study to conduct investigators should pay close attention to the major uncertainties that exist in relation to trial or intervention. A randomised pilot study may not be necessary to address these; in some cases it may not even be necessary to implement an intervention at all. Similarly, funders should look for a justification for the type of feasibility study that investigators propose. We have has also highlighted the need for better reporting of these studies. The CONSORT extension for randomised pilot studies that our group has developed are important in helping to address this need and will be reported separately. Nevertheless, further work will be necessary to extend or adapt these reporting guidelines for use for non-randomised pilot studies and for feasibility studies that are not pilot studies. There is also more work to be done in developing good practice guidance for the conduct of pilot and feasibility studies.

Supporting Information

S1 fig. search strategy to identify studies that authors described as pilot or feasibility studies..

https://doi.org/10.1371/journal.pone.0150205.s001

S2 Fig. Initial comprehensive diagrammatic representation of framework.

https://doi.org/10.1371/journal.pone.0150205.s002

Acknowledgments

We thank Alicia O’Cathain and Pat Hoddinot for discussions about the reporting of qualitative studies, and consensus participants for their views on our developing framework. Claire Coleman was funded by a National Institute for Health Research (NIHR) Research Methods Fellowship. This article presents independent research funded by the NIHR. The views expressed are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health.

Author Contributions

Conceived and designed the experiments: SE GL MC LT SH CB. Performed the experiments: SE GL MC LT SH CB CC. Analyzed the data: SE GL MC LT SH CB CC. Contributed reagents/materials/analysis tools: SE GL MC LT SH CB. Wrote the paper: SE GL MC LT SH CB CC.

• View Article
• PubMed/NCBI
• Google Scholar
• 5. National Institute for Health Research. NIHR Evaluation, Trials and Studies | Glossary 2015. Available: http://www.nets.nihr.ac.uk/glossary/feasibility-studies . Accessed 2015 Mar 17.
• 6. National Institute for Health Research. NIHR Evaluation, Trials and Studies | Pilot studies 2015. Available: http://www.nets.nihr.ac.uk/glossary/pilot-studies . Accessed 2015 Mar 17.
• 9. CLINVIVO. Clinvivo Limited 2015 [cited 2015 9 April]. Available: http://www.clinvivo.com/ .
• 13. In Conference Ltd. 2nd Clinical Trials Methodology Conference | 18–19 November 2013 EICC, Edinburgh, Scotland 2013. Available: http://www.methodologyconference2013.org.uk/ . Accessed 2015 Mar 17.
• 14. Oxford Dictionaries. Oxford Dictionaries | feasibility 2015. Available: http://www.oxforddictionaries.com/definition/english/feasibility . Accessed 2015 Mar 17.
• 15. Oxford Dictionaries. Oxford Dictionaries | feasibility study 2015. Available: http://www.oxforddictionaries.com/definition/english/feasibility-study . Accessed 2015 Mar 17.
• 16. Wikipedia. Feasibility study 2015 [cited 2015 17 March]. Available: http://en.wikipedia.org/wiki/Feasibility_study .
• 17. Oxford Dictionaries. Oxford Dictionaries | pilot 2015. Available from: http://www.oxforddictionaries.com/definition/english/pilot . Accessed 2015 Mar 17.
• 18. Collins. Collins English Dictionary | pilot study 2015. Available from: http://www.collinsdictionary.com/dictionary/english/pilot-study . Accessed 2015 Mar 17.
• 19. Wikipedia. Pilot experiment 2015. Available: http://en.wikipedia.org/wiki/Pilot_experiment . Accessed 2015 Mar 17.

Guide to Conducting a Feasibility Study

What is an operating model.

• Category Growth

So you’re thinking of a launching a new venture? Entering a new market? Launching a new product? It’s estimated that only one in fifty business ideas are actually commercially viable and so you’ll want to understand the viability of any proposed project before you invest your time, energy and money into it. That’s why you need a feasibility study.

Why do you need a feasibility study  .

With such a low success rate of new business ventures a business feasibility study is the best way to learn whether you have an idea that could work and guard against wastage of further investment. If the results are positive, then the outcome of the feasibility study can be used as the basis for a full business plan allowing your to proceed with a clearer view of the risks involved and move forward quicker. If it’s negative then you’ve skilfully avoided wasting time and money on a venture that wouldn’t have worded out.

What is a feasibility study?  

A feasibility study aims to make a recommendation as to the likely success of a venture. At the heart of any feasibility study is a hypothesis or question that you want to answer.  Examples include “is there a demand for a X new product or product feature”, “should we enter Y market” and “should we launch Z new venture”.

How to conduct a feasibility study?  

Once you’ve got a clear hypothesis or question that you want to answer, you need to look at five areas that will impact the feasibility of your idea. Let’s look at each of these in turn:

Market Feasibility

Is the market in question attractive? Are there high barrier to entry? Is it of a size that will support our ambitions? Is it growing? Are there any regulatory or legislative requirements to enter or participate in the market?

Technical Feasibility

What technical skills/ability/knowledge/equipment is required? Do you have or could you source the technical expertise required? Do you fully understand the technical requirements underpinning your hypothesis? Could you manufacture / develop the product or service with the resources you have available?

Business Model Feasibility

How will the idea make money? How will you attract users? What costs will you have to pay? Have you modelled the financials? Do you have access to the funding needed? What legal entity structure would you need?

Management Model Feasibility

Who will lead the venture? Do you have the skills and expertise required to manage and operate the venture/product/market? Does the team have the time needed to deliver the venture? If not, can they be recruited or are their skills hard to find?

Exit Feasibility

Do you have a plan to exit the venture and do you need one?

When competing a feasibility study each of the above areas should have a recommendation as to whether it’s feasible or not from that specific perspective factoring in the resources you have available.  This should conclude with a recommendation based on the analysis as to if the venture is or isn’t feasible and the key data points that underpin that recommendation.

Remember that a great feasibility study should not just give you a go / no-go decision. It should provide either a spring board to move forward, highlighting the key areas to focus on to achieve success or a useful analysis highlighting the key obstacles that make the venture unfeasible and should be considered for any future ideas. Even if the answer is no, it’s not a wasted effort, the analysis will leave you better informed for future decisions.

A feasibility study is an essential tool for anyone looking at a new venture. It’s very easy to get excited by a new idea of proposition and steam ahead spending time and money without having a clear view as to whether it’s viable or not. A feasibility study should be your first stop to maximise the returns on your time, energy and investment.

Best of luck with your feasibility studies!

Chris Purcell @ Prussel & Co

• Member Benefits
• Communities
• Grants and Scholarships
• Student Nurse Resources
• Member Directory
• Course Login
• Professional Development
• Organizations Hub
• ONS Course Catalog
• ONS Book Catalog
• ONS Oncology Nurse Orientation Program™
• Account Settings
• Help Center
• Print Membership Card
• Print NCPD Certificate
• Verify Cardholder or Certificate Status

• Trouble finding what you need?
• Check our search tips.

• Oncology Nursing Forum
• Number 5 / September 2018

Feasibility Studies: What They Are, How They Are Done, and What We Can Learn From Them

Anne M. Kolenic

Nursing clinical research is a growing field, and as more nurses become engaged in conducting clinical research, feasibility studies may be their first encounter. Understanding what they are, how to conduct them, and the importance of properly reporting their outcomes is vital to the continued advancement of nursing science.

Jump to a section

Many interventions, practices, and processes exist in the nursing field that are grounded in evidence; however, problems that do not appear to be linked to any strong evidence are encountered in daily practice. Nurses are left questioning, “Why do we do it this way?” or “Is there a better way to provide this intervention?” Sometimes these questions may be answered by performing a literature search and realizing that a novel approach exists to implement into their practice; however, if the literature search does not yield any results for an evidence-based practice change, then conducting research could be the next step. Conducting a large, well-designed study can be overwhelming and expensive and may require funding; it also may not be the appropriate first step in the research process (Morris & Rosenbloom, 2017). A feasibility study may be the appropriate first step to help identify whether a larger research study is warranted.

A feasibility study is often a critical step to be taken prior to conducting a larger study. The primary aim of a feasibility study is to assess the feasibility of conducting future conclusive randomized, controlled trials (RCTs) (Eldridge et al., 2016a). Feasibility studies do not have a primary focus on effectiveness or efficacy (Eldridge et al., 2016a); they can be viewed as a dry run to identify problems that may hinder or prevent success of a subsequent larger trial (Conn, Algase, Rawl, Zerwic, & Wyman, 2010). Feasibility studies can demonstrate that a research design is achievable and that recruitment for an anticipated larger study is possible (Morris & Rosenbloom, 2017). They also can supply data that often are required to receive funding and support for a larger RCT to demonstrate that the study approach is feasible and to make a case that the proposed study will answer the question that is being posed (Morris & Rosenbloom, 2017). They also permit testing of sampling strategies, intervention content, delivery methods, data collection, and analysis (Conn et al., 2010). The article “Nurse-Delivered Symptom Assessment for Individuals With Advanced Lung Cancer” (Flannery et al., 2018) provides an example of how a nurse took a clinical question and moved it into the research arena by conducting a feasibility study to assess an intervention strategy.

A feasibility study’s focus is not on efficacy or effectiveness, but the publication of the findings is beneficial and important to the development of science and must follow high standards, just as definitive trials do (Conn et al., 2010; Eldridge et al., 2016a). The Consolidated Standards of Reporting Trials (CONSORT) statement, last updated in 2010, is a guideline designed to improve the transparency and quality of the reporting of RCTs (Eldridge et al., 2016a). Eldridge et al. (2016a) presented an extension to that statement for randomized pilot and feasibility trials conducted in advance of a future definitive RCT. The development was motivated by the increasing number of studies that were described as pilot or feasibility studies and by research that identified weaknesses in the way they were being conducted and in their reporting (Eldridge et al., 2016b). Eldridge et al. (2016b) recognized that, although much of the information to be reported in these trials was similar to RCTs, key differences also were seen, and the CONSORT standards and checklists needed to be adapted to assist in improving the reporting of pilot and feasibility studies (Eldridge et al., 2016a). When conducting and reporting a feasibility study, of importance is that the guidelines, flowchart, and checklists provided in the 2016 extension of the CONSORT 2010 statement are used by the researcher to promote transparency and to improve the quality and standardization of the reporting (Eldridge et al., 2016a).

Many terms are used interchangeably to describe preliminary studies that are done before a larger study, but consensus is growing in the field of research that distinctions among them should be recognized and more consistently used (Morris & Rosenbloom, 2017). The rationale for needing increased consistency in usage is because the way terms are defined determines the necessary components of the study (Eldridge et al., 2016b; Morris & Rosenbloom, 2017). For example, the terms feasibility studies, pilot studies, pilot RCTs, pilot trials, and pilot work are used by many authors to reference a study done in advance of a future definitive RCT and whose primary aim is to assess feasibility (Eldridge et al., 2016b; Morris & Rosenbloom, 2017). This can be confusing when reading and searching the literature. Eldridge et al. (2016b) proposed the following definitions, which may be helpful when reading articles or when a researcher is deciding on which type of study to perform:

•  Feasibility study: Research conducted to determine whether something can or should be done and, if so, how

•  Randomized pilot study: A small-scale feasibility study, conducted with randomization of participants, that evaluates the practicability of carrying out all or part of an intervention and other processes to be undertaken in a future larger study; may or may not include alternative approaches

•  Nonrandomized pilot study: A small-scale feasibility study, conducted without randomization of participants, that evaluates the practicability of carrying out all or part of an intervention—and, possibly, other processes—to be undertaken in a future larger study

•  Feasibility study that is not a pilot study: A feasibility study that does not incorporate the intervention or other processes to be undertaken in a future trial but may address the development of interventions or processes

Regardless of the type of feasibility study that will be done, they all start the same way, with a question or a problem that a clinician has come up with, followed by a literature search. After that, the researcher must identify gaps in knowledge and in the literature, followed by revision and refinement of the original question into a specific research question. Next, the reasons for conducting the preliminary research need to be considered and then the form it should take determined. The focus of feasibility studies can be on any aspect of research, including the following (Morris & Rosenbloom, 2007):

•  Processes: Informed consent procedures, recruitment approaches, nonadherence

•  Resources: Budget allocation, equipment, data collection time, time requirements

•  Management: Data management, ease of data entry, overall study feasibility, and reporting procedures

•  Science: Treatment safety, dose levels and responses, and variance of treatment effect

After the focus and form are decided, the researcher must design the study, collaborate with stakeholders, carry out the study, and analyze the results. Finally, the researcher must relate the findings to plans for a future study and disseminate the findings.

The publication of feasibility studies provides important information to the scientific community. The results of feasibility studies focus on the value of outcomes for subsequent studies rather than on specific findings (Morris & Rosenbloom, 2017). These studies can provide detailed information that often is omitted from reports of large-scale trials because of space considerations, such as changes to the protocol or other modifications that were done because of findings during the pilot (Conn et al., 2010). Often, a larger trial does not happen after the pilot study is completed for one reason or another, so publication of the pilot results may be the only publicly available record that the intervention was tested (Conn et al., 2010). Flannery et al. (2018) reported that although delivering the intervention with fidelity was possible, the feasibility findings did not warrant intervention replication. This is an important finding to report because it will prevent additional researchers from wasting their time and resources testing that same intervention and process. So, even though these findings did not support the plan to conduct a future larger study, they still provide vital information concerning this vulnerable population. This article provides detailed information on how the feasibility study was designed and conducted, allowing future researchers to change the approach and test different interventions and delivery to this population to promote their well-being.

Feasibility studies are extremely important to advance the science of nursing because they allow for the planning of subsequent larger trials. Nurses often think of ideas and solutions to everyday clinical problems and issues but are challenged to move that idea into a full-scale study. Taking that idea or solution and conducting a feasibility study may be a first step into the area of research for many nurses.

About the Author(s)

Anne M. Kolenic, DNP, APRN, AOCNS®, is an ambulatory clinical nurse specialist at the University Hospitals Seidman Cancer Center in Cleveland, OH. No financial relationships to disclose. Kolenic can be reached at [email protected] , with copy to [email protected] .

Conn, V.S., Algase, D.L., Rawl, S.M., Zerwic, J.J., & Wyman, J.F. (2010). Publishing pilot intervention work. Western Journal of Nursing Research, 32, 994–1010. https://doi.org/10.1177/0193945910367229

Eldridge, S.M., Chan, C.L., Campbell, M.J., Bond, C.M., Hopewell, S., Thabane, L., & Lancaster, G.A. (2016a). CONSORT 2010 statement: Extension to randomised pilot and feasibility trials. Pilot and Feasibility Studies, 2, 64.

Eldridge, S.M., Lancaster, G.A., Campbell, M.J., Thabane, L., Hopewell, S., Coleman, C.L., & Bond, C.M. (2016b). Defining feasibility and pilot studies in preparation for randomized controlled trials: Development of a conceptual framework. PLOS ONE, 11(3), e0150205. https://doi.org/10.1371/journal.pone.0150205

Flannery, M., Stein, K.F., Dougherty, D.W., Mohile, S., Guido, J., & Wells, N. (2018). Nurse-delivered symptom assessment for individuals with advanced lung cancer. Oncology Nursing Forum, 45, 619–630. https://doi.org/10.1188/18.ONF.619-630

Morris, N.S., & Rosenbloom, D.A. (2017). CE: Defining and understanding pilot and other feasibility studies. American Journal of Nursing, 117(3), 38–46.

Related Articles

Cancer prevention and early detection: a changing field with an unchanging goal, palliative care: building a foundation for clinical oncology nursing practice, pandemic-related innovations in oncology nursing research: seeking the positive, related books.

Core Curriculum for Oncology Nursing (Seventh Edition)

Advanced Oncology Nursing Certification Review and Resource Manual (Third Edition)

Access Device Guidelines: Recommendations for Nursing Practice and Education (Fourth Edition)

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• My Account Login
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Clinical Study
• Open access
• Published: 17 April 2014

A feasibility study testing four hypotheses with phase II outcomes in advanced colorectal cancer (MRC FOCUS3): a model for randomised controlled trials in the era of personalised medicine?

• T S Maughan 1 ,
• A M Meade 2 ,
• R A Adams 3 ,
• S D Richman 4 ,
• R Butler 5 ,
• D Fisher 2 ,
• R H Wilson 6 ,
• B Jasani 7 ,
• G R Taylor 4 ,
• G T Williams 7 ,
• J R Sampson 7 ,
• M T Seymour 8 ,
• L L Nichols 2 ,
• S L Kenny 2 ,
• A Nelson 9 ,
• C M Sampson 9 ,
• E Hodgkinson 10 ,
• J A Bridgewater 11 ,
• D L Furniss 10 ,
• M J Pope 2   na1 ,
• J K Pope 2   na1 ,
• M Parmar 2 ,
• P Quirke 4 &
• R Kaplan 2  

British Journal of Cancer volume  110 ,  pages 2178–2186 ( 2014 ) Cite this article

3134 Accesses

25 Citations

3 Altmetric

Metrics details

• Clinical trials
• Colorectal cancer
• Personalized medicine

This article has been updated

Background:

Molecular characteristics of cancer vary between individuals. In future, most trials will require assessment of biomarkers to allocate patients into enriched populations in which targeted therapies are more likely to be effective. The MRC FOCUS3 trial is a feasibility study to assess key elements in the planning of such studies.

Patients and Methods:

Patients with advanced colorectal cancer were registered from 24 centres between February 2010 and April 2011. With their consent, patients’ tumour samples were analysed for KRAS/BRAF oncogene mutation status and topoisomerase 1 (topo-1) immunohistochemistry. Patients were then classified into one of four molecular strata; within each strata patients were randomised to one of two hypothesis-driven experimental therapies or a common control arm (FOLFIRI chemotherapy). A 4-stage suite of patient information sheets (PISs) was developed to avoid patient overload.

A total of 332 patients were registered, 244 randomised. Among randomised patients, biomarker results were provided within 10 working days (w.d.) in 71%, 15 w.d. in 91% and 20 w.d. in 99%. DNA mutation analysis was 100% concordant between two laboratories. Over 90% of participants reported excellent understanding of all aspects of the trial. In this randomised phase II setting, omission of irinotecan in the low topo-1 group was associated with increased response rate and addition of cetuximab in the KRAS, BRAF wild-type cohort was associated with longer progression-free survival.

Conclusions:

Patient samples can be collected and analysed within workable time frames and with reproducible mutation results. Complex multi-arm designs are acceptable to patients with good PIS. Randomisation within each cohort provides outcome data that can inform clinical practice.

Similar content being viewed by others

The Oncology Biomarker Discovery framework reveals cetuximab and bevacizumab response patterns in metastatic colorectal cancer

Fluoropyrimidine type, patient age, tumour sidedness and mutation status as determinants of benefit in patients with metastatic colorectal cancer treated with EGFR monoclonal antibodies: individual patient data pooled analysis of randomised trials from the ARCAD database

Critical evaluation of molecular tumour board outcomes following 2 years of clinical practice in a Comprehensive Cancer Centre

Cancer is the product of a somatic evolutionary process, in which successive advantageous genetic and epigenetic alterations drive the progression of the disease ( Greaves and Maley, 2012 ). Although current knowledge indicates many similar changes in different cancers, the number of possible combinations of changes even within a given anatomical/histological type such as colorectal cancer (CRC) is very large ( The Cancer Genome Network Atlas, 2012 ). This raises a major challenge in the search for effective therapies that target the properties of any given cancer, especially for advanced disease where clonal evolution and the selective pressure of prior therapies drive increasing diversity and resistance to subsequent therapy ( Sequist et al, 2011 ; Gerlinger et al, 2012 ). This emerging understanding of the heterogeneity of cancer is a major challenge to clinical trialists and demands new methodologies for testing novel therapies.

Fundamental to this challenge is the identification of biomarkers that help enrich the evaluated population for benefit from a specific therapy. In CRC, the use of epidermal growth factor receptor (EGFR)-targeted therapy has led to the discovery of the importance of KRAS and recently NRAS mutations ( Douillard et al, 2013 ) in prediction of lack of response to that therapy and association of BRAF mutation with a particularly poor prognosis in advanced CRC (ACRC; Lievre et al, 2006 ; Karapetis et al, 2008 ; Maughan et al, 2011 ). Further biomarker candidates under evaluation as potentially predicting lack of benefit from anti-EGFR therapy are PI3K mutations and loss of PTEN expression ( De Roock et al, 2010 ; Seymour et al, 2013 ).

This paper reports the results of the MRC FOCUS3 trial (ISRCTN83171665), a randomised feasibility trial for the selection of therapy for patients with ACRC based on their KRAS and BRAF mutation status as well as their topoisomerase 1 (topo-1) expression status.

Materials and Methods

Trial design.

Patients were registered on the day they provided written consent for the release of a tumour sample. Upon determination of their biomarker status, patients were allocated to one of four molecular subgroups for randomisation: (1) low topo-1 expression levels and both KRAS and BRAF wild type, (2) low topo-1 and either KRAS- or BRAF -activating mutations, (3) high topo-1 and both KRAS and BRAF wild type and (4) high topo-1 and either KRAS or BRAF mutations. These randomisation subgroups correspond to the prior hypotheses that: (1) in patients with low topo-1 tumours, FU alone is similarly effective and therefore preferable to irinotecan/FU combination ( Braun et al, 2008 ); (2) in patients with KRAS/BRAF wild-type tumours, anti-EGFR therapy improves outcomes ( Van Cutsem et al, 2009 ); (3) in patients with high topo-1 tumours, addition of oxaliplatin to irinotecan/FU improves outcomes ( Braun et al, 2008 ) and (4) in patients with KRAS/BRAF-mutated tumours, anti-VEGF therapy might improve outcomes. There was no specific rationale for a biologically targeted therapy in patients with KRAS mutations; however, there were data suggesting benefit of bevacizumab ( Ince et al, 2005 ).

Patients were randomised centrally by the MRC CTU via telephone using minimisation and allocated in a 1 : 1 : 1 ratio to the control arm (A) common to each of the four subgroups or one of two experimental regimens ( Figures 1 and 2 ). If either molecular test failed, patients could still be randomised in a 1 : 1 ratio based on the results available ( Figure 1 ). Treatment allocation was not masked. Randomisation was stratified by standard clinical prognostic factors.

Trial design.

Diagram in patient information sheet 1 – given to patients to explain the tests carried out on their tumour sample.

Eligibility criteria were age ⩾ 18 years, colorectal adenocarcinoma, inoperable metastatic or locoregional RECIST measurable disease, no previous chemotherapy for metastases, WHO performance status 0–2 and good organ function ( Maughan and Meade, 2010 ). Written informed consent for both molecular testing and randomisation was required.

Outcome measures and sample size

The primary outcome measures for FOCUS3 were process outcomes, namely, in this national multi-site setting, how frequently the target could be met of ⩽ 10 w.d. between the date of registration and: (1) the provision of results to the investigator and (2) randomisation.

The target sample size was 240 patients; if >226 tumour blocks were processed within 10 w.d., we could reliably state that ⩾ 90% samples could be analysed within that time frame. If <206 blocks were processed within 10 w.d., we could reliably exclude a turnaround rate of 90% (i.e., the upper 95% confidence limit would exclude 90%).

Secondary outcome measures included toxicity, response rates (RRs) and progression-free survival (PFS) of the different regimens within each molecular subgroup; reproducibility of biomarker results and attitudes of patients to the study design, the consent process and refusal rates for trial entry.

Informed consent and patients attitudes to the trial design

A staged set of patient information sheets (PISs) was developed with input from patients, carers and nursing staff: PIS 1 explained the need for further analyses of tumour tissue using a very simple diagram and no technical details (see Figure 2 ), PIS2, given to patients before results of their molecular tests were known, covered the general issues of a three arm RCT and treatment side-effects. PIS3, in four specific versions a–d, describing the three arm randomisation for each of the four molecular sub-types (1–4) was given to patients before randomisation. PIS4, versions a–e, contained full details of the five treatment regimens (A–E).

Patient understanding of the information was captured on a questionnaire delivered immediately following their reading of the stage 2 PIS.

Attitudes of participants to trial entry, understanding and experience, particularly to the proposed 2 weeks time for tumour testing before treatment allocation, were evaluated by one-to-one semi-structured interviews using interpretative phenomenological analysis in a subgroup of randomised patients ( Smith and Osborn, 2003 ).

Sample collection and analysis process

The clinical research nurse (CRN) at the recruiting hospital requested the patients’ diagnostic FFPE block. Histopathology agreements were in place between MRC and all diagnostic hospitals outlining the trial rationale and stressing the importance of sending blocks promptly to the central laboratories. The MRC CTU team actively tracked samples throughout the biomarker analysis process. Upon reconfirmation of eligibility, and with their consent, patients were randomised.

Biomarker analysis

Analysis of DNA extracted from macro-dissected FFPE sections of KRAS codons 12, 13 and 61 and BRAF codon 600 was each performed by Pyrosequencing (details in Supplementary Appendix ).

Topo-1 protein expression was identified using a topo-1 antibody (NCL-TOPO1; Leica, Wetzlar, Germany; details in Supplementary Appendix ). Each case was scored on the basis of the percentage of positive tumour cells (<10% scored low, >10% high).

Quality assurance of biomarker analysis

Fifty samples were blinded and exchanged between the two laboratories before the trial and analysed for KRAS and BRAF mutation status. Throughout the trial both laboratories took part in external quality assessment (UK NEQAS) for KRAS . Topo-1 IHC was compared between laboratories.

Interventions and assessments

The five treatment regimens were all based on the 2-weekly FOLFIRI regimen – folinic acid and irinotecan followed by bolus and infusional 5-fluouracil (5-FU; Douillard et al, 2000 ): (A) Control: FOLFIRI, (B) omits irinotecan: LV5FU2, (C) adds oxaliplatin: FOLFOXIRI (FOLFIRI and oxaliplatin), (D) FOLFIRI plus cetuximab and (E) FOLFIRI plus bevacizumab. Doses in (C) were dependent on patient age and WHO performance status. The chemotherapy regimens FOLFIRI and LV5FU2 are internationally recognised acronyms. The actual regimens used in FOCUS3 were established in the UK ( Cheeseman et al, 2002 ; Leonard et al, 2002 ). They have been used in large numbers of patients, have been shown to be both efficacious and safe ( Seymour et al, 2007 ) and will be referred to as FOLFIRI and LV5FU2 in this paper. The FOLFIRI regimen consisted of an IV infusion of 180 mg m −2 IV infusion over 30 min followed by 350 mg IV infusion d,l-folinic acid or 175 mg l-folinic acid over 2 h. A 400 mg m −2 IV bolus injection of 5-FU was then administered over 5 min followed by 2400 mg m −2 5-flurouracil IV infusion over 46 h. For the LV5FU2 regimen, irinotecan was omitted and the 5-fluourouracil IV infusion dose was increased to 2800 mg m −2 . There were three different FOLFOXIRI regimens, which were prescribed based on the patient’s age and WHO PS status. The regimen for patients aged 70 years or less and with PS=0–1 contained 180 mg m −2 irinotecan and 85 mg m −2 oxaliplatin, 400 mg m −2 5-fluorouracil bolus and 2400 mg m −2 5-fluorouracil infusion. The individual components were reduced to 80% of full dose for patients ⩾ 70 years or PS=2 and to 60% for patients ⩾ 70 years and PS=2. In arm D, cetuximab was administered before chemotherapy as an IV dose of 500 mg m −2 , whereas in arm E bevacizumab was administered first as a 5 mg kg −1 IV infusion. All of the regimens are described in detail in the FOCUS 3 protocol ( Maughan and Meade, 2010 ).

If molecular results were not confirmed by 2 weeks, patients could have one cycle of LV5FU2 before randomisation. Treatment continued for at least 24 weeks or until disease progression on treatment.

Patient symptoms were scored using National Cancer Institute Common Toxicity Criteria for Adverse Events version 3.0. SAEs and deaths, together with an assessment of causality, were continuously reported; and were reassessed by an experienced oncologist on behalf of the MRC.

CT scans were performed within 5 weeks before the start of treatment and then 12 weekly on treatment and evaluated using RECIST (v1.1) criteria. Responses were not confirmed by repeat scans and external radiological review was not undertaken.

Statistical methods

Analyses were conducted according to a predefined statistical analysis plan, which was approved by the FOCUS3 TMG before database lock (first analysed in August 2011, data updated for final analysis in May 2012).

For each of the co-primary process outcomes, an exact binomial 95% confidence interval was calculated around the result. Exploratory analyses of the efficacy end points were planned in relation to the four hypotheses stated above (Trial Design), which in each case involved factorial analysis of two relevant molecular subgroups, as illustrated in Figure 1 . Time-to-event curves for analysis of PFS were estimated using the Kaplan–Meier method. All statistical analyses were carried out using Stata version 12 (StataCorp, College Station, TX, USA).

Between February 2010 and April 2011, 332 patients from 24 centres in the UK were registered for the FOCUS3 trial.

Topo-1 status was determined in 306 patients (92%) and was highly expressed (2–3) in 244 (73%). KRAS and BRAF status were determined in 319 patients (96%), of whom 117 (37%) had a KRAS mutation alone, 25 (8%) BRAF mutation alone, 1 (<1%) both mutations, 169 (53%) were double wild type and 7 (2%) had a BRAF mutation but inconclusive KRAS status. No association was seen between topo-1 expression and KRAS / BRAF mutation status ( Table 1 ).

Of patients registered, 288 were eligible for randomisation, and ultimately 244 (85%) were randomised. The reasons why patients were not randomised are described in Figure 3 (Consort Diagram). The main baseline characteristics and treatment allocation of all randomised patients are shown in Table 2 (and in Supplementary Tables 1 and 2 ) and Figure 3 . The distribution of KRAS/BRAF and Topo-1 status both at registration and randomisation is shown in Table 1 .

CONSORT diagram.

Primary process outcomes

The two co-primary process outcome measures were not met. Of those patients randomised 180 (74%) had their biomarker results within 10 w.d. of registration (95% CI=68%, 79%). However, the results for 225 patients (92%) were available to investigators within 15 w.d. of randomisation (95% CI=88%, 95%). The interval between registration and randomisation was less than or equal to 10 w.d. in only 70 (29%) patients (95% CI=23%, 35%), which suggests delays due to clinical issues (such as visit scheduling after results were available) had a greater impact on timelines than delays in biomarker analysis ( Supplementary Table 3 ).

Reproducibility of biomarker results

100% concordance was achieved in the DNA mutation analysis results obtained between the two reference laboratories. Initial crossing over of topo-1 samples between the laboratories produced consistent results, although there were a higher proportion of ‘high’ expressing tumours than was observed in FOCUS. The Cardiff centre was not able to fully adopt the previously validated Leeds laboratory topo-1 protocol, and early in the trial it was realised that the protocols adopted at the two centres were not giving uniformly consistent results required for trial purposes. All subsequent sample testing for KRAS , BRAF and topo-1 was therefore performed at Leeds.

Patient understanding

In all, 90–95% of participants self-reported that they either fully or mostly understood all of the aspects of the trial, see Figure 4 . The areas that were least well understood were the need to wait 2 weeks before start of treatment, how treatment was allocated and what happens during treatment.

Patient understanding of the consent process. Q1: Understanding of PIS2. Q2: Understanding why tumour was tested. Q3: Understanding of different treatments. Q4: Understanding of why you had to wait 2 weeks. Q5: Understanding of how treatment was allocated. Q6: Understanding of what happens during treatment. Q7: Understanding of request to give blood, complete questionnaire, take part in an interview.

Qualitative research

In-depth, interviews with 14 randomised patients were analysed using interpretative phenomenological analysis and will be published in full elsewhere. The dominant issue for the majority of participants was that they were discussing the trial immediately following diagnosis of ACRC. This was a greater concern than trial entry itself. Two of the fourteen interviewees experienced delays with results from tumour testing, causing significant distress. The majority of patients expressed no concern with tumour testing times but highlighted distress caused by prior delays during diagnosis and treatment.

Relationships with family were key to ongoing practical and emotional support and particularly relevant to the decision to enrol on the trial and the processing of information. The multiple roles of the CRN emerged in relation to recruitment and the ongoing care of participants in the trial. Reasons for enrolling in FOCUS3 related to altruism, perception of the trial as offering personalised treatment and better care, finding a cure for cancer and being the only option available.

Treatment and follow-up

Of the 244 randomised patients, 4 did not commence treatment—2 from arm A and 2 from arm E. Of the remaining 240, two patients (0.8%) received a single initial cycle of LV5FU2 alone before commencing their allocated regimens. Full-dose FOLFOXIRI was initiated in the 86% of patients with high topo-1 who were <70 and PFS 0–1; the remainder commenced at lower doses as per protocol. The median number of cycles of treatment delivered was 12 (IQR=7–13).

Efficacy outcomes

Efficacy outcomes were assessed in May 2012 when the median duration of follow-up was 15.2 months (IQR=12.6–18.8 months).

In patients with low topo-1 (B vs A, n =30), 12-week RR was 60% with LV5FU2 alone and 47% with FOLFIRI, supporting the original hypothesis that irinotecan does not add benefit in this group. There was no evidence of a difference in PFS.

There was no improvement in RR (40% vs 45%) or in PFS (HR=1.08 (0.67–1.76)) with the addition of oxaliplatin ( n =127) to FOLFIRI (C vs A). The complex randomisation algorithm resulted in a gender imbalance with more males in this group, which has uncertain relevance.

In patients with KRAS and BRAF wild type (D vs A, n =92), the addition of cetuximab to FOLFIRI was associated with an increased RR (44% vs 66%) and PFS (HR=0.44 (0.23–0.82)), consistent with the results of the phase III Crystal trial ( Van Cutsem et al, 2009 , 2011 ).

For the addition of bevacizumab to FOLFIRI in patients with KRAS or BRAF mutations (E vs A, n =72), there was an observed increased RR (47% vs 33%). No PFS benefits were observed.

Kaplan–Meier survival curves are presented in Figure 5 and 12-week RR data are summarised in Table 3 .

Treatment comparisons – progression-free survival.

Toxicity observed was as expected for the LV5FU2, FOLFIRI, FOLFIRI+cetuximab and FOLFIRI+bevacizumab regimens. The anticipated increased toxicity of the FOLFOXIRI regimen was minimal, with only 27% grade 3+ neutropenia. This may be due to the reduced dosing schedule in the elderly/less fit patients ( n =9 of 127) previously described ( Supplementary Table 4 ).

The primary objective of FOCUS3 was to assess the feasibility of undertaking a complex biomarker-driven trial in a national multicentre setting. Although the study did not meet either of its ambitious pre-specified co-primary process outcome measures, the trial has shown that complex prospective biomarker-driven RCTs are possible on a substantial scale across the United Kingdom. Extra resources are required in the reference pathology laboratories to undertake the biomarker analyses, but within investigator sites and the trials office there is no requirement for special dedicated staff.

Potentially eligible patients were necessarily approached for consent at precisely the time when they had recently learned of the life-threatening status of their disease; our qualitative research showed this was the dominating concern in their minds. That we achieved our target patient number from 24 centres in 1 year demonstrated that the strategy for explaining the trial was successful and that, even under difficult circumstances, complex trials can be attractive to patients. Our four-step consent procedure was developed in consultation with patients and carers and was praised by the research ethics committee. The responses to the questionnaire administered after patients had read their stage 2 PIS showed high levels of understanding of the trial. The subsequent steps in the consent process, with specific patient consent forms for each molecular cohort and for each treatment, avoided information overload and provided only that information that was specifically relevant to the particular patient.

The logistics of retrieval of the FFPE blocks from the diagnostic hospitals was a major concern. Prior written agreement, a modest (£15) fee for retrieval and detailed sample tracking by CTU personnel minimised delay. The critical lessons were the need for excellent communication between all parties in the chain: from CRN to pathologist to the central laboratories to the coordinating trials unit.

A delay in reporting analysis results back to the MRC CTU was observed in 22 cases and was distressing to some patients. The delays were due to insufficient tumour in the block ( n =4), unexpected technical difficulties ( n =6), initial testing inconclusive or failed ( n =12). This was mitigated by allowing patients ( n =2) to start cycle one of chemotherapy using the infusional 5-FU and folinic acid backbone, which was common to all treatment protocols and then adding in the relevant additional agents for cycle 2 once the biomarker results were available.

Overall, the most important laboratory issue was reproducibility of IHC results. Although 100% concordance was achieved in the calling of KRAS and BRAF mutations between the two laboratories, it proved very difficult to perform and report the topo-1 IHC staining intensity in a sufficiently comparable way. Owing to technical- and manpower-based organisational limitations, it was not possible to completely replicate the manual staining methodology adopted initially by the Leeds laboratory in the Cardiff laboratory where an automated staining platform was used. Even what were deemed inconsequential differences between staining protocols contributed to this lack of consistency. For future studies, contributing diagnostic centres will use the same antibodies, protocol and automated staining platform. Detailed guidance on scoring, blinded replication in contributing centres with face to face comparison of discrepantly scored sections have been implemented for IHC tests in FOCUS4. On trial quality assurance by double reading of slides will ensure comparability of evaluation.

This trial was structured so that we could address four distinct hypotheses, any or all of which might be the subject of a subsequent phase III trial. Our first hypothesis, arising from the observation in the earlier FOCUS trial that patients with low topo-1 expression appear to gain no benefit from the addition of irinotecan to LV5FU2 ( Seymour et al, 2007 ; Braun et al, 2008 ), was supported and remains an intriguing one. Only 30 patients were randomised to this comparison because of the lower than expected rate of low topo-1 expression, but the high RR (60%) in the LV5FU2 only treated patients suggests further work in this area might be rewarding.

The second hypothesis proposed that patients with high topo-1 expression, who alone in FOCUS gained benefit from either irinotecan or oxaliplatin in comparison to 5-FU ( Braun et al, 2008 ), may derive additional benefit from the triple chemotherapy regimen. With the protocol-specified dose reductions, the regimen was well tolerated. However, in contrast to the international literature ( Falcone et al, 2007 , 2013 ), although patients had a minimally higher RR, there was no hint of a PFS benefit.

The third hypothesis, tested in 92 patients with KRAS and BRAF wild-type tumours, was that the addition of cetuximab would increase efficacy. This recapitulated the Crystal study ( Van Cutsem et al, 2009 , 2011 ) and benefits in PFS and RR were observed.

Finally, our fourth hypothesis for patients with KRAS or BRAF mutations (72 patients) was based on the limited data that bevacizumab retains efficacy in these patients ( Ince et al, 2005 ). No benefits on either RR or PFS were observed.

The FOCUS4 trial programme ( Kaplan et al, 2013 ) has recently opened to recruitment building on many of the lessons learned in FOCUS3. Patient and clinician enthusiasm for biomarker-stratified trials and the rapid accrual observed in FOCUS3 have encouraged us to be optimistic in our predicted recruitment targets: 2400 registered patients with over 1500 randomised into multiple biomarker-directed comparisons in 4 years for FOCUS4. Staged PISs have been designed with information given at the time of registration limited to that which is necessary for consent for release of tumour blocks, plus a minimal outline of the protocol so as to avoid information overload. Detailed quality assurance work has been undertaken between the two biomarker reference laboratories, especially for the IHC tests (PTEN and mismatch repair proteins). In FOCUS4, the allocation by biomarker to specific comparisons occurs for patients with stable or responding disease after 4 months of first-line chemotherapy. Knowing that in FOCUS3 we completed biomarker analysis in 99% of patients within 20 w.d. of consent, the FOCUS4 logistics (registration of patients up to 12 weeks into their first-line chemotherapy) should facilitate accrual. Detailed engagement with pathologists in referring hospitals and a relatively small (£15) payment per case enabled rapid release of blocks for central analysis in FOCUS3 and the same pattern has been used in FOCUS4. Perhaps most important is the strength of the team working established through FOCUS3, including patient representatives, clinicians, biomarker experts (including histopathologists, immunohistochemists, geneticists and technicians), statisticians, research nurses, pharmacists, trial managers and data managers. To this, we have added research network managers to ensure improved patient transfers between district general hospitals and experimental cancer medicine centres, who are required in FOCUS4 for some patients randomised to the novel agent combinations being studied.

The FOCUS3 trial was a feasibility study designed to address the challenges of patient acceptability, technical logistics, and to test a novel design for examining the predictive role of biomarkers for first-line therapy of ACRC. We have shown that such studies are feasible and very well received by participants. The central trial design concepts have been taken forward into a major UK trial programme FOCUS4-molecular selection of therapy in CRC: a molecularly stratified RCT programme, which opened to accrual in January 2014 ( Kaplan et al, 2013 ).

Change history

29 april 2014.

This paper was modified 12 months after initial publication to switch to Creative Commons licence terms, as noted at publication

Braun MS, Richman SD, Quirke P, Daly C, Adlard JW, Elliott F, Barrett JH, Selby P, Meade AM, Stephens RJ, Parmar MK, Seymour MT (2008) Predictive biomarkers of chemotherapy efficacy in colorectal cancer: results from the UK MRC FOCUS trial. J Clin Oncol 26 : 2690–2698.

Article   CAS   Google Scholar  

Cheeseman SL, Joel SP, Chester JD, Wilson G, Dent JT, Richards FJ, Seymour MT (2002) A /'modified de Gramont/' regimen of fluorouracil, alone and with oxaliplatin, for advanced colorectal cancer. Br J Cancer 87 : 393–399.

De Roock W, Claes B, Bernasconi D, De Schutter J, Biesmans B, Fountzilas G, Kalogeras KT, Kotoula V, Papamichael D, Laurent-Puig P, Penault-Llorca F, Rougier P, Vincenzi B, Santini D, Tonini G, Cappuzzo F, Frattini M, Molinari F, Saletti P, De Dosso S, Martini M, Bardelli A, Siena S, Sartore-Bianchi A, Tabernero J, Macarulla T, Di Fiore F, Gangloff AO, Ciardiello F, Pfeiffer P, Qvortrup C, Hansen TP, Van Cutsem E, Piessevaux H, Lambrechts D, Delorenzi M, Tejpar S (2010) Effects of KRAS, BRAF, NRAS, and PIK3CA mutations on the efficacy of cetuximab plus chemotherapy in chemotherapy-refractory metastatic colorectal cancer: a retrospective consortium analysis. Lancet Oncol 11 : 753–762.

Douillard JY, Cunningham D, Roth AD, Navarro M, James RD, Karasek P, Jandik P, Iveson T, Carmichael J, Alakl M, Gruia G, Awad L, Rougier P (2000) Irinotecan combined with fluorouracil compared with fluorouracil alone as first-line treatment for metastatic colorectal cancer: a multicentre randomised trial. Lancet 355 : 1041–1047.

Douillard JY, Oliner KS, Siena S, Tabernero J, Burkes R, Barugel M, Humblet Y, Bodoky G, Cunningham D, Jassem J, Rivera F, Kocákova I, Ruff P, Błasińska-Morawiec M, Šmakal M, Canon JL, Rother M, Williams R, Rong A, Wiezorek J, Sidhu R, Patterson SD (2013) Panitumumab-FOLFOX4 treatment and RAS mutations in colorectal cancer. N Engl J Med 369 (11): 1023–1034.

Falcone A, Cremolini C, Masi G, Lonardi S, Zagonel V, Salvatore L, Trenta P, Tomasello G, Ronzoni M, Ciuffreda L, Zaniboni A, Tonini G, Buonadonna A, Valsuani C, Chiara S, Carlomagno C, Boni C, Marcucci L, Boni L, Loupakis F (2013) FOLFOXIRI/bevacizumab (bev) versus FOLFIRI/bev as first-line treatment in unresectable metastatic colorectal cancer (mCRC) patients (pts): Results of the phase III TRIBE trial by GONO group. J Clin Oncol 31 (suppl): abstr 3505.

Google Scholar  

Falcone A, Ricci S, Brunetti I, Pfanner E, Allegrini G, Barbara C, Crino L, Benedetti G, Evangelista W, Fanchini L, Cortesi E, Picone V, Vitello S, Chiara S, Granetto C, Porcile G, Fioretto L, Orlandini C, Andreuccetti M, Masi G (2007) Phase III trial of infusional fluorouracil, leucovorin, oxaliplatin, and irinotecan (FOLFOXIRI) compared with infusional fluorouracil, leucovorin, and irinotecan (FOLFIRI) as first-line treatment for metastatic colorectal cancer: the Gruppo Oncologico Nord Ovest. J Clin Oncol 25 : 1670–1676.

Gerlinger M, Rowan AJ, Horswell S, Larkin J, Endesfelder D, Gronroos E, Martinez P, Matthews N, Stewart A, Tarpey P, Varela I, Phillimore B, Begum S, Mcdonald NQ, Butler A, Jones D, Raine K, Latimer C, Santos CR, Nohadani M, Eklund AC, Spencer-Dene B, Clark G, Pickering L, Stamp G, Gore M, Szallasi Z, Downward J, Futreal PA, Swanton C (2012) Intratumor heterogeneity and branched evolution revealed by multiregion sequencing. N Engl J Med 366 : 883–892.

Greaves M, Maley CC (2012) Clonal evolution in cancer. Nature 481 : 306–313.

Ince WL, Jubb AM, Holden SN, Holmgren EB, Tobin P, Sridhar M, Hurwitz HI, Kabbinavar F, Novotny WF, Hillan KJ, Koeppen H (2005) Association of k-ras, b-raf, and p53 status with the treatment effect of bevacizumab. J Natl Cancer Inst 97 : 981–989.

Kaplan R, Maughan T, Crook A, Fisher D, Wilson R, Brown L, Parmar M (2013) Evaluating many treatments and biomarkers in oncology: a new design. J Clin Oncol 31 : 4562–4568.

Article   Google Scholar  

Karapetis CS, Khambata-Ford S, Jonker DJ, O'callaghan CJ, Tu D, Tebbutt NC, Simes RJ, Chalchal H, Shapiro JD, Robitaille S, Price TJ, Shepherd L, Au HJ, Langer C, Moore MJ, Zalcberg JR (2008) K-ras mutations and benefit from cetuximab in advanced colorectal cancer. N Engl J Med 359 : 1757–1765.

Leonard P, Seymour MT, James R, Hochhauser D, Ledermann JA (2002) Phase II study of irinotecan with bolus and high dose infusional 5-FU and folinic acid (modified de Gramont) for first or second line treatment of advanced or metastatic colorectal cancer. Br J Cancer 87 : 1216–1220.

Lievre A, Bachet JB, Le Corre D, Boige V, Landi B, Emile JF, Cote JF, Tomasic G, Penna C, Ducreux M, Rougier P, Penault-Llorca F, Laurent-Puig P (2006) KRAS mutation status is predictive of response to cetuximab therapy in colorectal cancer. Cancer Res 66 : 3992–3995.

Maughan TS, Adams RA, Smith CG, Meade AM, Seymour MT, Wilson RH, Idziaszczyk S, Harris R, Fisher D, Kenny SL, Kay E, Mitchell JK, Madi A, Jasani B, James MD, Bridgewater J, Kennedy MJ, Claes B, Lambrechts D, Kaplan R, Cheadle JP (2011) Addition of cetuximab to oxaliplatin-based first-line combination chemotherapy for treatment of advanced colorectal cancer: results of the randomised phase 3 MRC COIN trial. Lancet 377 : 2103–2114.

Maughan TS, Meade AM (2010) FOCUS 3 (CR12) protocol: a study to determine the feasibility of molecular selection of therapy using KRAS, BRAK and topo-1 in patients with metastatic or locally advanced colorectal cancer. Avialable at: http://www.ctu.mrc.ac.uk/plugins/StudyDisplay/protocols/FOCUS%203%20Protocol%20and%20appendices%20v4%200%20Nov%202010.pdf .

Sequist LV, Waltman BA, Dias-Santagata D, Digumarthy S, Turke AB, Fidias P, Bergethon K, Shaw AT, Gettinger S, Cosper AK, Akhavanfard S, Heist RS, Temel J, Christensen JG, Wain JC, Lynch TJ, Vernovsky K, Mark EJ, Lanuti M, Iafrate AJ, Mino-Kenudson M, Engelman JA (2011) Genotypic and histological evolution of lung cancers acquiring resistance to EGFR inhibitors. Sci Transl Med 3 : 75ra26.

Seymour MT, Brown SR, Middleton G, Maughan T, Richman S, Gwyther S, Lowe C, Seligmann JF, Wadsley J, Maisey N, Chau I, Hill M, Dawson L, Falk S, O'callaghan A, Benstead K, Chambers P, Oliver A, Marshall H, Napp V, Quirke P (2013) Panitumumab and irinotecan versus irinotecan alone for patients with KRAS wild-type, fluorouracil-resistant advanced colorectal cancer (PICCOLO): a prospectively stratified randomised trial. Lancet Oncol 14 : 749–759.

Seymour MT, Maughan TS, Ledermann JA, Topham C, James R, Gwyther SJ, Smith DB, Shepherd S, Maraveyas A, Ferry DR, Meade AM, Thompson L, Griffiths GO, Parmar MK, Stephens RJ (2007) Different strategies of sequential and combination chemotherapy for patients with poor prognosis advanced colorectal cancer (MRC FOCUS): a randomised controlled trial. Lancet 370 : 143–152.

Smith JA, Osborn M (2003) Interpretative phenomenological analysis. In: Qualitative Psychology . A Practical Guide to Research Methods Smith JA, (ed), pp 51–80. Sage: London.

The Cancer Genome Network Atlas (2012) Comprehensive molecular characterization of human colon and rectal cancer. Nature 487 : 330–337.

Van Cutsem E, Köhne CH, Láng I, Folprecht G, Nowacki M, Cascinu S, Shchepotin I, Maurel J, Cunningham D, Tejpar S, Schlichting M, Zubel A, Celik I, Rougier P, Ciardiello F (2011) Cetuximab plus irinotecan, fluorouracil, and leucovorin as first-line treatment for metastatic colorectal cancer: updated analysis of overall survival according to tumor KRAS and BRAF mutation status. J Clin Oncol 29 : 2011–2019.

Van Cutsem E, Kohne CH, Hitre E, Zaluski J, Chang Chien CR, Makhson A, D'haens G, Pinter T, Lim R, Bodoky G, Roh JK, Folprecht G, Ruff P, Stroh C, Tejpar S, Schlichting M, Nippgen J, Rougier P (2009) Cetuximab and chemotherapy as initial treatment for metastatic colorectal cancer. N Engl J Med 360 : 1408–1417.

Download references

Acknowledgements

We are indebted to the 332 patients and their families who participated in FOCUS3.

The design of the Medical Research Council (MRC) FOCUS3 trial was conceived and developed by the National Cancer Research Institute (NCRI) advanced colorectal cancer group. The trial was funded by the MRC. Additional support was provided by Merck KGaA (free cetuximab), Pfizer and Roche (educational research grants for the MRC colorectal research portfolio). The topo-1 antibody was provided free from Leica. Laboratory work in Leeds was also supported by funding from Yorkshire Cancer Research and the Leeds Experimental Cancer Medicines Centre. All tumour samples from patients who consented for future CRC research are stored at the Wales Cancer Bank.

The MRC was the overall sponsor of the study. FOCUS3 was approved by the Medicines and Healthcare Regulatory Agency (MHRA) on 12 June 2009 and Research Ethics Committee for Wales on 26 May 2009. The trial was coordinated by the MRC Clinical Trials Unit (CTU) following the principles of GCP, conducted with a Trial Management Group (TMG), monitored by a Data Monitoring Committee (DMC) and overseen by an independent Trial Steering Committee. Data collection at UK sites was supported by staff funding from the National Cancer Research Networks. All statistical analyses were performed at the MRC CTU. The trial is registered as an International Standard Randomised Controlled trial, number ISRCTN83171665.

Trial Management Group : TS Maughan (chair), R Adams, RH Wilson, MT Seymour, B Jasani, R Butler, S Richman, P Quirke, AM Nelson, GT Williams, G Taylor, H Grabsch, I Frayling, J Sampson, E Hodgkinson, P Rogers, M Pope and MRC CTU staff.

MRC Clinical Trials Unit: AM Meade, R Kaplan, D Fisher, SL Kenny, JK Mitchell, LL Nichols, L Harper, K Letchemanan, M Parmar.

Data Monitoring Committee: AM Meade, R Kaplan, D Fisher, TS Maughan, MT Seymour.

Trial Steering Committee: C Parker (current chair), R Rudd, J Whelan.

Sponsor: Medical Research Council.

Clinical Investigators (Institution—(number of patients contributed)): Bridgewater J, King J, Aggarwal A, Harinarayanan S, Melcher L, Karp Stephen (North Middlesex Hospital (32)), Furniss D, Wadsley J, Walkington L, Simmons T, Hornbuckle J, Pledge S, Clenton S (Weston Park Hospital (30)), Roy R, Dhadda A (Castle Hill Hospital (26)), Adams R, Maughan T, Jones R, Brewster A, Iqbal N, Arif, Crosby T (Velindre Hospital (23)), Falk S, Garadi K, Hopkins K (Bristol Haematology and Oncology Centre (18)), Seymour M, Swinson D, Anthoney A, (St James’ University Hospital, Leeds (18)), Leonard P, Mohamed M, (Whittington Hospital (14)), Benstead K, Farrugia D, Shepherd S (Cheltenham General Hospital (11)), Blesing C, Hyde K, Grant W (Great Western Hospital (10)), Lowdell C, Cleator S, Riddle P, Kenny L, Ahmad R (Charing Cross Hospital (9)), Hill M, Bhattacharjee P, Sevitt T, Summers J, Shah R (Maidstone Hospital (9)), Whillis D, Nicholls A, Ireland H, Macgregor C (Raigmore Hospital (8)), Sizer B, Basu D (Essex County Hospital (7)), Dent J, Hofmann U (Huddersfield Royal Infirmary (6)), Roy R, Butt M, Iqbal M (Diana, Princess of Wales Hospital (6)), Dent J (Calderdale Royal Hospital (6)), Hickish T, Osborne R (Poole Hospital (3)), Hickish T, Astras G, Purandare L (Royal Bournemouth Hospital (2)), Tahir S, Srinivasan G (Broomfield Hospital (2)), Gollins S, Kodavatiganti R (Wrexham Maelor Hospital (2)), Bale C, Mullard A, Fuller C, Williams R, Stuart N (Ysbyty Gwynedd (1)), Gollins S, Neupane R (Glan Clwyd Hospital (1)), Bessell E, Potter V (Nottingham University Hospital (0)), Tsang D (Southend University Hospital (0)).

In addition to the above-named individuals, we acknowledge the contributions of a large number of clinicians, research nurses, data managers and other clinical and support staff at the participating centres.

Author information

M J Pope and J K Pope: Malcolm and Janet Pope are Consumer Representatives; they also represent Velindre Hospital, Patient Liaison Group, Cardiff CF14 2TL, UK

Authors and Affiliations

CRUK/MRC Oxford Institute for Radiation Oncology, University of Oxford, Oxford OX3 7DQ, UK

T S Maughan

MRC Clinical Trials Unit at UCL, Institute of Clinical Trials and Methodology, London, WC2B 6NH, UK

A M Meade, D Fisher, L L Nichols, S L Kenny, M J Pope, J K Pope, M Parmar & R Kaplan

Cardiff University and Velindre Cancer Centre, Cardiff, UK

Leeds Institute of Cancer and Pathology, University of Leeds, Leeds, LS9 7TF, UK

S D Richman, G R Taylor & P Quirke

University Hospital of Wales, Cardiff, CF14 4XW, UK

Centre for Cancer Research and Cell Biology, Queen’s University Belfast, Belfast, BT9 7AE, UK

Institute of Cancer and Genetics, Cardiff University, Cardiff, CF14 4XN, UK

B Jasani, G T Williams & J R Sampson

St James’s Institute of Oncology, University of Leeds, Leeds, LS9 7TF, UK

M T Seymour

Wales Cancer Trials Unit, Cardiff University, Cardiff, CF14 4YS, UK

A Nelson & C M Sampson

Sheffield Teaching Hospitals NHS Foundation Trust, Sheffield, S5 7AU, UK

E Hodgkinson & D L Furniss

UCL Cancer Institute, London, WC1E 6BT, UK

J A Bridgewater

Department of Oncology, Castle Hill Hospital, East Riding of Yorkshire, HU16 5JQ, UK

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to A M Meade .

Additional information

This work is published under the standard license to publish agreement. After 12 months the work will become freely available and the license terms will switch to a Creative Commons Attribution-NonCommercial-Share Alike 3.0 Unported License.

Supplementary Information accompanies this paper on British Journal of Cancer website

Supplementary information

Supplementary information (doc 401 kb), rights and permissions.

From twelve months after its original publication, this work is licensed under the Creative Commons Attribution-NonCommercial-Share Alike 3.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-sa/3.0/

Reprints and permissions

About this article

Cite this article.

Maughan, T., Meade, A., Adams, R. et al. A feasibility study testing four hypotheses with phase II outcomes in advanced colorectal cancer (MRC FOCUS3): a model for randomised controlled trials in the era of personalised medicine?. Br J Cancer 110 , 2178–2186 (2014). https://doi.org/10.1038/bjc.2014.182

Download citation

Received : 21 November 2013

Revised : 27 February 2014

Accepted : 13 March 2014

Published : 17 April 2014

Issue Date : 29 April 2014

DOI : https://doi.org/10.1038/bjc.2014.182

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• colorectal cancer
• multi-arm trials
• personalised medicine

This article is cited by

Patient understanding and acceptability of an early lung cancer diagnosis trial: a qualitative study.

• Hayley C. Prout
• Allan Barham
• Annmarie Nelson

Trials (2018)

A phase II study of weekly irinotecan in patients with locally advanced or metastatic HER2- negative breast cancer and increased copy numbers of the topoisomerase 1 (TOP1) gene: a study protocol

• Iben Kümler
• Eva Balslev
• Dorte Nielsen

BMC Cancer (2015)

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Advisory boards aren’t only for executives. Join the LogRocket Content Advisory Board today →

• Product Management
• Solve User-Reported Issues
• Find Issues Faster
• Optimize Conversion and Adoption

How to conduct a feasibility study: Template and examples

Opportunities are everywhere. Some opportunities are small and don’t require many resources. Others are massive and need further analysis and evaluation.

One of your key responsibilities as a product manager is to evaluate the potential success of those opportunities before investing significant money, time, and resources. A feasibility study, also known as a feasibility assessment or feasibility analysis, is a critical tool that can help product managers determine whether a product idea or opportunity is viable, feasible, and profitable.

So, what is a feasibility analysis? Why should product managers use it? And how do you conduct one?

What is a feasibility study?

A feasibility study is a systematic analysis and evaluation of a product opportunity’s potential to succeed. It aims to determine whether a proposed opportunity is financially and technically viable, operationally feasible, and commercially profitable.

A feasibility study typically includes an assessment of a wide range of factors, including the technical requirements of the product, resources needed to develop and launch the product, the potential market gap and demand, the competitive landscape, and economic and financial viability.

Based on the analysis’s findings, the product manager and their product team can decide whether to proceed with the product opportunity, modify its scope, or pursue another opportunity and solve a different problem.

Conducting a feasibility study helps PMs ensure that resources are invested in opportunities that have a high likelihood of success and align with the overall objectives and goals of the product strategy .

What are feasibility analyses used for?

Feasibility studies are particularly useful when introducing entirely new products or verticals. Product managers can use the results of a feasibility study to:

• Assess the technical feasibility of a product opportunity — Evaluate whether the proposed product idea or opportunity can be developed with the available technology, tools, resources, and expertise
• Determine a project’s financial viability — By analyzing the costs of development, manufacturing, and distribution, a feasibility study helps you determine whether your product is financially viable and can generate a positive return on investment (ROI)
• Evaluate customer demand and the competitive landscape — Assessing the potential market size, target audience, and competitive landscape for the product opportunity can inform decisions about the overall product positioning, marketing strategies, and pricing
• Identify potential risks and challenges — Identify potential obstacles or challenges that could impact the success of the identified opportunity, such as regulatory hurdles, operational and legal issues, and technical limitations
• Refine the product concept — The insights gained from a feasibility study can help you refine the product’s concept, make necessary modifications to the scope, and ultimately create a better product that is more likely to succeed in the market and meet users’ expectations

How to conduct a feasibility study

The activities involved in conducting a feasibility study differ from one organization to another. Also, the threshold, expectations, and deliverables change from role to role.

For a general set of guidelines to help you get started, here are some basic steps to conduct and report a feasibility study for major product opportunities or features.

1. Clearly define the opportunity

Imagine your user base is facing a significant problem that your product doesn’t solve. This is an opportunity. Define the opportunity clearly, support it with data, talk to your stakeholders to understand the opportunity space, and use it to define the objective.

2. Define the objective and scope

Each opportunity should be coupled with a business objective and should align with your product strategy.

Over 200k developers and product managers use LogRocket to create better digital experiences

Determine and clearly communicate the business goals and objectives of the opportunity. Align those objectives with company leaders to make sure everyone is on the same page. Lastly, define the scope of what you plan to build.

3. Conduct market and user research

Now that you have everyone on the same page and the objective and scope of the opportunity clearly defined, gather data and insights on the target market.

Include elements like the total addressable market (TAM) , growth potential, competitors’ insights, and deep insight into users’ problems and preferences collected through techniques like interviews, surveys, observation studies, contextual inquiries, and focus groups.

4. Analyze technical feasibility

Suppose your market and user research have validated the problem you are trying to solve. The next step should be to, alongside your engineers, assess the technical resources and expertise needed to launch the product to the market.

Dig deeper into the proposed solution and try to comprehend the technical limitations and estimated time required for the product to be in your users’ hands.

5. Assess financial viability

If your company hasa product pricing team, work closely with them to determine the willingness to pay (WTP) and devise a monetization strategy for the new feature.

Conduct a comprehensive financial analysis, including the total cost of development, revenue streams, and the expected return on investment (ROI) based on the agreed-upon monetization strategy.

6. Evaluate potential risks

Now that you have almost a complete picture, identify the risks associated with building and launching the opportunity. Risks may include things like regulatory hurdles, technical limitations, and any operational risks.

7. Decide, prepare, and share

Based on the steps above, you should end up with a report that can help you decide whether to pursue the opportunity or not. Either way, prepare your findings, including any recommended modifications to the product scope, and present your final findings and recommendations to your stakeholders.

Make sure to prepare an executive summary for your C-suite; they will be the most critical stakeholders and the decision-makers at the end of the meeting.

Feasibility study example

Imagine you’re a product manager at a digital software company that specializes in building project management tools.

Your team has identified a potential opportunity to expand the product offering by developing a new AI-based feature that can automatically prioritize tasks for users based on their deadlines, workload, and importance.

To assess the viability of this opportunity, you can conduct a feasibility study. Here’s how you might approach it according to the process described above:

• Clearly define the opportunity — In this case, the opportunity is the development of an AI-based task prioritization feature within the existing project management software
• Define the objective and scope — The business objective is to increase user productivity and satisfaction by providing an intelligent task prioritization system. The scope includes the integration of the AI-based feature within the existing software, as well as any necessary training for users to understand and use the feature effectively
• Conduct market and user research — Investigate the demand for AI-driven task prioritization among your target audience. Collect data on competitors who may already be offering similar features and determine the unique selling points of your proposed solution. Conduct user research through interviews, surveys, and focus groups to understand users’ pain points regarding task prioritization and gauge their interest in the proposed feature
• Analyze technical feasibility — Collaborate with your engineering team to assess the technical requirements and challenges of developing the AI-based feature. Determine whether your team has the necessary expertise to implement the feature and estimate the time and resources required for its development
• Assess financial viability — Work with your pricing team to estimate the costs associated with developing, launching, and maintaining the AI-based feature. Analyze the potential revenue streams and calculate the expected ROI based on various pricing models and user adoption rates
• Evaluate potential risks — Identify any risks associated with the development and implementation of the AI-based feature, such as data privacy concerns, potential biases in the AI algorithm, or the impact on the existing product’s performance
• Decide, prepare, and share — Based on your analysis, determine whether the AI-based task prioritization feature is a viable opportunity for your company. Prepare a comprehensive report detailing your findings and recommendations, including any necessary modifications to the product scope or implementation plan. Present your findings to your stakeholders and be prepared to discuss and defend your recommendations

Feasibility study template

The following feasibility study template is designed to help you evaluate the feasibility of a product opportunity and provide a comprehensive report to inform decision-making and guide the development process.

Remember that each study will be unique to your product and market, so you may need to adjust the template to fit your specific needs.

• Briefly describe the product opportunity or feature you’re evaluating
• Explain the problem it aims to solve or the value it will bring to users
• Define the business goals and objectives for the opportunity
• Outline the scope of the product or feature, including any key components or functionality
• Summarize the findings from your market research, including data on the target market, competitors, and unique selling points
• Highlight insights from user research, such as user pain points, preferences, and potential adoption rates
• Detail the technical requirements and challenges for developing the product or feature
• Estimate the resources and expertise needed for implementation, including any necessary software, hardware, or skills
• Provide an overview of the costs associated with the development, launch, and maintenance of the product or feature
• Outline potential revenue streams and calculate the expected ROI based on various pricing models and user adoption rates
• Identify any potential risks or challenges associated with the development, implementation, or market adoption of the product or feature
• Discuss how these risks could impact the success of the opportunity and any potential mitigation strategies
• Based on your analysis, recommend whether to proceed with the opportunity, modify the scope, or explore other alternatives
• Provide a rationale for your recommendation, supported by data and insights from your research
• Summarize the key findings and recommendations from your feasibility study in a concise, easily digestible format for your stakeholders

Overcoming stakeholder management challenges

The ultimate challenge that faces most product managers when conducting a feasibility study is managing stakeholders .

Stakeholders may interfere with your analysis, jumping to conclude that your proposed product or feature won’t work and deeming it a waste of resources. They may even try to prioritize your backlog for you.

Here are some tips to help you deal with even the most difficult stakeholders during a feasibility study:

• Use hard data to make your point — Never defend your opinion based on your assumptions. Always show them data and evidence based on your user research and market analysis
• Learn to say no — You are the voice of customers, and you know their issues and how to monetize them. Don’t be afraid to say no and defend your team’s work as a product manager
• Build stakeholder buy-in early on — Engage stakeholders from the beginning of the feasibility study process by involving them in discussions and seeking their input. This helps create a sense of ownership and ensures that their concerns and insights are considered throughout the study
• Provide regular updates and maintain transparency — Keep stakeholders informed about the progress of the feasibility study by providing regular updates and sharing key findings. This transparency can help build trust, foster collaboration, and prevent misunderstandings or misaligned expectations
• Leverage stakeholder expertise — Recognize and utilize the unique expertise and knowledge that stakeholders bring to the table. By involving them in specific aspects of the feasibility study where their skills and experience can add value, you can strengthen the study’s outcomes and foster a more collaborative working relationship

Final thoughts

A feasibility study is a critical tool to use right after you identify a significant opportunity. It helps you evaluate the potential success of the opportunity, analyze and identify potential challenges, gaps, and risks in the opportunity, and provides a data-driven approach in the market insights to make an informed decision.

By conducting a feasibility study, product teams can determine whether a product idea is profitable, viable, feasible, and thus worth investing resources into. It is a crucial step in the product development process and when considering investments in significant initiatives such as launching a completely new product or vertical.

LogRocket generates product insights that lead to meaningful action

Get your teams on the same page — try LogRocket today.

Share this:

• Click to share on Twitter (Opens in new window)
• Click to share on Reddit (Opens in new window)
• Click to share on LinkedIn (Opens in new window)
• Click to share on Facebook (Opens in new window)
• #product strategy

Stop guessing about your digital experience with LogRocket

Recent posts:.

The PR/FAQ method for product innovation

The PR/FAQ (press release/frequently asked questions) helps you to envision and define a new product before it’s developed.

Leader Spotlight: Transforming retail with GenAI, with Russell Goldman

Russell Goldman talks about the possibilities and challenges associated with bringing GenAI into a combined retail and software space.

A guide to product-led marketing

Product-led marketing is a strategy that focuses on the product itself as the main driver of customer acquisition, conversion, and retention.

A guide to asynchronous communication

Asynchronous communication is a style of communication where participants don’t need to be simultaneously present.

Leave a Reply Cancel reply

Here’s how you know

• U.S. Department of Health and Human Services
• National Institutes of Health

Pilot Studies: Common Uses and Misuses

Although pilot studies are a critical step in the process of intervention development and testing, several misconceptions exist on their true uses and misuses. NCCIH has developed a Framework for Developing and Testing Mind and Body Interventions that includes brief information on pilot studies. Here we offer additional guidance specifically on the do’s and don’ts of pilot work.

A pilot study is defined as “A small-scale test of the methods and procedures to be used on a larger scale” (Porta, Dictionary of Epidemiology , 5th edition, 2008). The goal of pilot work is not to test hypotheses about the effects of an intervention, but rather, to assess the feasibility/acceptability of an approach to be used in a larger scale study. Thus, in a pilot study you are not answering the question “Does this intervention work?” Instead you are gathering information to help you answer “Can I do this?”

Uses of Pilot Studies

There are many aspects of feasibility and acceptability to examine to address the “Can I do this?” question. Here are some examples:

You may be able to think of other feasibility questions relevant to your specific intervention, population, or design. When designing a pilot study, it is important to set clear quantitative benchmarks for feasibility measures by which you will evaluate successful or unsuccessful feasibility (e.g., a benchmark for assessing adherence rates might be that at least 70 percent of participants in each arm will attend at least 8 of 12 scheduled group sessions). These benchmarks should be relevant to the specific treatment conditions and population under study, and thus will vary from one study to another. While using a randomized design is not always necessary for pilot studies, having a comparison group can provide a more realistic examination of recruitment rates, randomization procedures, implementation of interventions, procedures for maintaining blinded assessments, and the potential to assess for differential dropout rates. Feasibility measures are likely to vary between “open-label” designs, where participants know what they are signing up for, versus a randomized design where they will be assigned to a group.

In addition to providing important feasibility data as described above, pilot studies also provide an opportunity for study teams to develop good clinical practices to enhance the rigor and reproducibility of their research. This includes the development of documentation and informed consent procedures, data collection tools, regulatory reporting procedures, and monitoring procedures.

The goal of pilot studies is not to test hypotheses; thus, no inferential statistics should be proposed. Therefore, it is not necessary to provide power analyses for the proposed sample size of your pilot study. Instead, the proposed pilot study sample size should be based on practical considerations including participant flow, budgetary constraints, and the number of participants needed to reasonably evaluate feasibility goals.

This testing of the methods and procedures to be used in a larger scale study is the critical groundwork we wish to support in PAR-14-182 , to pave the way for the larger scale efficacy trial. As part of this process, investigators may also spend time refining their intervention through iterative development and then test the feasibility of their final approach.

Misuses of Pilot Studies

Rather than focusing on feasibility and acceptability, too often, proposed pilot studies focus on inappropriate outcomes, such as determining “preliminary efficacy.” The most common misuses of pilot studies include:

• Attempting to assess safety/tolerability of a treatment,
• Seeking to provide a preliminary test of the research hypothesis, and
• Estimating effect sizes for power calculations of the larger scale study.

Why can’t pilot studies be used to assess safety and tolerability?

Investigators often propose to examine “preliminary safety” of an intervention within a pilot study; however, due to the small sample sizes typically involved in pilot work, they cannot provide useful information on safety except for extreme cases where a death occurs or repeated serious adverse events surface. For most interventions proposed by NCCIH investigators, suspected safety concerns are quite minimal/rare and thus, unlikely to be picked up in a small pilot study. If any safety concerns are detected, group-specific rates with 95 percent confidence intervals should be reported for adverse events. However, if no safety concerns are demonstrated in the pilot study, investigators cannot conclude that the intervention is safe.

Why can’t pilot studies provide a “preliminary test” of the research hypothesis?

We routinely see specific aims for feasibility pilot studies that propose to evaluate “preliminary efficacy” of intervention A for condition X. However, there are two primary reasons why pilot studies cannot be used for this purpose. First, at the time a pilot study is conducted, there is a limited state of knowledge about the best methods to implement the intervention in the patient population under study. Therefore, conclusions about whether the intervention “works” are premature because you don’t yet know whether you implemented it correctly. Second, due to the smaller sample sizes used in pilot studies, they are not powered to answer questions about efficacy. Thus, any estimated effect size is uninterpretable—you do not know whether the “preliminary test” has returned a true result, a false positive result, or a false negative result (see Figure 1).

Why can’t pilot studies estimate effect sizes for power calculations of the larger scale study?

Since any effect size estimated from a pilot study is unstable, it does not provide a useful estimation for power calculations. If a very large effect size was observed in a pilot study and it achieves statistical significance, it only proves that the true effect is likely not zero, but the observed magnitude of the effect may be overestimating the true effect. Power calculations for the subsequent trial based on such effect size would indicate a smaller number of participants than actually needed to detect a clinically meaningful effect, ultimately resulting in a negative trial. On the other hand, if the effect size estimated from the pilot study was very small, the subsequent trial might not even be pursued due to assumptions that the intervention does not work. If the subsequent trial was designed, the power calculations would indicate a much larger number of participants than actually needed to detect an effect, which may reduce chances of funding (too expensive), or if funded, would expose an unnecessary number of participants to the intervention arms (see Figure 1).

So what else can you do to provide effect sizes for power calculations?

Because pilot studies provide unstable estimates of effect size, the recommended approach is to base sample size calculations for efficacy studies on estimates of a clinically meaningful difference as illustrated in Figure 2. Investigators can estimate clincally meaningful differences by consideration of what effect size would be necessary to change clinical behaviors and/or guideline recommendations. In this process it might be beneficial to convene stakeholder groups to determine what type of difference would be meaningful to patient groups, clinicians, practitioners, and/or policymakers. In the determination of a clinically meaningful effect, researchers should also consider the intensity of the intervention and risk of harm vs. the expectation of benefit. Observational data and the effect size seen with a standard treatment can provide useful starting points to help determine clinically meaningful effects. For all of these methods, you should ask the question, “What would make a difference for you?” You might consider using several of these methods and determining a range of effect sizes as a basis for your power calculations.

Pilot studies should not be used to test hypotheses about the effects of an intervention. The “Does this work?” question is best left to the full-scale efficacy trial, and the power calculations for that trial are best based on clinically meaningful differences. Instead, pilot studies should assess the feasibility/acceptability of the approach to be used in the larger study, and answer the “Can I do this?” question. You can read more about the other steps involved in developing and testing mind and body interventions on our NCCIH Research Framework page.

.header_greentext{color:green!important;font-size:24px!important;font-weight:500!important;}.header_bluetext{color:blue!important;font-size:18px!important;font-weight:500!important;}.header_redtext{color:red!important;font-size:28px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;font-size:28px!important;font-weight:500!important;}.header_purpletext{color:purple!important;font-size:31px!important;font-weight:500!important;}.header_yellowtext{color:yellow!important;font-size:20px!important;font-weight:500!important;}.header_blacktext{color:black!important;font-size:22px!important;font-weight:500!important;}.header_whitetext{color:white!important;font-size:22px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;}.Green_Header{color:green!important;font-size:24px!important;font-weight:500!important;}.Blue_Header{color:blue!important;font-size:18px!important;font-weight:500!important;}.Red_Header{color:red!important;font-size:28px!important;font-weight:500!important;}.Purple_Header{color:purple!important;font-size:31px!important;font-weight:500!important;}.Yellow_Header{color:yellow!important;font-size:20px!important;font-weight:500!important;}.Black_Header{color:black!important;font-size:22px!important;font-weight:500!important;}.White_Header{color:white!important;font-size:22px!important;font-weight:500!important;} Additional Resources:

• Leon AC, Davis LL, Kraemer HC. The role and interpretation of pilot studies in clinical research.   Journal of Psychiatric Research. 2011;45(5):626-629.
• Kraemer HC, Mintz J, Noda A, et al. Caution regarding the use of pilot studies to guide power calculations for study proposals. Archives of General Psychiatry. 2006;63(5):484-489.
• Kistin C, Silverstein M. Pilot studies: a critical but potentially misused component of interventional research. JAMA. 2015;314(15):1561-1562.
• Keefe RS, Kraemer HC, Epstein RS, et al. Defining a clinically meaningful effect for the design and interpretation of randomized controlled trials. Innovations in Clinical Neuroscience. 2013;10(5-6 Suppl A):4S-19S.

A feasibility study testing four hypotheses with phase II outcomes in advanced colorectal cancer (MRC FOCUS3): a model for randomised controlled trials in the era of personalised medicine?

Affiliations.

• 1 CRUK/MRC Oxford Institute for Radiation Oncology, University of Oxford, Oxford OX3 7DQ, UK.
• 2 MRC Clinical Trials Unit at UCL, Institute of Clinical Trials and Methodology, London WC2B 6NH, UK.
• 3 Cardiff University and Velindre Cancer Centre, Cardiff, UK.
• 4 Leeds Institute of Cancer and Pathology, University of Leeds, Leeds LS9 7TF, UK.
• 5 University Hospital of Wales, Cardiff CF14 4XW, UK.
• 6 Centre for Cancer Research and Cell Biology, Queen's University Belfast, Belfast BT9 7AE, UK.
• 7 Institute of Cancer and Genetics, Cardiff University, Cardiff CF14 4XN, UK.
• 8 St James's Institute of Oncology, University of Leeds, Leeds LS9 7TF, UK.
• 9 Wales Cancer Trials Unit, Cardiff University, Cardiff CF14 4YS, UK.
• 10 Sheffield Teaching Hospitals NHS Foundation Trust, Sheffield S5 7AU, UK.
• 11 UCL Cancer Institute, London WC1E 6BT, UK.
• 12 Department of Oncology, Castle Hill Hospital, East Riding of Yorkshire HU16 5JQ, UK.
• PMID: 24743706
• PMCID: PMC4007241
• DOI: 10.1038/bjc.2014.182

Background: Molecular characteristics of cancer vary between individuals. In future, most trials will require assessment of biomarkers to allocate patients into enriched populations in which targeted therapies are more likely to be effective. The MRC FOCUS3 trial is a feasibility study to assess key elements in the planning of such studies.

Patients and methods: Patients with advanced colorectal cancer were registered from 24 centres between February 2010 and April 2011. With their consent, patients' tumour samples were analysed for KRAS/BRAF oncogene mutation status and topoisomerase 1 (topo-1) immunohistochemistry. Patients were then classified into one of four molecular strata; within each strata patients were randomised to one of two hypothesis-driven experimental therapies or a common control arm (FOLFIRI chemotherapy). A 4-stage suite of patient information sheets (PISs) was developed to avoid patient overload.

Results: A total of 332 patients were registered, 244 randomised. Among randomised patients, biomarker results were provided within 10 working days (w.d.) in 71%, 15 w.d. in 91% and 20 w.d. in 99%. DNA mutation analysis was 100% concordant between two laboratories. Over 90% of participants reported excellent understanding of all aspects of the trial. In this randomised phase II setting, omission of irinotecan in the low topo-1 group was associated with increased response rate and addition of cetuximab in the KRAS, BRAF wild-type cohort was associated with longer progression-free survival.

Conclusions: Patient samples can be collected and analysed within workable time frames and with reproducible mutation results. Complex multi-arm designs are acceptable to patients with good PIS. Randomisation within each cohort provides outcome data that can inform clinical practice.

Publication types

• Clinical Trial, Phase II
• Randomized Controlled Trial
• Research Support, Non-U.S. Gov't
• Aged, 80 and over
• Biomarkers, Tumor / analysis
• Colorectal Neoplasms / drug therapy*
• Colorectal Neoplasms / genetics*
• Colorectal Neoplasms / mortality
• DNA Mutational Analysis
• Disease-Free Survival
• Feasibility Studies
• Middle Aged
• Precision Medicine*
• Proto-Oncogene Proteins / genetics*
• Proto-Oncogene Proteins B-raf / genetics*
• Proto-Oncogene Proteins p21(ras)
• Treatment Outcome
• ras Proteins / genetics*
• Biomarkers, Tumor
• KRAS protein, human
• Proto-Oncogene Proteins
• BRAF protein, human
• Proto-Oncogene Proteins B-raf
• ras Proteins

Associated data

• ISRCTN/ISRCTN83171665

Grants and funding

• MCCC-FCO-11-C/MCCC_/Marie Curie/United Kingdom
• G0701770/MRC_/Medical Research Council/United Kingdom
• MC_UU_12023/3/MRC_/Medical Research Council/United Kingdom
• MC_U122861325/MRC_/Medical Research Council/United Kingdom
• 18815/CRUK_/Cancer Research UK/United Kingdom
• 15954/CRUK_/Cancer Research UK/United Kingdom

Help | Advanced Search

Astrophysics > Earth and Planetary Astrophysics

Title: feasibility study on retrieving exoplanetary cloud cover distributions using polarimetry.

Abstract: Context. As a new growing field, exocartography aims to map the surface features of exoplanets that are beyond the resolution of traditional observing techniques. While photometric approaches have been discussed extensively, polarimetry has received less attention despite its promising prospects. Aims. We demonstrate that the limb polarization of an exoplanetary atmosphere offers valuable insights into its cloud cover distribution. Specifically, we determine an upper limit for the polarimetric precision, which is required to extract information about the latitudinal cloud cover of temperate Jovian planets for scenarios of observations with and without host stars. Methods. To compute the scattered stellar radiation of an exoplanetary atmosphere and to study the polarization at various planetary phase angles, we used the three-dimensional Monte Carlo radiative transfer code POLARIS. Results. When the planetary signal can be measured separately from the stellar radiation, information about the latitudinal cloud cover for polar cap models is accessible at polarimetric sensitivities of $0.1$ %. In contrast, a precision of about $10^{-3}$ ppm is required when the stellar flux is included to gain this information.

Submission history

Access paper:.

• HTML (experimental)
• Other Formats

References & Citations

• Google Scholar
• Semantic Scholar

BibTeX formatted citation

Bibliographic and Citation Tools

Code, data and media associated with this article, recommenders and search tools.

• Institution

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs .

• Methodology
• Open access
• Published: 03 February 2021

Determining sample size for progression criteria for pragmatic pilot RCTs: the hypothesis test strikes back!

• M. Lewis   ORCID: orcid.org/0000-0001-5290-7833 1 , 2 ,
• K. Bromley 1 , 2 ,
• C. J. Sutton 3 ,
• G. McCray 1 , 2 ,
• H. L. Myers 2 &
• G. A. Lancaster 1 , 2  

Pilot and Feasibility Studies volume  7 , Article number:  40 ( 2021 ) Cite this article

42k Accesses

93 Citations

8 Altmetric

Metrics details

The current CONSORT guidelines for reporting pilot trials do not recommend hypothesis testing of clinical outcomes on the basis that a pilot trial is under-powered to detect such differences and this is the aim of the main trial. It states that primary evaluation should focus on descriptive analysis of feasibility/process outcomes (e.g. recruitment, adherence, treatment fidelity). Whilst the argument for not testing clinical outcomes is justifiable, the same does not necessarily apply to feasibility/process outcomes, where differences may be large and detectable with small samples. Moreover, there remains much ambiguity around sample size for pilot trials.

Many pilot trials adopt a ‘traffic light’ system for evaluating progression to the main trial determined by a set of criteria set up a priori. We construct a hypothesis testing approach for binary feasibility outcomes focused around this system that tests against being in the RED zone (unacceptable outcome) based on an expectation of being in the GREEN zone (acceptable outcome) and choose the sample size to give high power to reject being in the RED zone if the GREEN zone holds true. Pilot point estimates falling in the RED zone will be statistically non-significant and in the GREEN zone will be significant; the AMBER zone designates potentially acceptable outcome and statistical tests may be significant or non-significant.

For example, in relation to treatment fidelity, if we assume the upper boundary of the RED zone is 50% and the lower boundary of the GREEN zone is 75% (designating unacceptable and acceptable treatment fidelity, respectively), the sample size required for analysis given 90% power and one-sided 5% alpha would be around n = 34 (intervention group alone). Observed treatment fidelity in the range of 0–17 participants (0–50%) will fall into the RED zone and be statistically non-significant, 18–25 (51–74%) fall into AMBER and may or may not be significant and 26–34 (75–100%) fall into GREEN and will be significant indicating acceptable fidelity.

In general, several key process outcomes are assessed for progression to a main trial; a composite approach would require appraising the rules of progression across all these outcomes. This methodology provides a formal framework for hypothesis testing and sample size indication around process outcome evaluation for pilot RCTs.

Peer Review reports

The importance and need for pilot and feasibility studies is clear: “A well-conducted pilot study, giving a clear list of aims and objectives … will encourage methodological rigour … and will lead to higher quality RCTs” [ 1 ]. The CONSORT extension to external pilot and feasibility trials was published in 2016 [ 2 ] with the following key methodological recommendations: (i) investigate areas of uncertainty about the future definitive RCT; (ii) ensure primary aims/objectives are about feasibility, which should guide the methodology used; (iii) include assessments to address the feasibility objectives which should be the main focus of data collection and analysis; and (iv) build decision processes into the pilot design whether or how to proceed to the main study. Given that many trials incur process problems during implementation—particularly with regard to recruitment [ 3 , 4 , 5 ]—the need for pilot and feasibility studies is evident.

One aspect of pilot and feasibility studies that remains unclear is the required sample size. There is no consensus but recommendations vary from 10 to 12 per group through to 60–75 per group depending on the main objective of the study. Sample size may be based on precision of a feasibility parameter [ 6 , 7 ]; precision of a clinical parameter which may inform main trial sample size—particularly the standard deviation (SD) [ 8 , 9 , 10 , 11 ] but also event rate [ 12 ] and effect size [ 13 , 14 ]; or, to a lesser degree, for clinical scale evaluation [ 9 , 15 ]. Billingham et al. [ 16 ] reported that the median sample size of pilot and feasibility studies is around 30–36 per group but there is wide variation. Herbert et al. [ 17 ] reported that targets within internal as opposed to external pilots are often slightly larger and somewhat different, being based on percentages of the total sample size and timeline rather than any fixed sample requirement.

The need for a clear directive on sample size of studies is of upmost relevance. The CONSORT extension [ 2 ] reports that “Pilot size should be based on feasibility objectives and some rationale given” and states that a “confidence interval approach may be used to calculate and justify the sample size based on key feasibility objective(s)”. Specifically, item 7a (How sample size was determined: Rationale for numbers in the pilot trial) qualifies: “Many pilot trials have key objectives related to estimating rates of acceptance, recruitment, retention, or uptake … for these sorts of objectives, numbers required in the study should ideally be set to ensure a desired degree of precision around the estimated rate”. Item 7b (When applicable, explanation of any interim analyses and stopping guidelines) is generally an uncommon scenario for pilot and feasibility studies and is not given consideration here.

A key aspect of pilot and feasibility studies is to inform progression to the main trial, which has important implications for all key stakeholders (funders, researchers, clinicians and patients). The CONSORT extension [ 2 ] states that “decision processes about how to proceed needs to be built into the pilot design (which might involve formal progression criteria to decide whether to proceed, proceed with amendments, or not to proceed)” and authors should present “if applicable, the pre-specified criteria used to judge whether or how to proceed with a future definitive RCT; … implications for progression from pilot to future definitive RCT, including any proposed amendments”. Avery et al. [ 18 ] published recommendations for internal pilots emphasising a traffic light (stop-amend-go/red-amber-green) approach to progression with focus on process assessment (recruitment, protocol adherence, follow-up) and transparent reporting around the choice of trial design and the decision-making processes for stopping, amending or proceeding to a main trial. The review of Herbert et al. [ 17 ] reported that the use of progression criteria (including recruitment rate) and traffic light stop-amend-go as opposed to simple stop-go is increasing for internal pilot studies.

A common misuse of pilot and feasibility studies has been the application of hypothesis testing for clinical outcomes in small under-powered studies. Arain et al. [ 19 ] claimed that pilot studies were often poorly reported with inappropriate emphasis on hypothesis testing. They reviewed 54 pilot and feasibility studies published in 2007–2008, of which 81% incorporated hypothesis testing of clinical outcomes. Similarly, Leon et al. [ 20 ] stated that a pilot is not a hypothesis testing study: safety, efficacy and effectiveness should not be evaluated. Despite this, hypothesis testing has been commonly performed for clinical effectiveness/efficacy without reasonable justification. Horne et al. [ 21 ] reviewed 31 pilot trials published in physical therapy journals between 2012 and 2015 and found that only 4/31 (13%) carried out a valid sample size calculation on effectiveness/efficacy outcomes but 26/31 (84%) used hypothesis testing. Wilson et al. [ 22 ] acknowledged a number of statistical challenges in assessing potential efficacy of complex interventions in pilot and feasibility studies. The CONSORT extension [ 2 ] re-affirmed many researchers’ views that formal hypothesis testing for effectiveness/efficacy is not recommended in pilot/feasibility studies since they are under-powered to do so. Sim’s commentary [ 23 ] further contests such testing of clinical outcomes stating that treatment effects calculated from pilot or feasibility studies should not be the basis of a sample size calculation for a main trial.

However, when the focus of analysis is on confidence interval estimation for process outcomes, this does not give a definitive basis for acceptance/rejection of progression criteria linked to formal powering. The issue in this regard is that precision focuses on alpha ( α , type I error) without clear consideration of beta (β, type II error) and may therefore not reasonably capture true differences if a study is under-powered. Further, it could be argued that hypothesis testing of feasibility outcomes (as well as addressing both alpha and beta) is justified on the grounds that moderate-to-large differences (‘process-effects’) may be expected rather than small differences that would require large sample numbers. Moore et al. [ 24 ] previously stated that some pilot studies require hypothesis testing to guide decisions about whether larger subsequent studies can be undertaken, giving the following example of how this could be done for feasibility outcomes: asking the question “Is taste of dietary supplement acceptable to at least 95% of the target population?”, they showed that sample sizes of 30, 50 and 70 provide 48%, 78% and 84% power to reject an acceptance rate of 85% or lower if the true acceptance rate is 95% using a 1-sided α = 0.05 binomial test. Schoenfeld [ 25 ] advocates that, even for clinical outcomes, there may be a place for testing at the level of clinical ‘indication’ rather than ‘clinical evidence’. He suggested that preliminary hypothesis testing for efficacy could be conducted with high alpha (up to 0.25), not to provide definitive evidence but as an indication as to whether a larger study should be conducted. Lee et al. [ 14 ] also reported how type 1 error levels other than the traditional 5% could be considered to provide preliminary evidence for efficacy, although they did stop short of recommending doing this by concluding that a confidence interval approach is preferable.

Current recommendations for sample sizes of pilot/feasibility studies vary, have a single rather than a multi-criterion basis, and do not necessarily link directly to formal progression criteria. The purpose of this article is to introduce a simple methodology that allows sample size derivation and formal testing of proposed progression cut-offs, whilst offering suggestions for multi-criterion assessment, thereby giving clear guidance and sign-posting for researchers embarking on a pilot/feasibility study to assess uncertainty in feasibility parameters prior to a main trial. The suggestions within the article do not directly apply to internal pilot studies built into the design of a main trial, but given the similarities to external randomised pilot and feasibility studies, many of the principles outlined here for external pilots might also extend to some degree to internal pilots of randomised and non-randomised studies.

The proposed approach focuses on estimation and hypothesis testing of progression criteria for feasibility outcomes that are potentially modifiable (e.g. recruitment, treatment fidelity/ adherence, level of follow up). Thus, it aligns with the main aims and objectives of pilot and feasibility studies and with the progression stop-amend-go recommendations of Eldridge et al. [ 2 ] and Avery et al. [ 18 ].

Hypothesis concept

Let R UL denote the upper RED zone cut-off and G LL denote the lower GREEN zone cut-off. The concept is to set up hypothesis testing around progression criteria that tests against being in the RED zone (designating unacceptable feasibility—‘ STOP ’) based on an alternative of being in the GREEN zone (designating acceptable feasibility—‘ GO ’). This is analogous to the zero difference (null) and clinically important difference (alternative) in a main superiority trial. Specifically, we are testing against R UL when G LL is hypothesised to be true:

Null hypothesis: True feasibility outcome ( ε ) not greater than the upper “RED” stop limit ( R UL )

Alternative hypothesis: True feasibility outcome ( ε ) is greater than R UL

The test is a 1-tailed test with suggested alpha ( α ) of 0.05 and beta (β) of 0.05, 0.1 or 0.2, dependent on the required strength of evidence of the test. An example of a feasibility outcome might be percentage recruitment uptake.

Progression rules

Let E denote the observed point estimate (ranging from 0 to 1 for proportions, or for percentages 0–100%). Simple 3-tiered progression criteria would follow as:

E ≤ R UL [ P value non-significant ( P ≥ α )] -> RED (unacceptable—STOP)

R UL < E < G LL -> AMBER (potentially acceptable—AMEND)

E ≥ G LL [ P value significant ( P < α )] -> GREEN (acceptable—GO)

Sample size

Table 1 displays a quick look-up grid for sample size across a range of anticipated proportions for R UL and G LL for one-sample one-sided 5% alpha with typical 80% and 90% (as well as 95%) power for the normal approximation method with continuity correction (see Appendix for corresponding mathematical expression; derived from Fleiss et al. [ 26 ]). Table 2 is the same look-up grid relating to the Binomial exact approach with sample sizes derived using G*Power version 3.1.9.7 [ 27 ]. Clearly, as the difference between proportions R UL and G LL increases the sample size requirement is reduced.

Multi-criteria assessment

We recommend that progression for all key feasibility criteria should be considered separately, and hence overall progression would be determined by the worst-performing criterion, e.g. RED if at least one signal is RED, AMBER if none of the signals fall into RED but at least one falls into AMBER and GREEN if all signals fall into the GREEN zone. Hence, the GREEN signal to ‘GO’ across the set of individual criteria will give indication that progression to a main trial can take place without any necessary changes. A signal to ‘STOP’ and not proceed to a main trial is recommended if any of the observed estimates are ‘unacceptably’ low (i.e. fall within the RED zone). Otherwise, where neither ‘GO’ nor ‘STOP’ are signalled, the design of the trial will need amending by indication of subpar performance on one or more of the criteria.

Sample size requirements across multi-criteria will vary according to the designated parameters linked to the progression criteria, which may be set at different stages of the study on different numbers of patients (e.g. those screened, eligible, recruited and randomised, allocated to the intervention arm, total followed up). The overall size needed will be dictated by the requirement to power each of the multi-criteria statistical tests. Since these tests will yield separate conclusions in regard to the decision to ‘STOP’, ‘AMEND’ or ‘GO’ across all individual feasibility criteria there is no need to consider a multiple testing correction with respect to alpha. However, researchers may wish to increase power (and hence, sample size) to ensure adequate power to detect ‘GO’ signals across the collective set of feasibility criteria. For example, powering at 90% across three criteria (assumed independent) will ensure a collective power of 73% (i.e. 0.9 3 ), which may be considered reasonable, but 80% power across five criteria will reduce the power of the combined test to 33%. The final three columns of Table 1 cover the sample sizes required for 95% power, which may address collective multi-criteria assessment when considering keeping a high overall statistical power.

Further expansion of AMBER zone

Within the same sample size framework, the AMBER zone may be further split to indicate whether ‘minor’ or ‘major’ amendments are required according to the significance of the p value. Consider a 2-way split in the AMBER zone denoted by cut-off A C , which indicates the threshold for statistical significance, where an observed estimate below the cut-point will result in a non-significant result and an estimate at or above the cut-point a significant result. Let AMBER R denote the region of Amber zone adjacent to the RED zone between R UL and A C , and AMBER G denote the region of AMBER zone between A C and G LL adjacent to the GREEN zone. This would draw on two possible levels of amendment (‘major’ AMEND and ‘minor’ AMEND) and the re-configured approach would follow as:

R UL < E < G LL and P ≥ α { R UL < E < A c } -> AMBER R (major AMEND)

R UL < E < G LL and P < α { A c ≤ E < G LL } -> AMBER G (minor AMEND)

In Tables 1 and 2 in relation to designated sample sizes for different R UL and G LL and specified α and β, we show the corresponding cut-points for statistical significance ( p < 0.05) both in absolute terms of sample number ( n ) [ A C ] and as a percentage of the total sample sizes [ A C % ].

A motivating example (aligned to the normal approximation approach) is presented in Table 3 , which illustrates a pilot trial with three progression criteria. Table 4 presents the sample size calculations for the example scenario following the 3-tiered approach, and Table 5 gives the sample size calculations for the example scenario using the extended 4-tiered approach. Cut-points for the feasibility outcomes relating to the shown sample sizes are also presented to show RED, AMBER and GREEN zones for each of the three progression criteria.

Overall sample size requirement should be dictated by the multi-criteria approach. This is illustrated in Table 4 where we have three progression criteria each with a different denominator population. For recruitment uptake, the denominator denotes the total number of children screened and the numerator the number of children randomised; for follow-up, the denominator is the number of children randomised with the numerator being number of those randomised who are successfully followed up; and lastly for treatment fidelity, the denominator is the number allocated to the intervention arm with the numerator being the number of children who were administered the treatment correctly by the dietician. In the example in order to meet the individual ≥ 90% power requirement for all three criteria we would need: (i) for recruitment, the number to be screened to be 78; (ii) for treatment fidelity, the number in the intervention arm to be 34; and (iii) for follow up, the number randomised to be 44. In order to determine the overall sample size for the whole study, we base our decision on the criterion that requires the largest numbers, which is the treatment fidelity criterion which requires 68 to be randomised. We cannot base our decision on the 78 required to be screened for recruitment because this would give only an expected number of 28 randomised (i.e. 35% of 78). If we expect 35% recruitment uptake, then we need to inflate the total 68 (randomised) to be 195 (1/0.35 × 68) children to be screened (rounded to 200). This would give 99.9%, 90% and 98.8% power for criteria (i), (ii) and (iii), respectively (assuming 68 of the 200 screened are randomised), giving a very reasonable collective 88.8% power of rejecting the null hypotheses over the three criteria if the alternative hypotheses (for acceptable feasibility outcomes) are true in each case.

Inherent in our approach are the probabilities around sample size, power and hypothesised feasibility parameters. For example, taking the cut-offs from treatment fidelity as a feasibility outcome from Table 4 (ii), we set a lower GREEN zone limit of G LL = 0.75 (“acceptable” (hypothesised alternative value)) and an upper RED zone limit of R UL = 0.5 (“not acceptable” (hypothesised null value)) for rejecting the null for this criterion based on 90% power and a 1-sided 5% significance level (alpha). Figure 1 presents the normal probability density functions for ε , for the null and alternative hypotheses. In the illustration this would imply through normal sampling theory that if G LL holds true (i.e. true recruitment uptake ( ε ) = G LL ) there would be the following:

A probability of 0.1 (type II error probability β) of the estimate falling within RED/AMBER R zones (i.e. blue shaded area under the curve to the left of A C where the test result will be non-significant ( p ≥ 0.05))

Probability of 0.4 of it falling in the AMBER G zone (i.e. area under the curve to the right of A C but below G LL )

Probability of 0.5 of the estimate falling in the GREEN zone (i.e. G LL and above).

Illustration of power using the 1-tailed hypothesis testing against the traffic light signalling approach to pilot progression. E , observed point estimate; R UL , upper limit of RED zone; G LL , lower limit of GREEN zone; Ac , cut-off for statistical significance (at the 1-sided 5% level); α , type I error; β , type II error

If R UL (the null) holds true (i.e. true feasibility outcome ( ε ) = R UL ), there would be the following:

A probability of 0.05 (one-tailed type I error probability α ) of the statistic/estimate falling in the AMBER G /GREEN zones (i.e. pink shaded area under the curve to the right of A C where the test result will be significant ( p < 0.05) as shown within Fig. 1 )

Probability of 0.45 of it falling in the AMBER R zone (i.e. to the left of A C but above R UL )

Probability of 0.5 of the estimate falling in the RED zone (i.e. R UL and below)

Figure 1 also illustrates how changing the sample size affects the sampling distribution and power of the analysis around the set null value (at R UL ) when the hypothesised alternative ( G LL ) is true. The figure emphasises the need for a large enough sample to safeguard against under-powering of the pilot analysis (as shown in the last plot which has a wider bell-shape than the first two plots and where the size of the beta probability is increased).

Figure 2 plots the probabilities of making each type of traffic light decision as functions of the true parameter value (focused on the recruitment uptake example from Table 5 (i)). Additional file 1 presents the R code for reproducing these probabilities and enables readers to insert different parameter values.

Probability of traffic light given true underlying probability of an event using the example from Table 5 (i). Two plots are presented: a relating to normal approximation approach and b relating to binomial exact approach. Based on n = 200, R UL = 40 and G LL = 70

The methodology introduced in this article provides an innovative formal framework and approach to sample size derivation, aligning sample size requirement to progression criteria with the intention of providing greater transparency to the progression process and full engagement with the standard aims and objectives of pilot/feasibility studies. Through the use of both alpha and beta parameters (rather than alpha alone), the method ensures rigour and capacity to address the progression criteria by ensuring there is adequate power to detect an acceptable threshold for moving forward to the main trial. As several key process outcomes are assessed in parallel and in combination, the method embraces a composite multi-criterion approach that appraises signals for progression across all the targeted feasibility measures. The methodology extends beyond the requirement for ‘sample size justification but not necessarily sample size calculation’ [ 28 ].

The focus of the strategy reported here is on process outcomes, which align with the recommended key objectives of primary feasibility evaluation for pilot and feasibility studies [ 2 , 24 ] and necessary targets to address key issues of uncertainty [ 29 ]. The concept of justifying progression is key. Charlesworth et al. [ 30 ] developed a checklist for intended use in decision-making on whether pilot data could be carried forward to a main trial. Our approach builds on this philosophy by introducing a formalised hypothesis test approach to address the key objectives and pilot sample size. Though the suggested sample size derivation focuses around the key process objectives, it may also be the case that other objectives are also important, e.g. assessment of precision of clinical outcome parameters. In this case, researchers may also wish to ensure that the size of the study suitably covers the needs of those evaluations, e.g. to estimate the SD of the intended clinical outcome, then the overall sample size may be boosted to cover this additional objective [ 10 ]. This tallies with the review by Blatch-Jones et al. [ 31 ] who reported that testing recruitment, determining the sample size and numbers available, and the intervention feasibility were the most commonly used targets of pilot evaluations.

Hypothesis testing in pilot studies, particularly in the context of effectiveness/efficacy of clinical outcomes, has been widely criticised due to the improper purpose and lack of statistical power of such evaluations [ 2 , 20 , 21 , 23 ]. Hence, pilot evaluations of clinical outcomes are not expected to include hypothesis testing. Since the main focus is on feasibility the scope of the testing reported here is different and importantly relates back to the recommended objectives of the study whilst also aligning with nominated progression criteria [ 2 ]. Hence, there is clear justification for this approach. Further, for the simple 3-tiered approach hypothesis testing is somewhat hypothetical: there is no need to physically carry out a test since the zonal positioning of the observed sample statistic estimate for the feasibility outcome will determine the decision in regard to progression; thus adding to the simplicity of the approach.

The link between the sample size and need to adequately power the study to detect a meaningful feasibility outcome gives this approach the extra rigour over the confidence interval approach. It is this sample size-power linkage that is key to the determination of the respective probabilities of falling into the different zones and is a fundamental underpinning to the methodological approach. In the same way as for a key clinical outcome in a main trial where the emphasis is not just on alpha but also on beta thereby addressing the capacity to detect a clinically significant difference, similarly, our approach is to ensure there is sufficient capacity to detect a meaningful signal for progression to a main trial if it truly exists. A statistically significant finding in this context will at least provide evidence to reject RED (signifying a decision to STOP) and in the 4-tiered case it would fall above AMBER R (decision to major-AMEND); hence, the estimate will fall into AMBER G or GREEN (signifying a decision to minor-AMEND or GO, respectively). The importance of adequately powering the pilot trial to address a feasibility criterion can be simply illustrated. For example, if we take R UL as 50% and G LL as 75% but with two different sample sizes of n = 25 and n = 50; the former would have 77.5% power of rejecting RED on the basis of a 1-sided 5% alpha level whereas the larger sample size would have 97.8% power of rejecting RED. So, if G LL holds true, there would be 20% higher probability of rejecting the null and being in the AMBER G /GREEN zone for the larger sample giving an increased chance of progressing to the main trial. It will be necessary to carry out the hypothesis test for the extended 4-tier approach if the observed statistic ( E ) falls in the AMBER zone to determine statistical significance or not, which will inform whether the result falls into the ‘minor’ or ‘major’ AMBER sub-zones.

We provide recommended sample sizes within a look-up grid relating to perceived likely progression cut-points to aid quick access and retrievable sample sizes for researchers. For a likely set difference in proportions between hypothesised null and alternative parameters of 0.15 to 0.25 when α = 0.05 and β = 0.1 the corresponding total sample size requirements for the approach of normal approximation with continuity correction take the range of 33 to 100 (median 56) [similarly these are 33–98 (median 54) for the binomial exact method]. Note, for treatment fidelity/adherence/compliance particularly, the marginal difference could be higher, e.g. ≥ 25%, since in most situations we would anticipate and hope to attain a high value for the outcome whilst being prepared to make necessary changes within a wide interval of below par values (and providing the value is not unacceptably low). As this relates to an arm-specific objective (relating to evaluation of the intervention only), then a usual 1:1 pilot will require twice the size; hence, the arm-specific sample size powered for detecting a ≥ 25% difference from the null would be about 34 (or lower)—as depicted from our illustration (Table 4 (ii), equating to n ≤ 68 overall for a 1:1 pilot; intervention and control arms). Hence, we expect that typical pilot sizes of around 30–40 randomised per arm [ 16 ] would likely fit with the proposed methodology within this manuscript (the number needed for screening being extrapolated upward of this figure) but if a smaller marginal difference (e.g. ≤ 15%) is to be tested then these sample sizes may fall short. We stress that the overall required sample size needs to be carefully considered and determined in line with the hypothesis testing approach across all criteria ensuring sufficiently high power. In our paper, we have made recommendations regarding various sample sizes based on both the normal approximation (with continuity correction) and binomial exact approaches; these are conservative compared to the Normal approximation (without continuity correction).

Importantly, the methodology outlines the necessary multi-criterion approach to the evaluation of pilot and feasibility studies. If all progression criteria are performing as well as anticipated (highlighting ‘GO’ according to all criteria), then the recommendation of the pilot/feasibility study is that all criteria meet their desired levels with no need for adjustment and the main trial can proceed without amendment. However, if the worst signal (across all measured criteria) is an AMBER signal, then adjustment will be required against those criteria that fall within that signal. Consequently, there is the possibility that the criteria may need subsequent re-assessment to re-evaluate processes in line with updated performance for the criteria in question. If one or more of the feasibility statistics fall within the RED zone then this signals ‘STOP’ and concludes that a main trial is not feasible based on those criteria. This approach to collectively appraising progression based on the results of all feasibility outcomes assessed against their criteria will be conservative as the power of the collective will be lower than the individual power of the separate tests; hence, it is recommended that the power of the individual tests is set high enough (for example, 90–95%) to ensure the collective power is high enough (e.g. at least 70 or 80%) to detect true ‘GO’ signals across all the feasibility criteria.

In this article, we also expand the possibilities for progression criterion and hypothesis testing where the AMBER zone is sub-divided arbitrarily based on the significance of the p value. This may work well when the AMBER zone has a wide range and is intended to provide a useful and workable indication of the level of amendment (‘minor’ (non-substantive) or ‘major’ (substantive)) required to progress to the main trial. Examples of substantial amendments include study re-design with possible re-appraisal and change of statistical parameters, inclusion of several additional sites, adding further data recruitment methods, significant reconfiguration of exclusions, major change to the method of delivery of trial intervention to ensure enhanced treatment fidelity/adherence, enhanced measures to systematically ensure greater patient compliance with allocated treatment, additional mode(s) of collecting and retrieving data (e.g. use of electronic data collection methods in addition to postal questionnaires). Minor amendments include small changes to the protocol and methodology, e.g. addition of one or two sites for attaining a slightly higher recruitment rate, use of occasional reminders in regard to treatment protocol and adding a further reminder process for boosting follow up. For the most likely parametrisation of α = 0.05/β = 0.1, the AMBER zone division will be roughly at the midpoint. However, researchers can choose this point (the major/minor cut-point) based on decisive arguments around how major and minor amendments would align to the outcome in question. This should be factored within the process of sample size determination for the pilot. In this regard, a smaller sample size will move A C upwards (due to increased standard error/reduced precision) and hence increase the size of the AMBER R zone in relation to AMBER G (whereas a larger sample size will shift A C downwards and do the opposite, increasing the ratio of AMBER G :AMBER R ). From Table 1 , for smaller sample sizes (related to 80% power) the AMBER R zone makes up 56–69% of the total amber zone across presented scenarios, whereas this falls to 47–61% for samples (related to 90% power) and 41–56% for larger samples (related to 95% power) for the same scenarios. Beyond our proposed 4-tier approach, other ways of providing an indication of level of amendment could include evaluation and review of the point and interval estimates or by evaluating posterior probabilities via a Bayesian approach [ 14 , 32 ].

The methodology illustrated here focuses on feasibility outcomes presented as percentages/proportions, which is likely to be the most common form for progression criteria under consideration. However, the steps that have been introduced can be readily adapted to any feasibility outcomes taking a numerical format, e.g. rate of recruitment per month per centre, count of centres taking part in the study. Also, we point out that in the examples presented in the paper (recruitment, treatment fidelity and percent follow-up), high proportions are acceptable and low ones not. This would not be true for, say, adverse events where a reverse scale is required.

Biased sample estimates are a concern as they may result in a wrong decision being made. This systematic error is over-and-above the possibility of an erroneous decision being made on the basis of sampling error; the latter may be reduced through an increased pilot sample size. Any positive bias will inflate/overestimate the feasibility sample estimate in favour of progressing whereas a negative bias will deflate/underestimate it towards the null and stopping. Both are problematic for opposite reasons; for example, the former may inform researchers that the main trial can ‘GO’ ahead when in fact it will struggle to meet key feasibility targets, whereas the latter may caution against progression when in reality the feasibility targets of a main trial would be met. For example, in regard to the choice of centres (and hence practitioners and participants), a common concern is that the selection of feasibility trial centres might not be a fair and representative sample of the ‘population’ of centres to be used for the main trial. It may be that the host centre (likely used in pilot studies) recruits far better than others (positive bias), thus exaggerating the signal to progress and subsequent recruitment to the main trial. Beets et al. [ 33 ] ‘define “risk of generalizability biases” as the degree to which features of the intervention and sample in the pilot study are NOT scalable or generalizable to the next stage of testing in a larger, efficacy/effectiveness trial … whether aspects like who delivers an intervention, to whom it is delivered, or the intensity and duration of the intervention during the pilot study are sustained in the larger, efficacy/effectiveness trial.’ As in other types of studies, safeguards regarding bias should be addressed through appropriate pilot study design and conduct.

Issues relating to progression criteria for internal pilots may be different to those for external pilots and non-randomised feasibility studies. The consequence of a ‘stop’ within an internal pilot may be more serious for stakeholders (researchers, funders, patients) as it would bring an end to the planned continuation into the main trial phase, whereas there would be less at stake for a negative external pilot. By contrast, the consequence of a ‘GO’ signal may work the other way with a clear and immediate gain for the internal pilot whereas for an external pilot, the researchers would still need to apply and get the necessary funding and approvals to undertake an intended main trial. The chances of falling into the different traffic light zones are likely to be quite different between the two designs. Possibly external pilot and feasibility studies are more likely to have estimates falling in and around the RED zone than for internal pilots, reflecting the greater uncertainty in the processes for the former and greater confidence in the mechanisms for trial delivery for the latter. However, to counter this, there are often large challenges with recruitment within internal pilot studies where the target population is usually spread over more diverse sites than may be expected for an external pilot. Despite this possible imbalance, the interpretation of zonal indications remains consistent for external and internal pilot studies. As such, our focus with regard to the recommendations in this article are aligned to requirements for external pilots, though application of this methodology to a degree may similarly hold for internal pilots (and further, to non-randomised studies that can include progression criteria—including longitudinal observational cohorts with the omission of the treatment fidelity criterion).

Conclusions

We propose a novel framework that provides a paradigm shift towards formally testing feasibility progression criteria in pilot and feasibility studies. The outlined approach ensures rigorous and transparent reporting in line with CONSORT recommendations for evaluation of STOP-AMEND-GO criteria and presents clear progression sign-posting which should help decision-making and inform stakeholders. Targeted progression criteria are focused on recommended pilot and feasibility objectives, particularly recruitment uptake, treatment fidelity and participant retention, and these criteria guide the methodology for sample size derivation and statistical testing. This methodology is intended to provide a more definitive and rounded structure to pilot and feasibility design and evaluation than currently exists. Sample size recommendations will be dependent on the nature and cut-points for multiple key pre-defined progression criteria and should ensure a sufficient sample size for other feasibility outcomes such as review of the precision of clinical parameters to better inform main trial size.

Availability of data and materials

Not applicable.

Abbreviations

Significance level (Type I error probability)

AMBER sub-zone split adjacent to the GREEN zone (within 4-tiered approach)

AMBER sub-zone split adjacent to the RED zone (within 4-tiered approach)

AMBER-statistical significance threshold (within the AMBER zone) where an observed estimate below the cut-point will result in a non-significant result (p ≥ 0.05) and figures at or above the cut-point will be significant (p < 0.05)

A C expressed as a percentage of the sample size

Type II error probability

Estimate of feasibility outcome

True feasibility parameter

Lower Limit of GREEN zone

Sample size (n s = number of patients screened; n r = number of patients randomised; n i = number of patients randomised to the intervention arm only)

(1 – Type II error probability)

Upper Limit of RED zone

Lancaster GA, Dodd S, Williamson PR. Design and analysis of pilot studies: recommendations for good practice. J Eval Clin Pract. 2004;10(2):307–12.

Article   Google Scholar  

Eldridge SM, Chan CL, Campbell MJ, Bond CM, Hopewell S, Thabane L, et al. CONSORT 2010 statement: extension to randomised pilot and feasibility trials. Pilot Feasibility Stud. 2016;2:64.

McDonald AM, Knight RC, Campbell MK, Entwistle VA, Grant AM, Cook JA, et al. What influences recruitment to randomised controlled trials? A review of trials funded by two UK funding agencies. Trials. 2006;7:9.

Sully BG, Julious SA, Nicholl J. A reinvestigation of recruitment to randomised, controlled, multicenter trials: a review of trials funded by two UK funding agencies. Trials. 2013;14:166.

Walters SJ, Bonacho Dos Anjos Henriques-Cadby I, Bortolami O, Flight L, Hind D, Jacques RM, et al. Recruitment and retention of participants in randomised controlled trials: a review of trials funded and published by the United Kingdom Health Technology Assessment Programme. BMJ Open. 2017;7(3):e015276.

Julious SA. Sample size of 12 per group rule of thumb for a pilot study. Pharm Stat. 2005;4:287–91.

Thabane L, Ma J, Chu R, Cheng J, Ismaila A, Rios LP, et al. A tutorial on pilot studies: the what, why and how. BMC Med Res Methodol. 2010;10:1.

Browne RH. On the use of a pilot sample for sample size determination. Stat Med. 1995;14:1933–40.

Article   CAS   Google Scholar  

Hertzog MA. Considerations in determining sample size for pilot studies. Res Nurs Health. 2008;31(2):180–91.

Sim J, Lewis M. The size of a pilot study for a clinical trial should be calculated in relation to considerations of precision and efficiency. J Clin Epidemiol. 2012;65(3):301–8.

Whitehead AL, Julious SA, Cooper CL, Campbell MJ. Estimating the sample size for a pilot randomised trial to minimise the overall trial sample size for the external pilot and main trial for a continuous outcome variable. Stat Methods Med Res. 2016;25(3):1057–73.

Teare MD, Dimairo M, Shephard N, Hayman A, Whitehead A, Walters SJ. Sample size requirements to estimate key design parameters from external pilot randomised controlled trials: a simulation study. Trials. 2014;15:264.

Cocks K, Torgerson DJ. Sample size calculations for pilot randomized trials: a confidence interval approach. J Clin Epidemiol. 2013;66(2):197–201.

Lee EC, Whitehead AL, Jacques RM, Julious SA. The statistical interpretation of pilot trials: should significance thresholds be reconsidered? BMC Med Res Methodol. 2014;14:41.

Johanson GA, Brooks GP. Initial scale development: sample size for pilot studies. Edu Psychol Measurement. 2010;70(3):394–400.

Billingham SA, Whitehead AL, Julious SA. An audit of sample sizes for pilot and feasibility trials being undertaken in the United Kingdom registered in the United Kingdom Clinical Research Network database. BMC Med Res Methodol. 2013;13:104.

Herbert E, Julious SA, Goodacre S. Progression criteria in trials with an internal pilot: an audit of publicly funded randomised controlled trials. Trials. 2019;20(1):493.

Avery KN, Williamson PR, Gamble C, O’Connell Francischetto E, Metcalfe C, Davidson P, et al. Informing efficient randomised controlled trials: exploration of challenges in developing progression criteria for internal pilot studies. BMJ Open. 2017;7(2):e013537.

Arain M, Campbell MJ, Cooper CL, Lancaster GA. What is a pilot or feasibility study? A review of current practice and editorial policy. BMC Med Res Methodol. 2010;10:67.

Leon AC, Davis LL, Kraemer HC. The role and interpretation of pilot studies in clinical research. J Psychiatr Res. 2011;45(5):626–9.

Horne E, Lancaster GA, Matson R, Cooper A, Ness A, Leary S. Pilot trials in physical activity journals: a review of reporting and editorial policy. Pilot Feasibility Stud. 2018;4:125.

Wilson DT, Walwyn RE, Brown J, Farrin AJ, Brown SR. Statistical challenges in assessing potential efficacy of complex interventions in pilot or feasibility studies. Stat Methods Med Res. 2016;25(3):997–1009.

Sim J. Should treatment effects be estimated in pilot and feasibility studies? Pilot Feasibility Stud. 2019;5:107.

Moore CG, Carter RE, Nietert PJ, Stewart PW. Recommendations for planning pilot studies in clinical and translational research. Clin Transl Sci. 2011;4(5):332–7.

Schoenfeld D. Statistical considerations for pilot studies. Int J Radiat Oncol Biol Phys. 1980;6(3):371–4.

Fleiss JL, Levin B, Paik MC. Statistical methods for rates and proportions, Third Edition. New York: John Wiley & Sons; 2003. p. 32.

Book   Google Scholar  

Faul F, Erdfelder E, Lang AG, Buchner A. G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behav Res Methods. 2007;39:175–91.

Julious SA. Pilot studies in clinical research. Stat Methods Med Res. 2016;25(3):995–6.

Lancaster GA. Pilot and feasibility studies come of age! Pilot Feasibility Stud. 2015;1(1):1.

Charlesworth G, Burnell K, Hoe J, Orrell M, Russell I. Acceptance checklist for clinical effectiveness pilot trials: a systematic approach. BMC Med Res Methodol. 2013;13:78.

Blatch-Jones AJ, Pek W, Kirkpatrick E, Ashton-Key M. Role of feasibility and pilot studies in randomised controlled trials: a cross-sectional study. BMJ Open. 2018;8(9):e022233.

Willan AR, Thabane L. Bayesian methods for pilot studies. Clin Trials 2020;17(4):414-9.

Beets MW, Weaver RG, Ioannidis JPA, Geraci M, Brazendale K, Decker L, et al. Identification and evaluation of risk of generalizability biases in pilot versus efficacy/effectiveness trials: a systematic review and meta-analysis. Int J Behav Nutr Phys Act. 2020;17:19.

Download references

Acknowledgements

We thank Professor Julius Sim, Dr Ivonne Solis-Trapala, Dr Elaine Nicholls and Marko Raseta for their feedback on the initial study abstract.

KB was supported by a UK 2017 NIHR Research Methods Fellowship Award (ref RM-FI-2017-08-006).

Author information

Authors and affiliations.

Biostatistics Group, School of Medicine, Keele University, Room 1.111, David Weatherall Building, Keele, Staffordshire, ST5 5BG, UK

M. Lewis, K. Bromley, G. McCray & G. A. Lancaster

Keele Clinical Trials Unit, Keele University, Keele, Staffordshire, UK

M. Lewis, K. Bromley, G. McCray, H. L. Myers & G. A. Lancaster

Centre for Biostatistics, School of Health Sciences, University of Manchester, Manchester, Staffordshire, UK

C. J. Sutton

You can also search for this author in PubMed   Google Scholar

Contributions

ML and CJS conceived the original methodological framework for the paper. ML prepared draft manuscripts. KB and GMcC provided examples and illustrations. All authors contributed to the writing and provided feedback on drafts and steer and suggestions for article updating. All authors read and approved the final manuscript.

Corresponding author

Correspondence to M. Lewis .

Ethics declarations

Ethics approval and consent to participate, consent for publication, competing interests.

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1..

R codes used for Fig. 2 .

Mathematical formulae for derivation of sample size

The required sample size may be derived using normal approximation to binary response data—using a continuity correction, via Fleiss et al. [ 26 ] if the convention of np > 5 and n ( 1 − p ) > 5 holds true:

where R UL = upper limit of RED zone; G LL = lower limit of GREEN zone; z 1− α = one-sided statistical significance level (type I error probability); z 1−β = beta (type II error probability)

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Lewis, M., Bromley, K., Sutton, C.J. et al. Determining sample size for progression criteria for pragmatic pilot RCTs: the hypothesis test strikes back!. Pilot Feasibility Stud 7 , 40 (2021). https://doi.org/10.1186/s40814-021-00770-x

Download citation

Received : 23 April 2020

Accepted : 07 January 2021

Published : 03 February 2021

DOI : https://doi.org/10.1186/s40814-021-00770-x

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Outcome and process assessment
• Sample size, Statistics

Pilot and Feasibility Studies

ISSN: 2055-5784

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

• Study Protocol
• Open access
• Published: 25 May 2024

Protocol for CHAMPION study: a prospective study of maximal-cytoreductive therapies for patients with de novo metastatic hormone-sensitive prostate cancer who achieve oligopersistent metastases during systemic treatment with apalutamide plus androgen deprivation therapy

• Beihe Wang 1 , 2 , 3   na1 ,
• Jian Pan 1 , 2 , 3   na1 ,
• Tingwei Zhang 1 , 2 , 3   na1 ,
• Xudong Ni 1 , 2 , 3   na1 ,
• Yu Wei 1 , 2 , 3   na1 ,
• Xiaomeng Li 1 , 2 , 3   na1 ,
• Bangwei Fang 1 , 2 , 3   na1 ,
• Xiaoxin Hu 3 , 4   na1 ,
• Hualei Gan 3 , 5   na1 ,
• Junlong Wu 1 , 2 , 3   na1 ,
• Hongkai Wang 1 , 2 , 3   na1 ,
• Dingwei Ye 1 , 2 , 3   na1 &
• Yao Zhu 1 , 2 , 3   na1  

BMC Cancer volume  24 , Article number:  643 ( 2024 ) Cite this article

Metrics details

The proposed trial is to examine the feasibility of prostate-specific membrane antigen (PSMA) positron emission tomography/computed tomography (PET/CT)-guided cytoreduction plus apalutamide and androgen deprivation therapy (ADT) for newly diagnosed metastatic hormone-sensitive prostate cancer (mHSPC) at oligometastatic state.

CHAMPION (NCT05717582) is an open-label, single-arm, phase II trial, planning to enroll newly diagnosed mHSPC cases with oligometastases (≤ 10 distant metastatic sites in conventional imaging). Patients will receive 6 cycles of apalutamide plus ADT. Patients with oligometastatic disease at PSMA PET/CT after 3 treatment cycles will receive cytoreductive radical prostatectomy. PSMA PET/CT-guided metastasis-directed external radiation therapy will be determined by the investigators. Apalutamide plus ADT will be continued for 2 weeks postoperatively. The primary endpoint is the proportion of patients with undetectable prostate-specific antigen (PSA), no disease progression, and no symptom deterioration after 6 cycles of apalutamide plus ADT. Secondary endpoints include the percentage of patients with PSA ≤ 0.2 ng/mL and oligometastases by the end of 3 treatment cycles, PSA response rate, and safety. Fleming’s two-stage group sequential design will be adopted in the study, where the null hypothesis is that the rate of patients with an undetectable PSA is ≤ 40% after 6 cycles of treatment, while the alternate hypothesis is an undetectable PSA of > 60%; with one-sided α = 0.05, power = 0.80, and an assumed dropout rate of 10%, the required number of patients for an effective analysis is 47. Enrolment in the study commenced in May 2023.

The multi-modal therapy based on treatment response may improve the prognosis of newly diagnosed mHSPC patients with oligometastases.

Trial registration

The study is registered with Clinical Trials.Gov (NCT05717582). Registered on 8th February 2023.

Peer Review reports

Introduction

Prostate cancer is a lethal disease especially when progressing into a metastatic status [ 1 , 2 ]. The proportion of patients with newly diagnosed prostate cancer at advanced stages is higher in China than in Western countries, and most patients have distant metastases diagnosis [ 3 ]. Currently, novel hormone therapy (NHT) plus androgen deprivation therapy (ADT) has become the standard first-line treatment for patients with metastatic hormone-sensitive prostate cancer (mHSPC) [ 4 ]. However, the 5-year overall survival (OS) for patients receiving this regime remained unsatisfying [ 5 ]. Treatment strategies should be refined to improve the prognosis of mHSPC.

Oligometastatic prostate cancer is an intermediate phase between a primary localized state and an extensive metastatic state and differs from widely metastatic lesions. The dynamic response to therapeutic treatment emphasizes the importance of this treatment window to defer disease progression [ 6 , 7 ]. The definition of oligometastatic disease is ambiguous in prostate cancer with less than 3, 5, or 10 metastatic sites and free of visceral metastasis [ 8 , 9 , 10 , 11 , 12 , 13 , 14 , 15 ]. A phase II clinical trial demonstrated that the addition of local treatment (mostly cytoreductive radical prostatectomy [cRP], accounting for 85%) to systemic therapy significantly prolonged radiographic progression-free survival (rPFS) and OS in mHSPC patients with ≤ 5 metastases [ 16 ]. Recent evidence has showed survival benefits and cost-effectiveness for multi-modal therapy (MMT) in mHSPC patients with limited metastatic lesions [ 17 , 18 ]. Besides systemic therapy, MMT strategy further included local treatment (e.g., cRP and external beam radiotherapy for prostate) and metastasis-directed therapy (MDT, e.g., stereotactic body radiation therapy [SBRT]) [ 19 , 20 , 21 ]. However, current MMT evidence mostly include traditional ADT instead of NHT. The benefit of NHT in MMT strategy should be further evaluated.

Prostate-specific membrane antigen (PSMA) positron emission tomography/computed tomography (PET/CT) has displayed diagnostic accuracy in prostate cancer [ 22 ]. Although, more metastatic lesions could be identified by PSMA PET/CT, the migration from low- to high-volume disease only occurred in 22% of patients according to the CHAARTED criteria [ 23 ]. Therefore, the value of PSMA PET/CT for newly diagnosed mHSPC still needs to be evaluated by its application during natural disease course. In a prospective trial, about 63% of newly diagnosed oligometastatic prostate cancer cases had a complete radiological response to systemic therapy plus pelvic lymph node dissection [PLMD] guided by PSMA PET/CT and 47% of cases had a pathological complete response, suggesting the potential value of PSMA PET/CT in treatment evaluation and decision-making [ 24 ]. Furthermore, a previous study by our group found that about 50% of nonmetastatic castration-resistant prostate cancer on conventional imaging has been proved to be metastatic on PSMA PET/CT, and the incidence of metastatic disease was less frequent in patients who received SBRT compared with those who did not [ 25 ].

These results suggested that when systemic therapy had inhibited the function of most distant metastases, the functional residual lesions might need individualized MDT. MMT strategies based on treatment response evaluated by PSMA PET/CT might be optimal for managing mHSPC with oligometastatic state.

Trial overview

This study is a prospective, single-arm, open-labeled, phase II clinical trial that will be conducted from March 2023 to December 2024 at Fudan University Shanghai Cancer Center (FUSCC). Patients with newly diagnosed metastatic mHSPC and ≤ 10 distant metastatic sites will be enrolled for the study. The patients will receive 6 cycles of apalutamide in combination with ADT. Gallium-68 ( 68 Ga) or fluorine-18 ( 18 F) PSMA PET/CT will be used for viewing metastatic sites after 3 treatment cycles. In patients with metastatic oligoresidues (≤ 10 distant metastases), cRP will be performed. During prostatectomy, ADT will be maintained. Apalutamide plus ADT will be continued for 2 weeks after surgery, and PSMA PET/CT-guided metastasis-directed SBRT will be adopted as determined by the investigators (Fig.  1 ). This study protocol will follow the Good Clinical Practice standards and the Declaration of Helsinki principles. This prospective study has been approved by the Clinical Research Ethics Committee of FUSCC (number 1909207–12) and has been registered with ClinicalTrials.gov (NCT05188911). The written informed consent will be obtained from all participants.

The flowchart of the proposed trial. MDT: metastasis-directed therapy; cRP: cytoreductive radical prostatectomy; ECOG PS: Eastern Cooperative Oncology Group Performance Status; RT: radiation therapy; ADT: androgen deprivation therapy; PSMA-PET/CT: prostate-specific membrane antigen positron emission tomography/computed tomography; PSA: prostate-specific antigen; CTCAE: Common Terminology Criteria for Adverse Events

The primary endpoint is the proportion of patients with undetectable prostate-specific antigen (PSA), no disease progression, and no symptomatic deterioration after 6 treatment cycles. Undetectable PSA is defined as ≤ 0.2 ng/mL and confirmed by two successive evaluations at least 3 weeks apart. Disease progression will be evaluated by a blinded independent central review (ICR) according to the Response Evaluation Criteria in Solid Tumours (RECIST) version 1.1. Symptomatic deterioration is referred to as a situation where the treatment needs to be interrupted, but with no apparent disease progression.

The secondary endpoints include the following:

Percentage of cases with undetectable PSA after 3 treatment cycles.

PSA50 and PSA90 response rate after 3 and 6 treatment cycles. PSA50 and PSA 90 response rate are the percentages of patients who have a reduction of greater than 50% and 90% from baseline, respectively.

Comparisons of baseline conventional imaging and PSMA PET/CT imaging features, including tumor-node-metastasis (TNM) staging, molecular imaging TNM (miTNM) classification, number of metastases, and metastatic sites.

Proportion of patients with ≤ 10 metastases (except pelvic lymph node metastases without new metastases) assessed by PSMA PET/CT at the end of cycle 3.

Comparison of conventional imaging and PSMA PET/CT imaging features at the end of the 3 and 6 treatment cycles, including change in prostate volume, number and volume of prostate tumor nodules, distribution of metastatic lesions and number of metastatic lesions.

Safety assessment: including adverse events (graded according to National Cancer Institute's Common Terminology Criteria for Adverse Events 5.0 criteria) and surgical complications (graded according to Clavien-Dindo classification).

Assess the feasibility of implementing cRP ± MDT based on the number of metastases in PSMA PET/CT.

The exploratory endpoints include the following:

SUVmax and tumor volume in PSMA PET/CT after 3 treatment cycles, and their correlations to PSA response rate and demographic characteristics.

Subgroup analysis stratified by baseline SUVmax and number of metastases.

Pathological complete response (defined as the percentage of patients having no surviving tumor cells in prostatectomy specimens following preoperative treatment), minimal residual disease (defined as residual tumor ≤ 5 mm in prostatectomy specimens following preoperative treatment), and pathological tumor-node.

Time to PSA progression, defined as the time from the beginning of treatment to the first PSA progression based on Prostate Cancer Working Group 2 criteria.

rPFS, defined as the time from treatment initiation until radiographic progression based on RECIST 1.1 criteria and death from any cause.

Metastatic progression pattern in patients achieving a minimal residual state after treatment.

Natural course of disease in patients with multiple metastases before and after treatment.

Biomarker search: prostate cancer-related molecular markers.

Participants and methods

This study will enroll patients with newly diagnosed mHSPC and distant metastases of ≤ 10 detected by conventional imaging. All patients will receive apalutamide plus ADT.

Inclusion criteria

Participants who satisfy all the following will be eligible for the study.

Able to understand and willing to sign the informed consent.

Men aged ≥ 18 years.

With histologically or cytologically confirmed prostate adenocarcinoma (primary small cell carcinoma or signet-ring cell carcinoma of the prostate are not allowed, however adenocarcinoma with neuroendocrine differentiation accounting ≤ 10% is allowed).

Newly diagnosed prostate cancer (within 3 months prior to enrollment).

With distant metastatic disease (M1a/b staging) assessed via conventional imaging including bone imaging, conventional CT, or magnetic resonance imaging [MRI].

With ≤ 10 distant metastatic sites assessed via conventional imaging before systemic treatment.

Receipt of apalutamide plus ADT as first-line treatment is the next treatment option. Prior therapy with ADT or first-generation antiandrogen agent (e.g., bicalutamide, flutamide) or apalutamide + ADT for ≤ 2 months before enrollment are permitted.

Receipt of PSMA PET/CT within 6 weeks before receiving apalutamide plus ADT and without prior treatment of NHT or chemotherapy before PSMA PET/CT examination.

Willing to accept cRP and/or PLMD ± MDT.

With Eastern Cooperative Oncology Group performance status of 0–1.

With adequate organ function.

With life expectancy ≥ 12 months.

Exclusion criteria

Participants with any of the following will be excluded from the study.

History of hypersensitivity reaction or intolerance to any drug involved in the study.

Contraindication or intolerance to cRP or radiation therapy

With visceral metastasis assessed via conventional imaging.

Prior treatment with any of the following: ADT or first-generation antiandrogen agent (such as bicalutamide and flutamide) for > 2 months; other NHT (such as abiraterone, enzalutamide, and darutamide); chemotherapy; any form of local prostate therapy, including surgery or RT (such as external beam radiotherapy [EBRT], SBRT, brachytherapy, and radiopharmaceutical therapy); any form of metastatic treatment, including surgery or RT (such as EBRT, SBRT, brachytherapy, and radiopharmaceutical therapy), but prior transurethral resection of the prostate for benign prostatic hyperplasia is permitted; immunotherapy; targeted therapy.

History of seizures, medication that lowers the seizure threshold, or a seizure-inducing illness within 12 months before starting study treatment (including history of transient ischemic attack, stroke, and traumatic brain injury requiring hospitalization with disturbance of consciousness).

Receipt of major surgery within 4 weeks before starting study treatment.

With major cardio-cerebrovascular disease within 6 months before study treatment, including severe/unstable angina, myocardial infarction, congestive heart failure [New York Heart Association class III or IV], cerebrovascular accident, or arrhythmia requiring medical treatment.

With swallowing disorder, chronic diarrhea, intestinal obstruction, or other factors affecting drug ingestion and absorption.

With active infection, such as serologically positive for human immunodeficiency virus (HIV), positive HBV surface antigen (HBsAg) result, and positive for hepatitis C virus (HCV) antibody, which may affect the safety and efficacy of medication according to the investigator's judgment.

Other malignancy within the last 3 years or at the same time, adequately treated non-melanoma skin cancer is permitted.

With known brain metastases or active meningitis.

Participation in another clinical study with investigational or medical devices.

Unable to cooperate with treatment and follow-up procedures.

With concomitant diseases (such as poorly controlled hypertension, severe diabetes, neurological, and psychiatric diseases) or any other condition that could seriously compromise patient safety, confound the study results, or prevent patients from completing the study.

Study treatment

Patients will receive apalutamide in combination with ADT. Each treatment cycle will be 28 days, with a total of 6 treatment cycles. Apalutamide will be given orally at a dosage of 240 mg once daily, but this dose will be adjusted according to the instruction in case of intolerability [ 26 ]. The ADT regimen consists of a gonadotropin-releasing hormone analog (GnRHa), either a GnRHa agonist or a GnRHa antagonist. The type, frequency, and dosage of the ADT will be determined by the investigators, and the dosage will be adjusted according to the instruction if adverse events occur.

The PSMA PET/CT imaging will be performed in a 2-week window after 3 cycles of apalutamide plus ADT. Based on the PSMA PET/CT images, the investigators will decide on subsequent cytoreduction. The patients with ≤ 10 distant metastatic sites by PSMA PET/CT, will be treated with cRP with/without MDT. cRP and PLND will be performed based on the evaluation of preoperative PSMA PET/CT images. ADT will be maintained during the whole perioperative period, while apalutamide will be discontinued for at least 1 week before the day of surgery. Apalutamide plus ADT will be restored 2 weeks postoperatively based on the evaluation by investigators. A multidisciplinary team including investigators will reach a decision on the application of MDT. SBRT will be performed and the dosages for different body sites will vary based on the tolerance level of the organ with metastases and the surrounding normal tissue. The lesions will be exposed to radiation with a recommended protocol using guidelines and the clinical experience of the investigators. The patients will be concurrently administered apalutamide plus ADT during the radiotherapy. For patients undergoing cytoreduction (cRP with/without MDT), the entire treatment duration will not exceed 8 months.

Cases recovering from the oligometastatic state after three cycles of apalutamide plus ADT assessed by PSMA PET/CT imaging will continue to receive apalutamide plus ADT. Treatment will be terminated if the patients cannot benefit from the therapy, experience intolerant toxicity, or withdraw the informed consent. For patients who discontinue apalutamide treatment plus ADT within the first three months of the trial, subsequent treatment will be determined by the investigators.

Assessments

Serum PSA will be assessed, and PSMA PET/CT imaging will be performed once every 3 cycles during the systemic treatment [ 27 ]. If serum PSA level increases or patients suffer from osteodynia, chest, abdominal, and pelvic CT/MRI imaging will be performed. The participants will be followed up via telephone until the end of the study, death, or loss to follow-up, within ± 4 weeks (Table  1 ). 68 Ga or 18 F can be used as a tracer and the type of tracer for each patient will remain unchanged throughout the study.

Data management

This trial will use case report forms (CRF) to collect basic and clinical data. The investigators will maintain source files for each subject, including original medical records and other written data or records. The data entered from the CRF must be linked to source files for each subject. Data managers will write the data review reports following the trial protocol and data review criteria from databases. Primary investigators, statisticians, and data managers will attend the data review meetings, and the data will be reviewed, resolved, and signed by the representatives attending the meetings. After all personnel approve the data, data managers will organize and execute data-locking tasks. The locked data will be submitted to statisticians for statistical analysis.

An independent data and safety monitoring committee will be established to assess safety when serious adverse events occur. All adverse events will be recorded. A qualified independent auditor will be appointed to scrutinize all trial protocols, and the trial will be conducted before and during the treatment period based on a written procedure.

Statistical analysis

The primary endpoint is the rate of participants with an undetectable PSA after 6 cycles of the apalutamide plus ADT. Fleming's two-stage group sequential design will be employed in this study, where the null hypothesis is that the rate of participants with an undetectable PSA will be ≤ 40%, and the alternate hypothesis was an undetectable PSA of > 60%. The test will be one-sided, with an α of 0.05 and a power value of 0.80. The first stage of the study will enroll 21 patients; if the number of patients with undetectable PSA is ≤ 8 during this stage, this treatment strategy will be considered ineffective and the study will be terminated. In case the number of patients with undetectable PSA is > 8 at the initial stage, this study will enter its second stage, when more patients will be added to reach the sample size of 42. If the number of patients with undetectable PSA is ≤ 22 during this period, this protocol will be considered ineffective. With an assumption of a dropout rate of 10%, a total of 47 patients will be needed for the effective analysis.

The full analysis set (FAS) will be used for efficacy analysis. The FAS follows the intend-to-treat (ITT) principle, where patients receiving at least one treatment are included in the analysis. The safety analysis set will include all patients receiving at least one treatment in this study, using the safety records after the treatment.

Quantitative data with normal distribution or near normal distribution will be expressed as mean ± standard deviation; those with skewed distribution or heterogeneity of variance will be expressed as median and quartile. Categorical data will be expressed as numbers and percentages. Categorical data will be analyzed by chi-square test or Fisher's exact test. The Kaplan–Meier method will be used to calculate the median times (95% confidence intervals) for time-to-event variables and generate survival plots. Each variable will be analyzed against the baseline value of the screening period. Paired t-test, χ2-test, exact probability method, or non-parametric test will be utilized to assess the differences of determinants before and after the study.

Study status

The CHAMPION study commenced patient enrollment in March 2023 and is currently ongoing with patient recruitment.

Metastatic prostate cancer is a major cause of mortality among men [ 1 ]. The proportion of patients with advanced prostate cancer when first diagnosed and treatment is higher in China compared to other developed counties [ 3 ]. The 5-year OS rate of patients with newly diagnosed metastatic prostate cancer is only about 32.3% [ 5 ]. In metastatic prostate cancer, the oligometastatic state (low-tumor-burden disease state) differs in biological characteristics, clinical manifestations, and response to therapeutic intervention, compared to the advanced metastatic state, providing a good treatment window to prevent/slow disease progression [ 6 , 7 ]. Therefore, developing effective treatment strategies for prostate cancer with oligometastatic state would be beneficial to many patients with metastatic prostate cancer.

MMT consists of local therapeutic intervention and systemic treatment and is recommended for low-volume mHSPC [ 17 , 18 ]. Investigators at John Hopkins University first proposed total eradication therapy (TET) in 2019. The MMT protocol for TET comprised systematic treatment, local therapeutic intervention, and MDT, and preliminarily demonstrated the feasibility and efficacy in managing metastatic prostate cancer. The HORRAD and STAMPEDE clinical trials also confirmed the benefits of combined treatment with local therapeutic intervention and systemic treatment in mHSPC patients [ 21 , 28 ]. Systematic treatment mostly included conventional ADT; currently, the NHT containing apalutamide plus ADT has become the standard treatment option for mHSPC. However, the effect of this combined therapy in patients with low-tumor-burden mHSPC still needs to be explored.

Limitations on the number of distant metastases that determines whether patients should undergo MMT or not continuously vary with changes in treatment techniques and notions. In the STAMPEDE trial, the survival benefits and failure-free survival were dependent on the number of bone metastases based on the subgroup analysis [ 21 ]. In patients with ≤ 4 bone metastases, the OS was significantly prolonged after radiation therapy; patients with 4–8 bone metastases still benefited from the radiation therapy; patients with as many as 9 bone metastatic sites benefited only from failure-free survival. Assessing SBRT in patients with 4–10 metastases, a phase III clinical study (SABR-COMET-10) found SBRT feasible and effective in patients with pan-tumor oligometastases, including 4–10 prostate cancer oligometastases [ 14 ]. This proposed trial will incorporate patients with newly diagnosed mHSPC and ≤ 10 metastatic sites to determine the feasibility and potential benefit of MMT.

Furthermore, identifying the suitable lesion and selecting the optimal timing and in MDT treatment remains controversial. Since PSMA PET/CT has higher diagnostic accuracy than MRI, CT, and prostate ultrasonography it might serve as an effective evaluation method during follow-up [ 29 , 30 , 31 ]. Therefore, this study will evaluate the feasibility of using PSMA PET/CT and its diagnostic accuracy to guide decision-making for cytoreduction in patients with newly diagnosed mHSPC treated with apalutamide and ADT.

This study will have a few limitations to be considered. First, this is a single-arm clinical trial without comparators. Therefore, the benefit of the treatment strategies in this study should be verified in further studies. Secondly, this is an exploratory trial with a relatively small sample size, and a firm conclusion cannot be drawn. Nevertheless, the findings of this study will provide preliminary evidence for a large randomized controlled clinical trial.

In conclusion, this study will provide a new clinical strategy, consisting of NHT and cytoreduction guided by PSMA PET/CT-guided in treating patients with newly diagnosed mHSPC and ≤ 10 metastatic sites. Further, the feasibility of the individualized therapeutic intervention developed in this study, including systemic therapy, cRP and MDT will be explored for managing patients with mHSPC. The findings of this study might provide evidence of the clinical management pathway, to further improve prognosis in patients with mHSPC.

Availability of data and materials

No datasets were generated or analysed during the current study.

Abbreviations

Androgen deprivation therapy

Case report forms

Cytoreductive radical prostatectomy

External beam radiotherapy

Full analysis set

Gonadotropin-releasing hormone analog

Independent central review

Intend-to-treat

Metastasis-directed therapy

Metastatic hormone-sensitive prostate cancer

Multi-modal therapy

Magnetic resonance imaging

Novel hormone therapy

Overall survival

Radiographic progression-free survival

Stereotactic body radiation therapy

• Positron emission tomography/computed tomography

Pelvic lymph node dissection

Prostate-specific antigen

• Prostate-specific membrane antigen

Response Evaluation Criteria in Solid Tumours

Total eradication therapy

Tumor-node-metastasis

Sung H, Ferlay J, Siegel RL, Laversanne M, Soerjomataram I, Jemal A, et al. Global cancer statistics 2020: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin. 2021;71(3):209–49.

Article   PubMed   Google Scholar  

Pan J, Ye D, Zhu Y. Olaparib outcomes in metastatic castration-resistant prostate cancer: First real-world experience in safety and efficacy from the Chinese mainland. Prostate Int. 2022;10(3):142–7.

Article   PubMed   PubMed Central   Google Scholar  

Qiu H, Cao S, Xu R. Cancer incidence, mortality, and burden in China: a time-trend analysis and comparison with the United States and United Kingdom based on the global epidemiological data released in 2020. Cancer Commun (Lond). 2021;41(10):1037–48.

Kanesvaran R, Castro E, Wong A, Fizazi K, Chua MLK, Zhu Y, et al. Pan-Asian adapted ESMO clinical practice guidelines for the diagnosis, treatment and follow-up of patients with prostate cancer. ESMO Open. 2022;7(4): 100518.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Pan J, Zhao J, Ni X, Gan H, Wei Y, Wu J, et al. The prevalence and prognosis of next-generation therapeutic targets in metastatic castration-resistant prostate cancer. Mol Oncol. 2022;16(22):4011–22.

Katipally RR, Pitroda SP, Juloori A, Chmura SJ, Weichselbaum RR. The oligometastatic spectrum in the era of improved detection and modern systemic therapy. Nat Rev Clin Oncol. 2022;19(9):585–99.

Hellman S, Weichselbaum RR. Oligometastases. J Clin Oncol. 1995;13(1):8–10.

Article   CAS   PubMed   Google Scholar  

Glicksman RM, Metser U, Vines D, Valliant J, Liu Z, Chung PW, et al. Curative-intent metastasis-directed therapies for molecularly-defined oligorecurrent prostate cancer: a prospective phase II trial testing the oligometastasis hypothesis. Eur Urol. 2021;80(3):374–82.

Ost P, Reynders D, Decaestecker K, Fonteyne V, Lumen N, De Bruycker A, et al. Surveillance or metastasis-directed therapy for oligometastatic prostate cancer recurrence: a prospective, randomized, multicenter phase II trial. J Clin Oncol. 2018;36(5):446–53.

Phillips R, Shi WY, Deek M, Radwan N, Lim SJ, Antonarakis ES, et al. Outcomes of observation vs stereotactic ablative radiation for oligometastatic prostate cancer: the ORIOLE phase 2 randomized clinical trial. JAMA Oncol. 2020;6(5):650–9.

De Bleser E, Jereczek-Fossa BA, Pasquier D, Zilli T, Van As N, Siva S, et al. Metastasis-directed therapy in treating nodal oligorecurrent prostate cancer: a multi-institutional analysis comparing the outcome and toxicity of stereotactic body radiotherapy and elective nodal radiotherapy. Eur Urol. 2019;76(6):732–9.

Steuber T, Jilg C, Tennstedt P, De Bruycker A, Tilki D, Decaestecker K, et al. Standard of care versus metastases-directed therapy for PET-detected nodal oligorecurrent prostate cancer following multimodality treatment: a multi-institutional case-control study. Eur Urol Focus. 2019;5(6):1007–13.

Farolfi A, Hadaschik B, Hamdy FC, Herrmann K, Hofman MS, Murphy DG, et al. Positron emission tomography and whole-body magnetic resonance imaging for metastasis-directed therapy in hormone-sensitive oligometastatic prostate cancer after primary radical treatment: a systematic review. Eur Urol Oncol. 2021;4(5):714–30.

Ashram S, Bahig H, Barry A, Blanchette D, Celinksi A, Chung P, et al. Planning trade-offs for SABR in patients with 4 to 10 metastases: a substudy of the SABR-COMET-10 randomized trial. Int J Radiat Oncol Biol Phys. 2022;114(5):1011–5.

Connor MJ, Smith A, Miah S, Shah TT, Winkler M, Khoo V, et al. Targeting oligometastasis with stereotactic ablative radiation therapy or surgery in metastatic hormone-sensitive prostate cancer: a systematic review of prospective clinical trials. Eur Urol Oncol. 2020;3(5):582–93.

Dai B, Zhang S, Wan FN, Wang HK, Zhang JY, Wang QF, et al. Combination of androgen deprivation therapy with radical local therapy versus androgen deprivation therapy alone for newly diagnosed oligometastatic prostate cancer: a phase II randomized controlled trial. Eur Urol Oncol. 2022;5(5):519–25.

Reyes DK, Rowe SP, Schaeffer EM, Allaf ME, Ross AE, Pavlovich CP, et al. Multidisciplinary total eradication therapy (TET) in men with newly diagnosed oligometastatic prostate cancer. Med Oncol. 2020;37(7):60.

Reyes DK, Trock BJ, Tran PT, Pavlovich CP, Deville C, Allaf ME, et al. Interim analysis of companion, prospective, phase II, clinical trials assessing the efficacy and safety of multi-modal total eradication therapy in men with synchronous oligometastatic prostate cancer. Med Oncol. 2022;39(5):63.

Lester-Coll NH, Ades S, Yu JB, Atherly A, Wallace HJ 3rd, Sprague BL. Cost-effectiveness of prostate radiation therapy for men with newly diagnosed low-burden metastatic prostate cancer. JAMA Netw Open. 2021;4(1):e2033787.

Parker CC, James ND, Brawley CD, Clarke NW, Ali A, Amos CL, et al. Radiotherapy to the prostate for men with metastatic prostate cancer in the UK and Switzerland: long-term results from the STAMPEDE randomised controlled trial. PLoS Med. 2022;19(6):e1003998.

Parker CC, James ND, Brawley CD, Clarke NW, Hoyle AP, Ali A, et al. Radiotherapy to the primary tumour for newly diagnosed, metastatic prostate cancer (STAMPEDE): a randomised controlled phase 3 trial. Lancet. 2018;392(10162):2353–66.

Lengana T, Lawal IO, Boshomane TG, Popoola GO, Mokoala KMG, Moshokoa E, et al. (68)Ga-PSMA PET/CT replacing bone scan in the initial staging of skeletal metastasis in prostate cancer: a fait accompli? Clin Genitourin Cancer. 2018;16(5):392–401.

Barbato F, Fendler WP, Rauscher I, Herrmann K, Wetter A, Ferdinandus J, et al. PSMA-PET for the assessment of metastatic hormone-sensitive prostate cancer volume of disease. J Nucl Med. 2021;62(12):1747–50.

Huebner N, Rasul S, Baltzer P, Clauser P, Hermann Grubmuller K, Mitterhauser M, et al. Feasibility and optimal time point of [(68)Ga]gallium-labeled prostate-specific membrane antigen ligand positron emission tomography imaging in patients undergoing cytoreductive surgery after systemic therapy for primary oligometastatic prostate cancer: implications for patient selection and extent of surgery. Eur Urol Open Sci. 2022;40:117–24.

Pan J, Wei Y, Zhang T, Liu C, Hu X, Zhao J, et al. Stereotactic radiotherapy for lesions detected via (68)Ga-prostate-specific membrane antigen and (18)F-fluorodexyglucose positron emission tomography/computed tomography in patients with nonmetastatic prostate cancer with early prostate-specific antigen progression on androgen deprivation therapy: a prospective single-center study. Eur Urol Oncol. 2022;5(4):420–7.

Chi KN, Agarwal N, Bjartell A, Pereira de Santana Gomes AJ, Given R, et al. Apalutamide for metastatic, castration-sensitive prostate cancer. N Engl J Med. 2019;381(1):13–24.

Pan J, Zhang T, Chen S, Bu T, Zhao J, Ni X, et al. Nomogram to predict the presence of PSMA-negative but FDG-positive lesion in castration-resistant prostate cancer: a multicenter cohort study. Ther Adv Med Oncol. 2024;16:17588359231220506.

Boeve LMS, Hulshof M, Vis AN, Zwinderman AH, Twisk JWR, Witjes WPJ, et al. Effect on survival of androgen deprivation therapy alone compared to androgen deprivation therapy combined with concurrent radiation therapy to the prostate in patients with primary bone metastatic prostate cancer in a prospective randomised clinical trial: data from the HORRAD trial. Eur Urol. 2019;75(3):410–8.

Perera M, Papa N, Roberts M, Williams M, Udovicich C, Vela I, et al. Gallium-68 prostate-specific membrane antigen positron emission tomography in advanced prostate cancer-updated diagnostic utility, sensitivity, specificity, and distribution of prostate-specific membrane antigen-avid lesions: a systematic review and meta-analysis. Eur Urol. 2020;77(4):403–17.

Tan N, Oyoyo U, Bavadian N, Ferguson N, Mukkamala A, Calais J, et al. PSMA-targeted radiotracers versus (18)F fluciclovine for the detection of prostate cancer biochemical recurrence after definitive therapy: a systematic review and meta-analysis. Radiology. 2020;296(1):44–55.

Pan J, Zhao J, Ni X, Zhu B, Hu X, Wang Q, et al. Heterogeneity of [(68)Ga]Ga-PSMA-11 PET/CT in metastatic castration-resistant prostate cancer: genomic characteristics and association with abiraterone response. Eur J Nucl Med Mol Imaging. 2023;50(6):1822–32.

Download references

Acknowledgements

The investigator-initiated study received funding from Xian Janssen Pharmaceutical Ltd. The authors would like to thank the patients and their families, as well as all the investigators who will participate in the proposed study.

This study was supported by National Natural Science Foundation of China (82203106 and 82172621), Shanghai Sailing Program (21YF1408100), Shanghai Medical Innovation Research Special Project (21Y11904300), Shanghai Shenkang Research Physician Innovation and Transformation Ability Training Project (SHDC2022CRD035), Clinical Research Plan of SHDC (SHDC-2020CR2016B), Shanghai Academic/Technology Research Leader (23XD1420600), Shanghai Anti-Cancer Association Eyas Project (SACA-CY22A04), China Urological Oncology Research Foundation, and Oriental Scholar Professorship. The content is solely the responsibility of the authors and does not necessarily represent the official views of the funding bodies. The protocol in this manuscript has undergone peer-review as part of the process of obtaining external funding.

Author information

Beihe Wang and Jian Pan contributed equally to this work.

Authors and Affiliations

Department of Urology, Fudan University Shanghai Cancer Center, No. 270 Dong’an Road, Shanghai, 200032, People’s Republic of China

Beihe Wang, Jian Pan, Tingwei Zhang, Xudong Ni, Yu Wei, Xiaomeng Li, Bangwei Fang, Junlong Wu, Hongkai Wang, Dingwei Ye & Yao Zhu

Shanghai Genitourinary Cancer Institute, Shanghai, China

Department of Oncology, Shanghai Medical College, Fudan University, Shanghai, China

Beihe Wang, Jian Pan, Tingwei Zhang, Xudong Ni, Yu Wei, Xiaomeng Li, Bangwei Fang, Xiaoxin Hu, Hualei Gan, Junlong Wu, Hongkai Wang, Dingwei Ye & Yao Zhu

Department of Radiology, Fudan University Shanghai Cancer Center, Shanghai, China

Department of Pathology, Fudan University Shanghai Cancer Center, Shanghai, China

You can also search for this author in PubMed   Google Scholar

Contributions

B.W., J.P., D.Y. and Y.Z. conceived the study. B.W., J.P., T.Z., X.N., Y.W., X.L., B.F., X.H., H.G., J.W., H.W., D.Y. and Y.Z. formed the expert panel. J.P., B.W. and Y.Z. drafted the manuscript with assistance from all coauthors. All authors critically assessed the study design, enrolled patients in the study, edited the manuscript, and approved the final manuscript. The corresponding author had final responsibility for the decision to submit for publication.

Corresponding authors

Correspondence to Dingwei Ye or Yao Zhu .

Ethics declarations

Ethics approval and consent to participate.

This study protocol will follow the Good Clinical Practice standards and the Declaration of Helsinki principles. This prospective study has been approved by the Clinical Research Ethics Committee of FUSCC (number 1909207–12) and has been registered with ClinicalTrials.gov (NCT05188911). The written informed consent will be obtained from all participants.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary material 1., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Wang, B., Pan, J., Zhang, T. et al. Protocol for CHAMPION study: a prospective study of maximal-cytoreductive therapies for patients with de novo metastatic hormone-sensitive prostate cancer who achieve oligopersistent metastases during systemic treatment with apalutamide plus androgen deprivation therapy. BMC Cancer 24 , 643 (2024). https://doi.org/10.1186/s12885-024-12395-3

Download citation

Received : 15 December 2023

Accepted : 16 May 2024

Published : 25 May 2024

DOI : https://doi.org/10.1186/s12885-024-12395-3

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Hormone-sensitive prostate cancer
• Apalutamide
• Radical prostatectomy

ISSN: 1471-2407

• Submission enquiries: [email protected]
• General enquiries: [email protected]

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Am J Occup Ther

Nuts and Bolts of Conducting Feasibility Studies

Linda tickle-degnen.

Linda Tickle-Degnen, PhD, OTR/L, FAOTA, is Professor, Department of Occupational Therapy, Tufts University, 26 Winthrop Street, Medford, MA 02155; ude.stfut@nengeD_elkciT.adniL

Many factors can affect the successful implementation and validity of intervention studies. A primary purpose of feasibility and pilot studies is to assess the potential for successful implementation of the proposed main intervention studies and to reduce threats to the validity of these studies. This article describes a typology to guide the aims of feasibility and pilot studies designed to support the development of randomized controlled trials and provides an example of the studies underlying the development of one rehabilitation trial. The purpose of most feasibility and pilot studies should be to describe information and evidence related to the successful implementation and validity of a planned main trial. Null hypothesis significance testing is not appropriate for these studies unless the sample size is properly powered. The primary tests of the intervention effectiveness hypotheses should occur in the main study, not in the studies that are serving as feasibility or pilot studies.

If you have built castles in the air, your work need not be lost; that is where they should be. Now put foundations under them. ( Thoreau, 1854/2009 , para. 10)

Planning intervention research takes creativity and innovation, a bit like building castles in the air. Feasibility studies are studies designed to build the foundation for the planned intervention study. For several reasons, it is challenging to define feasibility studies for occupational therapy research. Although many types of feasibility studies could be applicable to intervention research in occupational therapy, no typology has been developed specifically for the research done in our field. Published feasibility study typologies are rare and typically focus on preparing for drug trials in which a single “active” ingredient is being tested, such as a chemical that is posited to be the causal effect of intervention outcomes.

As we know, occupational therapy is not implemented with the assumption of a single active factor inducing change in our clients. Our intervention outcomes are derived from “blended” active agents. Person, environment, and occupation factors compose an interactional network of potential agents that create the quality of occupational performance and health outcomes ( Law et al., 1996 ). Moreover, our outcomes are often theoretical constructs (e.g., occupational performance, quality of life) rather than direct observables (e.g., cure of disease or a change in bodily function), and they generally are measured along a conceptual continuum that is not a true count (e.g., cell counts, tumor size). Our interventions in actual practice are client centered and highly individualized as opposed to highly standardized.

Feasibility studies in occupational therapy must build the foundations that support intervention trials that involve a blending of active agents; a theoretical perspective that reflects an understanding of occupational performance and outcomes as being at the intersection of person, environment, and occupation; a measurement paradigm based on constructs and continua; and client-centered, individualized intervention. Feasibility studies are rare in our field. Many, if not most, occupational therapy pilot studies do not fulfill the definition of feasibility study according to the current paradigms emerging in the methodological literature on conducting intervention trials and as described in this article. Yet, feasibility studies are critical to the successful implementation of randomized controlled trials (RCTs), one of the top-tier designs for supporting intervention effectiveness.

Many factors can affect the internal, external, construct, and statistical validity of the design, implementation, and results of RCTs ( Shadish, Cook, & Campbell, 2002 ). The primary purposes of a feasibility study are to ensure that study implementation is practical and to reduce threats to the validity of the study’s outcomes. Drawing on the emerging methodological literature in this area, this article first describes and defines feasibility and pilot studies; then presents a feasibility and pilot study typology that was designed for drug trials but that, with minor modification, is relevant to occupational therapy; and, finally, provides examples from my own and my colleagues’ work in Parkinson’s disease of how we used feasibility and pilot studies to design and refine a rehabilitation RCT.

Distinguishing Feasibility and Pilot Studies

One of the clearer definitions of feasibility study and its differentiation from pilot study comes from the United Kingdom’s National Institute for Health Research Evaluation, Trials and Studies Coordination Centre (NETSCC; 2012), which states, “Feasibility studies are pieces of research done before a main study in order to answer the question ‘Can this study be done?’. . . used to estimate important parameters that are needed to design the main study” (Research Methods section, para. 3). According to the NETSCC, a feasibility study differs from a pilot study in that a feasibility study tries out pieces of the RCT, whereas the pilot study tries out the operation of all pieces as they will be implemented in the planned RCT. A pilot study is

a version of the main study that is run in miniature to test whether the components of the main study can all work together . . . [and resembles] the main study in many respects, including an assessment of the primary outcome. (Research Methods section, para. 6)

Although the literature on feasibility and pilot study designs is relatively new and is not consistent with respect to these definitions, the emerging methodological literature suggests that both feasibility and pilot studies should be addressed specifically to descriptively assessing the feasibility and validity of the RCT plan and not to testing the hypotheses of the main RCT ( Arain, Campbell, Cooper, & Lancaster, 2010 ; Leon, Davis, & Kraemer, 2011 ; Shanyinde, Pickering, & Weatherall, 2011 ; Thabane et al., 2010 ). Feasibility and pilot studies are not expected to have the large sample sizes that are needed to adequately power statistical null hypothesis testing. Indeed, pilot studies that are published often do not show statistically significant findings and rarely lead to larger trials to adequately power the hypothesis testing ( Arain et al., 2010 ; Shanyinde et al., 2011 ). The outcomes of most feasibility and pilot studies should be measured with descriptive statistics, qualitative analysis, and the compilation of basic data related to administrative and physical infrastructure.

Unfortunately, editorial publication bias and peer review norms often require investigators to perform null hypothesis significance testing even when it is not scientifically reasonable to conduct these tests ( Arain et al., 2010 ). In addition, reviewers and investigators alike tend to misinterpret nonsignificant statistical tests—those that fail to achieve the largely arbitrary criterion of p < .05 ( Cohen, 1994 )—of appropriately small-scale studies as indicative of the poor feasibility of future planned research or as the need for “more research” before research can be scaled up. These types of misguided conclusions can sidetrack or slow down the developmental progression of strong science and the rigorous testing of occupational therapy interventions.

Typology of Feasibility and Pilot Studies

A typology developed by a clinical epidemiology and biostatistics group at McMaster University ( Thabane et al., 2010 ) appears to be one of the most systematic and comprehensive typologies developed to date. It focuses on drug trials; however, with minor modification, it can be a suitable rehabilitation intervention typology. Thabane et al. (2010) outlined four primary purposes for both pilot studies and feasibility studies: to test the (1) process, (2) resources, (3) management, and (4) scientific basis of the planned RCT.

To demonstrate these purposes, I draw on the experience of a recently completed RCT ( Tickle-Degnen, Ellis, Saint-Hilaire, Thomas, & Wagenaar, 2010 ). The purpose of the RCT was to determine whether increasing hours of interdisciplinary self-management rehabilitation (physical, occupational, and speech therapy) had increasing benefits for health-related quality of life (HRQOL) in patients with Parkinson’s disease (PD) beyond best medical treatment, whether effects persisted at 2- and 6-mo follow-up, and whether targeted compared with nontargeted HRQOL domains responded more to rehabilitation. Participants on best medication therapy were randomized to one of three conditions for 6 wk of intervention: (1) 0 hr of rehabilitation; (2) 18 hr of clinic group rehabilitation plus 9 hr of attention control social sessions; and (3) 27 hr of rehabilitation, with 18 hr in clinic group rehabilitation and 9 hr in rehabilitation designed to transfer clinic training into home and community routines. Intervention was client centered in addressing participants’ specific quality-of-life concerns and also provided general strategies for addressing concerns that typically occur during the progression of the disease.

Process Assessment

Examples of questions for assessing the processes of a planned RCT include the following:

• Numbers of eligible members of the targeted population?
• Recruitment rates?
• Refusal rates for participation and for randomization?
• Retention and follow-up rates as the participants move through the trial?
• Adherence rates to study procedures, intervention attendance, and engagement?
• Eligibility criteria? Are criteria clear and sufficient or too inclusive or restrictive?
• Data collection assessments? Do participants understand the questions and other data collection methods? Do they respond with missing or unusable data?
• Amount of data collection? Do the participants have enough time and capacity to complete data collection procedures? Does the overall data collection plan involve a reasonable amount of time, or does it create a burden for the participants?

Process assessment is often documented in grant proposals in sections describing preliminary studies and human participant plans. At the time we were planning our RCT (2001–2002), we were unaware of a typology available to guide our preliminary planning. We did, however, have completed studies and collected evidence that we used to plan the RCT and support feasibility. Our neurological physician and nurse research team collected evidence about the number of eligible members of our targeted recruitment population—community-living adults with PD—whom the team followed in a movement disorder clinic. We developed potential recruitment, refusal, retention, and adherence expectations on the basis of a PD exercise trial led by our physical therapy investigators ( Ellis et al., 2005 ) and on the research experience of the neurological team. We conducted small clinical interventions in occupational therapy, physical therapy, and speech–language pathology that involved elements of the planned RCT and asked clients to rate their satisfaction with the intervention, and we recorded adherence rates to the intervention.

For the feasibility and suitability of eligibility criteria, the research team members reflected qualitatively on their clinical and research experiences and combined these reflections with published standards for clinical trials for older adult and PD populations. To assess the quality and burden of our data collection procedures, we used experience from our physical therapy trial, our collective research and clinical experience, and a literature review of gold-standard PD HRQOL measures. We estimated the number of hours our assessments would take and took into consideration the interaction of data collection procedures with PD medication timing and fatigue. We also outlined factors that could contribute to our population completing the assessments as planned. We used this information to build snack and bathroom breaks into the study protocol and created a written data collection protocol that involved verbal administration and supervision of all data collection by a trained assessment team (blind to intervention condition). The primary investigator team collectively assessed the protocol and informally tested various aspects of the timing and administration of the protocol.

Resources Assessment

Examples of questions for assessing the resources for implementing a planned RCT include the following:

Do we have the

• Physical capacity to handle the number of participants? What is the square footage as related to the stages and tasks of the procedures?
• Phone and communication technology capacity to stay in touch with and coordinate the participants? Is there Web and teleconferencing capability?
• Time to conduct each stage and aspect of the protocol? What are the time frames, and how do they coordinate with other responsibilities? How long does it take to connect with a participant or to send out mailings?
• Equipment in the correct place at the correct time? What equipment is needed, and is it available when needed?
• Ability to deal with broken, lost, or stolen equipment and materials? Are there backup plans for obtaining needed equipment and materials?
• Adequate software to capture and use data? What software is available for conducting the research?
• Institutional, departmental, and clinical centers’ willingness, motivation, and capacity to carry through with project-related tasks and to support investigator time and effort? What administrative services are in place for research at this level?
• Documented evidence indicating that these centers abide by their commitments? What are the challenges in fulfilling research support commitments?
• Access to basic services, such as copying, libraries, institutional technology, data servers, and purchasing?

Resource assessment often involves the collection and summarization of factual information at the investigators’ institutional settings. This information is often documented in grant proposals on facilities and resources forms and in the narrative description of research procedures. The gathering of resource evidence is the basis for determining what new materials, systems, and equipment must be obtained before the research activities can be implemented and for developing a study budget.

When planning our RCT, we measured the size of our labs and clinical spaces where assessments and clinic-based intervention would occur. We counted offices, desks, chairs, and computers; listed our software; located backup materials; and investigated and summarized all the services that our institutional settings provided for research such as ours. We spent considerable time assessing and developing institutional and departmental support for the research activities, clarifying and documenting decisions through e-mail. We secured rooms and obtained locking file cabinets for secure data storage and documented these in our research and human participant plans.

Although studies and documentation of resources may seem minor relative to process assessment and scientific development of the research plan, thorough resource assessment is fundamental to the success of research implementation. For example, Gardner, Gardner, MacLellan, and Osborne (2003) found that their otherwise thoughtfully conducted pilot study had not taken into consideration hospital process and environment factors that contributed to poor recruitment of participants and inadequate adherence of hospital staff to study protocol in the main study despite staff education related to project implementation. Consequently, the main study had to be aborted after startup. Such unfortunate events often entail great financial costs and potentially compromise future collaborations and the credibility and reputation of a research team.

Management Assessment

Examples of questions for assessing management issues in implementing a planned RCT include the following:

What are the challenges and strengths of

• The investigators’ administrative capacity to manage the planned RCT?
• Research investigator and staff capacities, expertise, and availability for the planned research activities?
• Formats and structures of forms that document participant progress through the trial?
• Accurate data entry into the computer? Are data lost, forgotten, or entered incorrectly? How are data files organized, named, and dated? Who is in charge of tracking the latest data entry and the quality of entry?
• Matching of data to participants from different sources (e.g., participant screen data, consent and entry into the RCT, adherence, and responses on outcome measures)?
• Management of the ethics of the research? To what extent do staff comply with the approved human participants protocol? How effectively are adverse events during implementation identified, documented, and reported? What happens if a participant experiences a clinical emergency or if family abuse is identified during the trial?

Management assessment is often documented in grant proposals in the investigator biosketches, data management and human participant plans, and budget justifications describing the specific responsibilities, activities, and roles of the research personnel. In our RCT, we drew on our collective experience to determine what our strengths and weaknesses were related to management and then planned specific activities and roles around investigator strengths to minimize weaknesses. Only after we started collecting data did we determine that our institution had a centralized data management service, and we were able to secure additional funding to use that service.

Because management of an RCT involves superb collaborative and communication skills and frequent collaboration and communication among research staff, these qualities must be assessed and documented before the initiation of the trial. For our RCT, our strongest systematic assessments involved compliance with human participant ethics and the storage of recruitment and screening data because, at the time, these were the most formalized aspects of research planning in our institution and in general practice. We completed other aspects of management assessment rather informally and on the basis of our collective research experience without a comprehensive typology guiding our planning. Current research planning practices demand more formal and systematic assessment.

Scientific Assessment

Examples of questions for assessing the scientific basis for implementing a planned RCT include the following:

• What is the level of safety of the procedures in the intervention or interventions?
• What is the level of safety and burdensomeness of the frequency, intensity, and duration of the intervention? Can these and other elements be standardized in a protocol without loss of a client-centered, individualized focus?
• What are the reliability, validity, and trustworthiness of the assessments for the targeted population for this specific intervention? Do the assessments capture individual participants’ needs and measure their responsiveness to these needs?
• What values constitute clinically meaningful differences on the primary outcome measures or assessment procedures?
• What is the expected degree of change (i.e., responsiveness) of the participants?
• What are the estimates of the intervention effect and the variance of that effect across the planned population?
• What are the expected subgroup effects (i.e., specificity effects or moderator variables)?

When we think of pilot studies, the questions listed above often are the ones that come to mind. They are the research questions that we most clearly identify as necessary to investigate before conducting an RCT. Notice that there is no mediation question in the above list. Mediation is what causes the presumable effectiveness of the intervention. A whole set of other research studies—descriptive, observational, and experimental—occur before the implementation of the feasibility and pilot studies that set the foundation for the RCT. The RCT is often planned around a theoretical model of causality that has already been tested as representative of an underlying “active ingredient” involved in improving health or minimizing disability. Causality may be too restrictive of a construct for occupational therapy; we often use theoretical models of intervention that are targeted to the reduction of a set of risk factors or promotion of a set of protective factors related to health and disability, without the assumption that any one factor or variable is the critical active ingredient. Occupational therapists call on a multiplicity of ingredients to create adaptive responses in clients.

Assessing the feasibility of the scientific basis of the RCT largely involves assessing whether elements of the RCT will be likely to operate with low degrees of error and threats to validity ( Shadish et al., 2002 ). In the case of our RCT, we drew on the emerging research literature on self-management of chronic disease and on theoretical models of health behavior and task performance to guide how we would promote HRQOL outcomes with a client-centered approach. We chose to conduct an interdisciplinary rather than a discipline-specific or multidisciplinary intervention on the basis of our scientific and clinical theory that health promotion would be greater with a task-specific than a discipline-specific or multidisciplinary approach. For example, if we wished to promote participants’ engagement in doing a favorite activity, we would most effectively do so by helping them integrate and manage their physical, occupational, and speech capacities in the service of doing the activity. Logically, this objective called for interdisciplinary intervention.

We conducted pilot studies to reduce threats to validity and documented these studies in grant proposal sections on preliminary studies, sample size estimation, the description and rationale for the research plan, and the human participant plan. These studies included our exercise trial ( Ellis et al., 2005 ) and two meta-analyses on the effectiveness of rehabilitation for PD, one on occupational therapy effectiveness ( Murphy & Tickle-Degnen, 2001 ) and one on physical therapy effectiveness ( de Goede, Keus, Kwakkel, & Wagenaar, 2001 ).

We also planned the RCT around the evidence found in the literature on movement and speech science and quality of life research in older adults with PD. From this investigation, we ascertained the probable safety of our intervention and decided on our primary outcome measure. At the time of the initiation of the RCT, no research studies were available on clinically meaningful differences for our measure of HRQOL. This information became available by the time of publication of our study, and we reported our results accordingly.

Our pilot studies provided us with estimates of the probable effect of our intervention, enabling us to estimate an adequately powered sample size for our study. At the time, the standards of sample size estimation did not include an estimation of the variance of the predicted effect; however, this estimate is now advisable for planning an RCT ( Lenth, 2001 ). Finally, we did not attempt to differentiate rehabilitation effects for subgroups of people with PD because little evidence was available in this respect. After completion of the trial, we performed post hoc analyses that suggested that more problematic baseline HRQOL predicted more participant responsiveness to the intervention, and from these tests we generated hypotheses to be tested in future intervention research.

Feasibility and pilot studies are important for building the foundation of large RCTs. These studies address all elements of the planned trial and ensure that the study is feasible and will be conducted in a manner that reduces threats to study validity. When conducted with proper aims and approaches, feasibility and pilot studies confront researchers with important facts before research stakeholders commit to a major investment in money and time for a large clinical trial. The publication of feasibility studies before a planned RCT, especially related to process, resources, and management of the RCT, was unusual in the past and is now becoming more common in the broader medical literature. Publication of these types of studies is rare in occupational therapy (e.g., Sturkenboom et al., 2012 ). The typology developed by Thabane et al. (2010) and slightly modified for occupational therapy intervention research provides guidance on how to approach pre-RCT studies systematically.

• Arain M., Campbell M. J., Cooper C. L., Lancaster G. A. What is a pilot or feasibility study? A review of current practice and editorial policy. BMC Medical Research Methodology. 2010; 10 (67) Retrieved from www.biomedcentral.com/1471-2288/10/67 . [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Cohen J. The earth is round ( p < .05) American Psychologist. 1994; 49 :997–1003. [ Google Scholar ]
• de Goede C. J. T., Keus S. H. J., Kwakkel G., Wagenaar R. C. The effects of physical therapy in Parkinson’s disease: A research synthesis. Archives of Physical Medicine and Rehabilitation. 2001; 82 :509–515. [ PubMed ] [ Google Scholar ]
• Ellis T., de Goede C., Feldman R. G., Wolters E. C., Kwakkel G., Wagenaar R. C. Efficacy of a physical therapy program in patients with Parkinson’s disease: A randomized controlled trial. Archives of Physical Medicine and Rehabilitation. 2005; 86 :626–632. [ PubMed ] [ Google Scholar ]
• Gardner G., Gardner A., MacLellan L., Osborne S. Reconceptualizing the objectives of a pilot study for clinical research. International Journal of Nursing Studies. 2003; 40 :719–724. [ PubMed ] [ Google Scholar ]
• Law M., Cooper B., Strong S., Stewart D., Rigby P., Letts L. The Person–Environment–Occupation model: A transactive approach to occupational performance. Canadian Journal of Occupational Therapy. 1996; 63 (1):9–23. [ PubMed ] [ Google Scholar ]
• Lenth R. V. Some practical guidelines for effective sample size determination. American Statistician. 2001; 55 :187–193. [ Google Scholar ]
• Leon A. C., Davis L. L., Kraemer H. C. The role and interpretation of pilot studies in clinical research. Journal of Psychiatric Research. 2011; 45 :626–629. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Murphy S., Tickle-Degnen L. The effectiveness of occupational therapy–related treatments for persons with Parkinson’s disease: A meta-analytic review. American Journal of Occupational Therapy. 2001; 55 :385–392. http://dx.doi.org/10.5014/ajot.55.4.385 . [ PubMed ] [ Google Scholar ]
• National Institute for Health Research. NIHR Evaluation, Trials and Studies Coordination Centre: Glossary. 2012. Retrieved from www.netscc.ac.uk/glossary/#glos6/ [ Google Scholar ]
• Shadish W. R., Cook T. D., Campbell D. T. Experimental and quasi-experimental designs for generalized causal inference. Boston: Houghton-Mifflin; 2002. [ Google Scholar ]
• Shanyinde M., Pickering R. M., Weatherall M. Questions asked and answered in pilot and feasibility randomized controlled trials. BMC Medical Research Methodology. 2011; 11 (117) Retrieved from www.biomedcentral.com/1471-2288/11/117 . [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Sturkenboom I. H., Graff M. J., Borm G. F., Veenhuizen Y., Bloem B. R., Munneke M., Nijhuis-van der Sanden M. W. The impact of occupational therapy in Parkinson’s disease: A randomized controlled feasibility study. Clinical Rehabilitation. Advance online publication. http://dx.doi.org/10.1177/0269215512448382 . [ PubMed ] [ Google Scholar ]
• Thabane L., Ma J., Chu R., Cheng J., Ismaila A., Rios L. P., Goldsmith C. H. A tutorial on pilot studies: The what, why and how. BMC Medical Research Methodology. 2010; 10 :1. Retrieved from www.biomedcentral.com/1471-2288/10/1 . [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Thoreau H. D. Walden. 2009. (Original work published 1854) Retrieved November 15, 2012, from www.transcendentalists.com/walden_conclusion.htm . [ Google Scholar ]
• Tickle-Degnen L., Ellis T., Saint-Hilaire M. H., Thomas C. A., Wagenaar R. C. Self-management rehabilitation and health-related quality of life in Parkinson’s disease: A randomized controlled trial. Movement Disorders. 2010; 25 :194–204. Retrieved from www.ncbi.nlm.nih.gov/pmc/articles/PMC2831132/ [ PMC free article ] [ PubMed ] [ Google Scholar ]

Telehealth cognitive behaviour therapy for the management of sleep disturbance in women with early breast cancer receiving chemotherapy: a feasibility study

• Open access
• Published: 23 May 2024
• Volume 32 , article number  375 , ( 2024 )

Cite this article

You have full access to this open access article

• Emma-Kate Carson   ORCID: orcid.org/0000-0001-9684-0247 1 , 2 , 3 ,
• Haryana M. Dhillon 4 , 5 ,
• Janette L. Vardy 1 , 5 , 6 ,
• Chris Brown 1 , 7 ,
• Kelly Ferrao Nunes-Zlotkowski 4 , 5 ,
• Stephen Della-Fiorentina 2 , 8 , 9 ,
• Sarah Khan 8 ,
• Andrew Parsonson 3 , 10 ,
• Felicia Roncoloato 1 , 2 , 3 ,
• Antonia Pearson 1 , 11 ,
• Tristan Barnes 10 , 11 &
• Belinda E. Kiely 1 , 2 , 3 , 6 , 7  

163 Accesses

4 Altmetric

Explore all metrics

Sleep quality commonly deteriorates in people receiving chemotherapy for breast cancer (BC). We aimed to determine feasibility and acceptability of telehealth-delivered cognitive behaviour therapy for insomnia (CBT-I) in people with early BC receiving (neo)adjuvant chemotherapy.

Multi-centre, single arm, phase 2 feasibility trial. People with stage I-III BC received 4 sessions of telehealth CBT-I over 8 weeks, during chemotherapy. Participants completed Pittsburgh Sleep Quality Index (PSQI) and other Patient Reported Outcome Measures (PROMs) at baseline, post-program (week 9) and post-chemotherapy (week 24); and an Acceptability Questionnaire at week 9. Primary endpoint was proportion completing 4 sessions of telehealth CBT-I.

In total, 41 participants were recruited: mean age 51 years (range 31–73). All 4 CBT-I sessions were completed by 35 (85%) participants. Acceptability of the program was high and 71% reported ‘ the program was useful’ . There was no significant difference in the number of poor sleepers (PSQI score ≥ 5) at baseline 29/40 (73%) and week 24 17/25 (68%); or in the mean PSQI score at baseline (7.43, SD 4.06) and week 24 (7.48, SD 4.41). From baseline to week 24, 7/25 (28%) participants had a ≥ 3 point improvement in sleep quality on PSQI, and 5/25 (20%) had a ≥ 3 point deterioration. There was no significant difference in mean PROM scores.

It is feasible to deliver telehealth CBT-I to people with early BC receiving chemotherapy. Contrary to literature predictions, sleep quality did not deteriorate. Telehealth CBT-I has a potential role in preventing and managing sleep disturbance during chemotherapy.

Australian New Zealand Clinical Trials Registry (ANZCTR) registration number: ACTRN12620001379909 and date 22/12/2020.

Similar content being viewed by others

Light-enhanced cognitive behavioural therapy for sleep and fatigue: study protocol for a randomised controlled trial during chemotherapy for breast cancer

AYA ‘Can-Sleep’ programme: protocol for a stepped-care, cognitive behavioural therapy-based approach to the management of sleep difficulties in adolescents and young adults with cancer

A randomized phase II trial evaluating two non-pharmacologic interventions in cancer survivors for the treatment of sleep-wake disturbances: NCCTG N07C4 (Alliance)

Avoid common mistakes on your manuscript.

Introduction

Breast cancer (BC) is the most common cancer in women [ 1 ] and up to 70% of patients with BC report sleep disturbance [ 2 ]. Sleep disturbance includes difficulty initiating or maintaining sleep, or circadian rhythm sleep–wake disorders, which can negatively affect an individual’s health and quality of life [ 3 ]. While sleep disturbance is common in the general population, it can be triggered or exacerbated by a cancer diagnosis and its treatment [ 4 , 5 ]. The causes of sleep disturbance in cancer populations include: changes in inflammatory cytokines and disruption of circadian rhythm; treatment-induced menopausal symptoms; hospitalisation; anxiety and distress in response to cancer; symptoms such as pain; and medications used to manage treatment side effects such as corticosteroids [ 5 , 6 ].

People with BC receiving chemotherapy have been found to experience poorer sleep, more fatigue and more depressive symptoms during treatment compared to before starting chemotherapy [ 7 ]. Transient disruption of the sleep–wake cycle progressively worsens with repeat chemotherapy cycles [ 8 ]. Impacts of sleep disturbance include: daytime fatigue, reduced cognitive performance, mood swings, and psychological distress [ 9 ]. Sleep disturbance is a significant public health problem, resulting in impaired daytime functioning, reduced productivity, diminished health-related quality of life (HRQOL) and increased use of health services [ 5 ].

Despite the negative impact of chemotherapy on sleep, few trials have assessed sleep interventions during chemotherapy. Cognitive behaviour therapy for insomnia (CBT-I) is generally regarded as a gold standard treatment for sleep disturbance, with effects lasting beyond treatment completion [ 10 ]. CBT-I addresses the cognitive elements of thoughts, beliefs, and emotional reactions that exacerbate poor sleep patterns, along with the behavioural components of Sleep Restriction Therapy and Stimulus Control Therapy, to improve sleep outcomes. Sleep Restriction Therapy aims to improve sleep efficiency by limiting the time spent in bed. Stimulus Control Therapy aims to strengthen the bed as a cue for sleep and weaken it as a cue for wakefulness [ 10 ].

A systematic review found CBT-I is associated with clinically and statistically significant improvements in subjective sleep outcomes in people with cancer, and may improve mood, fatigue, and quality of life [ 11 ]. In a randomised trial of CBT-I versus control in women who had completed primary treatment for BC and reported sleep disturbance, the CBT-I group reported subjective improvement of sleep, lower frequency of medicated nights, lower levels of depression and anxiety, and better quality of life compared to the control group, with benefits continued for up to 12 months of follow-up [ 12 ].

Although effective, accessing CBT-I is difficult for many people, due to cost, availability, and a shortage of trained providers. Telehealth is the provision of healthcare remotely using telecommunication technology, including telephone or videoconference. Telehealth CBT-I has been found to improve sleep, and reduce depression and anxiety symptoms in the general population [ 13 ], but there are few studies assessing telehealth delivery of CBT-I in the cancer population. One study demonstrated feasibility of telehealth CBT-I in BC survivors who had completed treatment [ 14 ], but no studies have assessed telehealth CBT-I in BC patients whilst receiving chemotherapy.

Telehealth is a promising model for remote delivery of CBT-I as it is potentially more cost effective and accessible than face-to-face [ 15 ]. The aim of this study was to determine the feasibility and acceptability of a telehealth CBT-I intervention to prevent and manage sleep disturbance in people with early BC receiving (neo)adjuvant chemotherapy.

Study design

This was a multi-centre, single arm, phase 2 feasibility trial of telehealth CBT-I in people with early BC receiving (neo)adjuvant chemotherapy. The study was approved by the Sydney Local Health District – Concord Zone Health Research Ethics Committee for all sites (CH62/6/2020–018), with governance approved at all participating sites.

Participants

The study population included people aged ≥ 18 years with stage one to three BC, scheduled to receive adjuvant or neo-adjuvant chemotherapy, and willing and able to complete study questionnaires in English. Participants did not need to report sleep disturbance to be eligible. The study excluded people: with an existing diagnosis of a sleep disorder (e.g., obstructive sleep apnoea, narcolepsy, restless legs syndrome or REM sleep disorder); currently using a continuous positive airway pressure (CPAP) machine; currently receiving CBT for any indication; and regularly working more than one overnight shift per fortnight.

Participants were recruited from outpatient cancer clinics at four hospitals in Sydney, Australia over a 12-month period.

Participants completed Patient Reported Outcome Measures (PROMs) (described below) prior to commencing the telehealth CBT-I program (baseline), after completing the program (week 9), and after completing chemotherapy (week 24). Participants also completed an Acceptability Questionnaire at week 9.

Participants completed a sleep diary recording their daily sleep pattern (bed and awake times), use of steroids, use of rescue medications for sleep, and use of alcohol. Participants completed the diary daily for seven days at each of the assessment points (baseline, week 9 and week 24). The questionnaires and diary could be completed on paper or online.

Intervention

The telehealth CBT-I program consisted of four 45–75 min sessions delivered every two weeks over eight weeks. Participants commenced the first session before their second chemotherapy cycle. The CBT-I program incorporated standard components of sleep education, sleep hygiene, stimulus control, sleep restriction, development and adjustment of the sleep schedule, cognitive restructuring, and relaxation training (Table  1 ) [ 10 ].

Participants had the option to participate in the CBT-I sessions by one or a combination of telephone or videoconference (Zoom Video Communication). CBT-I was provided by provisionally registered psychologists who were students in the University of Sydney Master of Clinical Psychology programme under the supervision of a senior psychologist. The psychologists had received prior training in CBT delivery, and for the trial received further in-person training on CBT-I and sleep disturbance in the breast cancer population supported by a treatment manual and individual session information guides.

Acceptability of the intervention was measured using an Acceptability Questionnaire adapted from the 16-item Internet Intervention Utility Questionnaire which has good internal reliability [ 16 ]. Participants answered questions on whether they found the intervention helpful, easy to use and if it improved their understanding of sleep disturbance. Responses are rated on 5-point Likert scales from 1 (not at all) to 5 (very) [ 17 ].

Sleep quality was measured using the Pittsburgh Sleep Quality Index (PSQI), a well-validated 19-item questionnaire, assessing sleep quality over the past month, with a test–retest reliability of 0.85 for the global score [ 18 ]. A higher score indicates worse sleep and a global score of 5 or greater indicates a ‘‘poor’’ sleeper with a diagnostic sensitivity of 99% and specificity of 84% as a marker for sleep disturbances in insomnia patients compared with controls [ 19 ]. A change of three points or more has been recommended as the minimal clinically important difference for the PSQI; therefore an improvement in sleep quality in an individual is defined as a ≥ 3 point decrease in their PSQI total score from baseline to 9 weeks, and deterioration in sleep quality as a ≥ 3 point increase [ 20 ].

HRQOL was measured using the Functional Assessment of Cancer Therapy-Breast (FACT-B), a validated, widely used breast cancer specific quality of life questionnaire. It incorporates the FACT-General (FACT-G) questionnaire which includes 37 items assessing physical, social/family, emotional and functional well-being, combined with items assessing breast cancer symptoms, body image, bothered by hair loss, worry about effect of stress on illness, worry about members of family getting breast cancer and bothered by weight change [ 21 ]. The Cronbach's alpha for the FACT-B total score is 0.90, with subscales ranging from 0.63 to 0.86 [ 21 ].

Fatigue was measured using the FACT-Fatigue subscale (FACT-F), a 13-item validated measure of fatigue [ 22 ]. The FACT-F subscale has strong internal consistency with a Cronbach alpha range of 0.93–0.95 [ 22 ]. Symptoms of anxiety and depression were measured using the Hospital Anxiety and Depression Scale (HADS); a validated and widely used 14 item scale to determine the presence and severity of anxiety and depression in both hospital and community settings [ 23 ]. The HADS has a Cronbach's alpha of 0.83 for anxiety and 0.82 for depression [ 24 ]. The Distress Thermometer is a validated rating scale (0–10) to determine level of distress in cancer patients, with a Cronbach's alpha of 0.80. [ 25 , 26 ].

Participants completed a study specific diary to record their daily sleep pattern (bedtime and awake time), use of rescue medications (prescription and over the counter) for sleep, and use of alcohol for sleep.

Demographic characteristics collected included age, country of birth, Eastern Cooperative Oncology Group (ECOG) performance status and menopausal status. Clinical characteristics included cancer receptor status, stage, and chemotherapy regimen received.

Statistical analysis

The primary objective was to determine the feasibility of delivering telehealth CBT-I during (neo)adjuvant chemotherapy. We defined the intervention as feasible and worthy of further study if at least 70% of participants completed all four sessions of CBT-I. Powering the study to reliably show “success” (80% power) if the true rate was 70%, was chosen by the investigators reflecting both (a) on prior studies reporting CBT-I completion rates in people with breast cancer ranging from 59–84% [ 27 , 28 , 29 ] and (b) the minimum completion the investigator team felt would be need for the intervention to be viable. Secondary objectives included: acceptability (proportion of women finding the intervention useful and easy to use); and activity (impact of CBT-I on: proportion of ‘poor sleepers’ [PSQI ≥ 5]; sleep quality; and use of rescue medications for sleep).

Descriptive statistics, such as means, standard deviation (SD) or count/proportions, were used to describe participant characteristics. Outcomes were compared between baseline and week 9 (telehealth CBT-I program completion) and week 24 (chemotherapy completion) using standard statistical methods, such as t-tests or appropriate non-parametric tests for continuous outcomes, and chi-square tests for categorical outcomes. Effect sizes and 95% confidence intervals were calculated where appropriate. All p-values computed are reported and not adjusted for multiple comparisons. Statistical analysis was conducted using SPSS version 28.

The sample size calculation was based on a Simon 2 stage Phase 2 design [ 30 ]. A sample of size of 41 participants provided > 80% power to reject the null hypothesis that the rate of completing all four CBT-I sessions was less than 50% (uninteresting rate). This design had a type I error rate of 5% and power of 80% if the true response rate was 70% (interesting rate). We allowed for up to 10% attrition in the design.

Baseline characteristics

Between July 2021 and July 2022, 52 participants were screened, and 41 participants were recruited with mean age of 51 years (SD 9.0, range 31–73). Of the 52 who were screened, one was not eligible (diagnosis of obstructive sleep apnoea) and one not interested. Overall, 50 participants were enrolled, nine decided not to proceed with the most common reasons being busy schedule, being overwhelmed or no longer interested.

Most (68%) had hormone receptor positive breast cancer, 46% had stage two disease, chemotherapy was adjuvant for 56% (Table  2 ). Most of the CBT-I sessions were delivered via telephone (58%), with the remainder by videoconference; 5% commenced before cycle one chemotherapy and 95% before cycle two. Most (73%) met criteria for poor sleep at baseline (PSQI score ≥ 5).

Primary outcome

Feasibility.

All four telehealth CBT-I sessions were completed by 35/41 (85%, 95% CI 72—93) participants during chemotherapy. This rejected the null hypothesis that the feasibility rate was less than 50% ( p  < 0.01).

Assessment completion rates decreased over time, with 98% and 92% completing the questionnaire and sleep diary respectively at baseline, 76% and 76% respectively at week 9, and 61% and 56% respectively at week 24.’

Secondary outcomes

Acceptability.

Of 31 participants completing the post-program questionnaire, 100% reported ‘the program was easy to understand’, 89% ‘convenient to use’, 83% ‘would recommend the program to others’, 74% ‘the program was useful’, and 66% believed ‘the program was generally effective’. In total, 70% reported ‘the program as a suitable treatment for getting to sleep’ and 60% ‘the program was a suitable treatment for staying asleep’ (Fig.  1 ).

Acceptability of receiving telehealth CBT-I during chemotherapy. Acceptability questionnaire assessed at week 9, with each item scored out of 5, percentage of patients in each category demonstrated in bar graph, n  = 35

There was no significant difference in the proportion of poor sleepers (PSQI score ≥ 5) at baseline (29/40; 73%) and week 9 (21/31; 68%), ( p  = 0.7) (Table  3 ). From baseline to week 9, 7/31 (23%) participants had an improvement in sleep quality (≥ 3 point decrease in their PSQI score) and 9/31 (29%) had a deterioration in sleep quality (≥ 3 point increase in PSQI scores) (Fig.  2 ).

Change in sleep quality (PSQI) from baseline to week 9 and 24. Change in sleep quality assessed with the Pittsburgh Sleep Quality Index (PSQI), bar graph demonstrating the percentage of participants with a PSQI score ≥ 3 point increase (deterioration in sleep quality), ≥ 3 point decrease (improvement in sleep quality) or < 3 point increase or decrease (stable sleep quality) compared to baseline (week 0) at week 9 and week 24

The use of recognised medications for sleep (melatonin, doxylamine, and temazepam) was low throughout the study and there was no significant difference in the use of rescue medication between baseline and week 9 or 24 (Table  4 ).

Sleep disturbance at week 24

There was no significant difference in the number of poor sleepers (PSQI ≥ 5) at baseline 29/40 (73%) and week 24 17/25 (68%), ( p  = 0.7); or in the mean PSQI score at baseline, 7.43 (SD 4.01) and week 24, 7.48 (SD 4.31) ( p  = 1.0) (Table  3 , Fig.  2 ).

Health related quality of life

There was no significant difference in the HRQOL total or domain scores at baseline, week 9 (total score, p  = 0.9), and week 24, (total score, p  = 0.3) (Table  3 ).

Use of alcohol for sleep

The reported use of alcohol was low throughout the study and there was no significant difference in the use of alcohol for sleep between baseline and week 9 or 24. The proportion of participants using alcohol for sleep was 4/38 (11%) at baseline, 1/31 (3%) at week 9 ( p  = 0.2) and 3/23 (13%) at week 24 ( p  = 0.8) (Table  4 ).

Sleep times

The mean sleep duration at baseline was 7:52 hh:mm (range 5:15–10:42), 8:15 (5:54–9:30) at week 9 and 8:06 (6:12–9:32) at week 24.

Corticosteroids

At baseline 71% of participants self-reported using corticosteroids at least once during the 7 day study diary period, with a mean number of corticosteroids days 1.9 (range 0–6), this decreased to 55% and 1.0 (0–5) respectively at week 9, and 17% and 0.6 (0–6) respectively at week 24.

Our study demonstrated it is feasible and acceptable to deliver telehealth CBT-I to people with early BC receiving chemotherapy, with 85% of participants completing all four sessions. Most participants found the program easy to understand, easy and convenient to use, and would recommend it to others. One quarter of our participants had improvement in their sleep, defined as a PSQI score decrease of ≥ 3.

Several studies have demonstrated receipt of chemotherapy is associated with progressively worse sleep in women with early BC with enduring impairments in sleep-wake activity rhythms measured by actigraphy, as well as significantly poorer (increased) PSQI scores from pre to post chemotherapy [ 8 , 31 , 32 , 33 ]. Based on these studies, and participants not needing to have sleep disturbance at baseline, we expected to see a deterioration in sleep quality in our study. Instead, we found sleep quality did not deteriorate during chemotherapy with few participants (29%) experiencing a decline in sleep quality, and no significant difference in mean PSQI scores or the proportion with a PSQI score of ≥5 (i.e., poor sleep). We did not expect or see a significant improvement in sleep quality.

A meta-analysis from 2016 demonstrated that cancer survivors treated with CBT-I showed improvements in sleep, which were sustained up to 6 months [ 34 ]. Most of the included studies were in BC survivors, after completion of primary treatment. One study determined it was feasible to deliver telehealth CBT-I to BC survivors with sleep disturbance but this study did not include participants currently receiving chemotherapy [ 14 ]. Other studies have determined feasibility and efficacy of video or internet-based self-help CBT-I in BC survivors who reported sleep disturbance [ 27 , 28 , 35 , 36 , 37 ], but the interventions were not delivered during chemotherapy. Delivering telehealth CBT-I to patients during chemotherapy may help them develop good sleep behaviours and may prevent sleep disturbance development.

A small number of studies have assessed behavioural interventions for sleep during chemotherapy in people with BC [ 29 , 38 , 39 , 40 , 41 ]. A variety of face-to-face and self-help behavioural interventions were assessed, including individualised sleep plans, brief behavioural therapy, and CBT-I combined with light therapy. Similar to our study, most aimed to prevent and manage sleep disturbance, so participants were not required to have sleep disturbance prior to participation. Subjective sleep assessments either improved or remained stable. While some of these studies have evaluated CBT-I during chemotherapy, to our knowledge, no trial has determined feasibility or activity of telehealth CBT-I in people with early BC whilst receiving chemotherapy.

In our study, we reported the mean PSQI scores, the proportion of poor sleepers (PSQI score of ≥ 5), as well as the proportion with a PSQI score increase or decrease of ≥ 3 (indicating sleep deterioration or improvement, respectively). Other studies assessing sleep in people with BC during chemotherapy, commonly only report mean scores. Some observational studies have reported sleep outcomes using a predefined threshold to define poor sleep (i.e., PSQI) or clinically significant insomnia (i.e., Insomnia Severity Index) [ 32 ]. No studies reported the proportion with pre-defined score changes, to indicate sleep deterioration or improvement. We believe reporting these proportions when assessing the impact of cancer treatment or interventions, is more clinically meaningful and relevant to individual patients than mean scores.

In our study, there was no significant difference in HRQOL scores, including the FACT-B global score or for each domain. There also was no significant difference in fatigue, anxiety, depression, and distress. This contrasts with other studies reporting HRQOL, fatigue, anxiety and depression all significantly deteriorate in women with BC during chemotherapy, returning to baseline after completion [ 7 , 31 , 33 , 42 , 43 ]. The idea CBT-I may prevent deterioration in HRQOL during chemotherapy is worthy of further study. This is supported by trials that have assessed behavioural interventions for sleep in women with BC during chemotherapy and have shown fatigue either improved or remained stable, with no change in depression or anxiety scores [ 29 , 38 , 39 , 40 ].

Overall, we found the use of both rescue sleep medication and alcohol for sleep were low and did not significantly change throughout the study. This is surprising given the majority (73%) had poor sleep (PSQI ≥ 5) at baseline. This may be due to patient concerns of interactions with chemotherapy. These rates are similar to another study in women with BC receiving adjuvant chemotherapy, which demonstrated 14–23% used at least one sleep medication [ 44 ]. They also reported 4% used alcohol, again similar to our population.

Strengths/limitations

This study is characterised by several strengths, including use of a longitudinal design with assessment of sleep and HRQOL before, during and after chemotherapy, recruitment from multiple sites, and use of the PSQI, a well validated sleep assessment. Our study addresses a common problem, and measured feasibility and acceptability of the intervention. We assessed a non-pharmacological intervention that is simple, convenient, and safe to deliver during chemotherapy without concern of drug interactions or side effects from psychotropic medications. This intervention permits access to a psychologist who does not have to be on site, particularly beneficial to cancer centres with a limited psychology service. The telehealth format is convenient, saving patients time travelling to and from appointments. Telehealth CBT-I may prevent sleep disturbance in this population before it becomes a chronic problem. It has a potential role in preventing and managing sleep disturbance not only in those receiving chemotherapy, but during survivorship and beyond, and further studies are warranted.

Our study has a number of limitations. Firstly, the CBT-I program was condensed into four sessions to aid delivery during chemotherapy. Standard CBT-I generally involves six to eight sessions and we do not know if the shorter duration compromised the intervention activity. Second, we do not have a pre-cycle one chemotherapy baseline assessment on all participants, with 30% of participants completing the baseline assessment pre-cycle one, and 70% pre-cycle two. Given the start of chemotherapy is often an overwhelming time, allowing the baseline assessment to be completed prior to cycle two reduced the burden on participants and improved recruitment. A potential selection bias was that participants with a pre-existing history of sleep disturbance may have been more likely to participate. Similarly, participants more interested in CBT or behavioural interventions may have been more likely to engage with the study. We did not include an objective sleep assessment such as actigraphy. Sleep disturbance may affect cognition, and we did not include a cognition assessment. Our assessment completion rate declined during the study with 61% completing the final week 24 assessment. This was despite participants receiving reminder emails and phone-call from the research team. CBT-I treatment fidelity was not monitored but could be included in a future trial to ensure treatment was delivered in accordance with the treatment manual.

This study provides proof of concept that it is feasible and acceptable to deliver telehealth CBT-I to people with early BC receiving chemotherapy. While the study was not designed to evaluate the efficacy of CBT-I, it is encouraging that we did not see a significant deterioration in sleep quality from baseline to post-chemotherapy. This suggests CBT-I has potential to prevent and manage sleep disturbance in this population. Prevention of sleep disturbance avoids the negative impacts such as fatigue and reduced HRQOL, and the potential need for pharmacological measures. Pharmacological measures provide immediate effects; however, CBT-I produces superior long-term outcomes [ 45 ]. Most participants (73%) were poor sleepers at baseline, demonstrating the need for effective, practical, non-pharmacological treatments to manage this common problem.

Future research is needed to assess this intervention in a larger phase 3 randomised study of telehealth CBT-I versus control of standard practice (i.e. sleep hygiene) or waitlist control. In addition to subjective assessments, it would be important to assess sleep objectively using actigraphy, as well as include longer follow-up to assess for long lasting benefits. Trials are also needed to assess this intervention in people with other cancer types treated with chemotherapy.

A stepped care approach comprising a hierarchy of interventions may be the ideal strategy to assess in a future trial, e.g., starting with a self-help intervention and adding expert CBT-I delivered face-to-face or telehealth for those with poorer sleep. This approach may be more cost-effective, and a more appropriate use of scarce resources, addressing the high demand and short supply of CBT-I [ 46 ]. This could provide therapy at lower cost, be more convenient, require the lowest treatment intensity and least specialist time. Alternatively, a trial with a matched approach, tailoring the intervention to the precipitating or perpetuating cause of sleep disturbance as well as the severity, using a single or combination of interventions may be more effective but more resource intensive.

Our results show it is feasible to deliver telehealth CBT-I to women with early BC receiving chemotherapy. Contrary to expectations, mean sleep quality did not deteriorate, with one quarter of participants having improvement in their sleep. Telehealth CBT-I may be a potential intervention for preventing and managing sleep disturbance and maintaining HRQOL during chemotherapy in this population.

Data availability

Data and material are available.

Savard J et al (2001) Prevalence, clinical characteristics, and risk factors for insomnia in the context of breast cancer. Sleep 24(5):583–590

Article   CAS   PubMed   Google Scholar  

Savard J et al (2009) Prevalence, natural course, and risk factors of insomnia comorbid with cancer over a 2-month period. J Clin Oncol 27(31):5233–5239

Article   PubMed   Google Scholar  

RE C (1990) Sleep disturbances. clinical methods: the history, physical, and laboratory examinations. 3rd edition., ed. H.W. Walker HK, Hurst JW, eds. Boston, Butterworths

Ohayon MM (2011) Epidemiological Overview of sleep Disorders in the General Population. Sleep Med Res 2(1):1–9

Article   Google Scholar  

Savard J, Morin CM (2001) Insomnia in the context of cancer: a review of a neglected problem. J Clin Oncol 19(3):895–908

Dahiya S, Ahluwalia MS, Walia HK (2013) Sleep disturbances in cancer patients: underrecognized and undertreated. Clevel Clin J Med 80(11):722–732

Liu L et al (2009) Pre-treatment symptom cluster in breast cancer patients is associated with worse sleep, fatigue and depression during chemotherapy. Psychooncology 18(2):187–194

Article   PubMed   PubMed Central   Google Scholar  

Savard J et al (2009) Breast cancer patients have progressively impaired sleep-wake activity rhythms during chemotherapy. Sleep 32(9):1155–1160

Davidson JR et al (2002) Sleep disturbance in cancer patients. Soc Sci Med 54(9):1309–1321

Mitchell MD et al (2012) Comparative effectiveness of cognitive behavioral therapy for insomnia: a systematic review. BMC Fam Pract 13(1):40

Garland SN et al (2014) Sleeping well with cancer: a systematic review of cognitive behavioral therapy for insomnia in cancer patients. Neuropsychiatr Dis Treat 10:1113–1124

PubMed   PubMed Central   Google Scholar  

Savard J et al (2005) Randomized study on the efficacy of cognitive-behavioral therapy for insomnia secondary to breast cancer, part I: Sleep and psychological effects. J Clin Oncol 23(25):6083–6096

Bastien CH et al (2004) Cognitive-behavioral therapy for insomnia: comparison of individual therapy, group therapy, and telephone consultations. J Consult Clin Psychol 72(4):653–659

McCarthy MS et al (2018) Feasibility of a telemedicine-delivered cognitive behavioral therapy for insomnia in rural breast cancer survivors. Oncol Nurs Forum 45(5):607–618

Brenes GA, Ingram CW, Danhauer SC (2011) Benefits and challenges of conducting psychotherapy by telephone. Prof Psychol Res Pr 42(6):543–549

Thorndike FP et al (2008) Development and perceived utility and impact of an internet intervention for insomnia. E J Appl Psychol 4(2):32–42

Ritterband LM et al (2008) Real world use of an Internet intervention for pediatric encopresis. J Med Internet Res 10(2):e16

Buysse DJ et al (1989) The pittsburgh sleep quality index: a new instrument for psychiatric practice and research. Psychiatry Res 28(2):193–213

Backhaus J et al (2002) Test-retest reliability and validity of the Pittsburgh Sleep Quality Index in primary insomnia. J Psychosom Res 53(3):737–740

Hughes CM et al (2009) Acupuncture and reflexology for insomnia: a feasibility study. Acupunct Med 27(4):163–168

Brady MJ et al (1997) Reliability and validity of the functional assessment of cancer therapy-breast quality-of-life instrument. J Clin Oncol 15(3):974–986

Yellen SB et al (1997) Measuring fatigue and other anemia-related symptoms with the Functional Assessment of Cancer Therapy (FACT) measurement system. J Pain Symptom Manage 13(2):63–74

Snaith RP (2003) The hospital anxiety and depression scale. Health Qual Life Outcomes 1(1):29

Bjelland I et al (2002) The validity of the hospital anxiety and depression scale. An updated literature review. J Psychosom Res 52(2):69–77

Ownby KK (2019) Use of the distress thermometer in clinical practice. J Adv Pract Oncol 10(2):175–179

Tang LL et al (2011) Validation and reliability of distress thermometer in chinese cancer patients. Chin J Cancer Res 23(1):54–58

Article   CAS   PubMed   PubMed Central   Google Scholar  

Dozeman E et al (2017) Guided web-based intervention for insomnia targeting breast cancer patients: Feasibility and effect. Internet Interv 9:1–6

Zachariae R et al (2018) Internet-delivered cognitive-behavioral therapy for insomnia in breast cancer survivors: a randomized controlled trial. J Natl Cancer Inst 110(8):880–887

Bean HR et al (2022) Light enhanced cognitive behavioral therapy for insomnia and fatigue during chemotherapy for breast cancer: a randomized controlled trial. Sleep 45(3):zsab246. https://doi.org/10.1093/sleep/zsab246

Simon R (1989) Optimal two-stage designs for phase II clinical trials. Control Clin Trials 10(1):1–10

Ancoli-Israel S et al (2014) Sleep, fatigue, depression, and circadian activity rhythms in women with breast cancer before and after treatment: a 1-year longitudinal study. Support Care Cancer 22(9):2535–2545

Fakih R et al (2018) Prevalence and severity of sleep disturbances among patients with early breast cancer. Indian J Palliat Care 24(1):35–38

Liu L et al (2013) Decreased health-related quality of life in women with breast cancer is associated with poor sleep. Behav Sleep Med 11(3):189–206

Johnson JA et al (2016) A systematic review and meta-analysis of randomized controlled trials of cognitive behavior therapy for insomnia (CBT-I) in cancer survivors. Sleep Med Rev 27:20–28

Ritterband LM et al (2012) Initial evaluation of an Internet intervention to improve the sleep of cancer survivors with insomnia. Psychooncology 21(7):695–705

Savard J et al (2011) Feasibility of a self-help treatment for insomnia comorbid with cancer. Psychooncology 20(9):1013–1019

Savard J et al (2014) Is a video-based cognitive behavioral therapy for insomnia as efficacious as a professionally administered treatment in breast cancer? Results of a randomized controlled trial. Sleep 37(8):1305–1314

Berger AM et al (2002) Feasibilty of a sleep intervention during adjuvant breast cancer chemotherapy. Oncol Nurs Forum 29(10):1431–1441

Berger AM et al (2009) Behavioral therapy intervention trial to improve sleep quality and cancer-related fatigue. Psychooncology 18(6):634–646

Palesh O et al (2018) Feasibility and acceptability of brief behavioral therapy for cancer-related insomnia: effects on insomnia and circadian rhythm during chemotherapy: a phase II randomised multicentre controlled trial. Br J Cancer 119(3):274–281

Marion LP, Ivers H, Savard J (2021) Feasibility of a preventive intervention for insomnia in women with breast cancer receiving chemotherapy. Behav Sleep Med 19(1):70–82

Sanford SD et al (2013) Longitudinal prospective assessment of sleep quality: Before, during, and after adjuvant chemotherapy for breast cancer. Support Care Cancer 21(4):959–967

Liu L et al (2012) Fatigue and sleep quality are associated with changes in inflammatory markers in breast cancer patients undergoing chemotherapy. Brain Behav Immun 26(5):706–713

Moore TA, Berger AM, Dizona P (2011) Sleep aid use during and following breast cancer adjuvant chemotherapy. Psychooncology 20(3):321–325

van der Zweerde T et al (2019) Cognitive behavioral therapy for insomnia: A meta-analysis of long-term effects in controlled studies. Sleep Med Rev 48:101208

Espie CA (2009) “Stepped Care”: A Health Technology Solution for Delivering Cognitive Behavioral Therapy as a First Line Insomnia Treatment. Sleep 32(12):1549–1558

Download references

Acknowledgements

This study was sponsored by The University of Sydney, Sydney Australia. We would like to thank the women who generously participated, the psychologists who provided the CBT-I and the staff at each site who contributed to this trial. We also thank Dr Shayla Gooley, Clinical Psychology Unit, The University of Sydney, who provided supervision for the study psychologists.

Open Access funding enabled and organized by CAUL and its Member Institutions This study was supported by a grant from Concord Cancer Centre, Concord Repatriation General Hospital, Sydney, New South Wales, Australia. The funder had no role in the study design, data collection, analysis, interpretation, or presentation of results. The authors declare that no funds, grants, or other support were received during the preparation of this manuscript.

Author information

Authors and affiliations.

Faculty of Medicine and Health, University of Sydney, Camperdown, NSW, Australia

Emma-Kate Carson, Janette L. Vardy, Chris Brown, Felicia Roncoloato, Antonia Pearson & Belinda E. Kiely

School of Medicine, Western Sydney University, Campbelltown, NSW, Australia

Emma-Kate Carson, Stephen Della-Fiorentina, Felicia Roncoloato & Belinda E. Kiely

Macarthur Cancer Therapy Centre, Campbelltown Hospital, Campbelltown, NSW, Australia

Emma-Kate Carson, Andrew Parsonson, Felicia Roncoloato & Belinda E. Kiely

Centre for Medical Psychology & Evidence-Based Decision-Making, University of Sydney, Sydney, NSW, Australia

Haryana M. Dhillon & Kelly Ferrao Nunes-Zlotkowski

Faculty of Science, School of Psychology, Psycho-Oncology Cooperative Research Group, University of Sydney, Sydney, NSW, Australia

Haryana M. Dhillon, Janette L. Vardy & Kelly Ferrao Nunes-Zlotkowski

Concord Cancer Centre, Concord Repatriation General Hospital, Concord, NSW, Australia

Janette L. Vardy & Belinda E. Kiely

NHMRC Clinical Trials Centre, University of Sydney, Camperdown, NSW, Australia

Chris Brown & Belinda E. Kiely

Southern Highlands Cancer Centre, Southern Highlands Private Hospital, Bowral, NSW, Australia

Stephen Della-Fiorentina & Sarah Khan

Cancer Services, South Western Sydney Local Health District, Liverpool, NSW, Australia

Stephen Della-Fiorentina

Faculty of Medicine, Health and Human Sciences, Macquarie University, Macquarie Park, NSW, Australia

Andrew Parsonson & Tristan Barnes

Northern Beaches Hospital, Frenchs Forest, NSW, Australia

Antonia Pearson & Tristan Barnes

You can also search for this author in PubMed   Google Scholar

Contributions

EC: study concept, design, data analysis, and writing of the manuscript. JV, HD, and BK: study concept and design. CB: data analysis. All authors reviewed and approved the final manuscript.

Corresponding author

Correspondence to Emma-Kate Carson .

Ethics declarations

Ethics approval.

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards (Institutional Human Research Ethics Committee protocol number HREC/14/CRGH/23). The study was approved by the Sydney Local Health District – Concord Zone Health Research Ethics Committee for all sites (CH62/6/2020–018), with governance approved at all participating sites.

Consent to participate

Informed consent was obtained from all individual participants included in the study.

Consent to publish

Informed consent to have de-identified data published was obtained from all individual participants included in the study.

Competing interests

Author EC has received speaker honoraria from Eli Lilly. Author BK has received speaker honoraria from Novartis, Eisai, MSD and Gilead, travel support from Novartis, Pfizer and MSD, and honoraria for serving on advisory boards for Novartis and Gilead. Author TB has received travel support from GSK and Novartis. Author HD, JV, CB, KNZ, SDF, SK, AP, FR, and AP declare they have no financial interests. The authors have no relevant non-financial interests to disclose.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 21 KB)

Rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Carson, EK., Dhillon, H.M., Vardy, J.L. et al. Telehealth cognitive behaviour therapy for the management of sleep disturbance in women with early breast cancer receiving chemotherapy: a feasibility study. Support Care Cancer 32 , 375 (2024). https://doi.org/10.1007/s00520-024-08554-8

Download citation

Received : 15 January 2024

Accepted : 07 May 2024

Published : 23 May 2024

DOI : https://doi.org/10.1007/s00520-024-08554-8

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Sleep disturbance
• Breast cancer
• Chemotherapy
• Quality of life
• Find a journal
• Publish with us
• Track your research

Log in using your username and password

• Search More Search for this keyword Advanced search
• Latest content
• Current issue
• BMJ Journals More You are viewing from: Google Indexer

You are here

• Online First
• The RELIEF feasibility trial: topical lidocaine patches in older adults with rib fractures
• Article Text
• Article info
• Citation Tools
• Rapid Responses
• Article metrics

• Madeleine Clout 1 ,
• Nicholas Turner 1 ,
• Clare Clement 2 ,
• Philip Braude 3 ,
• http://orcid.org/0000-0001-6131-0916 Jonathan Benger 4 ,
• James Gagg 5 ,
• Emma Gendall 6 ,
• Simon Holloway 7 ,
• Jenny Ingram 8 ,
• Rebecca Kandiyali 9 ,
• Amanda Lewis 1 ,
• Nick A Maskell 10 ,
• David Shipway 11 ,
• http://orcid.org/0000-0002-6143-0421 Jason E Smith 12 ,
• Jodi Taylor 13 ,
• Alia Darweish Medniuk 14 ,
• http://orcid.org/0000-0002-2064-4618 Edward Carlton 15 , 16
• 1 Population Health Sciences , University of Bristol , Bristol , UK
• 2 University of the West of England , Bristol , UK
• 3 CLARITY (Collaborative Ageing Research) , North Bristol NHS Trust , Westbury on Trym , UK
• 4 Faculty of Health and Life Sciences , University of the West of England , Bristol , UK
• 5 Department of Emergency Medicine , Somerset NHS Foundation Trust , Taunton , UK
• 6 Research and Innovation , Southmead Hospital , Bristol , UK
• 7 Pharmacy Clinical Trials and Research , Southmead Hospital , Bristol , UK
• 8 Bristol Medical School , University of Bristol , Bristol , UK
• 9 Warwick Clinical Trials Unit , Warwick Medical School , Coventry , UK
• 10 Academic Respiratory Unit , University of Bristol , Bristol , UK
• 11 Department of Medicine for Older People, Southmead Hospital , North Bristol NHS Trust , Bristol , UK
• 12 Emergency Department , University Hospitals Plymouth NHS Trust , Plymouth , UK
• 13 Bristol Trials Centre, Population Health Sciences , University of Bristol , Bristol , UK
• 14 Department of Anaesthesia and Pain Medicine , Southmead Hospital , Bristol , UK
• 15 Emergency Department , Southmead Hospital , Bristol , UK
• 16 Department of Emergency Medicine, Translational Health Sciences , University of Bristol , Bristol , UK
• Correspondence to Dr Edward Carlton, Emergency Department, Medicine Translational Health Sciences, Southmead Hospital, Bristol, BS10 5NB, UK; eddcarlton{at}gmail.com

Background Lidocaine patches, applied over rib fractures, may reduce pulmonary complications in older patients. Known barriers to recruiting older patients in emergency settings necessitate a feasibility trial. We aimed to establish whether a definitive randomised controlled trial (RCT) evaluating lidocaine patches in older patients with rib fracture(s) was feasible.

Methods This was a multicentre, parallel-group, open-label, feasibility RCT in seven hospitals in England and Scotland. Patients aged ≥65 years, presenting to ED with traumatic rib fracture(s) requiring hospital admission were randomised to receive up to 3×700 mg lidocaine patches (Ralvo), first applied in ED and then once daily for 72 hours in addition to standard care, or standard care alone. Feasibility outcomes were recruitment, retention and adherence. Clinical end points (pulmonary complications, pain and frailty-specific outcomes) and patient questionnaires were collected to determine feasibility of data collection and inform health economic scoping. Interviews and focus groups with trial participants and clinicians/research staff explored the understanding and acceptability of trial processes.

Results Between October 23, 2021 and October 7, 2022, 206 patients were eligible, of whom 100 (median age 83 years; IQR 74–88) were randomised; 48 to lidocaine patches and 52 to standard care. Pulmonary complications at 30 days were determined in 86% of participants and 83% of expected 30-day questionnaires were returned. Pulmonary complications occurred in 48% of the lidocaine group and 59% in standard care. Pain and some frailty-specific outcomes were not feasible to collect. Staff reported challenges in patient compliance, unfamiliarity with research measures and overwhelming the patients with research procedures.

Conclusion Recruitment of older patients with rib fracture(s) in an emergency setting for the evaluation of lidocaine patches is feasible. Refinement of data collection, with a focus on the collection of pain, frailty-specific outcomes and intervention delivery are needed before progression to a definitive trial.

Trial registration number ISRCTN14813929 .

• feasibility studies
• frail elderly

Data availability statement

Data are available on reasonable request. Further information and patient-facing materials (including model consent forms) are available at https://relief.blogs.bristol.ac.uk/ . Data available on request.

This is an open access article distributed in accordance with the Creative Commons Attribution 4.0 Unported (CC BY 4.0) license, which permits others to copy, redistribute, remix, transform and build upon this work for any purpose, provided the original work is properly cited, a link to the licence is given, and indication of whether changes were made. See:  https://creativecommons.org/licenses/by/4.0/ .

https://doi.org/10.1136/emermed-2024-213905

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

WHAT IS ALREADY KNOWN ON THIS TOPIC

Studies have evaluated the use of lidocaine patches in patients with rib fractures showing reductions in opioid use, improvements in pain scores and reductions in length of hospital stay.

Importantly, none has focused on older patients, who stand to gain the most benefit from improved analgesic regimens to reduce adverse pulmonary complications.

WHAT THIS STUDY ADDS

In this feasibility trial, prespecified progression criteria around recruitment, follow-up and adherence were met, demonstrating it is feasible to conduct randomised controlled trials in older patients, who are in pain, in an emergency setting.

There were challenges in data collection for pain and frailty-specific measures, together with treatment crossover, that require consideration in definitive trial design.

HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY

Researchers can adapt study processes to be inclusive of older patients in the emergency setting.

There are challenges in terms of data collection around pain and frailty-specific outcome measures which future research should consider.

Introduction

Rib fractures represent the most common non-spinal fracture in older people. 1 Age ≥65 years remains a predictor of morbidity and mortality in patients with rib fractures. 2 Pain can compromise normal respiratory function, with over 15% of older patients experiencing complications including pneumonia and death. 3

The mainstay for treatment of rib fracture pain remains strong opioid analgesia. However, as a result of poor physiological reserve, older patients are more vulnerable than younger people to the side effects of strong opioid medication such as nausea, constipation, sedation, delirium and respiratory depression. 4 Invasive approaches, such as thoracic epidural anaesthesia, have been used to reduce the likelihood of these side effects, but require specialist anaesthetic support, monitoring in a high-dependency environment and are only used in around 20% of admitted patients. 5 6

Lidocaine patches applied over rib fractures have been suggested as a non-invasive method of local anaesthetic delivery to improve respiratory function, reduce opioid consumption and consequently reduce pulmonary complications. 7 Studies have evaluated the use of lidocaine patches in patients with rib fractures showing reductions in opioid use, 8 improvements in pain scores 9 10 and reductions in length of hospital stay. 11 However, these studies are limited by retrospective design and low patient numbers with consequent bias and low precision. Importantly, none has focused on older patients, who are more susceptible to the development of pulmonary complications, 2 or tested lidocaine patches as an intervention in the ED where opioid analgesia is the mainstay of treatment.

Older people have often been excluded from research, relating to multiple long-term health conditions, social and cultural barriers and potentially impaired capacity to provide informed consent. 12 In addition, recruitment of older patients who are in pain in an emergency setting may pose further challenges around information provision and collection of clinical and patient-reported outcomes.

The aim of this trial was to establish whether a definitive randomised controlled trial (RCT) to evaluate the benefit of lidocaine patches, first applied in the ED, for older people requiring admission to hospital with rib fracture(s) is feasible.

Detailed methods, including detailed consent procedures, are described in full elsewhere. 13

Design, setting and participants

The Randomised Evaluation of topical Lidocaine patches in Elderly patients admitted to hospital with rib Fractures (RELIEF) study was a multicentre, parallel-group, open-label, individually randomised, feasibility RCT, conducted in seven NHS hospitals: five major trauma centres (Southmead Hospital; Royal Infirmary of Edinburgh; Derriford Hospital, Plymouth; Queen Elizabeth University Hospital, Glasgow; St George’s Hospital, London) and two trauma units (Musgrove Park Hospital, Taunton; Royal Devon and Exeter Hospital). The trial included a health economic scoping analysis and an integrated qualitative study. Patients were eligible for recruitment if they were aged ≥65 years, presented at any time after injury with traumatic rib fracture(s) (including multiple fractures, flail chest and traumatic haemothorax/pneumothorax even if this required intercostal chest drainage), confirmed radiologically (by CXR or CT conducted as part of routine care) and required hospital admission for ongoing care. Exclusion criteria are detailed in figure 1 .

• Download figure
• Open in new tab
• Download powerpoint

Exclusion criteria.

Randomisation and blinding

Participants were randomised in the ED by trained research or clinical staff, using an online randomisation system, with the randomisation sequence generated by Sealed Envelope (London, UK). Participants were allocated to the intervention or standard care in a 1:1 ratio. Randomisation was stratified by trial site and gender and blocked within strata. Allocations were blinded only to those performing central review of data for the assessment of outcomes.

Intervention

Participants randomised to the intervention received up to 3×700 mg lidocaine patches (Ralvo) at a time applied over the most painful area of rib injury. Patches were first applied in the ED, then once daily for 12 hours in accordance with the manufacturer’s (Grünenthal, Aachen, Germany) instructions. Treatment continued for up to 72 hours or until discharge from hospital. The intervention was additive to standard care (below). If participants subsequently underwent regional anaesthesia, patches were removed and no further patches were applied but data collection continued according to group allocation.

Standard care

All participants received standard local analgesic treatment for patients with rib fractures; this was not controlled for trial purposes. Data were collected on paracetamol, weak opioid, strong opioid and other non-opioid analgesia prescriptions in ED and for the 72-hour intervention period in both arms of the trial. 14

Patient and public involvement

Patient and public involvement was ensured at all stages of trial design, and continued throughout the trial’s lifetime via a patient advisory group and patient representation on the trial steering committees.

Clinical outcomes and measurement

Outcomes were measured at baseline, 72 hours (during or on completion of intervention) and 30-day postrandomisation. A full schedule of clinical data, questionnaires and end points is included in the published protocol. 13 Clinical end points were collected only to understand the feasibility of data collection and not to conduct hypothesis testing. Key clinical data and their measurement are briefly summarised as follows (further details on scales used are provided in the online supplemental material ):

Supplemental material

Demographics, injury details, relevant medical history and Clinical Frailty Scale (CFS) 15 : collected by researcher from clinical notes.

Retrospective pre-injury and baseline post-injury health EQ-5D-5L 16 : completed with participant/relative/carer.

Timed Up and Go test. 17

72 hours postrandomisation (intervention period) collected until discharge if sooner

Patient-reported pain scores: 4-hourly pain assessment using a Visual Analogue Scale (VAS) (scaled from 0 to 100). Recorded in a booklet provided to the patient.

Frailty-specific outcomes: Abbey Pain Scale, 18 4-AT delirium assessment tool, 19 constipation (Bristol Stool Chart), Timed Up and Go test. 17 Obtained by researchers.

Analgesia; ED and inpatient (72 hours) analgesic prescriptions, advanced analgesic provision (patient controlled analgesia (PCA), epidural, nerve block). Obtained by researchers from medical records.

30 days (+10 days) postrandomisation

Pulmonary complications: a priori proposed primary outcome for a definitive trial. Collected after review of medical records and adjudicated by site lead clinician.

Delirium: binary measure of any inpatient episode of delirium recorded in clinical notes.

Resource use: including admitted hospital length of stay, intensive care unit length of stay, unplanned readmission, discharge destination (notes review).

Questionnaires: booklets containing EQ-5D-5L and ICECAP-O 16 20 were sent by post to participants. Participants were permitted to complete these with the assistance of carers, although formal proxy versions of questionnaires were not provided.

Sample size

As this was a feasibility trial, it was not appropriate to calculate a sample size to detect a specified treatment effect size. In line with published ‘rules-of-thumb’, we determined that a total sample size of 100 would be sufficient to provide estimates of feasibility measures (recruitment, retention, data completion and adherence). 21 Recruitment was originally planned to take place over 18 months across three sites. However, trial set-up was delayed due to the COVID-19 pandemic. To achieve target recruitment within the funding period, the recruitment period was shortened to 12 months across seven sites.

Statistical methods

Feasibility measures were analysed and reported following the Consolidated Standards of Reporting Trials guidance extension for feasibility studies to include descriptive and summary statistics both overall and by treatment arm. 22

Descriptive statistics for participant characteristics and clinical outcome data were reported as means or medians with measures of dispersion for continuous outcomes and frequencies and percentages for categorical outcomes.

A priori thresholds for recruitment, follow-up and adherence were established to inform the feasibility of progression (table 2).

Integrated qualitative study

Telephone interviews were undertaken with trial participants around 1 month (and up to 90 days) postrandomisation. Interviews and focus groups were conducted with clinicians/research staff closely involved in the trial set-up, recruitment and follow-up. These explored trial participation experiences including understanding and acceptability of processes, pain control including perceived benefits of lidocaine patches and views on trial outcomes (topic guides are included in the online supplemental material ). Interviews and focus groups were audio-recorded, transcribed and analysed using thematic analysis. 23 Qualitative findings were integrated with other elements using a ‘following a thread’ approach. 24 This involved analysing each dataset and then using insights from the qualitative themes to contextualise and explain quantitative outcomes with data presented together.

Health economic scoping

An evaluation of the feasibility of identifying and measuring health economics outcome data was completed, with the focus on establishing the most appropriate outcome measures for inclusion in a future economic evaluation alongside the definitive trial. The EQ-5D-5L (health-related quality of life) patient-reported questionnaire 16 was completed at baseline, to capture retrospective pre-injury state and baseline post-injury state, and 30 days postrandomisation. In addition to the standard EQ-5D questionnaire, which typically elicits post-injury health status, we additionally assessed pre-injury status by making an approved change to the wording. The ICECAP-O (measure of capability in older people) 20 was also collected at 30 days. Information on key resources, including length of stay, intensive care use and medication prescribing, was also collected.

Between 23 October 2021 and 7 October 2022, 447 patients were assessed for eligibility, of which 206 were eligible; of these, 29 declined and 77 were not approached. Therefore, 100 patients were randomised; 48 participants were allocated to lidocaine patches and 52 to standard care ( figure 2 ). Six participants died prior to the 30-day follow-up timepoint and three participants withdrew from questionnaire completion, but had clinical data retained for analysis. Baseline characteristics were well balanced between groups ( table 1 ).

Screening, recruitment, allocation and follow-up (Consolidated Standards of Reporting Trials diagram).

• View inline

Baseline demographics and injury characteristics

Participants were predominantly women (47%), of white British ethnicity (92%), with a median age of 83 years (IQR 74–88). Participants were predominantly admitted from their own homes (92%), were independent (75%) but were living with very mild frailty (median CFS 4; IQR 3–5). The most common mechanism of injury was a fall from <2 m (81%). On average, participants sustained four rib fractures (SD 2.0)and they were at high risk of developing pulmonary complications at baseline (median STUMBL score 21 (IQR 16–33)), equating to a 70% risk. 3

Feasibility outcomes

Table 2 details the prespecified progression criteria around recruitment, follow-up and adherence together with observed results.

Prespecified progression criteria and observed results

Recruitment and consent

An average of 14 participants were recruited per site (range 3–37) in 12 months. Participants were predominately recruited from major trauma centres (n=87).

Agreement to participate was largely obtained from patients (70%): personal consultees (in England) or legal representatives (in Scotland) were approached in 27% of cases, and professional consultees were used in 3% of cases.

In the qualitative research, clinical and research staff closely involved in delivering the trial reported challenges in recruiting within the ED setting. These challenges included general ED pressures, reliance on referrals from wider clinical teams not directly engaged in the research, resource-intensive monitoring of ED attendances for potentially eligible patients, the necessity to rapidly attend ED (when not based in the department) to approach patients and lack of out-of-hours research staff (although some engaged clinicians were able to recruit out of hours). However, they were able to recruit well by raising awareness of the trial and fostering good collaborative relationships with the wider ED clinical team members, who were able to actively participate in patient identification. Insights from older patients were limited due to challenges with interview engagement (of 26 participants approached for interviews, 7 took part, 5 declined, 14 did not respond). However, older patients interviewed welcomed being approached and were willing to participate in the trial because they wanted to help, but were sometimes unsure of trial details. Staff needed to consider older patients’ vulnerability, and carefully manage consent processes to avoid overwhelming them, while ensuring their full understanding of involvement and the option not to participate.

Follow-up and data completeness

The proposed primary outcome of adverse pulmonary complications at 30 days was completed for 86% of participants (data missing in 14%, due to transfer to remote facilities or discharge home and no further records were available). For the 30-day patient-completed questionnaires, in total 71 were returned (fully or partially completed), 15 were unreturned despite repeated contact and 14 had reasons recorded for non-return (7 deaths, 4 remained unwell/confused, 3 withdrawals). This equates to an overall return rate of 71% but rising to 83% when return was anticipated. Qualitative findings regarding questionnaire completion highlighted the unblinded nature of the intervention, with standard care participants not feeling part of the trial, potentially impacting their understanding of completing questionnaires in future research.

Pain and frailty-specific outcomes (important secondary outcomes but not included in prespecified progression criteria) were not feasible to collect as completeness was <65%. Table 3 summarises data completeness on these measures and qualitative exploration of factors influencing data collection.

Pain and frailty-specific outcomes that were not feasible to collect and qualitative exploration of factors influencing data collection

In the intervention arm, 44/48 (92%) participants had at least one lidocaine patch applied in ED at a median time of 393.5 min after arrival. In the standard care arm, 17/52 (33%) participants also had a lidocaine patch applied in ED and were therefore classed as non-adherent. However, overall adherence was 79% meeting the prespecified green criteria for feasibility (>75%). Themes identified in the qualitative research with clinical/research staff addressing variation in care included standard care (some hospitals use patches as standard care, others do not), patch application (eg, where best to place patches in the presence of multiple fractures), provision of nerve blockade (the ongoing use of lidocaine patches when nerve blocks are subsequently used), equipoise (mixed views on the benefits of patches) and patch acceptability (perceived benefits of patches to patients) (see online supplemental material for details).

Clinical outcomes

72-hour outcomes

Data on ED and inpatient (72 hours) analgesic prescriptions, together with advanced analgesic provision (PCA, epidural, nerve blocks) were collected in >75% of participants ( table 4 ) Analgesic prescriptions within ED and as an inpatient were similar between arms. Overall, 33/97 (34%) participants had advanced analgesia with 21/97 (22%) receiving some form of nerve blockade and 12/97 (13%) receiving PCA within the 72-hour intervention period.

30-day outcomes

Overall, 46/86 (53%) participants with complete data met the outcome of composite pulmonary complications within 30 days; 20 (48%) in the lidocaine patch arm and 26 (59%) in the standard care arm. The median length of hospital stay was 9.1 days (IQR 5.2–15.4) and over 30% of participants did not return to their baseline level of function on discharge (requiring increased package of care, residential, nursing or rehabilitation). Descriptive data on all 30-day outcomes is included in table 4 .

We achieved our objectives in terms of piloting instruments of data collection: administration of EQ-5D-5L and ICECAP-O measures and case report forms to record length of stay, use of analgesia and discharge destination ( table 4 ).

As anticipated EQ-VAS at baseline (measuring overall health status with 100 being best imaginable health) were reported as higher pre-injury (median 80 (60–90)) compared with post-injury (median 50 (25–70)). At 30 days, EQ-5D-5L completeness was 44% and ICECAP-O was 65%. In terms of the trajectory of health status, as anticipated the baseline EQ-5D-5L post-injury tariff had the lowest median (0.44 (0.25–0.63)) while at 30 days these data indicated participants had only partially recovered in terms of health status (0.59 (0.27–0.74)) ( table 4 ). The overall median ICECAP-O tariff at 30 days was 0.77, which is slightly below a published population norm of 0.81. 25

This trial suggests it is feasible to recruit older patients with rib fracture(s) in an emergency setting. Consent processes modified for older patients were effective and acceptable to patients and carers. However, pain and frailty-specific outcomes were not feasible to collect. While these were not anticipated primary outcomes for a future trial, they are clearly important secondary outcomes in this population. Our qualitative work highlighted areas for improvement in this regard. These include bespoke training for researchers when unfamiliar with measures (Abbey Pain Scale, 4-AT delirium assessment tool), embedding measures such as 4-AT delirium assessment tool into clinical practice and increased recognition of the potential to overwhelm older injured patients through research procedures when designing trials. It should be noted that the World Hip Trauma Evaluation platform study appears to have overcome many of these barriers to data collection in a similar population. 26

Data collection for the suggested primary outcome of a definitive trial (adverse pulmonary complications) was feasible, and the high rates of this outcome within the population confirm that it remains a target outcome for early analgesic interventions in older patients with rib fracture(s).

Paper-based, mailed out, patient-completed questionnaires were returned at high rates, suggesting that this remains an acceptable option for older participants in research. This aligns with consensus recommendations that alternatives should be offered to digital data collection to avoid digital exclusion in older patients. 12 However, for those patients with cognitive impairment, consideration of formal proxy versions of questionnaires should be considered where available.

While adherence to the intervention was high and overall adherence was deemed feasible, significant crossover in the standard care arm was seen. This finding suggests clinicians may lack equipoise in sites where lidocaine patches are already in use; this was confirmed in our healthcare professional focus groups. However, these focus groups also highlighted discrepancies in prescribing/availability and a recognition of the potential harm of overuse of lidocaine patches (at the expense of other analgesic modalities). In order to overcome these challenges in equipoise, avoid crossover and fully understand the clinical effectiveness of topical lidocaine, a definitive trial would need to test active patches against placebo patches rather than standard care.

In this trial, older patients admitted to hospital with radiologically confirmed rib fracture(s) were living with very mild frailty (median CFS 4) and were predominantly injured after a fall from standing (<2 m), a finding consistent with previous reports. 27 Despite having isolated rib fracture(s), many participants had prolonged hospital stays (median 9 days) and >30% did not return to baseline functional status on discharge. STUMBL scores recorded at baseline suggested a population at high risk of developing adverse pulmonary complications and this finding was confirmed in 30-day outcome collection. Development of delirium appeared lower than reported in other cohorts, 6 but may reflect a lack of robust data collection. Notable findings that may provide targets for service improvements include prolonged times between injury and hospital arrival (20 hours) and low rates of prehospital analgesia administration. In addition, in-hospital (72 hours) analgesic prescriptions appear to rely heavily on strong opioid analgesia, with more advanced analgesic modalities being used in only around one-fifth of this vulnerable patient group.

Rib fracture(s) were diagnosed by CT in over 90% of cases. This may reflect a more liberal use of CT in older patients with suspected trauma following influential reports such as Trauma Audit Research Network Major Trauma in Older People 28 and the majority of sites being major trauma centres. However, this finding may also reflect selection bias towards more severely injured patients, given that our inclusion criteria required radiological confirmation of rib fracture(s) and prior studies have demonstrated a poor sensitivity of X-ray diagnosis, with only 40% accuracy in older patients. 29 Amending the inclusion criteria to include patients with clinically suspected (rather than radiologically confirmed) rib fractures may mitigate against this selection bias and also allow the inclusion of those patients who are less severely injured and potentially more frail.

Our health economic scoping revealed key findings to be considered in future research involving older adults in emergency settings. Modification of the standard EQ-5D to obtain retrospective pre-injury health status may be beneficial in assessing specific impacts of injury in economic modelling. However, since response rates to the ICECAP-O were higher than for the EQ-5D at 30 days, which may reflect a patient preference for completing a measure specifically designed for use in older people, it is possible that this is a more appropriate measure for use in a definitive trial.

Conclusions

This trial has demonstrated that recruitment of older patients with rib fracture(s) in an emergency setting for the evaluation of early analgesic interventions (in the form of lidocaine patches) is feasible. Refinement of data collection, with a focus on collecting pain and frailty-specific outcomes, as well as intervention delivery, is needed before progressing to a definitive trial.

Ethics statements

Patient consent for publication.

Not applicable.

Ethics approval

The protocol (V.4.0 4 March 2022) and other related participant-facing documents were approved by the UK Health Research Authority and UK Research Ethics Committees (REC): 21/SC/0019 (South Central—Oxford C REC; IRAS reference 285096) and 21/SS/0043 (Scotland A REC; IRAS reference 299793). Participants gave informed consent to participate in the study before taking part.

Acknowledgments

Sponsor: North Bristol NHS Trust (R&I reference: 4284). Trial management: this trial was designed and delivered in collaboration with the Bristol Trials Centre, a UKCRC registered clinical trials unit, which is in receipt of National Institute for Health Research CTU support funding. The trial management group included all authors and particular thanks are given to Gareth Williams who led patient and public contributions on the trial management group. Trial Steering Committee: the RELIEF trial team would like to thank all members of the independent members of the committee who gave up their time to provide oversight of this work: Fiona Lecky (Clinical Professor in Emergency Medicine and TSC Chair), Rachel Bradley (Consultant in General, Geriatric and Orthogeriatric Medicine), Sean Ewings (Associate Professor of Medical Statistics, Southampton Clinical Trials Unit, University of Southampton), Gordon Halford (Patient and Public Involvement Contributor). Participating sites: the RELIEF trial team would like to thank all staff involved at the seven participating sites (Southmead Hospital, North Bristol NHS Trust, Principal Investigator (PI): Edward Carlton, Associate PI: Fraser Birse; Royal Infirmary of Edinburgh, NHS Lothian, PI: Rachel O’Brien; Derriford Hospital, University Hospitals Plymouth NHS Trust, co-PIs: Jason Smith and Robert James, Associate PI: Rory Heath; Queen Elizabeth University Hospital, NHS Greater Glasgow and Clyde, co-PIs: Fraser Denny and David Lowe, Associate PI: Nathalie Graham; St George's Hospital London, St George's University Hospitals NHS Foundation Trust, PI: Melanie Lynn; Musgrove Park Hospital, Somerset NHS Foundation Trust, PI: James Gagg; Royal Devon and Exeter Hospital, Royal Devon University Healthcare NHS Foundation Trust, PI: Andy Appelboam).

• Barrett-Connor E ,
• Nielson CM ,
• Orwoll E , et al
• Newey L , et al
• Hutchings H ,
• Lovett S , et al
• Pai L , et al
• Van Aken HK
• Zamary K , et al
• Williams H ,
• Shipway D , et al
• Johnson M ,
• Ata A , et al
• Mayberry JC ,
• Peck EG , et al
• Ingalls NK ,
• Horton ZA ,
• Bettendorf M , et al
• Goodwin VA ,
• Quinn TJ , et al
• Benger J , et al
• Ventafridda V ,
• Ripamonti C , et al
• Rockwood K ,
• MacKnight C , et al
• ↵ EuroQol - a new facility for the measurement of health-related quality of life . Health Policy 1990 ; 16 : 199 – 208 . doi:10.1016/0168-8510(90)90421-9 OpenUrl CrossRef PubMed Web of Science
• Sirois M-J , et al
• De Bellis A , et al
• MacLullich AMJ ,
• Perneczky R
• University of Birmingham
• Dimairo M ,
• Shephard N , et al
• Eldridge SM ,
• Campbell MJ , et al
• O’Cathain A ,
• Hackert MQN ,
• van Exel J ,
• Brouwer WBF
• Griffin XL ,
• Parsons N , et al
• Bouamra O , et al
• Singleton JM ,
• Bilello LA ,
• Canham LS , et al

Supplementary materials

Supplementary data.

This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

• Data supplement 1

Handling editor Kirsty Challen

X @DrPhilipBraude, @eddcarlton

Presented at Results were presented in part at the Royal College of Emergency Medicine Annual Scientific Conference on 26 September 2023 and Age Anaesthesia Annual Scientific Meeting on 12 May 2023.

Contributors MC and NT have had full access to all data in the study and take full responsibility for the integrity of the data and accuracy of data analysis. Study concept and design: EC, NT, CC, PB, JB, JG, JI, RK, NAM, DS, JS, ADM. Analysis and interpretation of data: all authors. Drafting of manuscript: EC, CC, MC, RK, NT. Critical revision of manuscript for important intellectual content: all authors. Statistical analysis: NT. Obtained funding: EC, NT, CC, PB, JB, JG, JI, RK, NAM, DS, JS, ADM. EC is the guarantor of the study.

Funding This study is funded by the NIHR [Advanced Fellowship (NIHR300068)]. The views expressed are those of the author(s) and not necessarily those of the NIHR or the Department of Health and Social Care

Disclaimer The funder was not involved in the design, execution, analysis and interpretation of data or writing up of the trial.

Competing interests None declared.

Patient and public involvement Patients and/or the public were involved in the design, or conduct, or reporting, or dissemination plans of this research. Refer to the 'Methods' section for further details.

Provenance and peer review Not commissioned; externally peer reviewed.

Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

Read the full text or download the PDF:

• Open access
• Published: 22 May 2024

Feasibility and acceptability of a cohort study baseline data collection of device-measured physical behaviors and cardiometabolic health in Saudi Arabia: expanding the Prospective Physical Activity, Sitting and Sleep consortium (ProPASS) in the Middle East

• Abdulrahman I. Alaqil   ORCID: orcid.org/0000-0003-0458-2354 1 , 2 , 3 ,
• Borja del Pozo Cruz   ORCID: orcid.org/0000-0002-9728-1317 2 , 4 , 5 ,
• Shaima A. Alothman   ORCID: orcid.org/0000-0003-2739-0929 6 ,
• Matthew N. Ahmadi   ORCID: orcid.org/0000-0002-3115-338X 7 , 8 ,
• Paolo Caserotti 2 ,
• Hazzaa M. Al-Hazzaa   ORCID: orcid.org/0000-0002-3099-0389 6 , 9 ,
• Andreas Holtermann   ORCID: orcid.org/0000-0003-4825-5697 3 ,
• Emmanuel Stamatakis 7 , 8 &
• Nidhi Gupta 3  

BMC Public Health volume  24 , Article number:  1379 ( 2024 ) Cite this article

305 Accesses

5 Altmetric

Metrics details

Physical behaviors such physical activity, sedentary behavior, and sleep are associated with mortality, but there is a lack of epidemiological data and knowledge using device-measured physical behaviors.

To assess the feasibility of baseline data collection using the Prospective Physical Activity, Sitting, and Sleep consortium (ProPASS) protocols in the specific context of Saudi Arabia. ProPASS is a recently developed global platform for collaborative research that aims to harmonize retrospective and prospective data on device-measured behaviors and health. Using ProPASS methods for collecting data to perform such studies in Saudi Arabia will provide standardized data from underrepresented countries.

This study explored the feasibility of baseline data collection in Saudi Arabia between November and December 2022 with a target recruitment of 50 participants aged ≥ 30 years. Established ProPASS methods were used to measure anthropometrics, measure blood pressure, collect blood samples, carry out physical function test, and measure health status and context of physical behaviors using questionnaires. The ActivPal™ device was used to assess physical behaviors and the participants were asked to attend two sessions at (LHRC). The feasibility of the current study was assessed by evaluating recruitment capability, acceptability, suitability of study procedures, and resources and abilities to manage and implement the study. Exit interviews were conducted with all participants.

A total of 75 participants expressed an interest in the study, out of whom 54 initially agreed to participate. Ultimately, 48 participants were recruited in the study (recruitment rate: 64%). The study completion rate was 87.5% of the recruited participants; 95% participants were satisfied with their participation in the study and 90% reported no negative feelings related to participating in the study. One participant reported experiencing moderate skin irritation related to placement of the accelerometer. Additionally, 96% of participants expressed their willingness to participate in the study again.

Based on successful methodology, data collection results, and participants’ acceptability, the ProPASS protocols are feasible to administer in Saudi Arabia. These findings are promising for establishing a prospective cohort in Saudi Arabia.

Peer Review reports

Global data from 2023 indicate that an estimated 27.5% of adults do not meet physical activity guidelines and have poor physical behaviors (e.g., physical activity, sedentary behavior, and sleep) that are linked with an increased risk of morbidity and mortality [ 1 , 2 , 3 , 4 ]. Sufficient physical activity and sensible sedentary times are associated with better health outcomes (e.g., cardiovascular health, mental health, and physical function) [ 1 , 2 ]. Despite this fact, 50–90% of Saudi Arabian adults perform low or insufficient daily physical activity; about 50% spend at least five hours per day sitting [ 5 ]. Furthermore, around 33% of the population experiences sleep durations of less than 7 h per night [ 6 ]. These trends could be a reason why non-communicable diseases account for 73% of mortality and cardiovascular diseases account for 37% of all deaths among Saudi Arabian adults [ 7 ]. However, there have been few studies in Middle Eastern countries, and the evidence that links between physical behaviors and health outcomes is under-represented in Saudi Arabia [ 1 ].

Furthermore, within Saudi Arabia, the few studies exploring this connection often rely on self-reported physical behaviors that often do not provide the most accurate picture [ 5 , 8 , 9 , 10 , 11 ]. This lack of data necessitates studies that incorporate measurements from devices that directly track these behaviors among Saudi Arabian adults, which aligns with recent guidance from the World Health Organization (WHO) on the necessity of incorporating device-measured physical behaviors into future studies to explore their relationships with various health aspects [ 1 , 12 ]. By employing such a method, we can gain more precise insights into the dose-response relationships between different physical behaviors and various health outcomes among Saudi Arabian adults.

The Prospective Physical Activity, Sitting, and Sleep Consortium (ProPASS) is an initiative that aims to explore how thigh-based accelerometry measurement of physical behaviors influences a wide range of health outcomes. This initiative operates on a global scale and aims to harmonize data from both retrospective and future studies [ 13 ]. To fulfill the aim, ProPASS is developing methods for collecting prospective data and processing, harmonizing, and pooling data from previous and future studies [ 14 ]. To date, the methods of the ProPASS consortium have been used to harmonize data from large-scale epidemiological studies, such as the 1970 British Birth Cohort, the Australian Longitudinal Study on Women’s Health [ 15 ], and Norway’s Trøndelag Health Study (HUNT) [ 16 , 17 ]. As such, this study seeks to determine if the ProPASS methodologies will be effective in the context of data collection within Saudi Arabia. This will be beneficial because it will help to standardize the measurement of physical behaviors, enhance harmonization across studies, and create more a representative and valid understanding of the associations between physical behaviors and health globally, including under-represented countries such as Saudi Arabia.

This paper describes the feasibility of baseline ProPASS data collection in Saudi Arabia with prospectively harmonized data with the main resource. This feasibility study of baseline data collection will serve as a framework for a future cohort study that will investigate the associations between device-measured physical behavior (e.g., physical activity, sedentary behavior, and sleep) and cardiometabolic health in Saudi adults.

The study was approved by the Institutional Review Board at Princess Nourah Bint Abdul Rahman University, Riyadh, Saudi Arabia (IRB 22–0146), and was carried out in accordance with the principles of the Declaration of Helsinki.

Study design and procedures

Participants were informed about the study’s aims and asked to read and sign the consent form before any measurements were taken. After agreeing to participate, they were asked to attend two sessions at the Lifestyle and Health Research Center (LHRC) at the Health Sciences Research Center of Princess Nourah Bint Abdulrahman University. During the first visit, each participant’s anthropometric measurements (e.g., height, weight, waist circumference), blood pressure and heart rate, blood samples, and handgrip strength were measured. Next, the participants completed questionnaires on demographic information, dietary habits, self-rated health, self-reported smoking status, and the Global Physical Activity, Sedentary Behaviors, and Sleep behavior questionnaires. At the end of the first visit, the researcher attached the ActivPAL™ accelerometer device to their thigh which they were asked to wear for seven consecutive days. Participants were also provided with a diary to record their waking and sleeping hours [ 18 ]. On the 8th day of study, the participants were asked to attend the LHRC for session two where they returned the device and were interviewed (see Fig.  1 ).

Demonstration and summary of the study procedure

Participants and eligibility

The study aimed to recruit a total of 50 Saudi adults aged ≥ 30 years, which is generally considered a common sample size for feasibility studies [ 19 , 20 ]. The eligibility criteria were: (1) Saudi nationals (2), resident in Riyadh, and (3) aged ≥ 30 years old. The exclusion criteria were: (1) having a current medical condition that forces them to be chair-bound or bedridden for more than half of their waking hours (2), being allergic to plasters or adhesives (3), being allergic to low-density polyethylene (4), having a skin condition that would prevent them from wearing the monitor, and (5) those who may need to pass through a metal detector/security checkpoint during the duration of the study. The study’s aims, protocol, and procedures were clearly described to all participants before any measurements were taken.

Recruitment

Participant recruitment was carried out over the month of November 2022. Participants were recruited from different locations across Riyadh, Saudi Arabia, by using electronic flyers on social media (e.g., Twitter, WhatsApp) that provided information about the study and the researcher’s contact details. Prospective participants who were interested in joining the study were asked to provide their contact information via a link to Google Forms featured in the study description. The participants who initially expressed interest but later decided not to join were invited to share their reasons for non-participation through a physical or telephonic meeting.

Measurements based on ProPASS methodology

The current study employed the ProPASS method and protocol for new cohort studies that seek to join ProPASS prospectively [ 14 , 21 ]. All measurements were taken by researchers that were well-trained in the ProPASS protocol and methods. Blood pressure and hand grip strength measurements were taken three times, and the mean average was then calculated; all other measurements were taken only once.

Anthropometric measurements

Height (to the nearest 0.1 cm) and weight (to the nearest 0.1 kg) were measured with a stadiometer (SECA 284; Seca, Hamburg, Germany), and scale (SECA 284; Seca, Hamburg, Germany), respectively. Waist circumference (to the nearest 0.1 cm) was measured midway between the lower rib margin and the iliac crest at the end of a gentle expiration [ 22 ]. Body mass index (BMI) was calculated using the standard calculation (height in meters squared/body weight in kilograms).

Blood pressure and heart rate

Blood pressure was taken after resting for five minutes in a sitting position. Blood pressure was taken three times with one minute between measurements and the average reading was recorded [ 23 ]. Blood pressure and heart rate were measured using a Welch Allyn Connex 7300 Spot Vital Signs Monitor, which provides a high degree of accuracy [ 24 ]. Mean arterial pressure (MAP) was then calculated (MAP = 1/3 * SBP + 2/3 * DBP in mm Hg) using the average of both the SBP and DBP values [ 25 ].

Blood samples

Non-fasting finger-prick (capillary) blood samples (40 µL) were collected for analysis after warming the finger for five minutes. A drop of blood was taken directly from the heated finger to be analysed for blood glucose, triglycerides, total cholesterol, high-density lipoprotein cholesterol, and low-density lipoprotein cholesterol. A previously validated CardioChek PA analyser (CardioChek PA Blood Analyser, UK) was used to analyse the blood samples [ 26 , 27 ].

Medication use

Participants’ medication use was evaluated by the question: Do you currently use any prescription medicines ? If the answer was yes, the participants were asked which medications they use, such as medication for high blood pressure, high cholesterol, asthma, COPD, anxiety, depression, thyroid problems, allergies. They were also asked whether the medication was in the form of tablets, or nasal sprays, whether the medication was anti-inflammatory, chemotherapeutic, urological, birth control, or neurological, and the age at which the participants had begun using the medication.

Familial disease history

Familial disease history was assessed by the question: Do your parents, siblings or children have, or have they ever had, some of the following diseases before the age of 60 ? The responses included asthma, hay fever/nasal allergies, chronic bronchitis, emphysema or COPD, anxiety or depression, myocardial infarction (heart attack), diabetes, stroke or brain hemorrhage, and cancer. The responses were yes, no , and I don’t know .

Chronic health status

Participants’ chronic disease status and/or long-term health issues were assessed by the question: Have you had, or do you have any of the following diseases? The responses included angina, myocardial infarction (heart attack), heart failure, peripheral vascular disease, atrial fibrillation, stroke/brain hemorrhage, thrombosis, pulmonary embolism, asthma, COPD or emphysema, diabetes, hypothyroidism (low metabolism), hyperthyroidism (high metabolism), cancer, migraine, psoriasis, kidney disease, arthritis (rheumatoid arthritis), Bechterew’s disease, gout, mental health problems, osteoporosis, sleep apnea, arthrosis, nerve disease, hearing/ear disease, eye disease, and infection. Those who replied yes were asked a follow-up question: How old were you when you had it for the first time?

Mobility limitations

The questionnaire was based on three questions on performance-based measures of mobility, which had already been translated and culturally adapted into Arabic [ 28 ]. These three questions are valid and reliable tools to identify the early indications of disability and can be used as indicators to identify those at high risk of future disability [ 29 ]. Self-reported mobility was assessed via the following questions: (1)  Do you have difficulty in walking 2.0 km?  (2)  Do you have difficulty in walking 0.5 km ? and (3)  Do you have difficulty in walking up one flight of stairs? The five response options were: (1)  able to manage without difficulty  (2), able to manage with some difficulty  (3), able to manage with a great deal of difficulty  (4), able to manage only with the help of another person, and  (5)  unable to manage even with help.

Dietary habits

The dietary habits questionnaire was translated and culturally adapted into Arabic [ 28 ]. The questionnaire assessed the dietary habits of the participants was adapted from the Survey of Health, Aging, and Retirement in Europe (SHARE), which has been demonstrated to be a valid and reliable tool for assessing diet [ 30 ]. The questionnaire focused on the consumption of dairy products, legumes, eggs, meat, fruit and vegetables.

Self-rated health

A set of valid and reliable questions adapted from Idler et al.’s (1997) questionnaire was used to assess participants’ self-rated health by asking them to rate their health status using the following questions: (1)  In general, would you say your health is…: Excellent; Very good; Good; Fair; Poor;  (2)  Compared to one year ago, how would you rate your health in general now?: Much better now than one year ago; Somewhat better now than one year ago; About the same; Somewhat worse now than one year ago; Much worse now than one year ago [ 31 , 32 ].

Smoking habits

Self-report questions on smoking behavior were adapted from the UK Biobank questionnaire and were used to assess participants’ present and past smoking habits including at what age they began smoking. the number of cigarettes smoked per day, the type of tobacco used, the duration of smoking, and, among former smokers, the age when smoking ceased [ 33 ].

Physical behaviours

Physical behaviors such as physical activity, sedentary behavior, and sleep were measured by using (1) self-reported and (2) device-based measures:

Self-report measures

Physical activity was measured on a self-report basis via the Global Physical Activity Questionnaire (GPAQ) which was translated into Arabic and previously validated [ 34 ]. In addition, the Sedentary Behavior Questionnaire (SBQ), which had already been translated into Arabic [ 28 ], was used to subjectively assess participants’ sedentary behavior time [ 35 ]. Lastly, the Pittsburgh Sleep Quality Index was used to assess sleep quality and sleep disturbances over a one-month period [ 36 ].

Device-based measures

Physical behaviors were measured by wearing a thigh-worn accelerometer device (an ActivPAL™ Micro4, PAL technologies, Glasgow, Scotland) that participants wore continuously for 24 h for seven full days [ 37 ]. The Activpal™ device was sealed with a nitrile sleeve and attached with a medical waterproof 3 M Tegaderm transparent dressing on the front of the right mid-thigh on the muscle belly by a well-trained member of researcher team. The ActivPAL™ monitor is a valid and reliable measure of time spent walking [ 38 ], sitting, and standing time in healthy adults [ 39 ]. In addition, the participants were asked to fill in a recording sheet that included a sleep diary (times that the participant went to and got out of bed), as well as, the dates and times when the accelerometer fell off or was removed.

Physical function

Physical function was objectively measured using a digital hand-grip strength dynamometer (Takei Hand Grip Dynamometer 5401-C, Japan) via three successive hand-grip assessments for each hand (left and right); the mean value for each hand was then recorded. The instrument can measure hand-grip values from 5 to 100 kg; the minimum unit of measurement is 0.1 kg. The tool is a good health outcomes predictor [ 40 , 41 ].

Data collection evaluation of feasibility

Overall, the study evaluated feasibility in two main stages where feedback from the first six participants was used to resolve any unforeseen issues in the protocol implementation on the remaining participants. Any changes to the procedure were documented.

The current study evaluated the feasibility of Saudi adults’ participation based on the following constructs: (1) recruitment capability (2), acceptability and suitability of study procedures, and (3) resources and ability to manage and implement the study. Table  1 outlines the feasibility constructs, measures, outcome definitions, and methods employed. In evaluating feasibility, the current study followed the recommendations for a feasibility study as reported by Orsmond and Cohn, 2015 [ 42 ].

Overall, the study collected data on the feasibility constructs via tracking the registration, equipment availability, and time spent on various tasks performed (for example training researchers, performing various tasks like attaching the sensor) and completion rate (such as tracking diary entries, questionnaire entries and number of days with accelerometer data), via personal contacts (for information on barriers and facilitators of participation), via processing sensor data, and via interviews after the measurement (for example obtaining information on potential issues during measurement and willingness to participate).

Participant interviews after measurement

After the completion of the study, face-to-face semi-structured interviews were conducted with all participants who had completed the 7-day study period. The aim of these interviews was to collect comprehensive feedback regarding participants’ experiences with the study protocol, with the goal of capturing additional insights that was not captured by other feasibility measures. Some examples of such measures were motivations for joining the study, their expectations prior to participation, and their levels of satisfaction with the study procedures. A detailed interview guide is described in Appendix A [ 28 , 43 , 44 ].

Statistical analysis

Descriptive analysis summarized participants’ demographics, anthropometric measurements, health status, clinical measurements, physical behaviors characteristics, and interview questions responses. The continuous variables were characterized using mean ± standard deviations (SD), while categorical variables were presented using frequencies accompanied by percentages (%). The recruitment rate was calculated by the number of participants who participated and signed the consent form / total number of participants who registered in the study (see Fig.  2 ). Additional analyses were performed to compare participants who reported burden with those who reported no burden of participation (see supplementary materials). T-tests and Chi-square tests were employed for this comparison. IBM’s Statistical Package for the Social Sciences (SPSS) (version 27 SPSS, Inc. Chicago, Illinois) was used to conduct the qualitative analysis. The raw data of ActivPAL were analyzed by using the ActiPASS software (ActiPASS © 2021 - Uppsala University, Sweden).

Recruitment and study participant’s diagram

A total of 75 participants initially volunteered to participate. Ten participants were excluded from the study as they did not meet the inclusion criteria ( n  = 8) or could not be contacted ( n  = 2). In addition, 11 participants withdrew their interest in participating for various reasons: (1) excessive distance between the location of the study (LRHC) and their residence ( n  = 3) (2), hesitant about joining the study ( n  = 1) (3), believed that the ActivPAL™ device would interfere with his/her health ( n  = 1) (4), believed that the ActivPAL™ device would interfere with their regular exercise routine ( n  = 2) (5), had family and work commitments ( n  = 3), and (6) claimed that the timing was unsuitable ( n  = 1). Out of a total of 54 participants who had agreed to participate in the study, 48 participants from Riyadh, Saudi Arabia, attended and completed the consent form. However, four of those participants provided incomplete data (i.e., they completed the questionnaires only and did not wear an ActivPAL™ device). Therefore, a total of 44 participants out of 75 potential participants (59%) successfully completed the study (wore an ActivPAL™ device and completed all questionnaires). See Fig.  2 for the study’s recruitment flow.

Participants

Of the 48 participants, nearly half were female (47.9%). On average, the participants were 37 ± 7.3 years old, had a BMI of 28.3 ± 5.6, and a waist circumference of 86.9 ± 16.4 cm. Most participants were married, had college degrees, were employed as office workers and professionals, had never smoked, and did not use any medication (see Table  2 ). A total of 87.5% of participants had a family history of disease; 85.4%, 95.8%, and 89.6%, reported having no difficulty walking 2 km, 500 m, and up one flight of stairs, respectively. Approximately 48% of participants rated their health as very good , while 39.6% reported their health as about the same compared to one year ago . In terms of dietary habits, nearly half the participants reported consuming dairy products every day, 25% consumed legumes and eggs 3 to 6 times a week, 56.3% consumed meat every day, and 45.8% consumed fruits and vegeTables 3, 4, 5 and 6 times a week.

Table  3 presents the primary variables of the study: including average systolic, diastolic, and mean arterial pressure values of 121.13 ± 11.81 mmHg, 79.26 ± 8.92 mmHg, and 93.15 ± 9.20 mmHg, respectively. The mean resting heart rate was 74.3 ± 12.66. Furthermore, the non-fasting blood profile of the sample was analyzed and showed the following values: total cholesterol: 177.89 ± 33.79 mg/dL; HDL-cholesterol: 50.96 ± 13.02 mg/dL; triglycerides: 123.94 ± 68.92 mg/dL; LDL-cholesterol: 103 ± 29.89 mg/dL; TC/HDL-cholesterol ratio: 3.71 ± 1.11; LDL/HDL-cholesterol ratio: 2.19 ± 0.81; non-HDL-cholesterol: 127.06 ± 33.51 mg/dL; non-fasting glucose: 102.98 ± 35.36 mg/dL. Table  3 provides an overview of the participants’ physical activity related behaviors.

Feasibility evaluation

The following results highlight the approaches taken by the current study to assess the feasibility of baseline data collection using ProPASS methodology specifically in the context of Saudi Arabia.

The evaluation of the feasibility of the study protocol was conducted in two stages, initially involving six participants, whose feedback was used to refine and improve the protocol implementation for the remaining participants. Of the six selected participants, three were female. In the pre-evaluation, only two minor issues were encountered; (1) accessing the lab outside of working hours (16:00–22:00) as most participants were unable to attend during the day (07:00–16:00) due to work commitments. This issue was resolved in all subsequent data collection points by receiving approval for extended lab hours; (2) obtaining the required number of ActivPAL™ devices from the technical coordinator due to miscommunication and high demand by other researchers. To prevent further issues, the author obtained 30 devices in advance for the feasibility evaluation.

Recruitment capability

The recruitment rate was used to measure the feasibility of recruitment methodology to collect baseline ProPASS data; the results showed that 64% ( n  = 48) of participants signed the consent form and attended the LRHC lab (see Fig.  2 ). After screening the eligibility criteria, out of a total of 75 participants, 65 met the study criteria, and 11 were excluded from participating due to the reasons as detailed in Fig.  2 . As Fig.  2 illustrates, although 54 participants scheduled an appointment for the study, only 48 (64%) attended and signed the consent form. In the final stage of the recruitment process, around 59% ( n  = 44) of participants completed all the required measurements for the study.

Acceptability and suitability of study procedures

The adherence rate (i.e., the extent to which participants adhered to the outlined procedures in terms of the number of days with valid accelerometry data) was 5.7 days. Furthermore, participants provided sleep diary entries for 85.4% of days. All questionnaires were completed with a 100% response rate.

To assess the study’s time demands on participants, the length of time participants needed to complete all measurements was mean time of 25 min (23 min to complete the questionnaires and two minutes to attach the sensor). Additionally, the completion rates for the registered participants who completed all the required measurements (i.e., accelerometer measurement, diary registration, and questionnaires) was 91.6%. (See Table  4 ).

Resources and ability

The final feasibility outcomes (i.e., having the required resources and ability to manage and implement the study) are presented in Table  5 . This objective was assessed based on four domains: skin irritation, equipment availability, training requirements, and accelerometer loss (see Table  5 ). The first domain revealed that three participants experienced skin irritation during the study; of these, two participants had mild symptoms, such as itchiness and discomfort that lasted for the first three days but did not lead to their withdrawal from the study. However, one participant reported moderate irritation resulting in red skin which required them to withdraw from the study. The second domain, equipment availability, indicated that all the necessary equipment was available 100% of the time. The third domain was training requirements, and the researchers required four hours of training on how to use it correctly. Finally, in the accelerometer loss domain, the study recorded four failed devices out of 30 that did not generate data for seven days.

Participant interview after measurement

After completing the study, all participants were interviewed around five primary themes: (1)  motivation and expectations of participation  (2), participant satisfaction  (3), the burden of participation  (4), willingness to participate again , and (5)  perception of time usage (see Fig.  3 ).

Interview outcomes of participant’s experience with the study protocol

To determine the participants’ motivations for and expectations about joining the study, they were asked: What made you want to join this study? The results showed that 90% of participants were interested in learning about their physical behaviors and health status; 43% participated in supporting the researcher, and 14% reported that the final report attracted them to participate (see Fig.  3 a and the example of final report in supplementary material). Participant satisfaction was assessed via two questions: (1)  What was your overall experience of participating in the study? and (2)  Was it as you expected? The findings indicated that 62% of participants were satisfied that the study was as expected, 33% were more satisfied than expected, and 5% were unsatisfied and found the study below their expectations (see Fig.  3 b).

Regarding the overall burden of participation, 76% of participants reported that it was no burden , 5% reported that it was a burden , and 14% believed it was somewhat burdensome (see Fig.  3 c). Additionally, 79% of participants expressed their willingness to participate again in the future (see Fig.  3 d). Finally, regarding time usage, 67% of participants found it easy to complete the seven-day study without any concerns (see Fig.  3 h).

The feasibility of the baseline ProPASS data collection methodology was evaluated among Saudi adults who participated in this study. The findings revealed that the methodology was both feasible and acceptable, paving the way for large-scale prospective cohort research in Saudi Arabia. This research marks the first attempt to establish a prospective cohort study in Saudi Arabia using established ProPASS methods [ 13 , 15 ] and protocols. Conducting such a cohort study in Saudi Arabia is crucial due to the country’s high prevalence of non-communicable diseases that are mostly due to poor physical behaviors (e.g., lack of physical activity, sedentary behavior, and sleep) [ 7 ], due to recent enormous economic growth accompanied by technological transformations and urbanization [ 11 ].

The first aspect of feasibility evaluated of the baseline ProPASS data collection methodology was the capability to recruit participants. The findings indicated that the recruitment rate was 64% which is similar to prior studies [ 46 , 47 ]. One study indicated that a recruitment rate of at least between 20 and 40% is required to be deemed feasible [ 48 ]. Thus, the recruitment rate in the current study seems acceptable for creating a future cohort using ProPASS methods in Saudi Arabia. Additionally, in the current study, the refusal rate was only 15% which is significantly lower than in previous studies [ 45 , 49 ] where refusal rates ranged from 50 to 66%. One reason for the low refusal rate in the current study is that the recruitment was material specifically designed to motivate Saudi participants to join the study by indicating that the study would provide data and insight into their current state of health. For example, the results of the semi-structured interviews illustrated that 90% of participants joined the study because they wanted to know about their physical behaviors and health status (see Fig.  3 ). This result also indicates that our recruitment material might be suitable for ensuring high participation in the future cohort study.

The second aspect of feasibility for the baseline ProPASS data collection methodology that was evaluated in this study was the acceptability and suitability of the study procedures. Previous studies have shown that in order to obtain reliable estimates of adults’ habitual physical activity, it is necessary to record accelerometer data for 3–5 days [ 50 , 51 ] to gather valid data to perform analysis and provide information about the habitual physical behaviors. A recent study indicated that distributing accelerometers in person was associated with a high proposition of participants consenting to wear an accelerometer and meeting minimum wear criteria [ 21 ]. Our study was able to collect an average six days of valid data which was sufficient to obtain representative descriptions of the participants’ physical behaviors [ 52 ]. There were high general adherence rates for participant diary entries, questionnaires completion, and adherence to the study protocol, indicating that the ProPASS methods could be feasibly implemented with a larger study population. The study also assessed the time commitment necessary to complete the questionnaires and attach the ActivPAL™ devices to participants’ thighs. Completing the questionnaires took approximately 23 min (SD = 8). Prior studies have indicated that shorter questionnaires (e.g., 20 min) yield a higher response rate from participants, a finding that was consistent with our study [ 53 , 54 ]. Additionally, attaching the sensor to the participant’s thigh took about two minutes. These findings indicate that participation in this study was not burdensome, which was confirmed by the interviews that showed that 95% of participants felt that participating in the study (i.e., filling out all questionnaires and wearing the ActivPal™ device for 7 days) was not a burden. Overall, ProPASS methods appear to be less burdensome, well-suited, and readily accepted by participants.

The third aspect of feasibility for the baseline ProPASS data collection methodology was the availability of resources and the ability to manage and execute the study. As we aim to create a new cohort adhering to global (ProPASS) standards, protocol training was vital to obtain quality outcomes as per the ProPASS protocol. As a result, the protocol training took around four hours which was similar to a prior study [ 45 ]. In terms of the availability of resources, all essential equipment was always accessible. The study also considered skin irritation as an important factor. One study noted that 38% of participants stopped using ActivPal™ due to skin irritation from PALstickies or Tegaderm dressings [ 55 ]; another reported one discontinuation due to irritation associated with a Tegaderm dressing [ 56 ]. In the current study, there were three reported irritations, with two having mild initial discomfort that eventually subsided. One participant left the study due to moderate irritation. Nonetheless, it is important to note that the data collection occurred during colder winter periods (average 20 degrees Celsius). It is possible that instances of skin irritation could be more pronounced during Saudi Arabia’s hot summer season, characterized by temperatures of approximately 40 degrees Celsius. Future studies should investigate the feasibility of using devices and tape suitable for summer temperatures. In addition, the current study also had a low accelerometer failure rate: only four accelerometers failed to record, which is similar to previous studies [ 57 , 58 ]. All ActivPal™ devices were returned at the end of the study during visit two, ensuring that the ProPASS method is suitable to be used in future cohorts in Saudi Arabia.

Strengths and limitations of Study

This study represents the first of its kind to utilize device-based measures for assessing physical behaviors among adults in Saudi Arabia. The device-based measure has been shown to provide useful information about physical behaviors when compared to using self-report questionnaires [ 16 ]. Furthermore, it marks the initial examination of the ProPASS consortium method in the Middle East, particularly in Saudi Arabia. Nevertheless, the current study has certain limitations including recruiting among relatively young participants, presumably without any medical conditions and with postgraduate qualifications. This may limit the generalization of the findings to the entire population. The acceptability of the study in other age groups and among individuals with lower educational backgrounds is yet to be studied. In addition, the feasibility of the baseline ProPASS data collection methodology study was conducted during winter, which might have influenced the observed levels of physical behaviors in our sample. Similarly, the study was unable to evaluate the feasibility of utilizing 3 M Tegaderm dressings in hot summer months. Lastly, it’s important to note that our study employed a relatively small sample size; nonetheless, this size is considered acceptable for feasibility studies.

The baseline ProPASS data collection methodology and protocol for a future cohort study are both feasible and acceptable for implementation within the context of Saudi Arabia. This feasibility study represents the first step toward establishing a prospective ProPASS cohort study to examine the association between physical behaviors and cardiometabolic health among Saudi Arabian adults.

Availability of data and materials

The datasets used and/or analyzed during the current study available from the corresponding author on reasonable request.

Abbreviations

The Prospective Physical Activity, Sitting and Sleep consortium

Physical activity, sedentary behavior, and sleep

Bull FC, Al-Ansari SS, Biddle S, Borodulin K, Buman MP, Cardon G, et al. World health organization 2020 guidelines on physical activity and sedentary behaviour. Br J Sports Med. 2020;54(24):1451–62.

Article   PubMed   Google Scholar  

Chrysant SG, Chrysant GS. Association of physical activity and trajectories of physical activity with cardiovascular disease. Expert Rev Cardiovasc Ther. 2023;0(0):1–10.

Google Scholar  

Falck RS, Davis JC, Li L, Stamatakis E, Liu-Ambrose T. Preventing the ‘24-hour Babel’: the need for a consensus on a consistent terminology scheme for physical activity, sedentary behaviour and sleep. Br J Sports Med. 2022;56(7):367–8.

Guthold R, Stevens GA, Riley LM, Bull FC. Worldwide trends in insufficient physical activity from 2001 to 2016: a pooled analysis of 358 population-based surveys with 1·9 million participants. Lancet Global Health. 2018;6(10):e1077-1086.

Evenson KR, Alhusseini N, Moore CC, Hamza MM, Al-Qunaibet A, Rakic S, et al. Scoping review of Population-based physical activity and sedentary behavior in Saudi Arabia. J Phys Activity Health. 2023;20(6):471–86.

Article   Google Scholar  

Ahmed AE, Al-Jahdali F, AlALwan A, Abuabat F, Salih SB, Al-Harbi A, et al. Prevalence of sleep duration among Saudi adults. Saudi Med J. 2017;38(3):276–83.

Article   PubMed   PubMed Central   Google Scholar  

World Health Organization. Noncommunicable Diseases Progress Monitor 2022. 2022. Available from: https://www.who.int/publications-detail-redirect/9789240047761 . Cited 2023 Jun 22.

Besson H, Brage S, Jakes RW, Ekelund U, Wareham NJ. Estimating physical activity energy expenditure, sedentary time, and physical activity intensity by self-report in adults. Am J Clin Nutr. 2010;91(1):106–14.

Article   CAS   PubMed   Google Scholar  

Cerin E, Cain KL, Owen Oyeyemial, Conway N, Cochrane TL. Correlates of agreement between accelerometry and self-reported physical activity. Med Sci Sports Exerc. 2016;48(6):1075–84.

Klesges RC, Eck LH, Mellon MW, Fulliton W, Somes GW, Hanson CL. The accuracy of self-reports of physical activity. Med Sci Sports Exerc. 1990;22(5):690–7.

Al-Hazzaa HM. Physical inactivity in Saudi Arabia revisited: a systematic review of inactivity prevalence and perceived barriers to active living. Int J Health Sci (Qassim). 2018;12(6):50–64.

PubMed   Google Scholar  

DiPietro L, Al-Ansari SS, Biddle SJH, Borodulin K, Bull FC, Buman MP, et al. Advancing the global physical activity agenda: recommendations for future research by the 2020 WHO physical activity and sedentary behavior guidelines development group. Int J Behav Nutr Phys Act. 2020;17(1):143.

Stamatakis E, Koster A, Hamer M, Rangul V, Lee IM, Bauman AE, et al. Emerging collaborative research platforms for the next generation of physical activity, sleep and exercise medicine guidelines: the prospective physical activity, sitting, and Sleep consortium (ProPASS). Br J Sports Med. 2020;54(8):435–7.

The prospective physical activity, sitting and sleep consortium. Prospective Physical. 2022. ProPASS - prospective physical activity, sitting, and sleep consortium. Available from: https://www.propassconsortium.org . Cited 2022 May 20.

Wei L, Ahmadi MN, Chan HW, Chastin S, Hamer M, Mishra GD, et al. Association between device-measured stepping behaviors and cardiometabolic health markers in middle-aged women: the Australian longitudinal study on women’s Health. Scand J Med Sci Sports. 2023;33(8):1384–98.

Ahmadi MN, Blodgett JM, Atkin AJ, Chan HW, Pozo CB del, Suorsa K, et al. Device-measured physical activity type, posture, and cardiometabolic health markers: pooled dose-response associations from the ProPASS Consortium. medRxiv. 2023; 2023.07.31.23293468. Available from: https://www.medrxiv.org/content/10.1101/2023.07.31.23293468v1 . Cited 2023 Aug 28.

Blodgett JM, Ahmadi MN, Atkin AJ, Chastin S, Chan HW, Suorsa K, et al. Device measured sedentary behaviour, sleep, light and moderate-vigorous physical activity and cardio-metabolic health: A compositional individual participant data analysis in the ProPASS consortium. medRxiv. 2023:2023.08.01.23293499. Available from: https://www.medrxiv.org/content/10.1101/2023.08.01.23293499v1 . Cited 2023 Aug 28.

Inan-Eroglu E, Huang BH, Shepherd L, Pearson N, Koster A, Palm P, et al. Comparison of a thigh-worn accelerometer algorithm with diary estimates of time in bed and time asleep: the 1970 British cohort study. J Meas Phys Behav. 2021;4(1):60–7.

Lancaster GA, Dodd S, Williamson PR. Design and analysis of pilot studies: recommendations for good practice. J Eval Clin Pract. 2004;10(2):307–12.

Thabane L, Ma J, Chu R, Cheng J, Ismaila A, Rios LP, et al. A tutorial on pilot studies: the what, why and how. BMC Med Res Methodol. 2010;10(1):1.

Pulsford RM, Brocklebank L, Fenton SAM, Bakker E, Mielke GI, Tsai LT, et al. The impact of selected methodological factors on data collection outcomes in observational studies of device-measured physical behaviour in adults: a systematic review. Int J Behav Nutr Phys Act. 2023;20(1):26.

Ma WY, Yang CY, Shih SR, Hsieh HJ, Hung CS, Chiu FC, et al. Measurement of Waist circumference. Diabetes Care. 2013;36(6):1660–6.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Berenson GS, Srinivasan SR, Bao W, Newman WP, Tracy RE, Wattigney WA. Association between multiple cardiovascular risk factors and atherosclerosis in children and young adults. The Bogalusa heart study. N Engl J Med. 1998;338(23):1650–6.

Alpert BS, Quinn D, Kinsley M, Whitaker T, John TT. Accurate blood pressure during patient arm movement: the Welch allyn connex spot monitor’s SureBP algorithm. Blood Press Monit. 2019;24(1):42–4.

The Sixth Report of the Joint National Committee on Prevention. Detection, evaluation, and treatment of high blood pressure. Arch Intern Med. 1997;157(21):2413–46.

Panz VR, Raal FJ, Paiker J, Immelman R, Miles H. Performance of the CardioChek PA and Cholestech LDX point-of-care analysers compared to clinical diagnostic laboratory methods for the measurement of lipids. Cardiovasc J S Afr. 2005;16(2):112–7.

CAS   PubMed   Google Scholar  

PTS Diagnostics. CardioChek PA Analyzer. PTS Diagnostics. 2022. Available from: https://ptsdiagnostics.com/cardiochek-pa-analyzer/ . Cited 2022 Feb 26.

Alaqil AI, Gupta N, Alothman SA, Al-Hazzaa HM, Stamatakis E, del Pozo Cruz B. Arabic translation and cultural adaptation of sedentary behavior, dietary habits, and preclinical mobility limitation questionnaires: a cognitive interview study. PLOS One. 2023;18(6):e0286375.

Mänty M, Heinonen A, Leinonen R, Törmäkangas T, Sakari-Rantala R, Hirvensalo M, et al. Construct and predictive validity of a self-reported measure of preclinical mobility limitation. Arch Phys Med Rehabil. 2007;88(9):1108–13.

Börsch-Supan A, Brandt M, Hunkler C, Kneip T, Korbmacher J, Malter F, et al. Data Resource Profile: the Survey of Health, Ageing and Retirement in Europe (SHARE). Int J Epidemiol. 2013;42(4):992–1001.

Idler EL, Benyamini Y. Self-rated health and mortality: a review of twenty-seven community studies. J Health Soc Behav. 1997;38(1):21–37.

Lundberg O, Manderbacka K. Assessing reliability of a measure of self-rated health. Scand J Soc Med. 1996;24(3):218–24.

Peters SAE, Huxley RR, Woodward M. Do smoking habits differ between women and men in contemporary western populations? Evidence from half a million people in the UK Biobank study. BMJ Open. 2014;4(12):e005663.

Doyle C, Khan A, Burton N. Reliability and validity of a self-administered arabic version of the global physical activity questionnaire (GPAQ-A). J Sports Med Phys Fit. 2019;59(7):1221–8.

Rosenberg DE, Norman GJ, Wagner N, Patrick K, Calfas KJ, Sallis JF. Reliability and validity of the sedentary behavior questionnaire (SBQ) for adults. J Phys Act Health. 2010;7(6):697–705.

Buysse DJ, Reynolds CF, Monk TH, Berman SR, Kupfer DJ. The Pittsburgh sleep quality index: a new instrument for psychiatric practice and research. Psychiatry Res. 1989;28(2):193–213.

Crowley P, Skotte J, Stamatakis E, Hamer M, Aadahl M, Stevens ML, et al. Comparison of physical behavior estimates from three different thigh-worn accelerometers brands: a proof-of-concept for the prospective physical activity, sitting, and Sleep consortium (ProPASS). Int J Behav Nutr Phys Act. 2019;16(1):65.

Ryan CG, Grant PM, Tigbe WW, Granat MH. The validity and reliability of a novel activity monitor as a measure of walking. Br J Sports Med. 2006;40(9):779–84.

Kozey-Keadle S, Libertine A, Lyden K, Staudenmayer J, Freedson PS. Validation of wearable monitors for assessing sedentary behavior. Med Sci Sports Exerc. 2011;43(8):1561–7.

Altankhuyag I, Byambaa A, Tuvshinjargal A, Bayarmunkh A, Jadamba T, Dagvajantsan B, et al. Association between hand-grip strength and risk of stroke among Mongolian adults: results from a population-based study. Neurosci Res Notes. 2021;4(3Suppl):8–16.

Bohannon RW. Hand-grip dynamometry predicts future outcomes in aging adults. J Geriatr Phys Ther. 2008;31(1):3–10.

Garcia L, Ferguson SE, Facio L, Schary D, Guenther CH. Assessment of well-being using fitbit technology in college students, faculty and staff completing breathing meditation during COVID-19: a pilot study. Mental Health Prev. 2023;30:200280.

Al-Hazzaa HM, Alothman SA, Albawardi NM, Alghannam AF, Almasud AA. An arabic sedentary behaviors questionnaire (ASBQ): development, content validation, and pre-testing findings. Behav Sci. 2022;12(6):183.

Orsmond GI, Cohn ES. The distinctive features of a feasibility study: objectives and guiding questions. OTJR. 2015;35(3):169–77. https://doi.org/10.1177/1539449215578649 . (Cited 2022 Aug 4).

Marmash D, Ha K, Sakaki JR, Hair R, Morales E, Duffy VB, et al. A feasibility and pilot study of a personalized nutrition intervention in mobile food pantry users in Northeastern connecticut. Nutrients. 2021;13(9):2939.

Ouchi K, Lee RS, Block SD, Aaronson EL, Hasdianda MA, Wang W, Rossmassler S, Palan Lopez R, Berry D, Sudore R, Schonberg MA, Tulsky JA. An emergency department nurse led intervention to facilitate serious illness conversations among seriously ill older adults: A feasibility study. Palliat Med. 2023;37(5):730–9. https://doi.org/10.1177/02692163221136641 .

Bajwah S, Ross JR, Wells AU, Mohammed K, Oyebode C, Birring SS, et al. Palliative care for patients with advanced fibrotic lung disease: a randomised controlled phase II and feasibility trial of a community case conference intervention. Thorax. 2015;70(9):830–9.

Mosadeghi S, Reid MW, Martinez B, Rosen BT, Spiegel BMR. Feasibility of an immersive virtual reality intervention for hospitalized patients: an observational cohort study. JMIR Mental Health. 2016;3(2):e5801.

Papatzikis E, Elhalik M, Inocencio SAM, Agapaki M, Selvan RN, Muhammed FS, et al. Key challenges and future directions when running auditory Brainstem Response (ABR) Research Protocols with newborns: a Music and Language EEG Feasibility Study. Brain Sci. 2021;11(12):1562.

Trost SG, Mciver KL, Pate RR. Conducting accelerometer-based activity assessments in field-based research. Med Sci Sports Exerc. 2005;37(11):S531-543.

Wagnild JM, Hinshaw K, Pollard TM. Associations of sedentary time and self-reported television time during pregnancy with incident gestational diabetes and plasma glucose levels in women at risk of gestational diabetes in the UK. BMC Public Health. 2019;19(1):575.

Ham SA, Ainsworth BE. Disparities in data on healthy people 2010 physical activity objectives collected by accelerometry and self-report. Am J Public Health. 2010;100(S1):S263-268.

Marcus B, Bosnjak M, Lindner S, Pilischenko S, Schütz A. Compensating for low topic interest and long surveys: a field experiment on nonresponse in web surveys. Social Sci Comput Rev. 2007;25(3):372–83.

Sharma H. How short or long should be a questionnaire for any research? Researchers dilemma in deciding the appropriate questionnaire length. Saudi J Anaesth. 2022;16(1):65–8.

De Decker E, De Craemer M, Santos-Lozano A, Van Cauwenberghe E, De Bourdeaudhuij I, Cardon G. Validity of the ActivPAL ™ and the ActiGraph monitors in preschoolers. Med Sci Sports Exerc. 2013;45(10):2002.

Aguilar-Farias N, Martino-Fuentealba P, Chandia-Poblete D. Cultural adaptation, translation and validation of the Spanish version of past-day adults’ sedentary time. BMC Public Health. 2021;21(1):182.

Reid RER, Carver TE, Andersen KM, Court O, Andersen RE. Physical activity and sedentary behavior in bariatric patients long-term post-surgery. Obes Surg. 2015;25(6):1073–7.

Reid RER, Carver TE, Reid TGR, Picard-Turcot MA, Andersen KM, Christou NV, et al. Effects of neighborhood walkability on physical activity and sedentary behavior long-term post-bariatric surgery. Obes Surg. 2017;27(6):1589–94.

Download references

Acknowledgements

The authors would like to express gratitude to all participants for their involvement in the study. Additionally, we extend our appreciation to the research assistants (Rasil Alhadi, Ragad Alasiri, and Khalid Aldosari) who assisted in the data collection. Finally, we would like to thank the LHRC, Princess Nourah Bint Abdulrahman University for providing their site for collecting the data.

This research was supported by the Deanship of Scientific Research, Vice Presidency for Graduate Studies and Scientific Research, King Faisal University, Saudi Arabia [Project No. GrantA353]. The funders had no role in study design, data collection and analysis, the decision to publish, or the preparation of the manuscript.

Author information

Authors and affiliations.

Department of Physical Education, College of Education, King Faisal University, Al-Ahsa, 31982, Saudi Arabia

Abdulrahman I. Alaqil

Center for Active and Healthy Ageing (CAHA), Department of Sports Science and Clinical Biomechanics, University of Southern Denmark, Odense, 5230, Denmark

Abdulrahman I. Alaqil, Borja del Pozo Cruz & Paolo Caserotti

Department of Musculoskeletal Disorders and Physical Workload, National Research Centre for the Working Environment, Lersø Parkalle 105, Copenhagen, 2100, Denmark

Abdulrahman I. Alaqil, Andreas Holtermann & Nidhi Gupta

Faculty of Education, Department of Physical Education, University of Cádiz, Cádiz, Spain

Borja del Pozo Cruz

Biomedical Research and Innovation Institute of Cádiz (INiBICA) Research Unit, University of Cádiz, Cadiz, Spain

Lifestyle and Health Research Center, Health Sciences Research Center, Princess Nourah Bint Abdulrahman University, Riyadh, 11671, Saudi Arabia

Shaima A. Alothman & Hazzaa M. Al-Hazzaa

Mackenzie Wearables Research Hub, Charles Perkins Centre, The University of Sydney, Camperdown, NSW, Australia

Matthew N. Ahmadi & Emmanuel Stamatakis

School of Health Sciences, Faculty of Medicine and Health, The University of Sydney, Camperdown, NSW, Australia

School of Sports Sciences, University of Jordan, Amman, Jordan

Hazzaa M. Al-Hazzaa

You can also search for this author in PubMed   Google Scholar

Contributions

Conceptualization: AIA, NG, ES, and BdCMethodology: AIA, NG, ES, HMA, and BdCInvestigation: AIAData collection: AIAInterpretation of the findings: AIA, HMA, ES, NG, AH, PC, MNA, and BdCDrafting the paper: AIAReviewing and editing the draft: AIA, ES, HMA, BdC, SAA, PC, MNA, AH, and NGAll authors critically read, revised the draft for important intellectual content, approved the final version of the manuscript to be published, and agreed to be accountable for all aspects of the work.

Corresponding author

Correspondence to Abdulrahman I. Alaqil .

Ethics declarations

Ethics approval and consent to participate.

The Ethic approval was obtained from the Institutional Review Board at Princess Nourah Bint Abdul Rahman University, Riyadh, Saudi Arabia (IRB 22–0146). Written informed consent was obtained from participants. All methods were carried out in accordance with the Declaration of Helsinki.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary material 1., supplementary material 2., supplementary material 3., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Alaqil, A.I., del Pozo Cruz, B., Alothman, S.A. et al. Feasibility and acceptability of a cohort study baseline data collection of device-measured physical behaviors and cardiometabolic health in Saudi Arabia: expanding the Prospective Physical Activity, Sitting and Sleep consortium (ProPASS) in the Middle East. BMC Public Health 24 , 1379 (2024). https://doi.org/10.1186/s12889-024-18867-2

Download citation

Received : 12 September 2023

Accepted : 16 May 2024

Published : 22 May 2024

DOI : https://doi.org/10.1186/s12889-024-18867-2

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Feasibility
• Epidemiology
• Physical activity
• Physical behavior
• Sedentary behaviors
• Accelerometry
• Saudi adults

BMC Public Health

ISSN: 1471-2458

• General enquiries: [email protected]

Would you cross the Newport Pell Bridge on bike? A new study will examine the possibility

Could someone living in South Kingstown take their bike to their job in Newport? Not as it is now, but a new federally funded study could open up the possibilities of making the Pell and Verrazzano bridges bike and pedestrian-friendly.

In March, U.S. Sen. Sheldon Whitehouse announced that he helped secure $59.2 million in direct federal funding for 38 Rhode Island projects through a bipartisan minibus package of six fiscal 2024 appropriations bills. While one of the largest and most publicized projects included in this list was a $5 million grant for repairs to the Cliff Walk , a smaller project, labeled “feasibility study,” for the Pell and Verrazzano Bridges, was also included.

This $1.48 million feasibility study will investigate the possibility of adding bicycle lanes to both bridges, something Bike Newport and Whitehouse have been interested in for years. In 2021, Whitehouse and Bike Newport worked alongside graduate students from the Rhode Island School of Design to imagine how bike and pedestrian-friendly paths on the Pell Bridge might look.

“It’s a very exciting moment and we’re extremely grateful to Senator Whitehouse for shepherding this and President Biden for signing it,” Bike Newport Executive Director Bari Freeman said.

The conversation surrounding bike path on Pell Bridge goes back even further, including preliminary plans from the Rhode Island Bridge and Turnpike Authority in 2018 which concluded that adding pedestrian and bike access across the Pell, Verrazzano and Mount Hope bridges could be possible, but could cost between $25 and $50 million to complete.

These new federal funds for a new feasibility study will actually go to the Rhode Island Department of Transportation, which will distribute the funds to Bike Newport. It’s currently early in the process, with Bike Newport currently looking into who might be the best team to conduct the study.

“Whoever gets hired to do this is going to be given access to everything that’s happened until now, and that includes the RISD studies and other conversations, and there will be more community engagement and input,” Freeman said. “It’s going to be a very thorough study.”

Freeman anticipates wind as the main challenge that would impact bicyclists on a future possible Pell and Verrazzano bridge bike paths but said the RISD designs already considered wind as a factor.

“The first thing I see is opportunities,” Freeman said. “People need to be able to go across. They need to be able to do it in their own time on their own terms and not to have to schedule something in advance. That connectivity is going to be incredible.”

What are you looking for?

Most popular topics.

• Sustainable Aviation Fuel (SAF)

21 May 2024

Airbus, houston airports, center for houston’s future join forces to study feasibility of hydrogen hub at george bush intercontinental airport.

Houston, 21 May 2024 – Airbus, Houston Airports and the Center for Houston’s Future (CHF) have signed a Memorandum of Understanding to study the feasibility of a hydrogen hub at George Bush Intercontinental Airport (IAH). 

Airports are at the forefront of leading the transition towards the decarbonization of operations across the entire value chain.  Airbus’ “Hydrogen Hub at Airports'' concept brings together key airport ecosystem players to better understand hydrogen infrastructure needs for future aircraft and to develop a stepped approach to decarbonizing all airport-associated infrastructure using hydrogen. 

In this study, the partners will work together to rethink how their infrastructures could be designed and operated to reduce their overall environmental footprint and welcome future hydrogen-powered aircraft such as those Airbus plans to bring to the air in 2035. 

The study will focus on identifying opportunities for and barriers to hydrogen supply, infrastructure development, and usage at the airport and to potentially advance further developments. The scope of the study will include end-use in aviation, airport ground transportation, airport heating, and potentially supply to adjacent customers in transport and local industries.

The use of hydrogen to power future aircraft is not only expected to help eliminate aircraft CO2 emissions in the air, but could also help decarbonize air transport activities on the ground. This particular study reflects the partners’ ambition to use their respective expertise to support the decarbonization of the aviation industry:

• Airbus : Airbus pioneers sustainable aerospace for a safe and united world. The company constantly innovates to provide efficient and technologically-advanced solutions in aerospace. Airbus is currently developing the first hydrogen-powered commercial aircraft with the ambition to enter into service in 2035 and promoting the H2 Hubs at airport concept.
• Houston Airports:  The city provides a safe and dynamic air services network that fosters economic vitality for the transportation industry and facilitates a strong level of global connectivity for a diverse and growing population living throughout the greater Houston region. The Houston Airport System forms one of North America's largest public airport systems and positions Houston as the international passenger and cargo gateway to the south-central United States and a primary gateway to Latin America.
• Center for Houston’s Future:  The Center is leading a  broad-based effort to create a clean hydrogen ecosystem across the value chain  in the Houston region and across the Gulf Coast. The Center was a founding organizer of the HyVelocity Hydrogen Hub, selected by the U.S. Department of Energy as one of seven hydrogen hubs nationally.

“For hydrogen to meet its full potential, the entire airport ecosystem – including airport authorities, energy suppliers and regulatory authorities – needs to come together and collaborate,” said Karine Guenan, Airbus’ Vice President ZEROe Ecosystem. “This joint-study will help us better understand what hydrogen infrastructure would be needed at Houston’s airport to support hydrogen and low carbon aviation in the future. What we learn here can also support knowledge around the world.”

“Houston’s airports are experiencing tremendous growth, connecting our city to the world like never before,” said  Jim Szczesniak, the aviation director for the City of Houston. “As we continue to expand and modernize our facilities, participating in this sustainability study is crucial. Continuing to build a sustainable airport system will ensure a healthy future for Houston, attract top talent and businesses, and demonstrate our commitment to being a responsible global citizen. This study will provide us with valuable insights to guide our development and position Houston as a global leader in sustainable aviation innovation for generations to come.”

“The Center for Houston’s Future is pleased to have played a crucial role in bringing together the partners for this study. With Houston’s role as the world’s energy capital, our record of energy innovation and desire to lead in the business of low-carbon energy, Houston is the perfect place to develop our airports as North American clean hydrogen pioneers,” said Brett Perlman, the Center’s CEO and President.

The study in Houston is scheduled for completion at the end of March 2025.

Further information about the topic of hydrogen in aviation can be found at the following  link .  

@Airbus @IAH @futurehouston #hydrogen #ZEROe

Artist rendering: ZEROe hydrogen-powered aircraft over the city of Houston.

Your contact

Kristi Tucker

Airbus Americas, Inc.

Laura Goldberg

Center for Houston’s Future

Augusto Bernal

Houston Airports

Learn more about innovation at Airbus

Hydrogen hub at airport zeroe flying over the city of houston.

Png 5.48 MB

EN_PR_Houston H2 study May 2024

Pdf 4.91 MB

• Decarbonisation

See more Innovation news

New pilot assistance technologies take to the road with Airbus' Optimate demonstrator

Meet Optimate, an extra pair of eyes and ears for pilots

The Racer is a real winner!