research project management literature

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here .

Loading metrics

Open Access

Peer-reviewed

Research Article

How do project managers’ competencies impact project success? A systematic literature review

Roles Conceptualization, Supervision, Writing – review & editing

* E-mail: [email protected]

Affiliation ESPAE Graduate School of Management, Escuela Superior Politécnica del Litoral (ESPOL), Guayaquil, Ecuador

Roles Formal analysis, Methodology, Writing – original draft, Writing – review & editing

Roles Investigation, Methodology, Software, Writing – original draft

Roles Conceptualization, Formal analysis, Writing – review & editing

Roles Writing – review & editing

• Paola Ochoa Pacheco, 
• David Coello-Montecel, 
• Michelle Tello, 
• Virginia Lasio, 
• Alfredo Armijos

• Published: December 7, 2023
• https://doi.org/10.1371/journal.pone.0295417
• Peer Review
• Reader Comments

Despite the existence of systematic literature reviews focused on examining the factors contributing to project success, there remains a scarcity of reviews addressing the relationship between the project managers’ competencies and project success. To fill this gap in the literature, this review aimed to evaluate peer-reviewed articles, published between 2010 and 2022, and analyze the impact of project managers’ competencies on project success. The Web of Science, Scopus, ScienceDirect, and ProQuest electronic databases were first consulted in September 2021, with an update in August and October 2022. A total of 232 titles were analyzed. Ten articles met the criteria and were fully reviewed. A content analysis and a citation network were carried out to analyze the included articles. The analysis revealed that the existing literature has primarily explored the influence of competencies from the personal and social dimensions, such as leadership, communication, and emotional intelligence, on project success. Conversely, competencies from other dimensions have received less attention in the literature. In addition, this review contributes to the literature by providing a holistic categorization of competencies associated with project success and examining and organizing project success criteria into three dimensions.

Citation: Ochoa Pacheco P, Coello-Montecel D, Tello M, Lasio V, Armijos A (2023) How do project managers’ competencies impact project success? A systematic literature review. PLoS ONE 18(12): e0295417. https://doi.org/10.1371/journal.pone.0295417

Editor: Jamshid Ali, University of Tabouk: University of Tabuk, SAUDI ARABIA

Received: July 19, 2023; Accepted: November 21, 2023; Published: December 7, 2023

Copyright: © 2023 Ochoa Pacheco et al. This is an open access article distributed under the terms of the Creative Commons Attribution License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: All data are available either within the manuscript (Tables 1 , 2 , 3 , 4 ) or as supplementary files . Hyperlinks are provided within the manuscript in the reference list.

Funding: The authors received no specific funding for this work.

Competing interests: The authors declare no conflict of interest.

1. Introduction

The profound economic, technological, and social changes experienced in recent years [ 1 , 2 ] have compelled organizations to devise strategies and implement initiatives to adapt to uncertain environments [ 3 ]. Projects allow organizations to face these challenges by leveraging their expertise and capabilities to deliver solutions aligned with business objectives [ 4 ]. Project management (PM) has been acknowledged as a valuable discipline for managers and professionals implementing strategic organizational transformations [ 1 ]. Given the shortage of qualified talent to execute strategic initiatives and drive change [ 5 ], the project managers’ (PMGs) competencies have garnered significant attention from scholars [ 6 – 11 ] and PM institutions [ 12 , 13 ]. Consequently, a substantial body of literature has devoted considerable effort to delineating the competencies that have the potential to enhance projects’ positive outcomes [ 14 – 18 ].

There has been a growing interest in exploring the individual and organizational factors contributing to project success (PS). At the individual level, the PMGs’ leadership style [ 19 ], job satisfaction [ 20 ], trust [ 21 ], job crafting [ 22 ], and work-family conflict [ 23 ], among other factors have been associated with PS. At the organizational level, scholars have highlighted that PS can be influenced by innovative climate [ 24 ], organizational culture [ 25 ], cultural diversity [ 26 ], governance [ 27 ], knowledge sharing and perceived trust and cohesion of the team [ 28 , 29 ], among others.

Despite the existence of systematic literature reviews (SLRs) that summarize the available evidence regarding factors associated with PS [ 30 ], there remains a scarcity of SLRs focusing on PMGs’ competencies [ 31 , 32 ] and their impact on PS. Only a limited number of SLRs [ 33 ] have been dedicated to identifying the competencies essential for achieving PS. However, to the best of our knowledge, an SLR focused on analyzing the relationship between PMGs’ competencies and PS has not been conducted before. To fill this gap in the literature, this SLR analyzes the existing evidence regarding the relationship between PMGs’ competencies and PS. Therefore, the present SLR was designed to address the following research questions: (RQ1) Which PMGs’ competencies are the most examined in the existing literature? (RQ2) Which success criteria are the most considered when measuring PS in the existing literature? (RQ3) Which PMGs’ competencies have a relationship with PS?

This SLR contributes to the literature on the PM discipline in four ways. Firstly, it fills a gap in the existing literature by employing the SLR methodology to comprehensively synthesize the available evidence from published empirical studies concerning the relationship between PMGs’ competencies and PS. Secondly, it employs a thematic analysis and a holistic perspective to categorize the PMGs’ competencies associated with PS. This methodological approach provides a comprehensive framework for understanding the diverse competencies relevant to PS. Thirdly, it offers an insightful analysis of a graphical representation that showcases the primary authors and institutions that have significantly influenced the conceptualization of PMGs’ competencies. Lastly, it examines the criteria utilized for measuring PS in the included articles and organizes them into three dimensions, enhancing the understanding of the multifaceted nature of PS assessment. By addressing these aspects, this SLR contributes to advancing knowledge in PM.

The subsequent sections of this paper are structured as follows. Section 2 presents the conceptualization of PMGs’ competencies and PS. Section 3 outlines the procedure for conducting the SLR, encompassing the search strategy, study selection, data extraction, and analysis. The findings derived from the SLR are presented in Section 4. Lastly, the paper concludes by discussing the implications of the results, highlighting the strengths and limitations of the SLR, and offering final remarks.

2. Competencies and project success

This section provides an overview of the conceptualizations of competencies adopted in the PM literature, and briefly discuss the evolution of the PS dimensions.

2.1. Competencies

Various conceptualizations of competencies have been explored in the existing literature [ 16 , 34 – 38 ]. Within the PM discipline, several studies [ 18 , 39 – 42 ] have aligned with the classical definition proposed by Boyatzis [ 35 ]. According to his framework, competencies encompass the underlying characteristics of an individual, including knowledge, skills, abilities, attitudes, and more, that collectively enable the achievement of high performance. These elements have served as a foundational basis for scholars [ 9 , 43 , 44 ] and institutions [ 12 , 13 ], who have further expanded the scope to develop frameworks tailored explicitly to the domain of PM.

PM institutions, including the Project Management Institute (PMI) and the International Project Management Association (IPMA), have played a crucial role in the definition and development of various standards and frameworks that pertain to the competencies of PMGs [ 45 ]. Several studies [ 16 , 46 , 47 ] have employed these institutional standards to define competencies. The next paragraph provides a concise overview of these institutional frameworks.

According to the IPMA [ 13 ], competencies comprise the practical application of knowledges, skills, and abilities to achieve desired outcomes. This framework recognizes the interconnectedness of these elements, as proficiency entails acquiring relevant knowledge and developing skills that, when put into practice, enable professionals to manage projects effectively and successfully. Similarly, the PMI [ 12 ] defines competencies as the capability to carry out activities within a portfolio, program, or project setting that yield anticipated results based on established and accepted standards. This definition builds upon Boyatzis’ [ 35 ] elements and aligns with the IPMA [ 13 ] perspective, but it also emphasizes compliance by acknowledging the significance of adhering to current regulations and guidelines to meet stakeholders’ expectations. More recently, the PMI [ 48 ] introduced the concept of power skills , which refers to the abilities and behaviors that facilitate working with others and enable project professionals to succeed in the workplace, align projects to organizational objectives, and motivate teams to contribute value to the organization and its customers.

The scholarly literature [ 8 , 37 , 43 , 44 ] has significantly contributed to the conceptualization of the competencies required by PMGs by incorporating key elements from the PM discipline. For instance, Hanna et al. [ 43 ] emphasized the evolving nature of projects. They argued that competencies entail the demonstrated ability to perform project activities within a dynamic environment, leading to expected outcomes based on established standards. Building upon this perspective, Bashir et al. [ 44 ] defined competencies as a meta-ability that integrates skills, aptitudes, and abilities to perform throughout the project life cycle, from initiation to closing, intending to achieve expected results. Moreover, Crawford [ 49 ] posited a close relationship between PMGs’ competencies and PS. Recent literature has underscored the pivotal role of PMGs’ competencies in attaining higher levels of success, enhancing efficiency and effectiveness, and consequently increasing the likelihood of PS [ 8 ].

2.2. Project success

This section provides an overview of the historical development of the conceptualization of PS, tracing its progression from a unidimensional to a more comprehensive and multidimensional concept [ 50 ]. It also aims to identify the dimensions and criteria incorporated into the concept in recent years. Furthermore, it defines PS and examines its distinctions from related concepts, such as project performance and efficiency.

Traditionally, scholars [ 39 , 51 – 53 ] have viewed PS as a combination of success factors and criteria. On the one hand, success factors refer to the significant elements that enhance the probability of achieving success. On the other hand, success criteria comprise a set of measures used to evaluate if the project can be judged as successful [ 39 ]. This SLR specifically focuses on PS criteria.

The measurement criteria for assessing PS have undergone significant evolution to encompass the complex and dynamic nature of projects, resulting in the development of more comprehensive models [ 52 , 54 ]. Initially, PS frameworks primarily focused on efficiency criteria, commonly referred to as the “golden triangle,” “iron triangle,” or “holy trinity,” which encompassed elements such as time, cost, and quality [ 54 ]. Subsequent models expanded to incorporate dimensions of client and project team satisfaction [ 55 ]. From the year 2000 onwards, the emergence of integrative models took into account additional dimensions, including realized benefits to the business or organization [ 56 , 57 ], satisfaction levels of internal and external stakeholders such as end-users, suppliers, and other relevant parties [ 58 ], the impacts on the community and environment [ 59 ], long-term effects like the creation of new markets or product lines [ 56 , 60 ], and investment returns [ 61 ].

The conceptual boundaries between PS, project performance, project efficiency, and PM success have often been blurred. On the one hand, PM success represents a conventional measure of PS that primarily focuses on time, cost, and quality, assessed upon project completion [ 62 , 63 ]. These criteria are also called project efficiency [ 64 ]. On the other hand, project performance refers to the degree to which management practices and processes contribute to the achievement of goals and objectives, as well as the fulfillment of stakeholders’ expectations. It is typically evaluated throughout project execution and upon completion [ 54 , 65 ]. In contrast, PS represents a broader and multidimensional concept encompassing the achievement of goals and objectives determined by key stakeholders after project completion [ 63 , 64 ], as well as the long-term impacts of the project [ 66 ].

The SLR was undertaken to investigate the abovementioned research questions and followed the guidelines outlined in the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA). The protocol employed for conducting this SLR is elaborated next.

3.1. Search strategy

The Web of Science, Scopus, ScienceDirect, and ProQuest electronic databases were selected for this SLR. The databases were first consulted in September 2021, with an update in August and October 2022, by searching the following keywords in the title of the article: “competence,” “competency,” “competences,” “competencies,” “skill,” “skills,” and “project success,” without any additional constraint. The search was performed by two of the authors using the following search strings:

• Scopus database : TITLE ((competence) OR (competency) OR (competences) OR (competencies) OR (skill) OR (skills)) AND TITLE ((project AND success))
• Web of Science database : TI = (competence OR competency OR competences OR competencies OR skill OR skills) AND TI = (project success)
• ScienceDirect database : Title: (competence OR competency OR competences OR competencies OR skill OR skills) AND (project success)
• ProQuest database : title((competence OR competency OR competences OR competencies OR skill OR skills)) AND title((project success))

The metadata of the records (title, authors, document type, source title, author keywords, abstract, publication year, volume number, issue number, and DOI) was exported, compared, and saved on Microsoft Excel spreadsheets to remove duplicated studies and conduct the screening process.

3.2. Study selection

The study selection process comprised several stages to find relevant articles for the review. The initial research resulted in 232 articles. After removing duplicated records, 172 articles were considered for the next stages. The procedures followed by the authors are described below.

3.2.1. Inclusion and exclusion criteria.

The inclusion and exclusion criteria for document selection in this review were based on various factors, including publication timeline, document type, language, study type, population, and context. To be included in this review, documents had to meet the following criteria: (1) they had to be peer-reviewed scholarly research articles, (2) they had to be published between January 2010 and October 2022, (3) they had to be written in English, (4) they had to have a quantitative approach measuring PMG’s competencies as independent variable and PS as a dependent variable, (5) the study population had to consist of PMGs or similar positions (e.g., project director, project leader, senior PMG, department manager, functional manager, team leader), and (6) the research work had to be conducted in professional settings. The study selection process did not impose restrictions on industry, project type, or project size to ensure a broader scope and encompass various perspectives. This approach allowed for the retrieval of peer-reviewed scholarly articles that addressed the research questions of this SLR. Initially, 172 articles were evaluated, and after applying the inclusion criteria, 131 records were removed. Subsequently, 41 research articles remained for the screening process.

3.2.2. Article screening process.

After applying inclusion and exclusion criteria, the retained articles were screened by title, abstract, and full text. This process was conducted by two of the authors independently. The reasons for excluding articles were reported in each step. The exclusion criteria were objectively applied. Studies were excluded if the relationship between PMGs’ competencies and PS was not examined. Each reviewer’s number and list of excluded articles were compared after the screening. In those cases where there was disagreement between reviewers, a third author reviewed the article and discussed it with the other two authors to reach a consensus. Eligible articles were included in the final review. Ineligible articles were formally excluded, with the reasons for exclusion noted.

Out of 41 articles, seven were excluded based on the title. In this step, the main reasons for exclusion were: (a) the study was related to project-based learning ( n = 4), (b) the article was a literature review ( n = 3), and (c) the article was a case of study ( n = 1). The retained 34 articles were screened by abstract. After analyzing the abstract of each article, eight were removed because of the following: (a) the study had a qualitative design ( n = 1), (b) the article was a case of study ( n = 1), (c) the article analyzed only leadership styles ( n = 1), (d) the article was theoretical ( n = 2), (e) the study was not conducted in a PM professional setting ( n = 1), and (f) the article did not analyze the relationship between PMGs’ competencies and PS ( n = 2). Finally, the full-text screening was carried out on 26 articles. Thirteen records were excluded based on the following reasons: (a) PMGs’ competencies were not measured ( n = 3), (b) the article was theoretical ( n = 3), and (c) the study did not analyze the relationship between PMGs’ competencies and PS ( n = 7). After the whole screening process, 13 articles were considered for quality assessment.

3.2.3. Quality assessment.

The quality assessment focused on ten quality criteria statements: (1) The research questions, objectives, or hypothesis were appropriately established; (2) The study design was well described and appropriate for answering the research questions; (3) The sample and population of the study were clearly described, and its size was sufficient to carry out the proposed analysis; (4) The response rate was reported and above 50%; (5) The instruments used for measuring PMGs’ competencies were well described and design-based; (6) The instrument used for measuring PS was well described and design-based; (7) The statistical method was appropriate and sufficiently described to enable them to be repeated; (8) The research questions were adequately answered; (9) The statistical significance of associations was tested and reported; (10) The conclusions were clearly described and based on the results.

The abovementioned criteria were adapted from the Newcastle-Ottawa Quality Assessment Scale (adapted for cross-sectional studies), the Appraisal Tool for Cross-Sectional Studies (AXIS), and similar studies [ 67 ]. Each statement had three rating options coded as “Yes” (1 point), “No” (0 points), and “Partial” (0.5 points). Articles with a score of 7.5 points or higher were included in the final sample. The quality assessment was carried out by two authors independently. The results were compared, and the differences found were discussed to make a final decision. In this phase, three articles were removed. Ten articles were selected to conduct the analysis and answer the research questions of this SLR. Fig 1 summarizes the data extraction procedure through a PRISMA flow.

• PPT PowerPoint slide
• PNG larger image
• TIFF original image

https://doi.org/10.1371/journal.pone.0295417.g001

3.3. Data extraction and analysis

Three authors analyzed the articles for data extraction, including sample characteristics, country, setting, independent and outcome variable(s), data analysis procedures, and main findings. These data were synthesized in Table 4 .

A thematic analysis was conducted to identify the dimensions of PMGs’ competencies and PS criteria used in the included articles, following the procedures proposed by Nowell et al. [ 68 ]:

• Familiarization with the data . The authors read and analyzed the content of each article.
• Generation of initial code s. Each author generated a list of competencies and PS criteria extracted from each article. The resulting lists were compared and matched to get a final version.
• Creation of themes . Categories were created by grouping similar competencies and PS criteria. Each of the authors carried out this process individually. The resulting lists were compared and matched to get a final version as in the previous step.
• Definition and naming of themes . Once the final list of competencies and PS were obtained and the main categories were defined, each category was named based on theoretical foundations. This process was carried out jointly by the three authors.

When studying topics such as PMGs’ competencies, an important issue is how authors support their choice regarding what competencies to include in their work. This decision is important since it shapes the structure of the research field. Thus, a citation network analysis (CNA) was carried out to map the structure of the PMGs’ competencies research field. In CNA, research documents serve as nodes, and the connections between them are represented by citations [ 69 ]. CNA is a practical approach for identifying contributions to a specific topic and uncovering relationships within the scholarly literature, thereby revealing patterns of influence and collaboration [ 70 ]. In this SLR, the ten included articles relied on citations of prior works to select the pertinent PMGs’ competencies. These cited references were used to build a network representing the relevant frameworks in the included articles. The citation network was generated using the visNetwork package in RStudio.

4.1. Study characteristics

The main characteristics of the articles included in this SLR are shown in Table 1 . Data were collected from 11 countries across five regions: Asia, Europe, North America, Oceania, and South America. Notably, Pakistan emerged as the most prolific country, with five papers published between 2010 and 2022, followed by the USA ( n = 2) and Brazil ( n = 2). Most studies were published within the last five years ( n = 9). Out of the ten articles, eight were published in journals categorized in the Q1 ( n = 5) and Q2 ( n = 3) impact quartiles. In terms of study design, most articles employed a purely quantitative approach ( n = 8), while two utilized mixed methods. For instance, Sampaio et al. [ 71 ] conducted a systematic review to identify the competencies to be included in their subsequent questionnaire, while Podgórska and Pichlak [ 72 ] employed a mixed-method approach comprising semi-structured interviews and a survey questionnaire.

https://doi.org/10.1371/journal.pone.0295417.t001

4.2. Project managers’ competencies

4.2.1. the most influential theoretical frameworks..

The majority of articles ( n = 9) included in the SLR employed an existing framework to identify the PMGs’ competencies that were examined in their empirical analyses. However, in the study conducted by Sampaio et al. [ 71 ], a comprehensive literature review was undertaken to determine the specific competencies that should be considered for testing their impact on PS.

A CNA was conducted to explore the interrelationships among the ten articles included in this SLR and to identify the most influential frameworks for defining and determining the PMGs’ competencies. Fig 2 visually represents the articles included in the SLR as square nodes and the studies that have contributed to conceptualizing PMGs’ competencies as circle nodes. The size of each node reflects the number of citations it has received. The diagram layout was arranged such that the most frequently cited studies are positioned in the center, while less frequently cited ones are placed towards the periphery. A summary of the most influential works in PMGs’ competencies is provided below.

Notes. Square nodes represent the articles in the SLR ( n = 10), while circle nodes denote the studies that contributed to conceptualizing PMGs’ competencies. The number of citations gives the size of the node.

https://doi.org/10.1371/journal.pone.0295417.g002

Four articles included in this SLR [ 39 , 72 , 73 , 75 ] employed a common framework developed by Dulewicz and Higgs [ 79 ]. This framework encompasses 15 leadership competencies categorized into three dimensions: intellectual competencies (critical analysis and judgment, vision and imagination, strategic perspective), managerial competencies (managing resources, engaging communication, empowering, developing, achieving), and emotional competencies (self-awareness, emotional resilience, intuitiveness, interpersonal sensitivity, influence, motivation, conscientiousness). Additionally, two articles [ 15 , 77 ] drew upon Clarke’s [ 80 ] study, which identified four main PMGs’ competencies: communication, teamwork, attentiveness, and managing conflict. Other frameworks utilized in the SLR articles were proposed by Sunindijo [ 81 ], Katz [ 82 ], Nguyen and Hadikusumo [ 83 ], and Ofori [ 84 ]. These frameworks shared common elements, emphasizing the significance of communication, leadership, managing emotions, and interpersonal relationships as essential competencies for PMGs. Notably, the Project Manager Competency Development Framework [ 12 ] and the Individual Competence Baseline for Project Management [ 13 ] were among the most cited institutional frameworks employed in the SLR articles.

4.2.2. Categorization of project managers’ competencies.

Several common competencies were identified based on the review of competencies reported in each article. These competencies were categorized into four dimensions based in previous studies [ 11 , 31 , 85 ], as presented in Table 2 : cognitive, personal, social, and sustainability. It should be noted that not all competencies were consistently referred to by the same name across the included articles. Therefore, the names used to denote a specific competence in each article are listed in the third column of Table 2 .

https://doi.org/10.1371/journal.pone.0295417.t002

4.3. Project success criteria

Previous literature has traditionally focused on PS measures related to cost, time, and quality. However, the findings of the SLR indicate a growing tendency to incorporate a broader range of success criteria. Table 3 presents a categorization of the different success criteria reported in the included articles.

https://doi.org/10.1371/journal.pone.0295417.t003

The first dimension pertains to the impact on stakeholders, encompassing clients, users, providers, the project team, and other relevant parties. While stakeholder impact is commonly assessed through satisfaction measures, some studies consider alternative indicators such as the acceptability of the product, perceived benefits [ 73 ], or the fulfillment of stakeholder expectations [ 74 ]. Less frequently addressed are measures related to the impact on the organization, for which two criteria were identified: (i) visible short-term improvements in organizational outcomes or performance [ 73 , 75 , 76 ], and (ii) long-term improvements, such as the development of new technologies or the initiation of future projects [ 39 , 72 , 75 ]. Additional criteria related to the project management process were identified, encompassing project performance, achievement of the project’s primary objectives, other self-defined criteria related to project management, and compliance with procedures, safety regulations, and environmental standards. Project performance indicators include the traditional metrics of cost, time, and quality of the project’s deliverables [ 39 , 72 ].

4.4. Empirical analysis of the relationship between project managers’ competencies and project success

Table 4 presents a comprehensive overview of the research methods and results employed in the included studies. Several studies conducted correlational analyses to examine the relationship between competencies and various PS criteria [ 71 , 72 , 75 ], as well as overall PS [ 15 ]. Regression analysis was a common method to assess the predictive impact of PMGs’ competencies on PS criteria in the selected articles [ 39 , 72 , 74 , 75 ]. Additionally, some studies [ 73 , 74 , 76 , 78 ] employed structural equation modeling (SEM) or partial least squares (PLS) to analyze the predictive effect of PMGs’ competencies, modeled as second-order constructs, on PS.

https://doi.org/10.1371/journal.pone.0295417.t004

4.4.1. Relationship between cognitive competencies and project success.

Cognitive competencies encompassed creativity, decision-making, and strategic perspective. Findings from two studies revealed a positive correlation between creativity and various PS criteria, such as accomplishing project objectives, project efficiency, user satisfaction [ 71 ], and suppliers’ satisfaction [ 72 ]. While creativity significantly predicted project efficiency, its effect on achieving project objectives and user satisfaction was not statistically significant [ 71 ]. Two studies included in the SLR [ 39 , 72 ] provided evidence concerning the relationship between strategic perspective and PS. Firstly, Müller and Turner [ 39 ] found that this competence influences project efficiency and self-defined success criteria. Secondly, Podgórska & Pichlak [ 72 ] reported that strategic perspective is significantly associated with all the analyzed PS criteria except for project efficiency and self-defined success criteria. Regarding the decision-making competence, Müller and Turner [ 39 ] did not identify any significant predictive effects of this competence. However, Podgórska and Pichlak [ 72 ] observed significant positive correlations between decision-making and all the PS criteria, with the highest coefficients observed for self-defined success criteria, end-user satisfaction, and satisfaction of other stakeholders.

4.4.2. Relationship between personal competencies and project success.

Personal competencies included emotional intelligence, results orientation, and conscientiousness. Among these competencies, emotional intelligence has received significant attention in the included studies. Out of the ten studies, eight explored the relationship between emotional intelligence and PS. The evidence revealed direct and significant predictive effects of emotional intelligence on various PS criteria, such as end-user satisfaction [ 71 ], achievement of project objectives [ 39 ], and overall PS [ 15 ].

Regarding results orientation, Sampaio et al. [ 71 ] demonstrated that this competence had a predictive effect on project efficiency. Correlational analysis revealed a higher correlation between this competence with PS criteria related to user satisfaction [ 71 ] and satisfaction of other stakeholders [ 72 ]. The conscientiousness competence was examined in the studies conducted by Müller and Turner [ 39 ] and Podgórska and Pichlak [ 72 ]. This competence emerged as a significant predictor of team satisfaction and the achievement of project objectives [ 39 ]. Furthermore, the effects of conscientiousness could vary depending on the type and complexity of the project [ 72 ].

4.4.3. Relationship between social competencies and project success.

Based on the articles included in this review, social competencies, such as communication, leadership, interpersonal relations, conflict management, and teamwork, tend to be associated with PS. Leadership has been extensively studied in the project management literature and was addressed in seven out of the ten articles included in this review. Correlational analysis revealed that leadership shows significant associations with nearly all PS criteria, being its highest correlation with the user satisfaction criterion [ 71 , 72 ]. Regarding its predictive effect on PS, Sampaio et al. [ 71 ] reported a non-significant effect of this competence on some criteria, such as achieving project’s purpose, project efficiency, and stakeholders’ satisfaction, while other studies found a significant effect on team satisfaction criterion [ 39 ] and an overall PS measure [ 15 , 74 ].

Regarding communication, correlational analysis showed that this competence is highly correlated with stakeholders’ satisfaction criterion [ 71 , 72 ]. Maqbool et al. [ 15 ] found that communication had the strongest correlation with a general measure of PS among all competencies included in their study. The predictive effect of this competence on PS was confirmed by Lima and Quevedo-Silva [ 77 ], Khan et al. [ 78 ] and Podgórska and Pichlak [ 72 ]. However, Sampaio et al. [ 71 ] and Müller and Turner [ 39 ] reported non-significant effects of communication of PS criteria.

Interpersonal relations showed significant positive associations with PS criteria, with the strongest coefficients on achieving project’s purpose [ 72 ]. Müller and Turner [ 39 ] found that this competence has a significant predictive effect on other stakeholders’ satisfaction criteria, while two studies [ 77 , 78 ] reported its predictive effect on a general PS measure. Finally, significant positive associations of conflict management and teamwork with PS were reported by Maqbool et al. [ 15 ]. However, its predictive effect on individual PS criteria were not estimated on any of the included articles.

4.4.4. Relationship between sustainability competencies and project success.

According to Elmezain et al. [ 74 ], the capacity to demonstrate integrity, sincerity, and authenticity, and to inspire confidence and trust in others, is relevant for achieving PS. The authors emphasized that PMGs who possess integrity play a crucial role in the advancement of any project. Similarly, Sampaio et al. [ 71 ] highlighted ethics, conceptualized as transparency, integrity, and honesty, as the most significant competence for achieving PS in terms of goal attainment.

5. Discussion

This SLR examined the evidence pertaining to the relationship between PMGs’ competencies and PS. The analysis of the included studies yielded three key findings. Firstly, six distinct clusters of authors were identified, each contributing to the conceptualization and identification of PMGs’ competencies. Secondly, the conceptualization of PS has evolved from a traditional approach centered around criteria such as time, cost, and quality, to a more comprehensive, holistic, and multidimensional perspective. Lastly, through thematic analysis, a total of 12 competencies, organized into four dimensions, were identified as potential determinants of PS. Notably, the most significant competencies associated with PS were found within the personal and social dimensions. A brief discussion of these findings is presented below.

In relation to the first finding, this SLR identified six distinct clusters of authors whose work influenced the competence frameworks utilized in the included articles. These clusters represented conceptualizations proposed by scholars and reputable PM institutions. The in-depth content analysis revealed that the frameworks proposed by Dulewicz and Higgs [ 79 ] and Clarke [ 80 ] were the most prevalent among the examined articles. Conversely, frameworks developed by Sunindijo [ 81 ], Ofori [ 84 ], and Nguyen and Hadikusumo [ 83 ] were comparatively less frequently employed. Additionally, the PMI [ 12 ] emerged as a key institutional point of reference for identifying the competencies required in a PMG. For instance, Lima and Quevedo-Silva [ 77 ] and Maqbool et al. [ 15 ] studies adopted Clarke’s [ 80 ] framework, which was based on the PMI’s [ 12 ] (2017b) list of competencies. Elmezain et al. [ 74 ], who cited Sunindijo et al. [ 81 ] as their framework source, incorporated several competencies defined by the PMI [ 5 ], although the majority of these were technical.

Regarding the second finding, the articles examined in this SLR provided support for the view that PS should be understood as a multidimensional construct. This finding aligns with a recent study by Ika and Pinto [ 54 ] that revisited the conceptualization of PS. The results of this review indicate that project performance, encompassing time, cost, and quality, emerged as the most commonly considered criterion of success across all the articles. However, a significant number of the included articles also acknowledged additional criteria, leading to the identification of three dimensions of PS. The first dimension refers to the impact on stakeholders and includes criteria related to the satisfaction of various project stakeholders, including clients, users, suppliers, and the project team, among others. The second dimension focuses on the impact of the project on the organization, comprising both short- and long-term improvements. Lastly, the third dimension is related with the general management of the project. This dimension encompasses aspects such as project performance, which includes the traditional "iron triangle" of time, cost, and quality, as well as the achievement of project objectives, adherence to project-defined criteria, and compliance with safety and environmental protocols and regulations. This conceptualization supports the multidimensional nature of PS. However, as noted by Ika and Pinto [ 54 ], it is important to highlight that the majority of the included articles overlooked the inclusion of sustainability criteria. Among the entire sample of studies, only one [ 76 ] out of ten explicitly addressed compliance with safety and environmental regulations as a criterion of success.

The findings of this SLR have provided insights into the competencies that exhibit a significant relationship with PS. Specifically, the articles included in this review extensively examined competencies associated with the personal and social dimensions, such as leadership, communication, and emotional intelligence. These competencies have been extensively studied in previous literature [ 18 , 86 ], and their impact on PS was explored in the majority of the reviewed articles. Conversely, the influence of other competencies, such as ethics, received less attention and was not extensively explored. Moreover, the empirical evidence gathered in this review suggests that the effect of project management competencies on PS may vary depending on several factors. For instance, the type of project was found to be a significant factor influencing the relationship between competencies and PS [ 72 ]. Furthermore, individual and organizational factors were identified as potential mediating variables that could affect the relationship between competencies and PS [ 73 ]. These findings highlight the complexity and contextual nature of the relationship between competencies and PS. Next, a brief discussion will be presented to shed light on how these identified competencies can contribute to enhancing PS.

Leadership competence was one of the most studied competencies that improve PS. Although a few of the studies included in this SLR [ 71 , 73 ] reported that it does not have a significant effect on PS, a great number of the studies [ 15 , 39 , 72 , 74 , 76 ] suggested that PMGs’ leadership, conceived as their capacity to influence, empower and develop others, has a positive effect on PS. This finding agrees with the existing literature that has examined its influence on PS [ 87 – 90 ]. The development of competencies such as leadership allows PMGs to motivate their teams to be more productive [ 91 ], to show outstanding performance beyond expectations [ 89 ], to enhance team cohesion and engagement [ 92 ], to foster knowledge transfer across project teams [ 89 ], among other positive behaviors that would impact on projects’ outcomes.

The articles included in this SLR demonstrate a significant and positive relationship between communication and PS [ 15 , 72 , 77 , 78 ], in agreement with previous research findings [ 93 , 94 ]. The significance of this competence lies in its impact throughout various stages of a project [ 94 ]. Effective communication between PMGs and the project team’s members allows better collaboration [ 95 ], encourages knowledge sharing [ 96 ], and enhances the team’s motivation a sense of inclusivity [ 94 ], which contribute to the overall achievement of PS.

The influence of emotional intelligence on PS was assessed in most of the articles included in this SLR. Although some studies reported a non-significant relationship between emotional intelligence [ 71 , 73 , 77 ], there was evidence supporting a positive association between these two variables [ 15 , 39 , 72 , 75 ]. PMGs with high emotional intelligence are more likely to establish stronger relationships with their teams, thereby improving communication, clarity of mission, and support, ultimately enhancing PS [ 21 ]. In addition, the development of this competence allows PMGs to adequately regulate their emotions in complex situations, promoting positive behaviors such as empathy, respect, and leadership. These behaviors contribute to their ability to address challenges successfully and ensure higher PS [ 97 , 98 ].

Regarding the influence of PMGs’ ethics, a positive relationship was identified between this competence and PS criteria, particularly goal achievement [ 71 ]. Ethics has been acknowledged as a driving force for the advancement of the PM profession [ 48 ] and an essential competence that PMGs should possess [ 99 , 100 ]. However, empirical evidence on the impact of ethics on PS remains limited. Some related terms, such as honesty, integrity, and transparency [ 71 , 74 ], or ethical thinking [ 100 ], ethical decision-making, and ethics sensitivity have been addressed in previous studies. However, its effect on PS has rarely been estimated and reported. The evidence found on ethics in this SLR was obtained from information systems and construction projects. Future studies could explore the influence of this competence in different industries and countries.

6. Limitations and strengths

While this review contributes with some insights to the PM literature, it is important to mention its limitations. Firstly, the time frame of the review from 2010 to 2022 may have resulted in the exclusion of relevant articles that explore the relationship between PMGs’ competencies and PS. It is possible that some studies conducted outside this timeframe may provide further insights into the topic. Secondly, the use of specific search terms such as “competence,” “competency,” “competences,” “competencies,” “skill,” and “skills” may have excluded other studies [ 86 , 88 ] that examined the impact of different competencies individually. Including an exhaustive list of competencies in the search strings could have introduced significant heterogeneity into the reviewed articles, potentially limiting the ability to provide a comprehensive review of the existing literature.

Despite these limitations, this SLR makes several notable contributions to the PM discipline. First, it fills a gap in the existing literature by synthesizing available empirical evidence on the relationship between PMGs’ competencies and PS. Second, the review conducts a thematic analysis and adopts a holistic perspective to categorize the PMGs’ competencies that are associated with PS. Third, this review highlights the primary authors and PM institutions that have significantly influenced the conceptualization of PMGs’ competencies. Four, the review examines the criteria used to measure PS in the included articles and organizes them into three dimensions, offering a nuanced understanding of the multifaceted nature of PS measurement.

7. Conclusions

The present SLR extends the literature in project management concerning the influence of PMGs’ competencies on PS. Despite the growing interest in addressing the role of PMGs’ competencies to achieve higher success, to the best of our knowledge, there is a lack of systematic reviews that present an analysis of the available evidence on the relationship between PMGs’ competencies on PS. To fill this gap in the literature, this SLR analyzed the existing evidence regarding this relationship. Three main conclusions can be derived from the findings of this review. First, the existing literature has primarily explored the influence on PS of PMGs’ competencies from the personal and social dimensions, such as leadership, communication, and emotional intelligence. Second, PS is a multidimensional construct that comprises three main dimensions: impact on stakeholders, impact on the organization, and general project management. Third, the available data suggested that greater levels of PMGs’ competencies are associated with improved PS. These findings may support scholars and managers to understand the mechanisms through which individual characteristics, such as competencies, may allow PMGs to achieve better outcomes.

This SLR contributes to the existing literature in the PM discipline by offering a comprehensive synthesis of empirical evidence, providing a thorough overview of the current state of knowledge regarding the relationship between PMGs’ competencies and PS. In addition, this SLR identifies key contributors and sources of knowledge in the field, offering a valuable reference point for further research and exploration. The study also offers a review on how PS is conceptualized and measured. Moreover, it presents a classification of PMGs’ competencies that influence PS. Through a thematic analysis of the competencies examined in the included articles, this categorization provides valuable insights into the emphasis placed on different types of competencies. It highlights the significant attention given to personal and social competencies, while pointing out the relatively limited exploration of sustainability, cultural, or digital competencies [ 85 ].

Supporting information

S1 checklist. prisma 2020 checklist..

https://doi.org/10.1371/journal.pone.0295417.s001

S1 Table. Inclusion and exclusion criteria used in the SLR.

Notes: PMG = Project manager, PS = Project success.

https://doi.org/10.1371/journal.pone.0295417.s002

S2 Table. Quality assessment criteria scoring guide.

https://doi.org/10.1371/journal.pone.0295417.s003

S3 Table. Quality assessment results.

Notes: QC1 = Research questions; QC2 = Study design; QC3 = Sample representativeness; QC4 = Response rate; QC5 = PMG’s competencies measurement; QC6 = PS measurement; QC7 = Statistical analysis; QC8 = Results; QC9 = Statistical significance; QC10 = Conclusions; SLR = Systematic literature review.

https://doi.org/10.1371/journal.pone.0295417.s004

S4 Table. Brief description of the Project Managers’ competencies in included articles.

https://doi.org/10.1371/journal.pone.0295417.s005

S5 Table. Brief description of the project success criteria in included articles.

https://doi.org/10.1371/journal.pone.0295417.s006

• View Article
• Google Scholar
• 12. Project Management Institute. Project Manager Competency Development Framework. 3rd ed. Newtown Square (PA): Project Management Institute, Inc.; 2017.
• 13. International Project Management Association. Individual Competence Baseline for Project, Programme and Portfolio Management (Version 4.0.). Zurich: International Project Management Association; 2015.
• PubMed/NCBI
• 33. Iriarte C, Bayona Orè S. Soft Skills for IT Project Success: A Systematic Literature Review. In: Mejia J, Muñoz M, Rocha Á, Quiñonez Y, Calvo-Manzano J, editors. Trends and Applications in Software Engineering. Cham: Springer International Publishing; 2018. pp. 147–158. https://doi.org/10.1007/978-3-319-69341-5_14
• 34. Goleman D, Boyatzis R, McKee A. Primal Leadership: Unleashing the Power of Emotional Intelligence. Boston: Harvard Business Press; 2013.
• 35. Boyatzis R. The Competent Manager: A Model for Effective Performance. New York: John Wiley & Sons; 1982.
• 60. Shenhar AJ, Dvir D. Reinventing project management: the diamond approach to successful growth and innovation. Boston: Harvard Business Review Press; 2007.
• 65. Crawford L. Measuring Performance. In: Turner R, editor. Gower Handbook of Project Management. 5th ed. New York: Routledge; 2014. pp. 105–120.
• 85. Ochoa P, Jáuregui K, Gomes T, Ruiz B, Lasio V. Las competencias laborales en el mercado de profesionales de América Latina: ¿Qué destrezas demandarán las empresas del futuro en Chile, Colombia, Ecuador y el Perú? Lima: ESAN Ediciones; 2018.

• DOI: 10.1016/J.IJPROMAN.2015.02.001
• Corpus ID: 14119706

Managing collaborative research projects: A synthesis of project management literature and directives for future research

• J. Brocke , S. Lippe
• Published 1 July 2015
• Business, Education
• International Journal of Project Management

146 Citations

Features of research project management in european higher education institutes, project management practices for collaborative university-industry r&d: a hybrid approach, a quantitative study to assess a program and project management approach for collaborative university-industry r&d funded contracts, project management: a help or hindrance a collaborative research, configurations of project management practices to enhance the performance of open innovation r&d projects, benefits management in university-industry collaboration programs, value of project management in university–industry r&d collaborations, orchestrating collaborative projects: inside ict networks in horizon 2020, stakeholder management in university-industry collaboration programs: a case study, collaborative research competencies in supply chain management: the role of boundary spanning and reflexivity, 73 references, exploration and project management, managing collaborative r&d projects development of a practical management tool, a framework for structuring interdisciplinary research management, investigating differences among research projects and implications for managers, development of a project management model for a government research and development organization, systems thinking in innovation project management: a match that works, exploring the role of national culture in the management of large-scale international science projects, can research be project managed, inter-organisational projects in french innovation clusters: the construction of collaboration, defining uncertainty in projects – a new perspective, related papers.

Showing 1 through 3 of 0 Related Papers

Advertisement

How to conduct systematic literature reviews in management research: a guide in 6 steps and 14 decisions

• Review Paper
• Open access
• Published: 12 May 2023
• Volume 17 , pages 1899–1933, ( 2023 )

Cite this article

You have full access to this open access article

• Philipp C. Sauer   ORCID: orcid.org/0000-0002-1823-0723 1 &
• Stefan Seuring   ORCID: orcid.org/0000-0003-4204-9948 2  

26k Accesses

50 Citations

6 Altmetric

Explore all metrics

Systematic literature reviews (SLRs) have become a standard tool in many fields of management research but are often considerably less stringently presented than other pieces of research. The resulting lack of replicability of the research and conclusions has spurred a vital debate on the SLR process, but related guidance is scattered across a number of core references and is overly centered on the design and conduct of the SLR, while failing to guide researchers in crafting and presenting their findings in an impactful way. This paper offers an integrative review of the widely applied and most recent SLR guidelines in the management domain. The paper adopts a well-established six-step SLR process and refines it by sub-dividing the steps into 14 distinct decisions: (1) from the research question, via (2) characteristics of the primary studies, (3) to retrieving a sample of relevant literature, which is then (4) selected and (5) synthesized so that, finally (6), the results can be reported. Guided by these steps and decisions, prior SLR guidelines are critically reviewed, gaps are identified, and a synthesis is offered. This synthesis elaborates mainly on the gaps while pointing the reader toward the available guidelines. The paper thereby avoids reproducing existing guidance but critically enriches it. The 6 steps and 14 decisions provide methodological, theoretical, and practical guidelines along the SLR process, exemplifying them via best-practice examples and revealing their temporal sequence and main interrelations. The paper guides researchers in the process of designing, executing, and publishing a theory-based and impact-oriented SLR.

Similar content being viewed by others

The burgeoning role of literature review articles in management research: an introduction and outlook

On being ‘systematic’ in literature reviews

On being ‘systematic’ in literature reviews in IS

Avoid common mistakes on your manuscript.

1 Introduction

The application of systematic or structured literature reviews (SLRs) has developed into an established approach in the management domain (Kraus et al. 2020 ), with 90% of management-related SLRs published within the last 10 years (Clark et al. 2021 ). Such reviews help to condense knowledge in the field and point to future research directions, thereby enabling theory development (Fink 2010 ; Koufteros et al. 2018 ). SLRs have become an established method by now (e.g., Durach et al. 2017 ; Koufteros et al. 2018 ). However, many SLR authors struggle to efficiently synthesize and apply review protocols and justify their decisions throughout the review process (Paul et al. 2021 ) since only a few studies address and explain the respective research process and the decisions to be taken in this process. Moreover, the available guidelines do not form a coherent body of literature but focus on the different details of an SLR, while a comprehensive and detailed SLR process model is lacking. For example, Seuring and Gold ( 2012 ) provide some insights into the overall process, focusing on content analysis for data analysis without covering the practicalities of the research process in detail. Similarly, Durach et al. ( 2017 ) address SLRs from a paradigmatic perspective, offering a more foundational view covering ontological and epistemological positions. Durach et al. ( 2017 ) emphasize the philosophy of science foundations of an SLR. Although somewhat similar guidelines for SLRs might be found in the wider body of literature (Denyer and Tranfield 2009 ; Fink 2010 ; Snyder 2019 ), they often take a particular focus and are less geared toward explaining and reflecting on the single choices being made during the research process. The current body of SLR guidelines leaves it to the reader to find the right links among the guidelines and to justify their inconsistencies. This is critical since a vast number of SLRs are conducted by early-stage researchers who likely struggle to synthesize the existing guidance and best practices (Fisch and Block 2018 ; Kraus et al. 2020 ), leading to the frustration of authors, reviewers, editors, and readers alike.

Filling these gaps is critical in our eyes since researchers conducting literature reviews form the foundation of any kind of further analysis to position their research into the respective field (Fink 2010 ). So-called “systematic literature reviews” (e.g., Davis and Crombie 2001 ; Denyer and Tranfield 2009 ; Durach et al. 2017 ) or “structured literature reviews” (e.g., Koufteros et al. 2018 ; Miemczyk et al. 2012 ) differ from nonsystematic literature reviews in that the analysis of a certain body of literature becomes a means in itself (Kraus et al. 2020 ; Seuring et al. 2021 ). Although two different terms are used for this approach, the related studies refer to the same core methodological references that are also cited in this paper. Therefore, we see them as identical and abbreviate them as SLR.

There are several guidelines on such reviews already, which have been developed outside the management area (e.g. Fink 2010 ) or with a particular focus on one management domain (e.g., Kraus et al. 2020 ). SLRs aim at capturing the content of the field at a point in time but should also aim at informing future research (Denyer and Tranfield 2009 ), making follow-up research more efficient and productive (Kraus et al. 2021 ). Such standalone literature reviews would and should also prepare subsequent empirical or modeling research, but usually, they require far more effort and time (Fisch and Block 2018 ; Lim et al. 2022 ). To achieve this preparation, SLRs can essentially a) describe the state of the literature, b) test a hypothesis based on the available literature, c) extend the literature, and d) critique the literature (Xiao and Watson 2019 ). Beyond guiding the next incremental step in research, SLRs “may challenge established assumptions and norms of a given field or topic, recognize critical problems and factual errors, and stimulate future scientific conversations around that topic” (Kraus et al. 2022 , p. 2578). Moreover, they have the power to answer research questions that are beyond the scope of individual empirical or modeling studies (Snyder 2019 ) and to build, elaborate, and test theories beyond this single study scope (Seuring et al. 2021 ). These contributions of an SLR may be highly influential and therefore underline the need for high-quality planning, execution, and reporting of their process and details.

Regardless of the individual aims of standalone SLRs, their numbers have exponentially risen in the last two decades (Kraus et al. 2022 ) and almost all PhD or large research project proposals in the management domain include such a standalone SLR to build a solid foundation for their subsequent work packages. Standalone SLRs have thus become a key part of management research (Kraus et al. 2021 ; Seuring et al. 2021 ), which is also underlined by the fact that there are journals and special issues exclusively accepting standalone SLRs (Kraus et al. 2022 ; Lim et al. 2022 ).

However, SLRs require a commitment that is often comparable to an additional research process or project. Hence, SLRs should not be taken as a quick solution, as a simplistic, descriptive approach would usually not yield a publishable paper (see also Denyer and Tranfield 2009 ; Kraus et al. 2020 ).

Furthermore, as with other research techniques, SLRs are based on the rigorous application of rules and procedures, as well as on ensuring the validity and reliability of the method (Fisch and Block 2018 ; Seuring et al. 2021 ). In effect, there is a need to ensure “the same level of rigour to reviewing research evidence as should be used in producing that research evidence in the first place” (Davis and Crombie 2001 , p.1). This rigor holds for all steps of the research process, such as establishing the research question, collecting data, analyzing it, and making sense of the findings (Durach et al. 2017 ; Fink 2010 ; Seuring and Gold 2012 ). However, there is a high degree of diversity where some would be justified, while some papers do not report the full details of the research process. This lack of detail contrasts with an SLR’s aim of creating a valid map of the currently available research in the reviewed field, as critical information on the review’s completeness and potential reviewer biases cannot be judged by the reader or reviewer. This further impedes later replications or extensions of such reviews, which could provide longitudinal evidence of the development of a field (Denyer and Tranfield 2009 ; Durach et al. 2017 ). Against this observation, this paper addresses the following question:

Which decisions need to be made in an SLR process, and what practical guidelines can be put forward for making these decisions?

Answering this question, the key contributions of this paper are fourfold: (1) identifying the gaps in existing SLR guidelines, (2) refining the SLR process model by Durach et al. ( 2017 ) through 14 decisions, (3) synthesizing and enriching guidelines for these decisions, exemplifying the key decisions by means of best practice SLRs, and (4) presenting and discussing a refined SLR process model.

In some cases, we point to examples from operations and supply chain management. However, they illustrate the purposes discussed in the respective sections. We carefully checked that the arguments held for all fields of management-related research, and multiple examples from other fields of management were also included.

2 Identification of the need for an enriched process model, including a set of sequential decisions and their interrelations

In line with the exponential increase in SLR papers (Kraus et al. 2022 ), multiple SLR guidelines have recently been published. Since 2020, we have found a total of 10 papers offering guidelines on SLRs and other reviews for the field of management in general or some of its sub-fields. These guidelines are of double interest to this paper since we aim to complement them to fill the gap identified in the introduction while minimizing the doubling of efforts. Table 1 lists the 10 most recent guidelines and highlights their characteristics, research objectives, contributions, and how our paper aims to complement these previous contributions.

The sheer number and diversity of guideline papers, as well as the relevance expressed in them, underline the need for a comprehensive and exhaustive process model. At the same time, the guidelines take specific foci on, for example, updating earlier guidelines to new technological potentials (Kraus et al. 2020 ), clarifying the foundational elements of SLRs (Kraus et al. 2022 ) and proposing a review protocol (Paul et al. 2021 ) or the application and development of theory in SLRs (Seuring et al. 2021 ). Each of these foci fills an entire paper, while the authors acknowledge that much more needs to be considered in an SLR. Working through these most recent guidelines, it becomes obvious that the common paper formats in the management domain create a tension for guideline papers between elaborating on a) the SLR process and b) the details, options, and potentials of individual process steps.

Our analysis in Table 1 evidences that there are a number of rich contributions on aspect b), while the aspect a) of SLR process models has not received the same attention despite the substantial confusion of authors toward them (Paul et al. 2021 ). In fact, only two of the most recent guidelines approach SLR process models. First, Kraus et al. ( 2020 ) incrementally extended the 20-year-old Tranfield et al. ( 2003 ) three-stage model into four stages. A little later, Paul et al. ( 2021 ) proposed a three-stage (including six sub-stages) SPAR-4-SLR review protocol. It integrates the PRISMA reporting items (Moher et al. 2009 ; Page et al. 2021 ) that originate from clinical research to define 14 actions stating what items an SLR in management needs to report for reasons of validity, reliability, and replicability. Almost naturally, these 14 reporting-oriented actions mainly relate to the first SLR stage of “assembling the literature,” which accounts for nine of the 14 actions. Since this protocol is published in a special issue editorial, its presentation and elaboration are somewhat limited by the already mentioned word count limit. Nevertheless, the SPAR-4-SLR protocol provides a very useful checklist for researchers that enables them to include all data required to document the SLR and to avoid confusion from editors, reviewers, and readers regarding SLR characteristics.

Beyond Table 1 , Durach et al. ( 2017 ) synthesized six common SLR “steps” that differ only marginally in the delimitation of one step to another from the sub-stages of the previously mentioned SLR processes. In addition, Snyder ( 2019 ) proposed a process comprising four “phases” that take more of a bird’s perspective in addressing (1) design, (2) conduct, (3) analysis, and (4) structuring and writing the review. Moreover, Xiao and Watson ( 2019 ) proposed only three “stages” of (1) planning, (2) conducting, and (3) reporting the review that combines the previously mentioned conduct and the analysis and defines eight steps within them. Much in line with the other process models, the final reporting stage contains only one of the eight steps, leaving the reader somewhat alone in how to effectively craft a manuscript that contributes to the further development of the field.

In effect, the mentioned SLR processes differ only marginally, while the systematic nature of actions in the SPAR-4-SLR protocol (Paul et al. 2021 ) can be seen as a reporting must-have within any of the mentioned SLR processes. The similarity of the SLR processes is, however, also evident in the fact that they leave open how the SLR analysis can be executed, enriched, and reflected to make a contribution to the reviewed field. In contrast, this aspect is richly described in the other guidelines that do not offer an SLR process, leading us again toward the tension for guideline papers between elaborating on a) the SLR process and b) the details, options, and potentials of each process step.

To help (prospective) SLR authors successfully navigate this tension of existing guidelines, it is thus the ambition of this paper to adopt a comprehensive SLR process model along which an SLR project can be planned, executed, and written up in a coherent way. To enable this coherence, 14 distinct decisions are defined, reflected, and interlinked, which have to be taken across the different steps of the SLR process. At the same time, our process model aims to actively direct researchers to the best practices, tips, and guidance that previous guidelines have provided for individual decisions. We aim to achieve this by means of an integrative review of the relevant SLR guidelines, as outlined in the following section.

3 Methodology: an integrative literature review of guidelines for systematic literature reviews in management

It might seem intuitive to contribute to the debate on the “gold standard” of systematic literature reviews (Davis et al. 2014 ) by conducting a systematic review ourselves. However, there are different types of reviews aiming for distinctive contributions. Snyder ( 2019 ) distinguished between a) systematic, b) semi-systematic, and c) integrative (or critical) reviews, which aim for i) (mostly quantitative) synthesis and comparison of prior (primary) evidence, ii) an overview of the development of a field over time, and iii) a critique and synthesis of prior perspectives to reconceptualize or advance them. Each review team needs to position itself in such a typology of reviews to define the aims and scope of the review. To do so and structure the related research process, we adopted the four generic steps for an (integrative) literature review by Snyder ( 2019 )—(1) design, (2) conduct, (3) analysis, and (4) structuring and writing the review—on which we report in the remainder of this section. Since the last step is a very practical one that, for example, asks, “Is the contribution of the review clearly communicated?” (Snyder 2019 ), we will focus on the presentation of the method applied to the initial three steps:

(1) Regarding the design, we see the need for this study emerging from our experience in reviewing SLR manuscripts, supervising PhD students who, almost by default, need to prepare an SLR, and recurring discussions on certain decisions in the process of both. These discussions regularly left some blank or blurry spaces (see Table 1 ) that induced substantial uncertainty regarding critical decisions in the SLR process (Paul et al. 2021 ). To address this gap, we aim to synthesize prior guidance and critically enrich it, thus adopting an integrative approach for reviewing existing SLR guidance in the management domain (Snyder 2019 ).

(2) To conduct the review, we started collecting the literature that provided guidance on the individual SLR parts. We built on a sample of 13 regularly cited or very recent papers in the management domain. We started with core articles that we successfully used to publish SLRs in top-tier OSCM journals, such as Tranfield et al. ( 2003 ) and Durach et al. ( 2017 ), and we checked their references and papers that cited these publications. The search focus was defined by the following criteria: the articles needed to a) provide original methodological guidance for SLRs by providing new aspects of the guideline or synthesizing existing ones into more valid guidelines and b) focus on the management domain. Building on the nature of a critical or integrative review that does not require a full or representative sample (Snyder 2019 ), we limited the sample to the papers displayed in Table 2 that built the core of the currently applied SLR guidelines. In effect, we found 11 technical papers and two SLRs of SLRs (Carter and Washispack 2018 ; Seuring and Gold 2012 ). From the latter, we mainly analyzed the discussion and conclusion parts that explicitly developed guidance on conducting SLRs.

(3) For analyzing these papers, we first adopted the six-step SLR process proposed by Durach et al. ( 2017 , p.70), which they define as applicable to any “field, discipline or philosophical perspective”. The contrast between the six-step SLR process used for the analysis and the four-step process applied by ourselves may seem surprising but is justified by the use of an integrative approach. This approach differs mainly in retrieving and selecting pertinent literature that is key to SLRs and thus needs to be part of the analysis framework.

While deductively coding the sample papers against Durach et al.’s ( 2017 ), guidance in the six steps, we inductively built a set of 14 decisions presented in the right columns of Table 2 that are required to be made in any SLR. These decisions built a second and more detailed level of analysis, for which the single guidelines were coded as giving low, medium, or high levels of detail (see Table 3 ), which helped us identify the gaps in the current guidance papers and led our way in presenting, critically discussing, and enriching the literature. In effect, we see that almost all guidelines touch on the same issues and try to give a comprehensive overview. However, this results in multiple guidelines that all lack the space to go into detail, while only a few guidelines focus on filling a gap in the process. It is our ambition with this analysis to identify the gaps in the guidelines, thereby identifying a precise need for refinement, and to offer a first step into this refinement. Adopting advice from the literature sample, the coding was conducted by the entire author team (Snyder 2019 ; Tranfield et al. 2003 ) including discursive alignments of interpretation (Seuring and Gold 2012 ). This enabled a certain reliability and validity of the analysis by reducing the within-study and expectancy bias (Durach et al. 2017 ), while the replicability was supported by reporting the review sample and the coding results in Table 3 (Carter and Washispack 2018 ).

(4) For the writing of the review, we only pointed to the unusual structure of presenting the method without a theory section and then the findings in the following section. However, this was motivated by the nature of the integrative review so that the review findings at the same time represent the “state of the art,” “literature review,” or “conceptualization” sections of a paper.

4 Findings of the integrative review: presentation, critical discussion, and enrichment of prior guidance

4.1 the overall research process for a systematic literature review.

Even within our sample of only 13 guidelines, there are four distinct suggestions for structuring the SLR process. One of the earliest SLR process models was proposed by Tranfield et al. ( 2003 ) encompassing the three stages of (1) planning the review, (2) conducting a review, and (3) reporting and dissemination. Snyder ( 2019 ) proposed four steps employed in this study: (1) design, (2) conduct, (3) analysis, and (4) structuring and writing the review. Borrowing from content analysis guidelines, Seuring and Gold ( 2012 ) defined four steps: (1) material collection, (2) descriptive analysis, (3) category selection, and (4) material evaluation. Most recently Kraus et al. ( 2020 ) proposed four steps: (1) planning the review, (2) identifying and evaluating studies, (3) extracting and synthesizing data, and (4) disseminating the review findings. Most comprehensively, Durach et al. ( 2017 ) condensed prior process models into their generic six steps for an SLR. Adding the review of the process models by Snyder ( 2019 ) and Seuring and Gold ( 2012 ) to Durach et al.’s ( 2017 ) SLR process review of four papers, we support their conclusion of the general applicability of the six steps defined. Consequently, these six steps form the backbone of our coding scheme, as shown in the left column of Table 2 and described in the middle column.

As stated in Sect.  3 , we synthesized the review papers against these six steps but experienced that the papers were taking substantially different foci by providing rich details for some steps while largely bypassing others. To capture this heterogeneity and better operationalize the SLR process, we inductively introduced the right column, identifying 14 decisions to be made. These decisions are all elaborated in the reviewed papers but to substantially different extents, as the detailed coding results in Table 3 underline.

Mapping Table 3 for potential gaps in the existing guidelines, we found six decisions on which we found only low- to medium-level details, while high-detail elaboration was missing. These six decisions, which are illustrated in Fig.  1 , belong to three steps: 1: defining the research question, 5: synthesizing the literature, and 6: reporting the results. This result underscores our critique of currently unbalanced guidance that is, on the one hand, detailed on determining the required characteristics of primary studies (step 2), retrieving a sample of potentially relevant literature (step 3), and selecting the pertinent literature (step 4). On the other hand, authors, especially PhD students, are left without substantial guidance on the steps critical to publication. Instead, they are called “to go one step further … and derive meaningful conclusions” (Fisch and Block 2018 , p. 105) without further operationalizations on how this can be achieved; for example, how “meet the editor” conference sessions regularly cause frustration among PhDs when editors call for “new,” “bold,” and “relevant” research. Filling the gaps in the six decisions with best practice examples and practical experience is the main focus of this study’s contribution. The other eight decisions are synthesized with references to the guidelines that are most helpful and relevant for the respective step in our eyes.

The 6 steps and 14 decisions of the SLR process

4.2 Step 1: defining the research question

When initiating a research project, researchers make three key decisions.

Decision 1 considers the essential tasks of establishing a relevant and timely research question, but despite the importance of the decision, which determines large parts of further decisions (Snyder 2019 ; Tranfield et al. 2003 ), we only find scattered guidance in the literature. Hence, how can a research topic be specified to allow a strong literature review that is neither too narrow nor too broad? The latter is the danger in meta-reviews (i.e., reviews of reviews) (Aguinis et al. 2020 ; Carter and Washispack 2018 ; Kache and Seuring 2014 ). In this respect, even though the method would be robust, the findings would not be novel. In line with Carter and Washispack ( 2018 ), there should always be room for new reviews, yet over time, they must move from a descriptive overview of a field further into depth and provide detailed analyses of constructs. Clark et al. ( 2021 ) provided a detailed but very specific reflection on how they crafted a research question for an SLR and that revisiting the research question multiple times throughout the SLR process helps to coherently and efficiently move forward with the research. More generically, Kraus et al. ( 2020 ) listed six key contributions of an SLR that can guide the definition of the research question. Finally, Snyder ( 2019 ) suggested moving into more detail from existing SLRs and specified two main avenues for crafting an SLR research question that are either investigating the relationship among multiple effects, the effect of (a) specific variable(s), or mapping the evidence regarding a certain research area. For the latter, we see three possible alternative approaches, starting with a focus on certain industries. Examples are analyses of the food industry (Beske et al. 2014 ), retailing (Wiese et al. 2012 ), mining and minerals (Sauer and Seuring 2017 ), or perishable product supply chains (Lusiantoro et al. 2018 ) and traceability at the example of the apparel industry (Garcia-Torres et al. 2019 ). A second opportunity would be to assess the status of research in a geographical area that composes an interesting context from a research perspective, such as sustainable supply chain management (SSCM) in Latin America (Fritz and Silva 2018 ), yet this has to be justified explicitly, avoiding the fact that geographical focus is taken as the reason per se (e.g., Crane et al. 2016 ). A third variant addresses emerging issues, such as SCM, in a base-of-the-pyramid setting (Khalid and Seuring 2019 ) and the use of blockchain technology (Wang et al. 2019 ) or digital transformation (Hanelt et al. 2021 ). These approaches limit the reviewed field to enable a more contextualized analysis in which the novelty, continued relevance, or unjustified underrepresentation of the context can be used to specify a research gap and related research question(s). This also impacts the following decisions, as shown below.

Decision 2 concerns the option for a theoretical approach (i.e., the adoption of an inductive, abductive, or deductive approach) to theory building through the literature review. The review of previous guidance on this delivers an interesting observation. On the one hand, there are early elaborations on systematic reviews, realist synthesis, meta-synthesis, and meta-analysis by Tranfield et al. ( 2003 ) that are borrowing from the origins of systematic reviews in medical research. On the other hand, recent management-related guidelines largely neglect details of related decisions, but point out that SLRs are a suitable tool for theory building (Kraus et al. 2020 ). Seuring et al. ( 2021 ) set out to fill this gap and provided substantial guidance on how to use theory in SLRs to advance the field. To date, the option for a theoretical approach is only rarely made explicit, leaving the reader often puzzled about how advancement in theory has been crafted and impeding a review’s replicability (Seuring et al. 2021 ). Many papers still leave related choices in the dark (e.g., Rhaiem and Amara 2021 ; Rojas-Córdova et al. 2022 ) and move directly from the introduction to the method section.

In Decision 3, researchers need to adopt a theoretical framework (Durach et al. 2017 ) or at least a theoretical starting point, depending on the most appropriate theoretical approach (Seuring et al. 2021 ). Here, we find substantial guidance by Durach et al. ( 2017 ) that underlines the value of adopting a theoretical lens to investigate SCM phenomena and the literature. Moreover, the choice of a theoretical anchor enables a consistent definition and operationalization of constructs that are used to analyze the reviewed literature (Durach et al. 2017 ; Seuring et al. 2021 ). Hence, providing some upfront definitions is beneficial, clarifying what key terminology would be used in the subsequent paper, such as Devece et al. ( 2019 ) introduce their terminology on coopetition. Adding a practical hint beyond the elaborations of prior guidance papers for taking up established constructs in a deductive analysis (decision 2), there would be the question of whether these can yield interesting findings.

Here, it would be relevant to specify what kind of analysis is aimed for the SLR, where three approaches might be distinguished (i.e., bibliometric analysis, meta-analysis, and content analysis–based studies). Briefly distinguishing them, the core difference would be how many papers can be analyzed employing the respective method. Bibliometric analysis (Donthu et al. 2021 ) usually relies on the use of software, such as Biblioshiny, allowing the creation of figures on citations and co-citations. These figures enable the interpretation of large datasets in which several hundred papers can be analyzed in an automated manner. This allows for distinguishing among different research clusters, thereby following a more inductive approach. This would be contrasted by meta-analysis (e.g., Leuschner et al. 2013 ), where often a comparatively smaller number of papers is analyzed (86 in the respective case) but with a high number of observations (more than 17,000). The aim is to test for statistically significant correlations among single constructs, which requires that the related constructs and items be precisely defined (i.e., a clearly deductive approach to the analysis).

Content analysis is the third instrument frequently applied to data analysis, where an inductive or deductive approach might be taken (Seuring et al. 2021 ). Content-based analysis (see decision 9 in Sect.  4.6 ; Seuring and Gold 2012 ) is a labor-intensive step and can hardly be changed ex post. This also implies that only a certain number of papers might be analyzed (see Decision 6 in Sect.  4.5 ). It is advisable to adopt a wider set of constructs for the analysis stemming even from multiple established frameworks since it is difficult to predict which constructs and items might yield interesting insights. Hence, coding a more comprehensive set of items and dropping some in the process is less problematic than starting an analysis all over again for additional constructs and items. However, in the process of content analysis, such an iterative process might be required to improve the meaningfulness of the data and findings (Seuring and Gold 2012 ). A recent example of such an approach can be found in Khalid and Seuring ( 2019 ), building on the conceptual frameworks for SSCM of Carter and Rogers ( 2008 ), Seuring and Müller ( 2008 ), and Pagell and Wu ( 2009 ). This allows for an in-depth analysis of how SSCM constructs are inherently referred to in base-of-the-pyramid-related research. The core criticism and limitation of such an approach is the random and subjectively biased selection of frameworks for the purpose of analysis.

Beyond the aforementioned SLR methods, some reviews, similar to the one used here, apply a critical review approach. This is, however, nonsystematic, and not an SLR; thus, it is beyond the scope of this paper. Interested readers can nevertheless find some guidance on critical reviews in the available literature (e.g., Kraus et al. 2022 ; Snyder 2019 ).

4.3 Step 2: determining the required characteristics of primary studies

After setting the stage for the review, it is essential to determine which literature is to be reviewed in Decision 4. This topic is discussed by almost all existing guidelines and will thus only briefly be discussed here. Durach et al. ( 2017 ) elaborated in great detail on defining strict inclusion and exclusion criteria that need to be aligned with the chosen theoretical framework. The relevant units of analysis need to be specified (often a single paper, but other approaches might be possible) along with suitable research methods, particularly if exclusively empirical studies are reviewed or if other methods are applied. Beyond that, they elaborated on potential quality criteria that should be applied. The same is considered by a number of guidelines that especially draw on medical research, in which systematic reviews aim to pool prior studies to infer findings from their total population. Here, it is essential to ensure the exclusion of poor-quality evidence that would lower the quality of the review findings (Mulrow 1987 ; Tranfield et al. 2003 ). This could be ensured by, for example, only taking papers from journals listed on the Web of Science or Scopus or journals listed in quartile 1 of Scimago ( https://www.scimagojr.com/ ), a database providing citation and reference data for journals.

The selection of relevant publication years should again follow the purpose of the study defined in Step 1. As such, there might be a justified interest in the wide coverage of publication years if a historical perspective is taken. Alternatively, more contemporary developments or the analysis of very recent issues can justify the selection of very few years of publication (e.g., Kraus et al. 2022 ). Again, it is hard to specify a certain time period covered, but if developments of a field should be analyzed, a five-year period might be a typical lower threshold. On current topics, there is often a trend of rising publishing numbers. This scenario implies the rising relevance of a topic; however, this should be treated with caution. The total number of papers published per annum has increased substantially in recent years, which might account for the recently heightened number of papers on a certain topic.

4.4 Step 3: retrieving a sample of potentially relevant literature

After defining the required characteristics of the literature to be reviewed, the literature needs to be retrieved based on two decisions. Decision 5 concerns suitable literature sources and databases that need to be defined. Turning to Web of Science or Scopus would be two typical options found in many of the examples mentioned already (see also detailed guidance by Paul and Criado ( 2020 ) as well as Paul et al. ( 2021 )). These databases aggregate many management journals, and a typical argument for turning to the Web of Science database is the inclusion of impact factors, as they indicate a certain minimum quality of the journal (Sauer and Seuring 2017 ). Additionally, Google Scholar is increasingly mentioned as a usable search engine, often providing higher numbers of search results than the mentioned databases (e.g., Pearce 2018 ). These results often entail duplicates of articles from multiple sources or versions of the same article, as well as articles in predatory journals (Paul et al. 2021 ). Therefore, we concur with Paul et al. ( 2021 ) who underline the quality assurance mechanisms in Web of Science and Scopus, making them preferred databases for the literature search. From a practical perspective, it needs to be mentioned that SLRs in management mainly rely on databases that are not free to use. Against this limitation, Pearce ( 2018 ) provided a list of 20 search engines that are free of charge and elaborated on their advantages and disadvantages. Due to the individual limitations of the databases, it is advisable to use a combination of them (Kraus et al. 2020 , 2022 ) and build a consolidated sample by screening the papers found for duplicates, as regularly done in SLRs.

This decision also includes the choice of the types of literature to be analyzed. Typically, journal papers are selected, ensuring that the collected papers are peer-reviewed and have thus undergone an academic quality management process. Meanwhile, conference papers are usually avoided since they are often less mature and not checked for quality (e.g., Seuring et al. 2021 ). Nevertheless, for emerging topics, it might be too restrictive to consider only peer-reviewed journal articles and limit the literature to only a few references. Analyzing such rapidly emerging topics is relevant for timely and impact-oriented research and might justify the selection of different sources. Kraus et al. ( 2020 ) provided a discussion on the use of gray literature (i.e., nonacademic sources), and Sauer ( 2021 ) provided an example of a review of sustainability standards from a management perspective to derive implications for their application by managers on the one hand and for enhancing their applicability on the other hand.

Another popular way to limit the review sample is the restriction to a certain list of journals (Kraus et al. 2020 ; Snyder 2019 ). While this is sometimes favored by highly ranked journals, Carter and Washispack ( 2018 ), for example, found that many pertinent papers are not necessarily published in journals within the field. Webster and Watson ( 2002 ) quite tellingly cited a reviewer labeling the selection of top journals as an unjustified excuse for investigating the full body of relevant literature. Both aforementioned guidelines thus discourage the restriction to particular journals, a guidance that we fully support.

However, there is an argument to be made supporting the exclusion of certain lower-ranked journals. This can be done, for example, by using Scimago Journal quartiles ( www.scimagojr.com , last accessed 13. of April 2023) and restricting it to journals in the first quartile (e.g., Yavaprabhas et al. 2022 ). Other papers (e.g., Kraus et al. 2021 ; Rojas-Córdova et al. 2022 ) use certain journal quality lists to limit their sample. However, we argue for a careful check by the authors against the topic reviewed regarding what would be included and excluded.

Decision 6 entails the definition of search terms and a search string to be applied in the database just chosen. The search terms should reflect the aims of the review and the exclusion criteria that might be derived from the unit of analysis and the theoretical framework (Durach et al. 2017 ; Snyder 2019 ). Overall, two approaches to keywords can be observed. First, some guides suggest using synonyms of the key terms of interest (e.g., Durach et al. 2017 ; Kraus et al. 2020 ) in order to build a wide baseline sample that will be condensed in the next step. This is, of course, especially helpful if multiple terms delimitate a field together or different synonymous terms are used in parallel in different fields or journals. Empirical journals in supply chain management, for example, use the term “multiple supplier tiers ” (e.g., Tachizawa and Wong 2014 ), while modeling journals in the same field label this as “multiple supplier echelons ” (e.g., Brandenburg and Rebs 2015 ). Second, in some cases, single keywords are appropriate for capturing a central aspect or construct of a field if the single keyword has a global meaning tying this field together. This approach is especially relevant to the study of relatively broad terms, such as “social media” (Lim and Rasul 2022 ). However, this might result in very high numbers of publications found and therefore requires a purposeful combination with other search criteria, such as specific journals (Kraus et al. 2021 ; Lim et al. 2021 ), publication dates, article types, research methods, or the combination with keywords covering domains to which the search is aimed to be specified.

Since SLRs are often required to move into detail or review the intersections of relevant fields, we recommend building groups of keywords (single terms or multiple synonyms) for each field to be connected that are coupled via Boolean operators. To determine when a point of saturation for a keyword group is reached, one could monitor the increase in papers found in a database when adding another synonym. Once the increase is significantly decreasing or even zeroing, saturation is reached (Sauer and Seuring 2017 ). The keywords themselves can be derived from the list of keywords of influential publications in the field, while attention should be paid to potential synonyms in neighboring fields (Carter and Washispack 2018 ; Durach et al. 2017 ; Kraus et al. 2020 ).

4.5 Step 4: selecting the pertinent literature

The inclusion and exclusion criteria (Decision 6) are typically applied in Decision 7 in a two-stage process, first on the title, abstract, and keywords of an article before secondly applying them to the full text of the remaining articles (see also Kraus et al. 2020 ; Snyder 2019 ). Beyond this, Durach et al. ( 2017 ) underlined that the pertinence of the publication regarding units of analysis and the theoretical framework needs to be critically evaluated in this step to avoid bias in the review analysis. Moreover, Carter and Washispack ( 2018 ) requested the publication of the included and excluded sources to ensure the replicability of Steps 3 and 4. This can easily be done as an online supplement to an eventually published review article.

Nevertheless, the question remains: How many papers justify a literature review? While it is hard to specify how many papers comprise a body of literature, there might be certain thresholds for which Kraus et al. ( 2020 ) provide a useful discussion. As a rough guide, more than 50 papers would usually make a sound starting point (see also Paul and Criado 2020 ), while there are SLRs on emergent topics, such as multitier supply chain management, where 39 studies were included (Tachizawa and Wong 2014 ). An SLR on “learning from innovation failures” builds on 36 papers (Rhaiem and Amara 2021 ), which we would see as the lower threshold. However, such a low number should be an exception, and anything lower would certainly trigger the following question: Why is a review needed? Meanwhile, there are also limits on how many papers should be reviewed. While there are cases with 191 (Seuring and Müller 2008 ), 235 (Rojas-Córdova et al. 2022 ), or up to nearly 400 papers reviewed (Spens and Kovács 2006 ), these can be regarded as upper thresholds. Over time, similar topics seem to address larger datasets.

4.6 Step 5: synthesizing the literature

Before synthesizing the literature, Decision 8 considers the selection of a data extraction tool for which we found surprisingly little guidance. Some guidance is given on the use of cloud storage to enable remote team work (Clark et al. 2021 ). Beyond this, we found that SLRs have often been compiled with marked and commented PDFs or printed papers that were accompanied by tables (Kraus et al. 2020 ) or Excel sheets (see also the process tips by Clark et al. 2021 ). This sheet tabulated the single codes derived from the theoretical framework (Decision 3) and the single papers to be reviewed (Decision 7) by crossing out individual cells, signaling the representation of a particular code in a particular paper. While the frequency distribution of the codes is easily compiled from this data tool, the related content needs to be looked at in the papers in a tedious back-and-forth process. Beyond that, we would strongly recommend using data analysis software, such as MAXQDA or NVivo. Such programs enable the import of literature in PDF format and the automatic or manual coding of text passages, their comparison, and tabulation. Moreover, there is a permanent and editable reference of the coded text to a code. This enables a very quick compilation of content summaries or statistics for single codes and the identification of qualitative and quantitative links between codes and papers.

All the mentioned data extraction or data processing tools require a license and therefore are not free of cost. While many researchers may benefit from national or institutional subscriptions to these services, others may not. As a potential alternative, Pearce ( 2018 ) proposed a set of free open-source software (FOSS), including an elaboration on how they can be combined to perform an SLR. He also highlighted that both free and proprietary solutions have advantages and disadvantages that are worthwhile for those who do not have the required tools provided by their employers or other institutions they are members of. The same may apply to the literature databases used for the literature acquisition in Decision 5 (Pearce 2018 ).

Moreover, there is a link to Step 1, Decision 3, where bibliometric reviews and meta-analyses were mentioned. These methods, which are alternatives to content analysis–based approaches, have specific demands, so specific tools would be appropriate, such as the Biblioshiny software or VOSviewer. As we will point out for all decisions, there is a high degree of interdependence among the steps and decisions made.

Decision 9 looks at conducting the data analysis, such as coding against (pre-defined) constructs, in SLRs that rely, in most cases, on content analysis. Seuring and Gold ( 2012 ) elaborated in detail on its characteristics and application in SLRs. As this paper also explains the process of qualitative content analysis in detail, repetition is avoided here, but a summary is offered. Since different ways exist to conduct a content analysis, it is even more important to explain and justify, for example, the choice of an inductive or deductive approach (see Decision 2). In several cases, analytic variables are applied on the go, so there is no theory-based introduction of related constructs. However, to ensure the validity and replicability of the review (see Decision 11), it is necessary to explicitly define all the variables and codes used to analyze and synthesize the reviewed material (Durach et al. 2017 ; Seuring and Gold 2012 ). To build a valid framework as the SLR outcome, it is vital to ensure that the constructs used for the data analysis are sufficiently defined, mutually exclusive, and comprehensively exhaustive. For meta-analysis, the predefined constructs and items would demand quantitative coding so that the resulting data could be analyzed using statistical software tools such as SPSS or R (e.g., Xiao and Watson 2019 ). Pointing to bibliometric analysis again, the respective software would be used for data analysis, yielding different figures and paper clusters, which would then require interpretation (e.g., Donthu et al. 2021 ; Xiao and Watson 2019 ).

Decision 10, on conducting subsequent statistical analysis, considers follow-up analysis of the coding results. Again, this is linked to the chosen SLR method, and a bibliographic analysis will require a different statistical analysis than a content analysis–based SLR (e.g., Lim et al. 2022 ; Xiao and Watson 2019 ). Beyond the use of content analysis and the qualitative interpretation of its results, applying contingency analysis offers the opportunity to quantitatively assess the links among constructs and items. It provides insights into which items are correlated with each other without implying causality. Thus, the interpretation of the findings must explain the causality behind the correlations between the constructs and the items. This must be based on sound reasoning and linking the findings to theoretical arguments. For SLRs, there have recently been two kinds of applications of contingency analysis, differentiated by unit of analysis. De Lima et al. ( 2021 ) used the entire paper as the unit of analysis, deriving correlations on two constructs that were used together in one paper. This is, of course, subject to critique as to whether the constructs really represent correlated content. Moving a level deeper, Tröster and Hiete ( 2018 ) used single-text passages on one aspect, argument, or thought as the unit of analysis. Such an approach is immune against the critique raised before and can yield more valid statistical support for thematic analysis. Another recent methodological contribution employing the same contingency analysis–based approach was made by Siems et al. ( 2021 ). Their analysis employs constructs from SSCM and dynamic capabilities. Employing four subsets of data (i.e., two time periods each in the food and automotive industries), they showed that the method allows distinguishing among time frames as well as among industries.

However, the unit of analysis must be precisely explained so that the reader can comprehend it. Both examples use contingency analysis to identify under-researched topics and develop them into research directions whose formulation represents the particular aim of an SLR (Paul and Criado 2020 ; Snyder 2019 ). Other statistical tools might also be applied, such as cluster analysis. Interestingly, Brandenburg and Rebs ( 2015 ) applied both contingency and cluster analyses. However, the authors stated that the contingency analysis did not yield usable results, so they opted for cluster analysis. In effect, Brandenburg and Rebs ( 2015 ) added analytical depth to their analysis of model types in SSCM by clustering them against the main analytical categories of content analysis. In any case, the application of statistical tools needs to fit the study purpose (Decision 1) and the literature sample (Decision 7), just as in their more conventional applications (e.g., in empirical research processes).

Decision 11 regards the additional consideration of validity and reliability criteria and emphasizes the need for explaining and justifying the single steps of the research process (Seuring and Gold 2012 ), much in line with other examples of research (Davis and Crombie 2001 ). This is critical to underlining the quality of the review but is often neglected in many submitted manuscripts. In our review, we find rich guidance on this decision, to which we want to guide readers (see Table 3 ). In particular, Durach et al. ( 2017 ) provide an entire section of biases and what needs to be considered and reported on them. Moreover, Snyder ( 2019 ) regularly reflects on these issues in her elaborations. This rich guidance elaborates on how to ensure the quality of the individual steps of the review process, such as sampling, study inclusion and exclusion, coding, synthesizing, and more practical issues, including team composition and teamwork organization, which are discussed in some guidelines (e.g., Clark et al. 2021 ; Kraus et al. 2020 ). We only want to underline that the potential biases are, of course, to be seen in conjunction with Decisions 2, 3, 4, 5, 6, 7, 9, and 10. These decisions and the elaboration by Durach et al. ( 2017 ) should provide ample points of reflection that, however, many SLR manuscripts fail to address.

4.7 Step 6: reporting the results

In the final step, there are three decisions on which there is surprisingly little guidance, although reviews often fail in this critical part of the process (Kraus et al. 2020 ). The reviewed guidelines discuss the presentation almost exclusively, while almost no guidance is given on the overall paper structure or the key content to be reported.

Consequently, the first choice to be made in Decision 12 is regarding the paper structure. We suggest following the five-step logic of typical research papers (see also Fisch and Block 2018 ) and explaining only a few points in which a difference from other papers is seen.

(1) Introduction: While the introduction would follow a conventional logic of problem statement, research question, contribution, and outline of the paper (see also Webster and Watson 2002 ), the next parts might depend on the theoretical choices made in Decision 2.

(2) Literature review section: If deductive logic is taken, the paper usually has a conventional flow. After the introduction, the literature review section covers the theoretical background and the choice of constructs and variables for the analysis (De Lima et al. 2021 ; Dieste et al. 2022 ). To avoid confusion in this section with the literature review, its labeling can also be closer to the reviewed object.

If an inductive approach is applied, it might be challenging to present the theoretical basis up front, as the codes emerge only from analyzing the material. In this case, the theory section might be rather short, concentrating on defining the core concepts or terms used, for example, in the keyword-based search for papers. The latter approach is exemplified by the study at hand, which presents a short review of the available literature in the introduction and the first part of the findings. However, we do not perform a systematic but integrative review, which allows for more freedom and creativity (Snyder 2019 ).

(3) Method section: This section should cover the steps and follow the logic presented in this paper or any of the reviewed guidelines so that the choices made during the research process are transparently disclosed (Denyer and Tranfield 2009 ; Paul et al. 2021 ; Xiao and Watson 2019 ). In particular, the search for papers and their selection requires a sound explanation of each step taken, including the provision of reasons for the delimitation of the final paper sample. A stage that is often not covered in sufficient detail is data analysis (Seuring and Gold 2012 ). This also needs to be outlined so that the reader can comprehend how sense has been made of the material collected. Overall, the demands for SLR papers are similar to case studies, survey papers, or almost any piece of empirical research; thus, each step of the research process needs to be comprehensively described, including Decisions 4–10. This comprehensiveness must also include addressing measures for validity and reliability (see Decision 11) or other suitable measures of rigor in the research process since they are a critical issue in literature reviews (Durach et al. 2017 ). In particular, inductively conducted reviews are prone to subjective influences and thus require sound reporting of design choices and their justification.

(4) Findings: The findings typically start with a descriptive analysis of the literature covered, such as journals, distribution across years, or (empirical) methods applied (Tranfield et al. 2003 ). For modeling-related reviews, classifying papers against the approach chosen is a standard approach, but this can often also serve as an analytic category that provides detailed insights. The descriptive analysis should be kept short since a paper only presenting descriptive findings will not be of great interest to other researchers due to the missing contribution (Snyder 2019 ). Nevertheless, there are opportunities to provide interesting findings in the descriptive analysis. Beyond a mere description of the distributions of the single results, such as the distribution of methods used in the sample, authors should combine analytical categories to derive more detailed insights (see also Tranfield et al. 2003 ). The distribution of methods used might well be combined with the years of publication to identify and characterize different phases in the development of a field of research or its maturity. Moreover, there could be value in the analysis of theories applied in the review sample (e.g., Touboulic and Walker 2015 ; Zhu et al. 2022 ) and in reflecting on the interplay of different qualitative and quantitative methods in spurring the theoretical development of the reviewed field. This could yield detailed insights into methodological as well as theoretical gaps, and we would suggest explicitly linking the findings of such analyses to the research directions that an SLR typically provides. This link could help make the research directions much more tangible by giving researchers a clear indication of how to follow up on the findings, as, for example, done by Maestrini et al. ( 2017 ) or Dieste et al. ( 2022 ). In contrast to the mentioned examples of an actionable research agenda, a typical weakness of premature SLR manuscripts is that they ask rather superficially for more research in the different aspects they reviewed but remain silent about how exactly this can be achieved.

We would thus like to encourage future SLR authors to systematically investigate the potential to combine two categories of descriptive analysis to move this section of the findings to a higher level of quality, interest, and relevance. The same can, of course, be done with the thematic findings, which comprise the second part of this section.

Moving into the thematic analysis, we have already reached Decision 13 on the presentation of the refined theoretical framework and the discussion of its contents. A first step might present the frequencies of the codes or constructs applied in the analysis. This allows the reader to understand which topics are relevant. If a rather small body of literature is analyzed, tables providing evidence on which paper has been coded for which construct might be helpful in improving the transparency of the research process. Tables or other forms of visualization might help to organize the many codes soundly (see also Durach et al. 2017 ; Paul and Criado 2020 ; Webster and Watson 2002 ). These findings might then lead to interpretation, for which it is necessary to extract meaning from the body of literature and present it accordingly (Snyder 2019 ). To do so, it might seem needless to say that the researchers should refer back to Decisions 1, 2, and 3 taken in Step 1 and their justifications. These typically identify the research gap to be filled, but after the lengthy process of the SLR, the authors often fail to step back from the coding results and put them into a larger perspective against the research gap defined in Decision 1 (see also Clark et al. 2021 ). To support this, it is certainly helpful to illustrate the findings in a figure or graph presenting the links among the constructs and items and adding causal reasoning to this (Durach et al. 2017 ; Paul and Criado 2020 ), such as the three figures by Seuring and Müller ( 2008 ) or other examples by De Lima et al. ( 2021 ) or Tipu ( 2022 ). This presentation should condense arguments made in the assessed literature but should also chart the course for future research. It will be these parts of the paper that are decisive for a strong SLR paper.

Moreover, some guidelines define the most fruitful way of synthesizing the findings as concept-centric synthesis (Clark et al. 2021 ; Fisch and Block 2018 ; Webster and Watson 2002 ). As presented in the previous sentence, the presentation of the review findings is centered on the content or concept of “concept-centric synthesis.” It is accompanied by a reference to all or the most relevant literature in which the concept is evident. Contrastingly, Webster and Watson ( 2002 ) found that author-centric synthesis discusses individual papers and what they have done and found (just like this sentence here). They added that this approach fails to synthesize larger samples. We want to note that we used the latter approach in some places in this paper. However, this aims to actively refer the reader to these studies, as they stand out from our relatively small sample. Beyond this, we want to link back to Decision 3, the selection of a theoretical framework and constructs. These constructs, or the parts of a framework, can also serve to structure the findings section by using them as headlines for subsections (Seuring et al. 2021 ).

Last but not least, there might even be cases where core findings and relationships might be opposed, and alternative perspectives could be presented. This would certainly be challenging to argue for but worthwhile to do in order to drive the reviewed field forward. A related example is the paper by Zhu et al. ( 2022 ), who challenged the current debate at the intersection of blockchain applications and supply chain management and pointed to the limited use of theoretical foundations for related analysis.

(5) Discussion and Conclusion: The discussion needs to explain the contribution the paper makes to the extant literature, that is, which previous findings or hypotheses are supported or contradicted and which aspects of the findings are particularly interesting for the future development of the reviewed field. This is in line with the content required in the discussion sections of any other paper type. A typical structure might point to the contribution and put it into perspective with already existing research. Further, limitations should be addressed on both the theoretical and methodological sides. This elaboration of the limitations can be coupled with the considerations of the validity and reliability of the study in Decision 11. The implications for future research are a core aim of an SLR (Clark et al. 2021 ; Mulrow 1987 ; Snyder 2019 ) and should be addressed in a further part of the discussion section. Recently, a growing number of literature reviews have also provided research questions for future research that provide a very concrete and actionable output of the SLR (e.g. Dieste et al. 2022 ; Maestrini et al. 2017 ). Moreover, we would like to reiterate our call to clearly link the research implications to the SLR findings, which helps the authors craft more tangible research directions and helps the reader to follow the authors’ interpretation. Literature review papers are usually not strongly positioned toward managerial implications, but even these implications might be included.

As a kind of normal demand, the conclusion should provide an answer to the research question put forward in the introduction, thereby closing the cycle of arguments made in the paper.

Although all the works seem to be done when the paper is written and the contribution is fleshed out, there is still one major decision to be made. Decision 14 concerns the identification of an appropriate journal for submission. Despite the popularity of the SLR method, a rising number of journals explicitly limit the number of SLRs published by them. Moreover, there are only two guidelines elaborating on this decision, underlining the need for the following considerations.

Although it might seem most attractive to submit the paper to the highest-ranking journal for the reviewed topic, we argue for two critical and review-related decisions to be made during the research process that influence whether the paper fits a certain outlet:

The theoretical foundation of the SLR (Decision 3) usually relates to certain journals in which it is published or discussed. If a deductive approach was taken, the journals in which the foundational papers were published might be suitable since the review potentially contributes to the further validation or refinement of the frameworks. Overall, we need to keep in mind that a paper needs to be added to a discussion in the journal, and this can be based on the theoretical framework or the reviewed papers, as shown below.

Appropriate journals for publication can be derived from the analyzed journal papers (Decision 7) (see also Paul and Criado 2020 ). This allows for an easy link to the theoretical debate in the respective journal by submitting it. This choice is identifiable in most of the papers mentioned in this paper and is often illustrated in the descriptive analysis.

If the journal chosen for the submission was neither related to the theoretical foundation nor overly represented in the body of literature analyzed, an explicit justification in the paper itself might be needed. Alternatively, an explanation might be provided in the letter to the editor when submitting the paper. If such a statement is not presented, the likelihood of it being transferred into the review process and passing it is rather low. Finally, we want to refer readers interested in the specificities of the publication-related review process of SLRs to Webster and Watson ( 2002 ), who elaborated on this for Management Information Systems Quarterly.

5 Discussion and conclusion

Critically reviewing the currently available SLR guidelines in the management domain, this paper synthesizes 14 key decisions to be made and reported across the SLR research process. Guidelines are presented for each decision, including tasks that assist in making sound choices to complete the research process and make meaningful contributions. Applying these guidelines should improve the rigor and robustness of many review papers and thus enhance their contributions. Moreover, some practical hints and best-practice examples are provided on issues that unexperienced authors regularly struggle to present in a manuscript (Fisch and Block 2018 ) and thus frustrate reviewers, readers, editors, and authors alike.

Strikingly, the review of prior guidelines reported in Table 3 revealed their focus on the technical details that need to be reported in any SLR. Consequently, our discipline has come a long way in crafting search strings, inclusion, and exclusion criteria, and elaborating on the validity and reliability of an SLR. Nevertheless, we left critical areas underdeveloped, such as the identification of relevant research gaps and questions, data extraction tools, analysis of the findings, and a meaningful and interesting reporting of the results. Our study contributes to filling these gaps by providing operationalized guidance to SLR authors, especially early-stage researchers who craft SLRs at the outset of their research journeys. At the same time, we need to underline that our paper is, of course, not the only useful reference for SLR authors. Instead, the readers are invited to find more guidance on the many aspects to consider in an SLR in the references we provide within the single decisions, as well as in Tables 1 and 2 . The tables also identify the strongholds of other guidelines that our paper does not want to replace but connect and extend at selected occasions, especially in SLR Steps 5 and 6.

The findings regularly underline the interconnection of the 14 decisions identified and discussed in this paper. We thus support Tranfield et al. ( 2003 ) who requested a flexible approach to the SLR while clearly reporting all design decisions and reflecting their impacts. In line with the guidance synthesized in this review, and especially Durach et al. ( 2017 ), we also present a refined framework in Figs.  1 and 2 . It specifically refines the original six-step SLR process by Durach et al. ( 2017 ) in three ways:

Enriched six-step process including the core interrelations of the 14 decisions

First, we subdivided the six steps into 14 decisions to enhance the operationalization of the process and enable closer guidance (see Fig.  1 ). Second, we added a temporal sequence to Fig.  2 by positioning the decisions from left to right according to this temporal sequence. This is based on systematically reflecting on the need to finish one decision before the following. If this need is evident, the following decision moves to the right; if not, the decisions are positioned below each other. Turning to Fig.  2 , it becomes evident that Step 2, “determining the required characteristics of primary studies,” and Step 3, “retrieving a sample of potentially relevant literature,” including their Decisions 4–6, can be conducted in an iterative manner. While this contrasts with the strict division of the six steps by Durach et al. ( 2017 ), it supports other guidance that suggests running pilot studies to iteratively define the literature sample, its sources, and characteristics (Snyder 2019 ; Tranfield et al. 2003 ; Xiao and Watson 2019 ). While this insight might suggest merging Steps 2 and 3, we refrain from this superficial change and building yet another SLR process model. Instead, we prefer to add detail and depth to Durach et al.’s ( 2017 ) model.

(Decisions: D1: specifying the research gap and related research question, D2: opting for a theoretical approach, D3: defining the core theoretical framework and constructs, D4: specifying inclusion and exclusion criteria, D5: defining sources and databases, D6: defining search terms and crafting a search string, D7: including and excluding literature for detailed analysis and synthesis, D8: selecting data extraction tool(s), D9: coding against (pre-defined) constructs, D10: conducting a subsequent (statistical) analysis (optional), D11: ensuring validity and reliability, D12: deciding on the structure of the paper, D13: presenting a refined theoretical framework and discussing its contents, and D14: deriving an appropriate journal from the analyzed papers).

This is also done through the third refinement, which underlines which previous or later decisions need to be considered within each single decision. Such a consideration moves beyond the mere temporal sequence of steps and decisions that does not reflect the full complexity of the SLR process. Instead, its focus is on the need to align, for example, the conduct of the data analysis (Decision 9) with the theoretical approach (Decision 2) and consequently ensure that the chosen theoretical framework and the constructs (Decision 3) are sufficiently defined for the data analysis (i.e., mutually exclusive and comprehensively exhaustive). The mentioned interrelations are displayed in Fig.  2 by means of directed arrows from one decision to another. The underlying explanations can be found in the earlier paper sections by searching for the individual decisions in the text on the impacted decisions. Overall, it is unsurprising to see that the vast majority of interrelations are directed from the earlier to the later steps and decisions (displayed through arrows below the diagonal of decisions), while only a few interrelations are inverse.

Combining the first refinement of the original framework (defining the 14 decisions) and the third refinement (revealing the main interrelations among the decisions) underlines the contribution of this study in two main ways. First, the centrality of ensuring validity and reliability (Decision 11) is underlined. It becomes evident that considerations of validity and reliability are central to the overall SLR process since all steps before the writing of the paper need to be revisited in iterative cycles through Decision 11. Any lack of related considerations will most likely lead to reviewer critique, putting the SLR publication at risk. On the positive side of this centrality, we also found substantial guidance on this issue. In contrast, as evidenced in Table 3 , there is a lack of prior guidance on Decisions 1, 8, 10, 12, 13, and 14, which this study is helping to fill. At the same time, these underexplained decisions are influenced by 14 of the 44 (32%) incoming arrows in Fig.  2 and influence the other decisions in 6 of the 44 (14%) instances. These interrelations among decisions to be considered when crafting an SLR were scattered across prior guidelines, lacked in-depth elaborations, and were hardly explicitly related to each other. Thus, we hope that our study and the refined SLR process model will help enhance the quality and contribution of future SLRs.

Data availablity

The data generated during this research is summarized in Table 3 and the analyzed papers are publicly available. They are clearly identified in Table 3 and the reference list.

Aguinis H, Ramani RS, Alabduljader N (2020) Best-practice recommendations for producers, evaluators, and users of methodological literature reviews. Organ Res Methods. https://doi.org/10.1177/1094428120943281

Article   Google Scholar  

Beske P, Land A, Seuring S (2014) Sustainable supply chain management practices and dynamic capabilities in the food industry: a critical analysis of the literature. Int J Prod Econ 152:131–143. https://doi.org/10.1016/j.ijpe.2013.12.026

Brandenburg M, Rebs T (2015) Sustainable supply chain management: a modeling perspective. Ann Oper Res 229:213–252. https://doi.org/10.1007/s10479-015-1853-1

Carter CR, Rogers DS (2008) A framework of sustainable supply chain management: moving toward new theory. Int Jnl Phys Dist Logist Manage 38:360–387. https://doi.org/10.1108/09600030810882816

Carter CR, Washispack S (2018) Mapping the path forward for sustainable supply chain management: a review of reviews. J Bus Logist 39:242–247. https://doi.org/10.1111/jbl.12196

Clark WR, Clark LA, Raffo DM, Williams RI (2021) Extending fisch and block’s (2018) tips for a systematic review in management and business literature. Manag Rev Q 71:215–231. https://doi.org/10.1007/s11301-020-00184-8

Crane A, Henriques I, Husted BW, Matten D (2016) What constitutes a theoretical contribution in the business and society field? Bus Soc 55:783–791. https://doi.org/10.1177/0007650316651343

Davis J, Mengersen K, Bennett S, Mazerolle L (2014) Viewing systematic reviews and meta-analysis in social research through different lenses. Springerplus 3:511. https://doi.org/10.1186/2193-1801-3-511

Davis HTO, Crombie IK (2001) What is asystematicreview? http://vivrolfe.com/ProfDoc/Assets/Davis%20What%20is%20a%20systematic%20review.pdf . Accessed 22 February 2019

De Lima FA, Seuring S, Sauer PC (2021) A systematic literature review exploring uncertainty management and sustainability outcomes in circular supply chains. Int J Prod Res. https://doi.org/10.1080/00207543.2021.1976859

Denyer D, Tranfield D (2009) Producing a systematic review. In: Buchanan DA, Bryman A (eds) The Sage handbook of organizational research methods. Sage Publications Ltd, Thousand Oaks, CA, pp 671–689

Google Scholar  

Devece C, Ribeiro-Soriano DE, Palacios-Marqués D (2019) Coopetition as the new trend in inter-firm alliances: literature review and research patterns. Rev Manag Sci 13:207–226. https://doi.org/10.1007/s11846-017-0245-0

Dieste M, Sauer PC, Orzes G (2022) Organizational tensions in industry 4.0 implementation: a paradox theory approach. Int J Prod Econ 251:108532. https://doi.org/10.1016/j.ijpe.2022.108532

Donthu N, Kumar S, Mukherjee D, Pandey N, Lim WM (2021) How to conduct a bibliometric analysis: an overview and guidelines. J Bus Res 133:285–296. https://doi.org/10.1016/j.jbusres.2021.04.070

Durach CF, Kembro J, Wieland A (2017) A new paradigm for systematic literature reviews in supply chain management. J Supply Chain Manag 53:67–85. https://doi.org/10.1111/jscm.12145

Fink A (2010) Conducting research literature reviews: from the internet to paper, 3rd edn. SAGE, Los Angeles

Fisch C, Block J (2018) Six tips for your (systematic) literature review in business and management research. Manag Rev Q 68:103–106. https://doi.org/10.1007/s11301-018-0142-x

Fritz MMC, Silva ME (2018) Exploring supply chain sustainability research in Latin America. Int Jnl Phys Dist Logist Manag 48:818–841. https://doi.org/10.1108/IJPDLM-01-2017-0023

Garcia-Torres S, Albareda L, Rey-Garcia M, Seuring S (2019) Traceability for sustainability: literature review and conceptual framework. Supp Chain Manag 24:85–106. https://doi.org/10.1108/SCM-04-2018-0152

Hanelt A, Bohnsack R, Marz D, Antunes Marante C (2021) A systematic review of the literature on digital transformation: insights and implications for strategy and organizational change. J Manag Stud 58:1159–1197. https://doi.org/10.1111/joms.12639

Kache F, Seuring S (2014) Linking collaboration and integration to risk and performance in supply chains via a review of literature reviews. Supp Chain Mnagmnt 19:664–682. https://doi.org/10.1108/SCM-12-2013-0478

Khalid RU, Seuring S (2019) Analyzing base-of-the-pyramid research from a (sustainable) supply chain perspective. J Bus Ethics 155:663–686. https://doi.org/10.1007/s10551-017-3474-x

Koufteros X, Mackelprang A, Hazen B, Huo B (2018) Structured literature reviews on strategic issues in SCM and logistics: part 2. Int Jnl Phys Dist Logist Manage 48:742–744. https://doi.org/10.1108/IJPDLM-09-2018-363

Kraus S, Breier M, Dasí-Rodríguez S (2020) The art of crafting a systematic literature review in entrepreneurship research. Int Entrep Manag J 16:1023–1042. https://doi.org/10.1007/s11365-020-00635-4

Kraus S, Mahto RV, Walsh ST (2021) The importance of literature reviews in small business and entrepreneurship research. J Small Bus Manag. https://doi.org/10.1080/00472778.2021.1955128

Kraus S, Breier M, Lim WM, Dabić M, Kumar S, Kanbach D, Mukherjee D, Corvello V, Piñeiro-Chousa J, Liguori E, Palacios-Marqués D, Schiavone F, Ferraris A, Fernandes C, Ferreira JJ (2022) Literature reviews as independent studies: guidelines for academic practice. Rev Manag Sci 16:2577–2595. https://doi.org/10.1007/s11846-022-00588-8

Leuschner R, Rogers DS, Charvet FF (2013) A meta-analysis of supply chain integration and firm performance. J Supply Chain Manag 49:34–57. https://doi.org/10.1111/jscm.12013

Lim WM, Rasul T (2022) Customer engagement and social media: revisiting the past to inform the future. J Bus Res 148:325–342. https://doi.org/10.1016/j.jbusres.2022.04.068

Lim WM, Yap S-F, Makkar M (2021) Home sharing in marketing and tourism at a tipping point: what do we know, how do we know, and where should we be heading? J Bus Res 122:534–566. https://doi.org/10.1016/j.jbusres.2020.08.051

Lim WM, Kumar S, Ali F (2022) Advancing knowledge through literature reviews: ‘what’, ‘why’, and ‘how to contribute.’ Serv Ind J 42:481–513. https://doi.org/10.1080/02642069.2022.2047941

Lusiantoro L, Yates N, Mena C, Varga L (2018) A refined framework of information sharing in perishable product supply chains. Int J Phys Distrib Logist Manag 48:254–283. https://doi.org/10.1108/IJPDLM-08-2017-0250

Maestrini V, Luzzini D, Maccarrone P, Caniato F (2017) Supply chain performance measurement systems: a systematic review and research agenda. Int J Prod Econ 183:299–315. https://doi.org/10.1016/j.ijpe.2016.11.005

Miemczyk J, Johnsen TE, Macquet M (2012) Sustainable purchasing and supply management: a structured literature review of definitions and measures at the dyad, chain and network levels. Supp Chain Mnagmnt 17:478–496. https://doi.org/10.1108/13598541211258564

Moher D, Liberati A, Tetzlaff J, Altman DG (2009) Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med 6:e1000097. https://doi.org/10.1371/journal.pmed.1000097

Mukherjee D, Lim WM, Kumar S, Donthu N (2022) Guidelines for advancing theory and practice through bibliometric research. J Bus Res 148:101–115. https://doi.org/10.1016/j.jbusres.2022.04.042

Mulrow CD (1987) The medical review article: state of the science. Ann Intern Med 106:485–488. https://doi.org/10.7326/0003-4819-106-3-485

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, Shamseer L, Tetzlaff JM, Akl EA, Brennan SE, Chou R, Glanville J, Grimshaw JM, Hróbjartsson A, Lalu MM, Li T, Loder EW, Mayo-Wilson E, McDonald S, McGuinness LA, Stewart LA, Thomas J, Tricco AC, Welch VA, Whiting P, Moher D (2021) The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. J Clin Epidemiol 134:178–189. https://doi.org/10.1016/j.jclinepi.2021.03.001

Pagell M, Wu Z (2009) Building a more complete theory of sustainable supply chain management using case studies of 10 exemplars. J Supply Chain Manag 45:37–56. https://doi.org/10.1111/j.1745-493X.2009.03162.x

Paul J, Criado AR (2020) The art of writing literature review: What do we know and what do we need to know? Int Bus Rev 29:101717. https://doi.org/10.1016/j.ibusrev.2020.101717

Paul J, Lim WM, O’Cass A, Hao AW, Bresciani S (2021) Scientific procedures and rationales for systematic literature reviews (SPAR-4-SLR). Int J Consum Stud. https://doi.org/10.1111/ijcs.12695

Pearce JM (2018) How to perform a literature review with free and open source software. Pract Assess Res Eval 23:1–13

Rhaiem K, Amara N (2021) Learning from innovation failures: a systematic review of the literature and research agenda. Rev Manag Sci 15:189–234. https://doi.org/10.1007/s11846-019-00339-2

Rojas-Córdova C, Williamson AJ, Pertuze JA, Calvo G (2022) Why one strategy does not fit all: a systematic review on exploration–exploitation in different organizational archetypes. Rev Manag Sci. https://doi.org/10.1007/s11846-022-00577-x

Sauer PC (2021) The complementing role of sustainability standards in managing international and multi-tiered mineral supply chains. Resour Conserv Recycl 174:105747. https://doi.org/10.1016/j.resconrec.2021.105747

Sauer PC, Seuring S (2017) Sustainable supply chain management for minerals. J Clean Prod 151:235–249. https://doi.org/10.1016/j.jclepro.2017.03.049

Seuring S, Gold S (2012) Conducting content-analysis based literature reviews in supply chain management. Supp Chain Mnagmnt 17:544–555. https://doi.org/10.1108/13598541211258609

Seuring S, Müller M (2008) From a literature review to a conceptual framework for sustainable supply chain management. J Clean Prod 16:1699–1710. https://doi.org/10.1016/j.jclepro.2008.04.020

Seuring S, Yawar SA, Land A, Khalid RU, Sauer PC (2021) The application of theory in literature reviews: illustrated with examples from supply chain management. Int J Oper Prod Manag 41:1–20. https://doi.org/10.1108/IJOPM-04-2020-0247

Siems E, Land A, Seuring S (2021) Dynamic capabilities in sustainable supply chain management: an inter-temporal comparison of the food and automotive industries. Int J Prod Econ 236:108128. https://doi.org/10.1016/j.ijpe.2021.108128

Snyder H (2019) Literature review as a research methodology: an overview and guidelines. J Bus Res 104:333–339. https://doi.org/10.1016/j.jbusres.2019.07.039

Spens KM, Kovács G (2006) A content analysis of research approaches in logistics research. Int Jnl Phys Dist Logist Manage 36:374–390. https://doi.org/10.1108/09600030610676259

Tachizawa EM, Wong CY (2014) Towards a theory of multi-tier sustainable supply chains: a systematic literature review. Supp Chain Mnagmnt 19:643–663. https://doi.org/10.1108/SCM-02-2014-0070

Tipu SAA (2022) Organizational change for environmental, social, and financial sustainability: a systematic literature review. Rev Manag Sci 16:1697–1742. https://doi.org/10.1007/s11846-021-00494-5

Touboulic A, Walker H (2015) Theories in sustainable supply chain management: a structured literature review. Int Jnl Phys Dist Logist Manage 45:16–42. https://doi.org/10.1108/IJPDLM-05-2013-0106

Tranfield D, Denyer D, Smart P (2003) Towards a methodology for developing evidence-informed management knowledge by means of systematic review. Br J Manag 14:207–222. https://doi.org/10.1111/1467-8551.00375

Tröster R, Hiete M (2018) Success of voluntary sustainability certification schemes: a comprehensive review. J Clean Prod 196:1034–1043. https://doi.org/10.1016/j.jclepro.2018.05.240

Wang Y, Han JH, Beynon-Davies P (2019) Understanding blockchain technology for future supply chains: a systematic literature review and research agenda. Supp Chain Mnagmnt 24:62–84. https://doi.org/10.1108/SCM-03-2018-0148

Webster J, Watson RT (2002) Analyzing the past to prepare for the future: writing a literature review. MIS Q 26:xiii–xxiii

Wiese A, Kellner J, Lietke B, Toporowski W, Zielke S (2012) Sustainability in retailing: a summative content analysis. Int J Retail Distrib Manag 40:318–335. https://doi.org/10.1108/09590551211211792

Xiao Y, Watson M (2019) Guidance on conducting a systematic literature review. J Plan Educ Res 39:93–112. https://doi.org/10.1177/0739456X17723971

Yavaprabhas K, Pournader M, Seuring S (2022) Blockchain as the “trust-building machine” for supply chain management. Ann Oper Res. https://doi.org/10.1007/s10479-022-04868-0

Zhu Q, Bai C, Sarkis J (2022) Blockchain technology and supply chains: the paradox of the atheoretical research discourse. Transp Res Part E Logist Transp Rev 164:102824. https://doi.org/10.1016/j.tre.2022.102824

Download references

Open Access funding enabled and organized by Projekt DEAL.

Author information

Authors and affiliations.

EM Strasbourg Business School, Université de Strasbourg, HuManiS UR 7308, 67000, Strasbourg, France

Philipp C. Sauer

Chair of Supply Chain Management, Faculty of Economics and Management, The University of Kassel, Kassel, Germany

Stefan Seuring

You can also search for this author in PubMed   Google Scholar

Contributions

The article is based on the idea and extensive experience of SS. The literature search and data analysis has mainly been performed by PCS and supported by SS before the paper manuscript has been written and revised in a common effort of both authors.

Corresponding author

Correspondence to Stefan Seuring .

Ethics declarations

Conflict of interest.

The authors have no competing interests to declare that are relevant to the content of this article.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Sauer, P.C., Seuring, S. How to conduct systematic literature reviews in management research: a guide in 6 steps and 14 decisions. Rev Manag Sci 17 , 1899–1933 (2023). https://doi.org/10.1007/s11846-023-00668-3

Download citation

Received : 29 September 2022

Accepted : 17 April 2023

Published : 12 May 2023

Issue Date : July 2023

DOI : https://doi.org/10.1007/s11846-023-00668-3

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Methodology
• Replicability
• Research process
• Structured literature review
• Systematic literature review

JEL Classification

• Find a journal
• Publish with us
• Track your research

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• v.20; 2019 Oct 25

Managing Ideas, People, and Projects: Organizational Tools and Strategies for Researchers

Samuel pascal levin.

1 Beverly, MA 01915, USA

Michael Levin

2 Allen Discovery Center at Tufts University, Suite 4600, 200 Boston Avenue, Medford, MA 02155-4243, USA

Primary Investigators at all levels of their career face a range of challenges related to optimizing their activity within the constraints of deadlines and productive research. These range from enhancing creative thought and keeping track of ideas to organizing and prioritizing the activity of the members of the group. Numerous tools now exist that facilitate the storage and retrieval of information necessary for running a laboratory to advance specific project goals within associated timelines. Here we discuss strategies and tools/software that, together or individually, can be used as is or adapted to any size scientific laboratory. Specific software products, suggested use cases, and examples are shown across the life cycle from idea to publication. Strategies for managing the organization of, and access to, digital information and planning structures can greatly facilitate the efficiency and impact of an active scientific enterprise. The principles and workflow described here are applicable to many different fields.

Graphical Abstract

Information Systems; Knowledge Management

Introduction

Researchers, at all stages of their careers, are facing an ever-increasing deluge of information and deadlines. Additional difficulties arise when one is the Principal Investigator (PI) of those researchers: as group size and scope of inquiry increases, the challenges of managing people and projects and the interlocking timelines, finances, and information pertaining to those projects present a continuous challenge. In the immediate term, there are experiments to do, papers and grants to write, and presentations to construct, in addition to teaching and departmental duties. At the same time, however, the PI must make strategic decisions that will impact the future direction(s) of the laboratory and its personnel. The integration of deep creative thought together with the practical steps of implementing a research plan and running a laboratory on a day-to-day basis is one of the great challenges of the modern scientific enterprise. Especially difficult is the fact that attention needs to span many orders of scale, from decisions about which problems should be pursued by the group in the coming years and how to tackle those problems to putting out regular “fires” associated with the minutiae of managing people and limited resources toward the committed goals.

The planning of changes in research emphasis, hiring, grant-writing, etc. likewise occur over several different timescales. The optimization of resources and talent toward impactful goals requires the ability to organize, store, and rapidly access information that is integrated with project planning structures. Interestingly, unlike other fields such as business, there are few well-known, generally accepted guidelines for best practices available to researchers. Here we lay out a conceptual taxonomy of the life cycle of a project, from brainstorming ideas through to a final deliverable product. We recommend methods and software/tools to facilitate management of concurrent research activities across the timeline. The goal is to optimize the organization, storage, and access to the necessary information in each phase, and, crucially, to facilitate the interconnections between static information, action plans, and work product across all phases. We believe that the earlier in the career of a researcher such tools are implemented and customized, the more positive impact they will exert on the productivity of their enterprise.

This overview is intended for anyone who is conducting research or academic scholarship. It consists of a number of strategies and software recommendations that can be used together or independently (adapted to suit a given individual's or group's needs). Some of the specific software packages mentioned are only usable on Apple devices, but similar counterparts exist in the Windows and Linux ecosystems; these are indicated in Table 1 (definitions of special terms are given in Table 2 ). These strategies were developed (and have been continuously updated) over the last 20 years based on the experiences of the Levin group and those of various collaborators and other productive researchers. Although very specific software and platforms are indicated, to facilitate the immediate and practical adoption by researchers at all levels, the important thing is the strategies illustrated by the examples. As software and hardware inevitably change over the next few years, the fundamental principles can be readily adapted to newer products.

Software Packages and Alternatives

A Glossary of Special Terms

Basic Principles

Although there is a huge variety of different types of scientific enterprises, most of them contain one or more activities that can be roughly subsumed by the conceptual progression shown in Figure 1 . This life cycle progresses from brainstorming and ideation through planning, execution of research, and then creation of work products. Each stage requires unique activities and tools, and it is crucial to establish a pipeline and best practices that enable the results of each phase to effectively facilitate the next phase. All of the recommendations given below are designed to support the following basic principles:

• • Information should be easy to find and access, so as to enable the user to have to remember as little as possible—this keeps the mind free to generate new, creative ideas. We believe that when people get comfortable with not having to remember any details and are completely secure in the knowledge that the information has been offloaded to a dependable system and will be there when they need it, a deeper, improved level of thinking can be achieved.
• • Information should be both organized hierarchically (accessible by drill-down search through a rational structure) and searchable by keywords.
• • Information should be reachable from anywhere in the world (but secure and access restricted). Choose software that includes a cell phone/tablet platform client.
• • No information should ever be lost—the systems are such that additional information does not clog up or reduce efficiency of use and backup strategies ensure disaster robustness; therefore, it is possible to save everything.
• • Software tools optimized for specific management tasks should be used; select those tools based on interoperability, features, and the ability to export into common formats (such as XML) in case it becomes expedient someday to switch to a newer product.
• • One's digital world should be organized into several interlocking categories, which utilize different tools: activity (to-dos, projects, research goals) and knowledge (static information).
• • One's activity should be hierarchically organized according to a temporal scale, ranging from immediate goals all the way to career achievement objectives and core mission.
• • Storage of planning data should allow integration of plans with the information needed to implement them (using links to files and data in the various tools).
• • There should be no stored paper—everything should be obtained and stored in a digital form (or immediately digitized, using one of the tools described later in this document).
• • The information management tasks described herein should not occupy so much time as to take away from actual research. When implemented correctly, they result in a net increase in productivity.

The Life Cycle of Research Activity

Various projects occupy different places along a typical timeline. The life cycle extends from creative ideation to gathering information, to formulating a plan, to the execution for the plan, and then to producing a work product such as a grant or paper based on the results. Many of these phases necessitate feedback to a prior phase, shown in thinner arrows (for example, information discovered during a literature search or attempts to formalize the work plan may require novel brainstorming). This diagram shows the product (end result) of each phase and typical tools used to accomplish them.

These basic principles can be used as the skeleton around which specific strategies and new software products can be deployed. Whenever possible, these can be implemented via external administration services (i.e., by a dedicated project manager or administrator inside the group), but this is not always compatible with budgetary constraints, in which case they can readily be deployed by each principal investigator. The PIs also have to decide whether they plan to suggest (or insist) that other people in the group also use these strategies, and perhaps monitor their execution. In our experience, it is most essential for anyone leading a complex project or several to adopt these methods (typically, a faculty member or senior staff scientist), whereas people tightly focused on one project and with limited concurrent tasks involving others (e.g., Ph.D. students) are not essential to move toward the entire system (although, for example, the backup systems should absolutely be ensured to be implemented among all knowledge workers in the group). The following are some of the methods that have proven most effective in our own experience.

Information Technology Infrastructure

Several key elements should be pillars of your Information Technology (IT) infrastructure ( Figure 2 ). You should be familiar enough with computer technology that you can implement these yourself, as it is rare for an institutional IT department to be able to offer this level of assistance. Your primary disk should be a large (currently, ∼2TB) SSD drive or, better, a disk card (such as the 2TB SSD NVMe PCIe) for fast access and minimal waiting time. Your computer should be so fast that you spend no time (except in the case of calculations or data processing) waiting for anything—your typing and mouse movement should be the rate-limiting step. If you find yourself waiting for windows or files to open, obtain a better machine.

Schematic of Data Flow and Storage

Three types of information: data (facts and datasets), action plans (schedules and to-do lists), and work product (documents) all interact with each other in defining a region of work space for a given research project. All of this should be hosted on a single PC (personal computer). It is accessed by a set of regular backups of several types, as well as by the user who can interact with raw files through the file system or with organized data through a variety of client applications that organize information, schedules, and email. See Table 2 for definitions of special terms.

One key element is backups—redundant copies of your data. Disks fail—it is not a question of whether your laptop or hard drive will die, but when. Storage space is inexpensive and researchers' time is precious: team members should not tolerate time lost due to computer snafus. The backup and accessibility system should be such that data are immediately recoverable following any sort of disaster; it only has to be set up once, and it only takes one disaster to realize the value of paranoia about data. This extends also to laboratory inventory systems—it is useful to keep (and back up) lists of significant equipment and reagents in the laboratory, in case they are needed for the insurance process in case of loss or damage.

The main drive should be big enough to keep all key information (not primary laboratory data, such as images or video) in one volume—this is to facilitate cloning. You should have an extra internal drive (which can be a regular disk) of the same size or bigger. Use something like Carbon Copy Cloner or SuperDuper to set up a nightly clone operation. When the main disk fails (e.g., the night before a big grant is due), boot from the clone and your exact, functioning system is ready to go. For Macs, another internal drive set up as a Time Machine enables keeping versions of files as they change. You should also have an external drive, which is likewise a Time Machine or a clone: you can quickly unplug it and take it with you, if the laboratory has to be evacuated (fire alarm or chemical emergency) or if something happens to your computer and you need to use one elsewhere. Set a calendar reminder once a month to check that the Time Machine is accessible and can be searched and that your clone is actually updated and bootable. A Passport-type portable drive is ideal when traveling to conferences: if something happens to the laptop, you can boot a fresh (or borrowed) machine from the portable drive and continue working. For people who routinely install software or operating system updates, I also recommend getting one disk that is a clone of the entire system and applications and then set it to nightly clone the data only , leaving the operating system files unchanged. This guarantees that you have a usable system with the latest data files (useful in case an update or a new piece of software renders the system unstable or unbootable and it overwrites the regular clone before you notice the problem). Consider off-site storage. CrashPlan Pro is a reasonable choice for backing up laboratory data to the cloud. One solution for a single person's digital content is to have two extra external hard drives. One gets a clone of your office computer, and one is a clone of your home computer, and then you swap—bring the office one home and the home one to your office. Update them regularly, and keep them swapped, so that should a disaster strike one location, all of the data are available. Finally, pay careful attention (via timed reminders) to how your laboratory machines and your people's machines are being backed up; a lot of young researchers, especially those who have not been through a disaster yet, do not make backups. One solution is to have a system like CrashPlan Pro installed on everyone's machines to do automatic backup.

Another key element is accessibility of information. Everyone should be working on files (i.e., Microsoft Word documents) that are inside a Dropbox or Box folder; whatever you are working on this month, the files should be inside a folder synchronized by one of these services. That way, if anything happens to your machine, you can access your files from anywhere in the world. It is critical that whatever service is chosen, it is one that s ynchronizes a local copy of the data that live on your local machine (not simply keeps files in the cloud) —that way, you have what you need even if the internet is down or connectivity is poor. Tools that help connect to your resources while on the road include a VPN (especially useful for secure connections while traveling), SFTP (to transfer files; turn on the SFTP, not FTP, service on your office machine), and Remote Desktop (or VNC). All of these exist for cell phone or tablet devices, as well as for laptops, enabling access to anything from anywhere. All files (including scans of paper documents) should be processed by OCR (optical character recognition) software to render their contents searchable. This can be done in batch (on a schedule), by Adobe Acrobat's OCR function, which can be pointed to an entire folder of PDFs, for example, and left to run overnight. The result, especially with Apple's Spotlight feature, is that one can easily retrieve information that might be written inside a scanned document.

Here, we focus on work product and the thought process, not management of the raw data as it emerges from equipment and experimental apparatus. However, mention should be made of electronic laboratory notebooks (ELNs), which are becoming an important aspect of research. ELNs are a rapidly developing field, because they face a number of challenges. A laboratory that abandons paper notebooks entirely has to provide computer interfaces anywhere in the facility where data might be generated; having screens, keyboards, and mice at every microscope or other apparatus station, for example, can be expensive, and it is not trivial to find an ergonomically equivalent digital substitute for writing things down in a notebook as ideas or data appear. On the other hand, keeping both paper notebooks for immediate recording, and ELNs for organized official storage, raises problems of wasted effort during the (perhaps incomplete) transfer of information from paper to the digital version. ELNs are also an essential tool to prevent loss of institutional knowledge as team members move up to independent positions. ELN usage will evolve over time as input devices improve and best practices are developed to minimize the overhead of entering meta-data. However, regardless of how primary data are acquired, the researcher will need specific strategies for transitioning experimental findings into research product in the context of a complex set of personal, institutional, and scientific goals and constraints.

Facilitating Creativity

The pipeline begins with ideas, which must be cultivated and then harnessed for subsequent implementation ( Altshuller, 1984 ). This step consists of two components: identifying salient new information and arranging it in a way that facilitates novel ideas, associations, hypotheses, and strategic plans for making impact.

For the first step, we suggest an automated weekly PubCrawler search, which allows Boolean searches of the literature. Good searches to save include ones focusing on specific keywords of interest, as well as names of specific people whose work one wants to follow. The resulting weekly email of new papers matching specific criteria complements manual searches done via ISI's Web of Science, Google Scholar, and PubMed. The papers of interest should be immediately imported into a reference manager, such as Endnote, along with useful Keywords and text in the Notes field of each one that will facilitate locating them later. Additional tools include DevonAgent and DevonSphere, which enable smart searches of web and local resources, respectively.

Brainstorming can take place on paper or digitally (see later discussion). We have noticed that the rate of influx of new ideas is increased by habituating to never losing a new idea. This can be accomplished by establishing a voicemail contact in your cell phone leading to your own office voicemail (which allows voice recordings of idea fragments while driving or on the road, hands-free) and/or setting up Endnote or a similar server-synchronized application to record (and ideally transcribe) notes. It has been our experience that the more one records ideas arising in a non-work setting, the more often they will pop up automatically. For notes or schematics written on paper during dedicated brainstorming, one tool that ensures that nothing is lost is an electronic pen. For example, the Livescribe products are well integrated with Evernote and ensure that no matter where you are, anything you write down becomes captured in a form accessible from anywhere and are safe no matter what happens to the original notebook in which they were written.

Enhancing scientific thought, creative brainstorming, and strategic planning is facilitated by the creation of mind maps: visual representations of spatial structure of links between concepts, or the mapping of planned activity onto goals of different timescales. There are many available mind map software packages, including MindNode; their goal is to enable one to quickly set down relationships between concepts with a minimum of time spent on formatting. Examples are shown in Figures 3 A and 3B. The process of creating these mind maps (which can then be put on one's website or discussed with the laboratory members) helps refine fuzzy thinking and clarifies the relationships between concepts or activities. Mind mappers are an excellent tool because their light, freeform nature allows unimpeded brainstorming and fluid changes of idea structure but at the same time forces one to explicitly test out specific arrangements of plans or ideas.

Mind Mapping

(A and B) The task of schematizing concepts and ideas spatially based on their hierarchical relationships with each other is a powerful technique for organizing the creative thought process. Examples include (A), which shows how the different projects in our laboratory relate to each other. Importantly, it can also reveal disbalances or gaps in coverage of specific topics, as well as help identify novel relationships between sub-projects by placing them on axes (B) or even identify novel hypotheses suggested by symmetry.

(C) Relationships between the central nervous system (CNS) and regeneration, cancer, and embryogenesis. The connecting lines in black show typical projects (relationships) already being pursued by our laboratory, and the lack of a project in the space between CNS and embryogenesis suggests a straightforward hypothesis and project to examine the role of the brain in embryonic patterning.

It is important to note that mind maps can serve a function beyond explicit organization. In a good mapped structure, one can look for symmetries (revealing relationships that are otherwise not obvious) between the concepts involved. An obvious geometric pattern with a missing link or node can help one think about what could possibly go there, and often identifies new relationships or items that had not been considered ( Figure 3 C), in much the same way that gaps in the periodic table of the elements helped identify novel elements.

Organizing Information and Knowledge

The input and output of the feedback process between brainstorming and literature mining is information. Static information not only consists of the facts, images, documents, and other material needed to support a train of thought but also includes anything needed to support the various projects and activities. It should be accessible in three ways, as it will be active during all phases of the work cycle. Files should be arranged on your disk in a logical hierarchical structure appropriate to the work. Everything should also be searchable and indexed by Spotlight. Finally, some information should be stored as entries in a data management system, like Evernote or DevonThink, which have convenient client applications that make the data accessible from any device.

Notes in these systems should include useful lists and how-to's, including, for example:

• • Names and addresses of experts for specific topics
• • Emergency protocols for laboratory or animal habitats
• • Common recipes/methods
• • Lists and outlines of papers/grants on the docket
• • Information on students, computers, courses, etc.
• • Laboratory policies
• • Materials and advice for students, new group members, etc.
• • Lists of editors, and preferred media contacts
• • Lists of Materials Transfer Agreements (MTAs), contract texts, info on IP
• • Favorite questions for prospective laboratory members

Each note can have attachments, which include manuals, materials safety sheets, etc. DevonThink needs a little more setup but is more robust and also allows keeping the server on one's own machine (nothing gets uploaded to company servers, unlike with Evernote, which might be a factor for sensitive data). Scientific papers should be kept in a reference manager, whereas books (such as epub files and PDFs of books and manuscripts) can be stored in a Calibre library.

Email: A Distinct Kind of Information

A special case of static information is email, including especially informative and/or actionable emails from team members, external collaborators, reviewers, and funders. Because the influx of email is ever-increasing, it is important to (1) establish a good infrastructure for its management and (2) establish policies for responding to emails and using them to facilitate research. The first step is to ensure that one only sees useful emails, by training a good Bayesian spam filter such as SpamSieve. We suggest a triage system in which, at specific times of day (so that it does not interfere with other work), the Inbox is checked and each email is (1) forwarded to someone better suited to handling it, (2) responded quickly for urgent things that need a simple answer, or (3) started as a Draft email for those that require a thoughtful reply. Once a day or a couple of times per week, when circumstances permit focused thought, the Draft folder should be revisited and those emails answered. We suggest a “0 Inbox” policy whereby at the end of a day, the Inbox is basically empty, with everything either delegated, answered, or set to answer later.

We also suggest creating subfolders in the main account (keeping them on the mail server, not local to a computer, so that they can be searched and accessed from anywhere) as follows:

• • Collaborators (emails stating what they are going to do or updating on recent status)
• • Grants in play (emails from funding agencies confirming receipt)
• • Papers in play (emails from journals confirming receipt)
• • Waiting for information (emails from people for whom you are waiting for information)
• • Waiting for miscellaneous (emails from people who you expect to do something)
• • Waiting for reagents (emails from people confirming that they will be sending you a physical object)

Incoming emails belonging to those categories (for example, an email from an NIH program officer acknowledging a grant submission, a collaborator who emailed a plan of what they will do next, or someone who promised to answer a specific question) should be sorted from the Inbox to the relevant folder. Every couple of weeks (according to a calendar reminder), those folders should be checked, and those items that have since been dealt with can be saved to a Saved Messages folder archive, whereas those that remain can be Replied to as a reminder to prod the relevant person.

In addition, as most researchers now exchange a lot of information via email, the email trail preserves a record of relationships among colleagues and collaborators. It can be extremely useful, even years later, to be able to go back and see who said what to whom, what was the last conversation in a collaboration that stalled, who sent that special protocol or reagent and needs to be acknowledged, etc. It is imperative that you know where your email is being stored, by whom, and their policy on retention, storage space limits, search, backup, etc. Most university IT departments keep a mail server with limited storage space and will delete your old emails (even more so if you move institutions). One way to keep a permanent record with complete control is with an application called MailSteward Pro. This is a front-end client for a freely available MySQL server, which can run on any machine in your laboratory. It will import your mail and store unlimited quantities indefinitely. Unlike a mail server, this is a real database system and is not as susceptible to data corruption or loss as many other methods.

A suggested strategy is as follows. Keep every single email, sent and received. Every month (set a timed reminder), have MailSteward Pro import them into the MySQL database. Once a year, prune them from the mail server (or let IT do it on their own schedule). This allows rapid search (and then reply) from inside a mail client for anything that is less than one year old (most searches), but anything older can be found in the very versatile MailStewardPro Boolean search function. Over time, in addition to finding specific emails, this allows some informative data mining. Results of searches via MailStewardPro can be imported into Excel to, for example, identify the people with whom you most frequently communicate or make histograms of the frequency of specific keywords as a function of time throughout your career.

With ideas, mind maps, and the necessary information in hand, one can consider what aspects of the current operations plan can be changed to incorporate plans for new, impactful activity.

Organizing Tasks and Planning

A very useful strategy involves breaking down everything according to the timescales of decision-making, such as in the Getting Things Done (GTD) philosophy ( Figure 4 ) ( Allen, 2015 ). Activities range from immediate (daily) tasks to intermediate goals all the way to career-scale (or life-long) mission statements. As with mind maps, being explicit about these categories not only force one to think hard about important aspects of their work, but also facilitate the transmission of this information to others on the team. The different categories are to be revisited and revised at different rates, according to their position on the hierarchy. This enables you to make sure that effort and resources are being spent according to priorities.

Scales of Activity Planning

Activities should be assigned to a level of planning with a temporal scale, based on how often the goals of that level get re-evaluated. This ranges from core values, which can span an entire career or lifetime, all the way to tactics that guide day-to-day activities. Each level should be re-evaluated at a reasonable time frame to ensure that its goals are still consistent with the bigger picture of the level(s) above it and to help re-define the plans for the levels below it.

We also strongly recommend a yearly personal scientific retreat. This is not meant to be a vacation to “forget about work” but rather an opportunity for freedom from everyday minutiae to revisit, evaluate, and potentially revise future activity (priorities, action items) for the next few years. Every few years, take more time to re-map even higher levels on the pyramid hierarchy; consider what the group has been doing—do you like the intellectual space your group now occupies? Are your efforts having the kind of impact you realistically want to make? A formal diagram helps clarify the conceptual vision and identify gaps and opportunities. Once a correct level of activity has been identified, it is time to plan specific activities.

A very good tool for this purpose, which enables hierarchical storage of tasks and subtasks and their scheduling, is OmniFocus ( Figure 5 ). OmniFocus also enables inclusion of files (or links to files or links to Evernote notes of information) together with each Action. It additionally allows each action to be marked as “Done” once it is complete, providing not only a current action plan but a history of every past activity. Another interesting aspect is the fact that one can link individual actions with specific contexts: visualizing the database from the perspective of contexts enables efficient focus of attention on those tasks that are relevant in a specific scenario. OmniFocus allows setting reminders for specific actions and can be used for adding a time component to the activity.

Project Planning

This figure shows a screenshot of the OmniFocus application, illustrating the nested hierarchy of projects and sub-projects, arranged into larger groups.

The best way to manage time relative to activity (and to manage the people responsible for each activity) is to construct Gantt charts ( Figure 6 ), which can be used to plan out project timelines and help keep grant and contract deliverables on time. A critical feature is that it makes dependencies explicit, so that it is clear which items have to be solved/done before something else can be accomplished. Gantt charts are essential for complex, multi-person, and/or multi-step projects with strict deadlines (such as grant deliverables and progress reports). Software such as OmniPlanner can also be used to link resources (equipment, consumables, living material, etc.) with specific actions and timelines. Updating and evaluation of a Gantt chart for a specific project should take place on a time frame appropriate to the length of the next immediate phase; weekly or biweekly is typical.

Timeline Planning

This figure shows a screenshot of a typical Gantt chart, in OmniPlan software, illustrating the timelines of different project steps, their dependencies, and specific milestones (such as a due date for a site visit or grant submission). Note that Gantt software automatically moves the end date for each item if its subtasks' timing changes, enabling one to see a dynamically correct up-to-date temporal map of the project that adjusts for the real-world contingencies of research.

In addition to the comprehensive work plan in OmniFocus or similar, it is helpful to use a Calendar (which synchronizes to a server, such as Microsoft Office calendar with Exchange server). For yourself, make a task every day called “Monday tasks,” etc., which contains all the individual things to be accomplished (which do not warrant their own calendar reminder). First thing in the morning, one can take a look at the day's tasks to see what needs to be done. Whatever does not get done that day is to be copied onto another day's tasks. For each of the people on your team, make a timed reminder (weekly, for example, for those with whom you meet once a week) containing the immediate next steps for them to do and the next thing they are supposed to produce for your meeting. Have it with you when you meet, and give them a copy, updating the next occurrence as needed based on what was decided at the meeting to do next. This scheme makes it easy for you to remember precisely what needs to be covered in the discussion, serves as a record of the project and what you walked about with whom at any given day (which can be consulted years later, to reconstruct events if needed), and is useful to synchronize everyone on the same page (if the team member gets a copy of it after the meeting).

Writing: The Work Products

Writing, to disseminate results and analysis, is a central activity for scientists. One of the OmniFocus library's sections should contain lists of upcoming grants to write, primary papers that are being worked on, and reviews/hypothesis papers planned. Microsoft Word is the most popular tool for writing papers—its major advantage is compatibility with others, for collaborative manuscripts (its Track Changes feature is also very well implemented, enabling collaboration as a master document is passed from one co-author to another). But Scrivener should be seriously considered—it is an excellent tool that facilitates complex projects and documents because it enables WYSIWYG text editing in the context of a hierarchical structure, which allows you to simultaneously work on a detailed piece of text while seeing the whole outline of the project ( Figure 7 ).

Writing Complex Materials

This figure shows a screenshot from the Scrivener software. The panel on the left facilitates logical and hierarchical organization of a complex writing project (by showing where in the overall structure any given text would fit), while the editing pane on the right allows the user to focus on writing a specific subsection without having to scroll through (but still being able to see) the major categories within which it must fit.

It is critical to learn to use a reference manager—there are numerous ones, including, for example, Endnote, which will make it much easier to collaborate with others on papers with many citations. One specific tip to make collaboration easier is to ask all of the co-authors to set the reference manager to use PMID Accession Number in the temporary citations in the text instead of the arbitrary record number it uses by default. That way, a document can have its bibliography formatted by any of the co-authors even if they have completely different libraries. Although some prefer collaborative editing of a Google Doc file, we have found a “master document” system useful, in which a file is passed around among collaborators by email but only one can make (Tracked) edits at a time (i.e., one person has the master doc and everyone makes edits on top of that).

One task most scientists regularly undertake is writing reviews of a specific subfield (or Whitepapers). It is often difficult, when one has an assignment to write, to remember all of the important papers that were seen in the last few years that bear on the topic. One method to remedy this is to keep standing document files, one for each topic that one might plausibly want to cover and update them regularly. Whenever a good paper is found, immediately enter it into the reference manager (with good keywords) and put a sentence or two about its main point (with the citation) into the relevant document. Whenever you decide to write the review, you will already have a file with the necessary material that only remains to be organized, allowing you to focus on conceptual integration and not combing through literature.

The life cycle of research can be viewed through the lens of the tools used at different stages. First there are the conceptual ideas; many are interconnected, and a mind mapper is used to flesh out the structure of ideas, topics, and concepts; make it explicit; and share it within the team and with external collaborators. Then there is the knowledge—facts, data, documents, protocols, pieces of information that relate to the various concepts. Kept in a combination of Endnote (for papers), Evernote (for information fragments and lists), and file system files (for documents), everything is linked and cross-referenced to facilitate the projects. Activities are action items, based on the mind map, of what to do, who is doing what, and for which purpose/grant. OmniFocus stores the subtasks within tasks within goals for the PI and everyone in the laboratory. During meetings with team members, these lists and calendar entries are used to synchronize objectives with everyone and keep the activity optimized toward the next step goals. The product—discovery and synthesis—is embodied in publications via a word processor and reference manager. A calendar structure is used to manage the trajectory from idea to publication or grant.

The tools are currently good enough to enable individual components in this pipeline. Because new tools are continuously developed and improved, we recommend a yearly overview and analysis of how well the tools are working (e.g., which component of the management plan takes the most time or is the most difficult to make invisible relative to the actual thinking and writing), coupled to a web search for new software and updated versions of existing programs within each of the categories discussed earlier.

A major opportunity exists for software companies in the creation of integrated new tools that provide all the tools in a single integrated system. In future years, a single platform will surely appear that will enable the user to visualize the same research structure from the perspective of an idea mind map, a schedule, a list of action items, or a knowledge system to be queried. Subsequent development may even include Artificial Intelligence tools for knowledge mining, to help the researcher extract novel relationships among the content. These will also need to dovetail with ELN platforms, to enable a more seamless integration of project management with primary data. These may eventually become part of the suite of tools being developed for improving larger group dynamics (e.g., Microsoft Teams). One challenge in such endeavors is ensuring the compatibility of formats and management procedures across groups and collaborators, which can be mitigated by explicitly discussing choice of software and process, at the beginning of any serious collaboration.

Regardless of the specific software products used, a researcher needs to put systems in place for managing information, plans, schedules, and work products. These digital objects need to be maximally accessible and backed up, to optimize productivity. A core principle is to have these systems be so robust and lightweight as to serve as an “external brain” ( Menary, 2010 )—to maximize creativity and deep thought by making sure all the details are recorded and available when needed. Although the above discussion focused on the needs of a single researcher (perhaps running a team), future work will address the unique needs of collaborative projects with more lateral interactions by significant numbers of participants.

Acknowledgments

We thank Joshua Finkelstein for helpful comments on a draft of the manuscript. M.L. gratefully acknowledges support by an Allen Discovery Center award from the Paul G. Allen Frontiers Group (12171) and the Barton Family Foundation.

• Allen D. Revised edition. Penguin Books; 2015. Getting Things Done: The Art of Stress-free Productivity. [ Google Scholar ]
• Altshuller G.S. Gordon and Breach Science Publishers; 1984. Creativity as an Exact Science: The Theory of the Solution of Inventive Problems. [ Google Scholar ]
• Menary R. MIT Press; 2010. The Extended Mind. [ Google Scholar ]

• Advanced Search

Project management logics for agile public strategic management: : Propositions from the literature and a research agenda

New citation alert added.

This alert has been successfully added and will be sent to:

You will be notified whenever a record that you have chosen has been cited.

To manage your alert preferences, click on the button below.

New Citation Alert!

Please log in to your account

Information & Contributors

Bibliometrics & citations, view options, recommendations, information systems management in public sector organizations.

With the mounting pressure on public sector organizations to perform, information systems (IS) management in public sector is gaining importance. Public sector organizations are trying to emulate the IS practices of the private sector to achieve the ...

Strategic Project Management in Nigerian Public Research Organisations: The Gap in Practice

The aim of this study was to assess the application of strategic project management SPM in Nigerian public research organisations. A case study approach involving four R and D organisations in Nigeria was used. A total of 213 questionnaires were ...

Corporate Entrepreneurship and Strategic Management: Insights from a Process Study

This paper presents a model of the strategic process concerning entrepreneurial activity in large, complex organizations. Previous empirical and theoretical findings can be integrated in this new conceptual framework.

The paper makes the following key ...

Information

Published in.

Netherlands

Publication History

Author tags.

• Strategic agility
• agile government
• strategic management
• project management
• public sector
• problematizing literature review
• Research-article

Contributors

Other metrics, bibliometrics, article metrics.

• 0 Total Citations
• 0 Total Downloads
• Downloads (Last 12 months) 0
• Downloads (Last 6 weeks) 0

View options

Login options.

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Share this publication link.

Copying failed.

Share on social media

Affiliations, export citations.

• Please download or close your previous search result export first before starting a new bulk export. Preview is not available. By clicking download, a status dialog will open to start the export process. The process may take a few minutes but once it finishes a file will be downloadable from your browser. You may continue to browse the DL while the export process is in progress. Download
• Download citation
• Copy citation

We are preparing your search results for download ...

We will inform you here when the file is ready.

Your file of search results citations is now ready.

Your search export query has expired. Please try again.

• Ask a Librarian

ECET 380: Project Management

• Library and Online Resources

Literature Review

• Citation Resources
• Recommended Websites
• Terminology
• Instructable Examples
• Places to Purchase Materials
• Zotero Basics
• Free & Open Source Images

Further Resources

“A literature review is a document or section of a document that collects key sources on a topic and discusses those sources in conversation with each other (also called synthesis).” -- Purdue OWL

Literature Review Assignment Details

Your group will write the literature review together. 

• Include 4-5 professional sources (journal article, professional association article, specification details, etc.).
• IEEE citation format.
• A cover page with title, every team member listed. 
• Title and page number on each page, upper right hand corner.
• A References page.
• Assignment is due January 27, 11:59 PM.

All of your resources will be synthesized into one literature review. Follow the suggested steps for writing your literature review.

• Review the article themes.
• Notice the relationships between the themes and the different articles.
• Write down the overarching story between the articles.
• Introduction (what are you doing and why -- what problem are you going to solve?)
• Body (how will you do it -- what technology, materials etc. -- this is where patents, standards, articles will all inform those decisions)
• Conclusion (reiterate what, why, how)
• When writing your review, remember to use quotes sparingly and always cite where you got your information. Review signal phrases as a way to always give credit to authors.
• Learn the basics of a literature review from Purdue Owl's Writing a Literature Review
• Want to learn about the different types of literature reviews, check out Review: Outline, Strategies, and Examples ; by Study Corgi

To read example literature reviews:

1. Purdue Owl gives this example using APA.

2. And here is a paper within research gate that has a literature review section in IEEE (you just need to see the lit review as an example).

• << Previous: Library and Online Resources
• Next: Citation Resources >>
• Last Edited: May 29, 2024 4:28 PM
• URL: https://guides.lib.purdue.edu/entertain_tech

Information

• Author Services

Initiatives

You are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.

All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess .

Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.

Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.

Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.

Original Submission Date Received: .

• Active Journals
• Find a Journal
• Proceedings Series
• For Authors
• For Reviewers
• For Editors
• For Librarians
• For Publishers
• For Societies
• For Conference Organizers
• Open Access Policy
• Institutional Open Access Program
• Special Issues Guidelines
• Editorial Process
• Research and Publication Ethics
• Article Processing Charges
• Testimonials
• Preprints.org
• SciProfiles
• Encyclopedia

Article Menu

• Subscribe SciFeed
• Recommended Articles
• Google Scholar
• on Google Scholar
• Table of Contents

Find support for a specific problem in the support section of our website.

Please let us know what you think of our products and services.

Visit our dedicated information section to learn more about MDPI.

JSmol Viewer

Strategic digital city: multiple projects for sustainable urban management.

1. Introduction

2. literature review and background, 2.1. strategic digital city (sdc) concept and model, 2.2. strategic digital city (sdc) background, 2.3. sustainability, 2.4. strategic digital city and sustainability relationship, 3. materials and methods, 4. sdc multiple projects, 5. discussion, 6. conclusions, author contributions, institutional review board statement, informed consent statement, data availability statement, acknowledgments, conflicts of interest.

• Pira, S. The social issues of smart home: A review of four European cities’ experiences. Eur. J. Futures Res. 2021 , 9 , 3. [ Google Scholar ] [ CrossRef ]
• Yavuz, N. Gender differences in perception and usage of public transit technologies: Implications for digital government. Inf. Polity 2022 , 27 , 97–113. [ Google Scholar ] [ CrossRef ]
• Rezende, D.A. Strategic digital city: Concept, model, and research cases. J. Infrastruct. Policy Dev. 2023 , 7 , 2177. [ Google Scholar ] [ CrossRef ]
• Cucchiella, F.; Rotilio, M. Planning and prioritizing of energy retrofits for the cities of the future. Cities 2021 , 116 , 103272. [ Google Scholar ] [ CrossRef ]
• Simone, C.; Iandolo, F.; Fulco, I.; Loia, F. Rome was not built in a day. Resilience and the eternal city: Insights for urban management. Cities 2021 , 110 , 103070. [ Google Scholar ] [ CrossRef ]
• Chien, H.; Saito, O. Evaluating social-ecological fit in urban stream management: The role of governing institutions in sustainable urban ecosystem service provision. Ecosyst. Serv. 2021 , 49 , 101285. [ Google Scholar ] [ CrossRef ]
• Guerriero, A.; Busio, F.; Saidani, M.; Boje, C.; Mack, N. Combining Building Information Model and Life Cycle Assessment for Defining Circular Economy Strategies. Sustainability 2024 , 16 , 4561. [ Google Scholar ] [ CrossRef ]
• Hatem, F.M. Making cities smarter for an inclusive green transition towards a long-term sustainable development: A critical literature review. IET Smart Cities 2023 , 5 , 243–253. [ Google Scholar ] [ CrossRef ]
• Kornberger, M.; Meyer, R.E.; Höllerer, M.A. Exploring the long-term effect of strategy work: The case of Sustainable Sydney 2030. Urban Stud. 2021 , 58 , 3316–3334. [ Google Scholar ] [ CrossRef ]
• Manda, M.I. Power, politics, and the institutionalisation of information systems for promoting digital transformation in the public sector: A case of the South African’s government digital transformation journey. Inf. Polity 2022 , 27 , 311–329. [ Google Scholar ] [ CrossRef ]
• Christou, O.; Manou, D.B.; Armenia, S.; Franco, E.; Blouchoutzi, A.; Papathanasiou, J. Fostering a whole-institution approach to sustainability through systems thinking: An analysis of the state-of-the-art in sustainability integration in higher education institutions. Sustainability 2024 , 16 , 2508. [ Google Scholar ] [ CrossRef ]
• Gudmundsdottir, S.; Sigurjonsson, T.O. A need for standardized approaches to manage sustainability strategically. Sustainability 2024 , 16 , 2319. [ Google Scholar ] [ CrossRef ]
• Nahmo, L.; Rwizi, L.; Mpandeli, S.; Botai, J.; Magidi, J.; Tazvinga, H.; Sobratee, N.; Liphadzi, S.; Naidoo, D.; Modi, A.T.; et al. Urban nexus and transformative pathways towards a resilient Gauteng City-Region, South Africa. Cities 2021 , 116 , 103266. [ Google Scholar ] [ CrossRef ]
• Han, H.; Wu, Y.; Su, Z.; Zurlo, F. Design-driven innovation in urban context—Exploring the sustainable development of city design weeks. Sustainability 2024 , 16 , 1299. [ Google Scholar ] [ CrossRef ]
• Drumaux, A.; Ravet, J. European strategy 2020: What about the carrot? CEB Work. Pap. 2016 , 16 , 1–14. [ Google Scholar ] [ CrossRef ]
• Senousi, A.M.; Zhang, J.; Shi, W.; Liu, X. A proposed framework for identification of indicators to model high-frequency cities. ISPRS Int. J. Geo-Inf. 2021 , 10 , 317. [ Google Scholar ] [ CrossRef ]
• Naterer, A.; Žižek, A.; Lavrič, M. The quality of integrated urban strategies in light of the Europe 2020 strategy: The case of Slovenia. Cities 2018 , 72 , 369–378. [ Google Scholar ] [ CrossRef ]
• Perveen, S.; Kamruzzaman, M.; Yigitcanlar, T. Developing policy scenarios for sustainable urban growth management: A Delphi approach. Sustainability 2017 , 9 , 1787. [ Google Scholar ] [ CrossRef ]
• Benito, B.; Guillamón, M.; Ríos. A. Transparency and efficient management in local governments. Cities 2021 , 115 , 103234. [ Google Scholar ] [ CrossRef ]
• Meričková, B.M.; Muthová, N.J. Innovative concept of providing local public services based on ICT. NISPAcee J. Public Adm. Policy 2021 , 14 , 135–167. [ Google Scholar ] [ CrossRef ]
• Fang, Y.; Shan, Z. Optimising smart city evaluation: A people-oriented analysis method. IET Smart Cities 2023 , 6 , 41–53. [ Google Scholar ] [ CrossRef ]
• Uskuplu, T.; Colakoglu, B. Using social network data and space syntax analyses for developing urban strategies: Kadikoy case. Megaron 2019 , 14 , 269–278. [ Google Scholar ] [ CrossRef ]
• Ahvenniemi, H.; Huovila, A.; Pinto-Seppä, I.; Airaksinen, M. What are the differences between sustainable and smart cities? Cities 2017 , 60 , 234–245. [ Google Scholar ] [ CrossRef ]
• Krampe, F.; Hegazi, F.; Van Deveer, S.D. Sustaining peace through better resource governance: Three potential mechanisms for environmental peacebuilding. World Dev. 2021 , 144 , 105508. [ Google Scholar ] [ CrossRef ]
• Fumagalli, L.A.W.; Rezende, D.A.; Guimarães, T.A. Data intelligence in public transportation: Sustainable and equitable solutions to urban modals in strategic digital city subproject. Sustainability 2022 , 14 , 4683–4699. [ Google Scholar ] [ CrossRef ]
• Rezende, D.A.; Procopiuck, M.; Figueiredo, F.C. Public policy and a strategic digital city project: A case study of the Brazilian Municipality of Vinhedo. J. Urban Technol. 2015 , 22 , 63–83. [ Google Scholar ] [ CrossRef ]
• González-Galván, O.S. Understanding government discourses on social media: Lessons from the use of YouTube at local level. Inf. Polity 2022 , 27 , 343–356. [ Google Scholar ] [ CrossRef ]
• Jiang, H.; Geertman, S.; Witte, P. Smart urban governance: An urgent symbiosis? Inf. Polity 2019 , 24 , 245–269. [ Google Scholar ] [ CrossRef ]
• Diaz-Balteiro, L.; González-Pachón, J.; Romero, C. Sustainability as a multi-criteria concept: New developments and applications. Sustainability 2020 , 12 , 7527. [ Google Scholar ] [ CrossRef ]
• Esashika, D.; Masiero, G.; Mauger, Y. An investigation into the elusive concept of smart cities: A systematic review and meta-synthesis. Technol. Anal. Strateg. Manag. 2021 , 33 , 957–969. [ Google Scholar ] [ CrossRef ]
• Bastidas, V.; Oti-Sarpong, K.; Nochta, T.; Wan, L.; Tang, J.; Schooling, J. Leadership of urban digital innovation for public value: A competency framework. IET Smart Cities 2023 , 1–16. [ Google Scholar ] [ CrossRef ]
• Huang, Y.; Peng, H.; Wen, L.; Xing, T. Using digital technologies to plan and manage the pipelines network in city. IET Smart Cities 2023 , 5 , 95–110. [ Google Scholar ] [ CrossRef ]
• Dirsehan, T.; Van Zoonen, L. Smart city technologies from the perspective of technology acceptance. IET Smart Cities 2022 , 4 , 197–210. [ Google Scholar ] [ CrossRef ]
• Flores, C.C.; Rezende, D.A. Crowdsourcing framework applied to strategic digital city projects. J. Urban Manag. 2022 , 11 , 467–478. [ Google Scholar ] [ CrossRef ]
• Ribeiro, S.S.; Rezende, D.A.; Yao, J. Toward a model of the municipal evidence-based decision process in the strategic digital city context. Inf. Polity 2019 , 24 , 305–324. [ Google Scholar ] [ CrossRef ]
• Teixeira, A.V.; Rezende, D.A. A multidimensional information management framework for strategic digital cities: A comparative analysis of Canada and Brazil. Glob. J. Flex. Syst. Manag. 2023 , 24 , 107–121. [ Google Scholar ] [ CrossRef ]
• Rezende, D.A. Digital city projects: Information and public services offered by Chicago (USA) and Curitiba (Brazil). Int. J. Knowl. Soc. Res. 2016 , 7 , 16–30. [ Google Scholar ] [ CrossRef ]
• Sikdar, S. Measures for sustainability. Clean Technol. Environ. Policy 2020 , 22 , 279–280. [ Google Scholar ] [ CrossRef ]
• Kashani, S.J.; Hajian, M. 14—Indicators of sustainability. Sustain. Resour. Manag. 2021 , 317–334. [ Google Scholar ] [ CrossRef ]
• Nautiyal, H.; Goel, V.; Nautiyal, H.; Goel, V. Chapter 3—Sustainability assessment: Metrics and methods. Methods Sustain. Sci. 2021 , 27–46. [ Google Scholar ] [ CrossRef ]
• Le, X.; Shao, X.; Gao, K. The relationship between urbanization and consumption upgrading of rural residents under the sustainable development: An empirical study based on mediation effect and threshold effect. Sustainability 2023 , 15 , 8426. [ Google Scholar ] [ CrossRef ]
• Gonzáles, L.R.; Gale, F. Sustainability as economic value pluralism: Implications for urban politics and policy. Cities 2023 , 134 , 104167. [ Google Scholar ] [ CrossRef ]
• Adams, T.; Jameel, S.M.; Goggins, J. Education for sustainable development: Mapping the SDGs to University Curricula. Sustainability 2023 , 15 , 8340. [ Google Scholar ] [ CrossRef ]
• Dabard, C.H.; Mann, C. Sustainability innovations: A proposal for an analytical framework and its empirical application in the Schorfheide-Chorin Biosphere Reserve. Sustain. Sci. 2023 , 18 , 1085–1098. [ Google Scholar ] [ CrossRef ]
• Gargiulo, C.; Zucaro, F. A Method proposal to adapt urban open-built and green spaces to climate change. Sustainability 2023 , 15 , 8111. [ Google Scholar ] [ CrossRef ]
• Rosário, A.T.; Dias, J.C. Sustainability and the digital transition: A literature review. Sustainability 2022 , 14 , 4072. [ Google Scholar ] [ CrossRef ]
• Ramani, S.V.; Hettiarachchi, H. (Eds.) SDG11, Sustainable Cities and Communities , 1st ed.; Routledge: Delhi, India, 2022. [ Google Scholar ] [ CrossRef ]
• United Nations—SDGS-11. 2023. Available online: https://sdgs.un.org/goals/goal11 (accessed on 24 November 2023).
• Grainger-Brown, J.; Malekpour, S.; Raven, R.; Taylor, E. Exploring urban transformation to inform the implementation of the sustainable development goals. Cities 2022 , 131 , 103928. [ Google Scholar ] [ CrossRef ]
• Cash-Gibson, L.; Isart, F.M.; Martínez-Herrera, E.; Herrera, J.M.; Benach, J. Towards a systemic understanding of sustainable wellbeing for all in cities: A conceptual framework. Cities 2023 , 133 , 104143. [ Google Scholar ] [ CrossRef ]
• Serbanica, C.; Constantin, D.L. Misfortunes never come singly: A holistic approach to urban resilience and sustainability challenges. Cities 2023 , 134 , 104177. [ Google Scholar ] [ CrossRef ]
• Khan, M.; Khan, S. Proactively effecting community engagement in PPP projects: Lessons from the Tama Plaza redevelopment project, Yokohama. Sustainability 2024 , 16 , 180. [ Google Scholar ] [ CrossRef ]
• Morin, J.; Olsson, C.; Atikcan, E.Ö. Research Methods in the Social Sciences: An a-z of Key Concepts ; Oxford University Press: New York, NY, USA, 2021. [ Google Scholar ]
• Nichols, A.L.; Edlund, J. The Cambridge Handbook of Research Methods and Statistics for the Social and Behavioral Sciences ; Cambridge University Press: Cambridge, UK, 2023. [ Google Scholar ]
• Nachmias, F.C.; Nachmias, D. Research Methods in the Social Sciences , 7th ed.; Worth: New York, NY, USA, 2008. [ Google Scholar ]
• Tashakkori, A.M.; Johnson, R.B.; Teddlie, C.B. Foundations of Mixed Methods Research: Integrating Quantitative and Qualitative Approaches in the Social and Behavioral Sciences (Applied Social Research Methods) , 2nd ed.; Sage Publications: Thousand Oaks, CA, USA, 2020. [ Google Scholar ]
• Yin, R.K. Case Study Research and Applications: Design and Methods , 6th ed.; Sage Publications: Thousand Oaks, CA, USA, 2017. [ Google Scholar ]
• From, D.A. Model for Providing Municipal Public Services Connected through the Internet of Things in the Context of the Strategic Digital City. Ph.D. Thesis, Pontifícia Universidade Católica do Paraná (PUCPR), Curitiba, Brazil, 2022. [ Google Scholar ]
• Flores, C.C. Crowdsourcing Model for Strategic Digital City Projects. Ph.D. Thesis, Pontifícia Universidade Católica do Paraná (PUCPR), Curitiba, Brazil, 2021. [ Google Scholar ]
• Araujo, R.C.A. Smart Agent-Based Public Intelligence Model in the Strategic Digital City Services Context. Ph.D. Thesis, Pontifícia Universidade Católica do Paraná (PUCPR), Curitiba, Brazil, 2020. [ Google Scholar ]
• Teixeira, A.V. Multidimensional Information Management Model for the Strategic Digital City. Ph.D. Thesis, Pontifícia Universidade Católica do Paraná (PUCPR), Curitiba, Brazil, 2018. [ Google Scholar ]
• Ribeiro, S.S. Municipal Decision-Making Process Model in the Strategic Digital City Context. Ph.D. Thesis, Pontifícia Universidade Católica do Paraná (PUCPR), Curitiba, Brazil, 2018. [ Google Scholar ]
• Kanufre, R.A.M. Intersectoral Public Management Analysis Model in the Strategic Digital City Context RAK. Ph.D. Thesis, Pontifícia Universidade Católica do Paraná (PUCPR), Curitiba, Brazil, 2017. [ Google Scholar ]
• Ferreira, L.C. Model for Basic Sanitation Management in the Strategic Digital City Context. Ph.D. Thesis, Pontifícia Universidade Católica do Paraná (PUCPR), Curitiba, Brazil, 2015. [ Google Scholar ]
• Franco, L.M.G. Public Accounts Model in the Strategic Digital City Context. Ph.D. Thesis, Pontifícia Universidade Católica do Paraná (PUCPR), Curitiba, Brazil, 2015. [ Google Scholar ]
• Figueiredo, F.C. Digital Participatory Budget Model in the Strategic Digital City Context. Ph.D. Thesis, Pontifícia Universidade Católica do Paraná (PUCPR), Curitiba, Brazil, 2014. [ Google Scholar ]
• Leite, L.O. Municipal Government Performance Management: Proposal and Analysis of an Electronic Government Model as a Strategic Resource. Ph.D. Thesis, Pontifícia Universidade Católica do Paraná (PUCPR), Curitiba, Brazil, 2012. [ Google Scholar ]
• Baú. D. Access Model for Urban Workers and Strategic Digital City. Ph.D. Thesis, Pontifícia Universidade Católica do Paraná (PUCPR), Curitiba, Brazil, 2023.

Share and Cite

Rezende, D.A.; Almeida, G.G.F.; Fumagalli, L.A.W. Strategic Digital City: Multiple Projects for Sustainable Urban Management. Sustainability 2024 , 16 , 5450. https://doi.org/10.3390/su16135450

Rezende DA, Almeida GGF, Fumagalli LAW. Strategic Digital City: Multiple Projects for Sustainable Urban Management. Sustainability . 2024; 16(13):5450. https://doi.org/10.3390/su16135450

Rezende, Denis Alcides, Giovana Goretti Feijó Almeida, and Luis André Wernecke Fumagalli. 2024. "Strategic Digital City: Multiple Projects for Sustainable Urban Management" Sustainability 16, no. 13: 5450. https://doi.org/10.3390/su16135450

Article Metrics

Further information, mdpi initiatives, follow mdpi.

Subscribe to receive issue release notifications and newsletters from MDPI journals

• SUGGESTED TOPICS
• The Magazine
• Newsletters
• Managing Yourself
• Managing Teams
• Work-life Balance
• The Big Idea
• Data & Visuals
• Reading Lists
• Case Selections
• HBR Learning
• Topic Feeds
• Account Settings
• Email Preferences

Share Podcast

Why Project-Based Work Fails — and How to Get It Right

If your team is prioritizing project-based work, this episode is for you.

• Apple Podcasts

Companies of every size across the world are basing more of their work around projects than at any time in the past. But research shows that nearly two-thirds of those efforts fail.

Antonio Nieto-Rodriguez , who has studied projects and project management for decades, argues that at least some of the blame for these failures lies with executives who misunderstand the fundamentals of projects and fail to dedicate enough of their time to those they sponsor.

In this episode, Nieto-Rodriguez explains how to get better outcomes from project-based work. He also discusses how to frame projects, structure organizations around them, and avoid common pitfalls.

Key episode topics include: strategy, project management, operations strategy, organizational change.

HBR On Strategy curates the best case studies and conversations with the world’s top business and management experts, to help you unlock new ways of doing business. New episodes every week.

• Listen to the full HBR IdeaCast episode: The Future of Work Is Projects—So You’ve Got to Get Them Right (2021)
• Find more episodes of HBR IdeaCast
• Discover 100 years of Harvard Business Review articles, case studies, podcasts, and more at HBR.org

HANNAH BATES: Welcome to HBR On Strategy, case studies and conversations with the world’s top business and management experts, hand-selected to help you unlock new ways of doing business.

Companies of every size, in every industry across the world are basing more of their work around projects than any time in the past. But research shows that nearly two-thirds of those efforts fail.

Antonio Nieto-Rodriguez, who has studied projects and project management for decades, argues that at least some of the blame for these failures lies with executives – who misunderstand the fundamentals of projects and fail to dedicate enough of their time to the projects they sponsor.

In this episode, you’ll learn how to get better outcomes from project-based work. You’ll also learn how to frame projects, structure organizations around them, and avoid key pitfalls.

If your team is taking on project-based work or if you’re leading a new project, this episode is for you. It originally aired on HBR IdeaCast in November 2021. Here it is.

ALISON BEARD: Welcome to the HBR IdeaCast from Harvard Business Review. I’m Alison Beard. If the 20th century was all about operational efficiency in businesses, the 21st century is all about organizational change. And how do new initiatives, products and services, strategies or business models advance? Through project work. It’s what our guest today calls the project economy, and it’s estimated to generate $20 trillion in economic activity and employ 88 million people in project management related roles by 2027.

That’s across every industry and size of company in every part of the world, and yet research indicates that only 35% of projects are successful. At this increasingly critical business function, most of us are doing a pretty terrible job, so how do we get better at it going forward? Antonio Nieto-Rodriguez is the former chairman of the Project Management Institute, founder of Projects & Co. and the author of the HBR Project Management Handbook. He’s here to talk about emerging best practices for companies and the people in them. Antonio, welcome.

ANTONIO NIETO-RODRIGUEZ: Thank you, Alison. It’s a pleasure to be here.

ALISON BEARD: Project management seems like a clear idea, but how do you define it and think about it in a way that might be different than what people assume?

ANTONIO NIETO-RODRIGUEZ: Well, I think one of the challenges with project management that I face personally in my career is that as soon as you talk project management, senior executives and people who are not experts in project management, they think, “Oh, this is something very technical, very tactical. It’s nothing for me,” so I’ve been facing that kind of discontent or disinterest in project management for 25 years. So, for me, I want to move out from that project management term and move it up into projects, and we all do projects. And for me, the definition is anything that has to deal with change, that’s projects. You can manage them through project management, Agile methods, design thinking, product management. But I want to really, I think we need to elevate and say, “Well, all what goes around change, that’s projects,” and we need to manage them.

ALISON BEARD: And how has project work changed over the past few decades?

ANTONIO NIETO-RODRIGUEZ: Well, project work has changed in two big areas. One is on a macro level. I’ve been doing research, and of course we all talk about the Marshall Plan after the Second War and all the projects that came from U.S. funding to develop Europe, reconstruct Europe, that was about $13 billion. Then we talk about the financial crisis in 2008 and ’09, we were talking about $3 trillion of projects. And now after the pandemic, we’re talking about $15, $20 trillion of projects. I think the world will never see as many projects as what we’re going to see in the next decade. We need to reconstruct countries, healthcare systems, economies, so that’s from a macro perspective.

From a micro perspective, from the way work is organized in companies, in businesses, it has evolved significantly in the sense that so far, operations have been prime in most of the organizations over the past 80 years. That’s what I say, the world driven by efficiency, where most of the activities were around doing things cheaper, faster, more automated, more volumes. Companies have been organized for that. That’s why you have hierarchies, that’s where cultures like command and control have been in place and so on, but since a few years when artificial intelligence and robots are taking over a big chunk of operations, the type of work is shifted to project based. So, I think the biggest, biggest disruption that happens in the world of projects is what we’re experiencing now. A radical shift from operations to project based work.

ALISON BEARD: And that’s because projects are about sort of discovering the new innovating, and the pace of change is such in every industry now that every company needs to learn how to do this well?

ANTONIO NIETO-RODRIGUEZ: Absolutely, and one of the challenges I have to admit, Alison, I’m a big of course, expert in project, a big advocate of project management, but our performance, like you mentioned in introduction, has been poor or appalling. I think project management has not delivered the expected results. We need to find better ways to addressing the change. The future 10 years ago maybe was five years from now, right? So, you would have a project that would last for three years expecting to get some benefits maybe in three, four years, a digital transformation, a new M&A activity, a new business unit, but today, the future is so fast.

So, your future is tomorrow, right? So, that means the acceleration of project based work has to go faster. Let me give you a quick example. Here in Brussels, they were setting, establishing a hospital from scratch, Greenfield, start of the construction in 2016, completion of the hospital in 2020. So, four years of construction, state of the art, but to my surprise, the hospital was open in 2018 before it was co completed. So, I think there’s no company in the world can wait four years to get any benefits from the projects. The future is now, and we need to address that. That’s why you see exploding the number of projects in organizations. I come across companies where they have more projects than people.

ALISON BEARD: And I do want to get to how to do it better, but first, that failure rate is so high. What are some of the most common challenges or problems that projects run into? Why are we getting it so wrong right now?

ANTONIO NIETO-RODRIGUEZ: Let me highlight just three. First, I think senior leaders, they don’t have the competencies to be effective sponsorship. Over the years …

ALISON BEARD: They’re not going to like hearing that.

ANTONIO NIETO-RODRIGUEZ: I’m sure. I’m sure, but I’m sorry. I always am hundred percent honest on both on my thinking, but I think sponsors have not realized the role is crucial in sponsoring projects. It’s not about how many projects you sponsor, that has been the kind of, “I sponsor 20 projects. I’m the most important person in this company.” Well, now it’s about less is more, and it has been proven. When you are an executive, the CEO, the VP, and you dedicate time to your project, time means not just one hour per month, but a half a day per week. If this is the future of your business, I don’t understand why senior leaders don’t dedicate so much time. They’re all driven by operations and day to day urgency so very few leaders make the space. And second, they don’t understand the fundamentals of projects.

Most of the executives come from a path marketing, finance, operation strategy, and it requires for them to understand that projects are different. That you work in projects in a matrix, that is not so much the hierarchical approach, but this team working and collaboration. So, it’s hard to give you a number, Alison, but I would say 30% to and 40% of the success of the project is if the senior leaders is engaged and understands and drives the project. Alison, the second point, I realized that in the area of change in projects, we are always running with all methods. It happened in the past with IT projects, I started implementing big ERP systems, we were trying to apply some very traditional project management.

It didn’t work. Then Agile came and said, “Well, now we are going to use Agile for every project,” and that, we see today with digital transformations, AI implementations, that doesn’t work. The failure keeps there. The third reason, so I think the role of the project manager, the project management profession has not taken ownership of the results. It has been very focused on process, very focused on documentation. It did make a lot of sense in the sixties, in the fifties where you would do a lot of public sector projects where you want to document everything, but I think the reason that the third reason for me is that project management didn’t evolve to embrace the new reality. And second, project managers have been more a deliver type of role.

In project management, we always said, “Well, who’s accountable for delivering the projects? Who’s accountable for delivering the benefits?” Right? Well, it’s the sponsor. We project managers were responsible of delivering the project on time, on budget, on scope, and that has been the cradle for project management for the last 40 years. And we’ve missed to focus on the outcomes. We’ve missed to focus on the benefits. We’ve missed to take accountability of the results. It’s easy to make a project charter, but what companies are looking for is delivering value, either financial, either social, either sustainability. So, I’m asking my community of project managers to step up, to take ownership, to say, “No, it’s not just the plan. It’s not just delivery on time. What matters actually even more is delivering the benefits, whatever they are, and faster, please.”

ALISON BEARD: So, for an organization that does have existing operations that need to be managed, but then also wants to pursue change and innovation through project work, how does that company change its structure or culture to be able to do both well?

ANTONIO NIETO-RODRIGUEZ: Yeah. Well, great question, Alison. I’ve seen so many companies struggling because I’m not saying, “Let’s forget about what you’re doing right today. Let’s forget about that organization that you’ve built so successful for that world driven by efficiency with hierarchies, with yearly plans, with deep expertise, deep technical expertise,” but how can we address change? And change that’s going very fast and our products are just lasting less and less. In the past, we last five years, now five weeks or maybe five months. So, how can we mix that? And it’s a struggle. You cannot say, “Let’s forget my hierarchy and let’s move everything into flat teams and Agile structures and project basing.” That doesn’t work, so I think in the challenge for the leaders, the senior leaders, the executives, is finding that balance. And I always say you need to experiment.

You cannot just go and say, “Well, half of the organization is working without job descriptions. They’re all working project based.” I think my approach, my suggestion is, what are your top five projects? What are the five most important projects that your organization has to deliver? Extract those projects from your daily operations. Extract them. They should not be done by people working in operations. They should have a different structure. They should have a different culture. Put them aside, put them independent. They are own entities, and of course, strong sponsorship. Executives, you need to spend time on them. By extracting for those five top projects already, and moving out to that from that hierarchical structure, that operational activities, that you can see already, quite a lot of acceleration in the way you deliver projects.

ALISON BEARD: Often though, it seems as if particularly project leaders do have operational responsibilities as well, and then sort of, they’re expected to tack the project on top of that. So, how are companies that you work with navigating that balance? Are they giving the executives that time to take away for the project work?

ANTONIO NIETO-RODRIGUEZ: Great, great question. This is really the core. One of the core problems I always raise when I do workshops with senior leaders is, how come you cannot extract people from your day to day job and put them in a project? It’s one of the biggest challenges that I see. Even companies which have 10,000 people, they are not able to free up 50 people to carry out the project. The best projects I’ve seen in a research, one of them, of course the iPhone, the first one which I research very much in detail, at that time, they were able to take the day to day people, the senior leader, the best people of Apple at that time and extract them for two years and a half to develop.

And people who were in the operations side said, “Well, I love to join this project, but who’s going to do my day to day activities?” And we were saying, “Don’t bother. Anybody can do your day to day activity. You have a deputy when you’re gone,” said, “We’ll put those people. We’ll promote them. We’ll create more talent, but you, you are the best person in these companies. How come you’re not working in the most strategic project in the future of your work?” Right? It doesn’t make sense, but companies struggle so much and there’s nothing worse that you can do, Alison, than have half time people working in your projects. I work one hour per week, then I work two days per week, then it’s a mess. It’s not how you deliver great projects. At least try to get the best people around.

ALISON BEARD: I think that makes sense when you sort of have a clear idea of what the future’s going to look like, and you know exactly which five projects are the most important, but isn’t the issue in many cases that organizations sort of have 30 projects on the go, and aren’t really sure what’s going to pan out, and they can’t take all of those people away from their day to day activities? So, how do companies prioritize?

ANTONIO NIETO-RODRIGUEZ: I’m sure everybody that’s listening this, they can’t relate to that point. Companies have way too many projects. I think that if there’s a core skill for leaders in current times, is focused and prioritization. Knowing what is the big path, and unfortunately, it’s just very hard to see when you see more projects than employees. And like you’re saying, how can they do their day to day job plus three, four time projects? That’s where people get overwhelmed. I am sure that the big reset is linked to this, so many projects plus day to day activities. It’s just stressing everybody out, and I think that when you work with companies where the priorities are clear, where people know, these are our top three, their top five, and we know where we’re going, this is the focus, that’s where I think executives need to work on. On really making the tough decisions.

ALISON BEARD: What are some best practices for putting project teams together?

ANTONIO NIETO-RODRIGUEZ: Well, Alison, the formula for engagement is super simple. The most engaged people in a project, you know which one is it? Volunteers. Let me put you an example. Maybe in HBR, you are launching a new project. Why don’t you ask who wants to join?

ALISON BEARD: Makes sense. It’s so simple, but it makes so much sense.

ANTONIO NIETO-RODRIGUEZ: It’s so simple, because there’s different things that happen here. First, if nobody wants to volunteer in that project, that project is terrible. Don’t start it. Don’t start it because it’s just, people are going to be forced to do it, so ask for volunteers. Nobody shows up, don’t start it. You don’t need a business case of three months hiring consultants to make you, “Yes, this is …” If nobody jumps on it, terrible. Don’t even start it. It’s just a five minutes test and you save three months of work. Second, if the project talks about business case, very few people get excited, yeah? Who wants to work in a project that delivers 10% return on investment? Yes, nobody. Right? 15%, nobody. Who wants to work in a project which is going to make a more sustainable world? Who’s going to work in a project who’s going to increase the customer experience and make customers more happy, and deliver better value to a customer?

Who wants to work in a project who’s going to create our employees or make our employees more happy, and make us a top company? Lots of people. So, we have been, when we were talking about some of the issues, I think project manage has been focused on talking about things that don’t matter to most of the stakeholders, like a business case. Business case is super important. It’s the return investment, for sure, but that’s not what engages people. The purpose engages people. When you have volunteers, they will dream about your projects. They will do whatever they can to make it happen, and it can be because of the purpose, it can be because they like to work with you, they see a big opportunity to learn. Of course, as a project leader, you need to balance that. But as simple as that, Alison, “Who wants to volunteer?”

ALISON BEARD: How does the rise of project driven work relate to the gig economy? Is your sense that companies are hiring contractors and freelancers to get a lot of this done? Is it a balance or are they trying to handle most of it in-house?

ANTONIO NIETO-RODRIGUEZ: I think when we started to hear about the gig economy, I think yes, one of the reasons was, “Let’s hire external resources to work in our projects because we are so rigid internally, like I cannot free, from my 10,000 people, I cannot free 10 of them because they’re so busy in day to day operations plus other projects,” so it started like that. What I see now is that companies are finally taking the step of shifting resources to more project based work. Again, when I use the word project, I include Agile teams, self directed, so very flat project driven teams. So, that’s happening to the point that I talk about it is that companies are canceling job descriptions. We all had job descriptions like, Alison, most of the people listening, probably they had a job description, which tried to describe like, where do you fit in this box? Right? And just do those activities in this box, in that operational field.

That’s your box. If you do it right in two, three years, you just go up in the structure. But many large companies and small companies are realizing that people don’t work in boxes anymore, and job descriptions are not needed anymore. It’s a thing from that world driven by efficiency that together with the chief operating officer in this role, so I think they will not last very long. So, I think the project driven world is now being and embraced by organization where companies like Alibaba or other major players are really embracing this type of work where yeah, they’re looking for people who can have an idea, who can develop the idea, who can implement the project, and who can run the idea of the product or the business and generate value for the companies. This is what I call end-to-end players or strategy implementation professionals. We want this type of end-to-end players who can work transversely in organizations.

ALISON BEARD: Are there lessons from your project management world that might be helpful for people doing more traditional ongoing work?

ANTONIO NIETO-RODRIGUEZ: I think project managers have been a bit not very proud about their work. They’ve been seeing like, “Okay, you are not very modern. Agile teams are better, or innovation people,” so I think as a project manager, you need to believe on what you’re doing. Second, I think we need to take more ownership. I’ve been working 25 years in this space and managing large transformation M&A, and I always was waiting for the sponsor. I know the sponsor was very important for my projects, but I was kind of waiting and hoping that the sponsor will learn and follow training on how to do it or make some time for my projects. And I’ve learned the lesson is that the first thing I do in my projects is I go to the sponsor and talk frankly with the sponsor.

“Listen, are you ready to put time on this project? It’s very important. I need you, and I’m happy to coach you. I’m happy to tell you how projects work and what do we need to focus on, but I need your time, and I need a couple of hours per month. Let’s say an hour every two weeks. I need to talk to you. I need decisions from you.” So, I’m very much proactive because I know that role is very important and these people are really busy. One of the biggest lesson learned was being proactive with my project. The second maybe is I talk to many project managers and we are very technical to the point of sometimes difficult to understand, slash boring, right? Who wants to talk to a project manager? Come on. Do you have something more interesting? No, but that’s …

ALISON BEARD: You’re more interesting than I imagine, than my sort of vision of what the project manager is.

ANTONIO NIETO-RODRIGUEZ: You see? Because I don’t talk about project management, I don’t talk about Gantt charts, I don’t think that’s my kitchen. That’s what I do when I need to think about making a plan, but you are interested on the bigger picture. You are interested on how my ideas will contribute to our needs as an organization, so I do this exercise with project managers, “Tell your partner what you do without mentioning the words projects and project management,” and they say, “Oh, I’m struggling. What do I do?” And then they start talking about the value they bring, and that’s what people want to hear.

You covered this topic broadly in HBR, but talking, adapting, understanding the language of your stakeholders, using it. That’s how you get their engagement. That’s how you get their attention. That’s how they appreciate your value, and that’s the second big learning. When I did that, things changed for me. Senior leaders wanted to talk to me. When I forced them to prioritize in key projects, they were saying, “Antonio, we want another meeting with you,” was the CEO of the bank, because I force them. I force them to create value. I force them to have strategic dialogue, so I would say if you’re listening, you’re working in this space, move on into that space. Move on on the value creation, on your stakeholder, and things will change very fast.

ALISON BEARD: Well, Antonio, I learned a ton today. Thanks so much for coming on the show.

ANTONIO NIETO-RODRIGUEZ: A pleasure.

HANNAH BATES: That was project management expert Antonio Nieto-Rodriguez in conversation with Alison Beard on the HBR IdeaCast. He’s the author of the Harvard Business Review Project Management Handbook.

We’ll be back next Wednesday with another hand-picked conversation about business strategy from Harvard Business Review. If you found this episode helpful, share it with your friends and colleagues, and follow our show on Apple Podcasts, Spotify, or wherever you get your podcasts. While you’re there, be sure to leave us a review. And when you’re ready for more podcasts, articles, case studies, books, and videos with the world’s top business and management experts, find it all at HBR.org.

This episode was produced by Mary Dooe, Anne Saini, and me, Hannah Bates. Ian Fox is our editor. And special thanks to Rob Eckhardt, Adam Buchholz, Maureen Hoch, Nicole Smith, Erica Truxler, Ramsey Khabbaz, Anne Bartholomew, and you – our listener. See you next week.

• Subscribe On:

Latest in this series

This article is about strategy.

• Project management
• Operations strategy
• Organizational change

Partner Center

Communicative Sciences and Disorders

• Online Learners: Quick Links
• ASHA Journals
• Research Tip 1: Define the Research Question
• Reference Resources
• Evidence Summaries & Clinical Guidelines
• Drug Information
• Health Data & Statistics
• Patient/Consumer Facing Materials
• Images/Streaming Video
• Database Tutorials
• Crafting a Search
• Cited Reference Searching
• Research Tip 4: Find Grey Literature
• Research Tip 5: Save Your Work
• Cite and Manage Your Sources
• Critical Appraisal
• What are Literature Reviews?
• Conducting & Reporting Systematic Reviews
• Finding Systematic Reviews
• Tutorials & Tools for Literature Reviews
• Point of Care Tools (Mobile Apps)

Database Searching & Strategy Development

• YouTube Series: Carrie Price's Searching School This playlist of videos, created by librarian Carrie Price, details the nuances of systematic searching based on guidance documents like the PRISMA-S and the PRESS Checklist.
• Advanced Searching with CINAHL Subject Headings This guide explains what subject headings and subheadings are and demonstrates how to use CINAHL headings in command-line searching to build one-line simple or complex searches.
• PubMed Help This page offers tips, tricks, and tutorials for searching PubMed, which is a free resource developed and maintained by the National Center for Biotechnology Information (NCBI), at the U.S. National Library of Medicine (NLM).
• PubMed: Building a Search [Video] This tutorial from the Welch Medical Library at Johns Hopkins details the steps to building a systematic search strategy in PubMed with the building block method, including the use of Boolean operators, truncation, quotations, and field tags.
• Medline via Ovid Database Guide This guide describes how Medline information is structured in the Ovid interface. Jump down to Advanced Searching for tips on using Ovid syntax and limits to optimize a search strategy.
• PsycINFO via Ovid Database Guide This guide describes how PsycINFO information is structured in the Ovid interface. Jump down to Advanced Searching for tips on using Ovid syntax and limits to optimize a search strategy.

A spreadsheet template designed to help you keep track of your literature search terms during a systematic search.

1. Log in with your NYU credentials

2. Open and "Make a Copy" to create your own tracker for your literature search strategies

Translating Searches Between Databases

Searching in a comprehensive, systematic way requires authors to execute analogous searches in multiple databases, but not all databases accept the same search syntax, and most databases use different vocabulary for subject headings (or don't use subject headings at all).

As such, once a search strategy has been developed in one database, it is necessary to 'translate' it into a form that will work in a different database.

Here is the same search criteria (diabetes + self management), executed with database-specific search queries for three different databases. 

(diabetes OR diabetic* OR (MH "Diabetes Mellitus+")) AND (“self management” OR “self care” OR “self monitoring” OR “self regulation” OR (MH "Self-Management") OR (MH "Self Care+"))

(“diabetes”[tiab] OR “diabetic*”[tiab] OR "Diabetes Mellitus"[Mesh]) AND (“self management”[tiab] OR “self care”[tiab] OR “self monitoring”[tiab] OR “self regulation”[tiab] OR "Self-Management"[Mesh])

• Web of Science

(diabetes OR diabetic*) AND (“self management” OR “self care” OR “self monitoring” OR “self regulation”)

Resources for Translating Search Queries

• Cochrane Database Syntax Guide (PDF) (152KB) A summary of the different syntax used to structure queries in health databases.
• Cornell University Library Guide - Translate Search Strategies This section of Cornell's guide to evidence synthesis covers key syntax differences between databases.
• UniSA Systematic Reviews Guide: Run Your Search on Other Databases This page has links to documents describing how to translate a search from Ovid MEDLINE into other database syntax

Citation Management Tools

For help choosing a citation management tool: .

See comparison list

For help using a citation management tool: 

See schedule of RefWorks, EndNote, and Zotero classes (coded in pink)

• EndNote NYU has a campus-wide license to this software, which you download (at no cost to you) to your laptop or desktop computer. The software includes a less developed web-based component called EndNoteWeb, which can be used to sync across multiple personal computers.
• RefWorks NYU purchases a campus-wide subscription to this web-based service that covers all members of the NYU community. Because RefWorks is web-based, you can use it from any Internet-connected computer/device on campus or off.
• Zotero A free download that works with your web browser. There are two options: Zotero for Firefox is a Firefox-specific extension that offers the full range of functionality. Zotero Standalone -- a separate browser-independent application that can be used with Chrome, Safari, or Opera -- offers slightly less functionality than the Firefox extension.

Backing Up Bibliographic Data

If you are using a citation manager to store bibliographic data related to your evidence synthesis project, it is recommended that you maintain back up copies of your data.  

Recommended steps for backing up bibliographic data will vary depending on your software of choice (Zotero, EndNote or RefWorks) - more information can be found on the guide for Data Management Planning - Storage & Backup . 

Article Screening - Covidence

Covidence works with reference managers (e.g.,EndNote, Zotero, Refworks, Mendeley) to screen results for the purposes of systematic reviews and other research projects.

Link to Covidence to request a Covidence account using your NYU email address. Accept email invitation and Sign In;

(do NOT click the "free trial" account; do NOT "sign in with Cochrane" Select the option for NYU access).

Covidence Trainings & Support

• Covidence Knowledge Base The Covidence Knowledge Base contains a suite of articles offering overviews of the software, help getting started, and information about review settings, importing records, data extraction, exporting data, FAQs and more.
• Getting Started with Covidence - Video Collection This collection of video tutorials covers: -Signing in and out -Creating a new review and inviting co-reviewers -Importing citations -Screening titles & abstracts -How to breeze through screening
• Getting Started with Covidence (Webinar - 1 Hour) The Covidence 101 training webinar includes a live demo providing an overview of the systematic review workflow, and showcasing some of the most popular features.

Tutorials: Exporting Records from Databases into Covidence

Covidence Knowledge Base: Importing references in Covidence

Video Overviews by Database

• Education Source with ERIC is also hosted on EBSCO
• PsycINFO and MEDLINE are also hosted on EMBASE

PRISMA - Checklist & Diagram

PRISMA ( P referred R eporting I tems for S ystematic R eviews and M eta- A nalyses) is an evidence-based set of minimum items for reporting information in systematic reviews and meta-analyses.

Specific PRISMA resources include:

Prisma checklist.

• A 27 item checklist whose items refer to the preferred content of a for reporting systematic reviews and meta-analyses, including suggested content for the title, abstract, methods, results, discussion and funding.

PRISMA Extensions

• Extensions to the PRISMA Checklist to facilitate the reporting of different types or aspects of systematic reviews/meta-analyses (e.g., checklists for reporting scoping reviews, individual patient data)

PRISMA Diagram Templates

• Word Document Templates  - PRISMA Website
• This file is View Only. To create your own diagram in GoogleDrawings using this template: "File> Make a copy"
• Image can be downloaded as PDF, PNG, JPG, or SVG
• PRISMA Diagram Generator - ShinyApp.io

PRISMA Data Table

• This view-only GoogleDoc shows a table you can use to track the basic data that is needed to complete a PRISMA diagram, including an example table.

Example PRISMA Diagram 

Source: 

Stotz, S. A., McNealy, K., Begay, R. L., DeSanto, K., Manson, S. M., & Moore, K. R. (2021). Multi-level diabetes prevention and treatment interventions for Native people in the USA and Canada: A scoping review.  Current Diabetes Reports, 2 (11), 46. https://doi.org/10.1007/s11892-021-01414-3

Video Overview: Filling Out a PRISMA Flow Diagram (2020 Version; 8 minutes, 32 seconds)

Still have questions about constructing a PRISMA diagram using the 2020 template? 

This article, published in the Journal of the Medical Library Association, answers some frequently asked questions about using the PRISMA 2020 format. 

Rethlefsen, M. L., & Page, M. J. (2022). PRISMA 2020 and PRISMA-S: common questions on tracking records and the flow diagram.  Journal of the Medical Library Association : JMLA ,  110 (2), 253–257. https://doi.org/10.5195/jmla.2022.1449 

• << Previous: Finding Systematic Reviews
• Next: Point of Care Tools (Mobile Apps) >>
• Last Updated: Jun 26, 2024 3:00 PM
• URL: https://guides.nyu.edu/speech