case study innovation policy

The Reporter

Mechanisms and Impacts of Innovation Policy

The importance of innovation to job creation and economic growth — especially in young, high-growth firms — is widely accepted among economists as well as members of the business and policy communities. There is also a recognition that, at least at some times or in certain settings, the private sector underinvests in innovation, creating an opportunity for the public sector to step into the breach.

The longstanding problem is how. What tools are most effective?

There are myriad opportunities for government programs to fail. For example, if a program subsidizes only the “best projects,” those that would likely have gone forward with private capital regardless of government involvement, this is likely to be a poor use of taxpayer dollars. Alternatively, if only poor-quality projects are supported, they might fail even with government support.

In my research, I seek to understand the effects of, and mechanisms behind, common policy tools that subsidize high-growth entrepreneurship and innovation in the United States. In doing so, I hope to inform policymaking and shed light on the constraints and trade-offs of the innovation process.

Three key themes emerge in my work. First, program design appears to be more important than the amount of funding. For example, it is important to enable innovators to pivot and to control the commercialization pathway of their ideas. Second, effectiveness depends on which firms decide to apply for support. Programs need to target firms with the potential to benefit, and succeed in getting them to apply for support. Finally, direct federal funding plays an important role in our innovation ecosystem and is not always substitutable with private or privately intermediated alternatives.

The Evaluation Challenge

Economists have long been interested in evaluating government innovation programs, but it has been hard to identify causal effects. Program administrators are typically loath to run experiments. My work has addressed this challenge by employing several empirical approaches.

The most important of these methods is a regression discontinuity design (RDD) in which I compare winning and losing applicants within a competition for a grant or contract. I control for the rank that the program assigns to each applicant. Importantly, the cutoff decision determining which ranks win is exogenous to the ranking process. The key insight is that near the cutoff for winning, winners and losers should be similar, creating a natural experiment.

In other work, I use staggered program rollout designs, while addressing potential bias from pretreatment observations being considered by the model as controls. A final method is to instrument for funding using plausibly exogenous shocks. All three of these methods can be applied in many policy evaluation settings, and if carefully executed can reveal causal effects.

Design and Selection: Evidence from the SBIR Program

The US Small Business Innovation Research (SBIR) program, which was established in 1982, is the main vehicle by which the federal government directly supports innovation at small firms and encourages them to enter the federal contracting pipeline. It is available at 11 federal agencies and always has two stages. Firms first apply to a subsector- or topic-specific Phase 1 competition for awards, usually about $150,000. Phase 1 winners may then apply nine months later for $1 million Phase 2 awards. The SBIR program has been imitated around the world, and thus represents a particularly important research setting.

In a project using data from the SBIR program at the Department of Energy (DOE), I conducted the first quasi-experimental, large-sample evaluation of R&D grants to private firms. 1 Using the RDD approach, I found strong effects of the Phase 1 awards: they dramatically increased citation-weighted patenting, the chance of raising venture capital (VC) investment, revenue, and survival. On average, the early-stage grants did not crowd out private capital and instead enabled new technologies to go forward.

The picture was not so rosy for Phase 2. This larger grant had no measurable effect, except for a small positive effect on citation-weighted patents. I found evidence of adverse selection in Phase 2 applications. Almost 40 percent of Phase 1 winners did not apply to Phase 2, and these were disproportionately VC recipients. Phase 2 eligibility criteria, which include requirements that the firm not change its business strategy and not be more than 50 percent investor owned, apparently generated this adverse selection. This finding underscores the general theme that who decides to apply —i.e., selection — is a powerful force determining the effectiveness of a program.

Selection also plays a role in my work with John Van Reenen, Jason Rathje, and Jun Wong, which explores the design of public sector innovation procurement initiatives. 2 A key decision is whether to take a centralized approach where the desired innovation is tightly specified or to take a more open, decentralized approach where applicants are given leeway to suggest solutions. We compare these two approaches using a quasi-experiment conducted by the US Air Force SBIR program.

That program holds multiple competitions about every four months in which firms apply to develop military technologies. The Conventional Program approach is to hold competitions with highly specific topics such as “Affordable, Durable, Electrically Conductive Coating or Material Solution for Silver Paint Replacement on Advanced Aircraft.” After 2018, the Air Force also included an Open Program competition that ran alongside the Conventional model, wherein firms could propose anything they thought the Air Force would need.

We found that winning an open topic competition had positive and significant effects on three outcomes desired by the program administrators: the chances of the military adopting the new technology, the probability of subsequent VC investment, and patenting and patent originality. By contrast, winning a Conventional award had no measurable effect on any of these outcomes. Nor were there any causal impacts of winning a Conventional award between 2003 and 2017, before the Open Program was introduced.

Both selection and decentralization played a role in the Open Program’s success. It reached firms with startup characteristics that were less likely to have had previous defense contracts — a selection effect. At the same time, however, we also found that openness matters. For example, there were significantly more positive effects of Open awards even among the firms that applied to both the Open and Conventional Programs. Also, when a Conventional topic was less specific and thus closer to the Open Program’s approach, winning an award for that topic significantly increased innovation.

The Open Program seems to work in part because it provides firms with an avenue to identify technological opportunities of which the government is not yet fully aware, and it enables firms to pursue their private and government commercialization pathways simultaneously. These results are relevant beyond the Air Force, as governments and private firms increasingly turn to open or decentralized approaches to soliciting innovation.

Incentives: Who Is Funding?

I also found benefits of openness in a different setting: university research. Unlike the two projects focusing on important government programs, this project explored what happened when federal funding declined, shedding light on substitutability with private funding.

Together with Tania Babina, Alex He, Elisabeth Perlman, and Joseph Staudt, I asked whether declines in federal R&D funding affected the innovation outputs of academic research. 3 We linked data on all employees of all grants at 22 universities to career outcomes of individuals in the US Census Bureau’s IRS W-2 files, patent inventors, and publication authors in the PubMed database.

We found that a negative federal funding shock nearly halved a researcher’s chance of founding a high-tech startup, but doubled their chance of being an inventor on a patent. The shock also reduced the number of publications, especially those that are more basic, more cited, and in higher-impact journals.

What could explain these seemingly puzzling findings? We found evidence that they were in part driven by a shift from federal to private funders. While federal awards typically assert no property rights to research outcomes, private firms have incentives to appropriate research outputs, and for that reason employ complex legal contracts with researchers. As the composition of research funding shifts from federal to private sources, outputs are more often commercialized by the private funder, rather than disseminated openly in publications or taken to a startup by the researcher.

In all the programs discussed thus far, the government directly targets the operating firm or innovator. A popular alternative approach is to target financial intermediaries, such as VC funds — as is done in Israel, Canada, Singapore, China, and some other countries — or angel investors.

More than 14 countries and most US states offer angel investor tax credits. Matthew Denes, Filippo Mezzanotti, Xinxin Wang, Ting Xu, and I studied these credits. 4 They offer several promising features: no need for government to pick winners, low administrative burdens, and market incentives with investors retaining skin in the game.

Angel tax credits increase the number of angel investments by approximately 18 percent and the number of individual angel investors by 32 percent. Surprisingly, however, we found that angel tax credits do not appear to generate high-tech firm entry or job creation.

One reason for this outcome appears to be selection: additional investment flows to relatively low-growth firms. The angel investments appear to crowd out investment that would have happened otherwise, as common informal equity stakes — often made by insiders in the firm or family members of the entrepreneur — are labeled as “angel.”

Another reason emerges from the theory of investment in early stage, high-growth firms. These investments have fat-tailed returns. We find that as the right tail grows fatter, professional investors become less sensitive to the tax credits. This limits the ability of the policy to reach its intended targets — potentially high-growth startups. In the words of one survey respondent explaining why angel tax credits do not affect decision-making, “I’m more focused on the big win than offsetting a loss.”

Spillovers and Financial Constraints

Both the university research and angel tax credit projects highlight the role of decision-maker incentives, which determine the projects that get funded and their pathways to commercialization. While private funders and private intermediaries have attractive features, notably reducing the burden on government and costly taxpayer dollars, they have different incentive structures relative to government funders. In the programs I have studied, private sector actors have incentives to select projects with fewer knowledge spillovers.

My work also highlights that effective programs target financially constrained firms. The strong positive effects of the SBIR programs stem from awards to small, young firms that are new to SBIR and to government contracting. J. David Brown and I show that the small firms that benefit from SBIR awards use the funds in part to pay employees, especially those with long tenure at the firm. 5 These financially constrained firms appear to finance themselves in part by engaging in back-loaded wage contracts with their workers. By alleviating constraints, an effective program paves the way for future investment and growth.

In contrast, in both the DOE and Air Force settings, it seems that SBIR awards crowd out private investment among larger firms that win many such awards. Similarly, angel tax credit programs crowd out private activity because investors often use them in deals that would have occurred regardless of the program.

While money is of course fungible, my research suggests that the source of innovation funds and program design — especially design features that affect who applies to the program — matter a great deal.

Researchers

InnovationStrategy →

No results found in working knowledge.

Were any results found in one of the other content buckets on the left?
Try removing some search filters.
Use different search filters.

U.S. Science and Innovation Policy

Aims to strengthen the U.S. research and innovation enterprise to advance national goals.

Roundtable on Black Men and Black Women in Science, Engineering, and Medicine

Established in 2019, the Roundtable on Black Men and Black Women in STEM convenes a broad array of stakeholders that focus on the barriers and opportunities encountered by Black men and Black women as they navigate the pathways from K-12 and postsecondary education to careers in science, engineering and medicine.

U.S. Science and Innovation Policy is one of four themes under Policy and Global Affairs (PGA) .

Upcoming Events

Multiday Event | May 30-31, 2024

11:00AM - 12:00PM (ET)

June 4, 2024

Multiday Event | June 6-7, 2024

1:00PM - 2:30PM (ET)

June 10, 2024

3:00PM - 5:00PM (ET)

June 26, 2024

GUIRR at 40: Reimagining the Triple Helix of Innovation, Investments, and Partnerships. A Workshop
Understanding the Implications of The SCOTUS Affirmative Action Decision: A Symposium Series
Roles, Responsibilities, and Expectations of STEM Graduate Students and Postdoctoral Scholars: A Conversation Series
Inventorship, Entrepreneurship, and Systems for Nurturing Black People in Applied and Translation Fields: A Workshop
National Science, Technology, and Security Roundtable - Capstone Event
Interpreting the Axioms of Innovation - A Workshop
Engaging and Supporting Leaders to Develop an Inclusive and Equitable STEMM Research Ecosystem in Higher Education: A Workshop
Supporting K-12 STEM education to create the foundations for innovation: A Workshop
Assessing Research Security Efforts in Higher Education: Experts Meeting and Workshop Series
Lighting the Spark for Pathways to Greatness for Black People in SEM: A Workshop
The Impact of Burnout on Gender Equity in Science, Engineering, and Medicine: A Workshop
Developing New Approaches to Promote Equitable and Inclusive Implementation of Open Scholarship Policies: A Workshop
On Leading a Lab: Strengthening Scientific Leadership in Responsible Research, A Workshop
Working Group 1: On Leading A Lab, Guidance for Lab Leaders

Publications

People with disabilities are the largest minority group in the United States. While nothing about science, technology, engineering, and mathematics (STEM) education, jobs, or workplaces would seem to inherently exclude people with disabilities, in practice, stigma and discrimination continue to limit opportunities for disabled people to fully contribute to and be successful in the STEM ecosystem. The planning committee for Beyond Compliance: Promoting the Success of People with Disabilities in the STEM Workforce of the National Academies of Sciences, Engineering, and Medicine, with funding from the U.S. National Science Foundation, organized a hybrid national leadership summit and virtual workshop series to address and explore issues of accessibility and inclusivity in STEM workplaces. Across the 5 days of workshops, dozens of panelists spoke about their personal and professional experiences of ableism and barriers to full participation in the STEM workforce, as well as identified positive examples of mentorship and efforts to create fully inclusive STEM spaces in education, labs, the private sector, and professional development settings.

Disrupting Ableism and Advancing STEM: Promoting the Success of People with Disabilities in the STEM Workforce: Proceedings of a Workshop Series

The Roundtable on Aligning Incentives for Open Scholarship of the National Academies of Sciences, Engineering, and Medicine brings together stakeholders to discuss the effectiveness of current incentives for adopting open scholarship practices, barriers to adoption, and ways to move forward. On June 26, 2023, the Roundtable convened a virtual workshop to hear from a range of stakeholders on the challenges and opportunities in expanding open scholarship, with a focus on the impact of recent U.S. government initiatives to open the research ecosystem and broaden access to publicly-funded research. This publication summarizes the presentation and discussion of the workshop.

Promoting Equitable and Inclusive Implementation of Open Scholarship Policies: Proceedings of a Workshop—in Brief

The National Academies hosted a public webinar in February 2024 to discuss issues around contraception accessibility and implications for reproductive health of Americans following the overturn of the Roe v. Wade U.S. Supreme Court decision. This event was the final webinar in a 5-part series designed to consider society-wide effects of reproductive health care limits and challenges to health and well-being since the 2022 Dobbs v. Jackson Womens Health Organization ruling.

Pressing Issues Around Contraception Access Following the Repeal of Roe v. Wade: Proceedings of a Workshop–in Brief

On April 20, 2023, the Board on Research Data and Information of the National Academies of Sciences, Engineering, and Medicine convened a workshop to bring together stakeholders to explore new initiatives to support FAIR (findable-accessible-interoperable-reusable) data sharing, as well as the need for innovative approaches, potential obstacles to success, and how obstacles might be overcome. Participants included researchers and representatives from institutions, federal agencies, private funders, and professional societies.

Accelerating and Deepening Approaches to FAIR Data Sharing: Proceedings of a Workshop—in Brief

Family caregiving is not simply an outside obligation that has no bearing on the workings of academic science, technology, engineering, mathematics, and medicine (STEMM) as it affects the lives of so many people working and studying in colleges and universities around the country. Caregiving responsibilities often clash with ingrained norms in academic STEMM environments, which demand that STEMM students and workers demonstrate immense devotion to their fields and are always available and visibly working.

Supporting Family Caregivers in STEMM describes the ways in which the labor and contributions of caregivers are often invisible and undervalued, with a specific focus on the academic STEMM ecosystem, including undergraduate and graduate students, postdoctoral scholars, resident physicians and other trainees, tenure-track and non-tenure-track faculty, staff, and researchers. This report reviews policies and practices that support caregivers, locally and nationally, and describes best practices in policy implementation and design. Supporting Family Caregivers in STEMM also highlights innovative practices and offers actionable recommendations to higher education institutions, public and private funders, and the federal government.

Supporting Family Caregivers in STEMM: A Call to Action

A planning committee of the National Academies of Sciences, Engineering, and Medicine convened a series of three hybrid workshops to examine the key issues highlighted in the National Academies 2022 consensus study report, Defense Research Capacity at Historically Black Colleges and Universities and Other Minority Institutions: Transitioning from Good Intentions to Measurable Outcomes. As Department of Defense and other partners sought to implement the 2022 report recommendations, key questions remained to be explored, particularly related to how to seek ways of building research capacity at minority institutions (MIs) and develop true partnerships between MIs, other institutions of higher education, and federal agencies. The workshops featured commissioned research and literature reviews as well as case studies to illuminate problems, barriers, and approaches to increase research capacity at Historically Black Colleges and Universities.

Building Defense Research Capacity at Historically Black Colleges and Universities, Tribal Colleges and Universities, and Minority-Serving Institutions: Proceedings of Three Town Halls

Increasing the number of Black men and Black women who enter the fields of science, engineering, and medicine (SEM) will benefit the social and economic health of the nation. On May 2-3, 2022 the Roundtable on Black Men and Black Women in Science, Engineering, and Medicine of the National Academies of Sciences, Engineering, and Medicine convened a workshop to highlight promising financial and supportive services and programs throughout various stages of career development. Sessions followed student progression through the major stages of education and career development, and identified policies and practices that aim to mitigate and alleviate long-standing barriers to the full participation of Black students in SEM at the K–12, undergraduate, and graduate and professional levels.

Supporting Black Students Through Their Science, Engineering, and Medicine Career Journeys: Proceedings of a Workshop

In December 2023, the National Academies hosted a public webinar in which medical and human rights experts explored concerns related to harassment, threats, and physical attacks against health care professionals working to provide essential reproductive health care. The event was the fourth in a webinar series designed to consider society-wide effects of limits to reproductive health care access in the U.S. following the 2022 Supreme Court Decision in Dobbs v. Jackson Womens Health Organization. This Proceedings of a Workshop-in Brief highlights the presentations and discussions that occurred at the webinar.

Harassment and Violence Against Health Professionals Who Provide Reproductive Care: Proceedings of a Workshop–in Brief

On March 14-15, 2023, the Board on Science, Technology, and Economic Policy of the National Academies of Sciences, Engineering, and Medicine held a 2-day workshop in Washington, DC to explore the use of data, research, and experiments to improve the processes for and outcomes of federal funding of scientific research. The workshop brought together researchers in the science of science funding and practitioners from government and the private sector with experience supporting or carrying out experimentation and evaluation to discuss illustrative examples of the use of experimentation from the United States and abroad; consider methods of evaluation; and foster relationships for future experimentation.

Experimental Approaches to Improving Research Funding Programs: Proceedings of a Workshop

Despite the significant contributions of graduate students and postdoctoral scholars to scientific research, the environments in which they work and learn often offer inconsistent mentorship and professional development opportunities and only intermittently address their mental, financial, and social well-being.

To examine mentorship, well-being, and professional development and the challenge of translating existing mentorship theory into effective interventions, The Roundtable on Mentorship, Well-being, and Professional Development convened a workshop on November 1 and 2, 2023. This publication summarizes the presentation and discussion of the workshop.

Mentorship, Well-Being, and Professional Development in STEMM: Addressing the "Knowing-Doing Gap": Proceedings of a Workshop—in Brief

The National Academies Standing Committee on Reproductive Health, Equity, and Society hosted a virtual public webinar in October 2023 to explore state-level legal and political strategies to increase access to reproductive health care services, including abortion care, following the Dobbs v. Jackson Womens Health Organization ruling, which overturned the 1972 Roe v. Wade decision. Discussions included updates on state and federal legal challenges to abortion bans, the role of ballot initiatives in reproductive rights, additional strategic avenues such as legislative advocacy, how the legal landscape affects the science of reproductive health care, and more.

State-Level Legal and Political Strategies Following the Repeal of Roe v. Wade: Proceedings of a Workshop–in Brief

There are more than 400 public and private, two- and four-year minority institutions (MIs) across every U.S. state and territory, including many in areas near Department of Defense (DoD) facilities. Many already conduct cutting-edge research in areas of high priority to the U.S. government, while others could be positioned to do so with strategic investments. With their diverse populations, MIs support students and faculty that have a wealth of knowledge and talent to support diversifying STEM research, and ultimately strengthening national security. DoD supports and recognizes the potential contributions of many MI programs, but there are significant discrepancies in the amount, duration, and type of DoD funding, as well as the research infrastructure at MIs, when compared to non-MIs.

To explore opportunities for the DoD and historically underresourced minority institutions to adapt in support of increasing the engagement of MIs in the defense research ecosystem, the National Academies convened a nine-member committee of STEM professionals across sectors and disciplines to develop a series of three town hall workshops. This proceedings-in-brief serves to highlight points made by presenters in the series.

Building Defense Research Capacity at U.S. Minority Institutions : Proceedings of a Workshop Series—in Brief

Innovation is a major contributor to national security, productivity, competitiveness, and economic growth. Recent legislation and policies, most notably the CHIPS + Science Act of 2022, provide significant support to transform the science and research enterprise, with a focus on strengthening commercialization ecosystems and providing opportunities for historically underserved students and communities to be engaged in science, technology, engineering, and mathematics.

On June 13-14, 2023, the Government-University-Research Roundtable of the National Academies of Sciences, Engineering, and Medicine held a workshop to consider the axioms of innovation - the underpinning norms, conditions, and culture of innovation environments in the United States and globally. Participants discussed new trends in the theory and practice of innovation, as well as innovation-fostering partnerships and collaboration across sectors and disciplines. This publication summarizes the presentations and discussion of the workshop.

Interpreting the Axioms of Innovation: Proceedings of a Workshop—in Brief

Before the COVID-19 pandemic, research demonstrated a significant impact of family caregiving responsibilities on the careers of professionals who work in science, technology, engineering, mathematics, and medicine (STEMM) - especially women. The COVID-19 pandemic increased public recognition of the challenges experienced by caregivers in STEMM fields and created a sense of urgency among many stakeholders. As part of the information-gathering stage of a forthcoming consensus study to address the barriers faced by caregivers, of all genders, in STEMM fields, the Committee on Policies and Practices for Supporting Family Caregivers Working in Science, Engineering, and Medicine organized two national symposia on February 27 and March 27, 2023. Scholars and leaders from a range of sectors convened to identify evidence-based interventions that can support the recruitment, retention, and advancement of caregivers in science, engineering, and medical fields. This publication summarizes the presentations and discussion of the symposia.

Barriers, Challenges, and Supports for Family Caregivers in Science, Engineering, and Medicine: Proceedings of Two Symposia

State laws restricting abortion often contain some degree of exception to save the life of the pregnant person, but varying legal interpretations of those exceptions may create a confusing legal landscape for clinicians and lead to delays providing lifesaving treatment. The National Academies Standing Committee on Reproductive Health, Equity, and Society held a public workshop in an After Roe series to explore the current challenges of, and potential solutions to, delays in the provision of lifesaving care for people experiencing pregnancy complications or serious illness or injury while pregnant. Speakers considered the myriad of legal, policy, and provider challenges and the effect of these challenges on the delivery of care and clinician well-being. This Proceedings of a Workshop-in Brief summarizes the discussions held during the workshop.

Challenges in the Provision of Lifesaving Care for Pregnant Patients Following the Overturn of Roe v. Wade: Proceedings of a Workshop–in Brief

Foreign-funded language and culture institutes exist on U.S. campuses beyond Confucius Institutes (CIs)—Chinese government-funded centers established by the Chinese Communist Party to extend the reach of Chinese language and culture and to enhance worldwide opinion of China through offering classes in Mandarin Chinese and highlighting positive aspects of Chinese culture. Regardless of the sponsoring nation, foreign-funded language and culture institutes may pose risks for U.S. host institutions regarding academic freedom, freedom of expression, governance, and national security. This is particularly true if the values of the sponsoring nation do not align with the democratic values held in the United States and if the sponsoring nation is suspected of engaging in activities adversely affecting human rights, academic freedom, freedom of expression, association, dissent, and U.S. national security.

This report explores the role of other foreign-funded institutes at U.S. institutions of higher education, describing characteristics and features of such institutes; determining characteristics and features of foreign-funded institutes at U.S. institutions of higher education that could be flags for institutions to engage in further deliberation and vetting prior to entering into a partnership; identifying implementable practices for U.S. institutions of higher education to ensure appropriate operations; and continuing exploration of what role the sensitivity of the research conducted on campus should play in determining which foreign-funded partnerships are appropriate. Foreign-Funded Language and Culture Institutes at U.S. Institutions of Higher Education recommends actions that U.S. colleges and universities can take to minimize risks associated with hosting foreign-funded language and culture institutions, such as a CI, on or near campus and protect academic freedom and national security.

Foreign-Funded Language and Culture Institutes at U.S. Institutions of Higher Education: Practices to Assess and Mitigate Risk

The National Academies Standing Committee on Reproductive Health, Equity, and Society and the National Academy of Medicine, committed to equitable access to quality reproductive health, hosted a webinar, After Roe: Physician Perspectives and Workforce Implications, in May 2023. Discussions increased awareness and promoted dialogue in the medical, public health, societal, and general population. Speakers explored clinician workforce impacts of legal restrictions on the provision of reproductive health services in the U.S. Practicing physicians from obstetrics-gynecology, maternal-fetal medicine, family medicine, emergency medicine, and oncology provided their perspectives on the effects of the legal limitations on their well-being (e.g., moral distress), professional futures, and institutional supports. Perspectives included individuals from a range of states with varying legal restrictions. This proceedings document summarizes the discussions held during the webinar.

Physician Perspectives and Workforce Implications Following the Repeal of Roe v. Wade: Proceedings of a Workshop–in Brief

On February 7-8, 2023, the Government-University-Industry Research Roundtable of the National Academies of Science, Engineering, and Medicine convened a workshop for its membership and invited guests to discuss how to foster the health and productivity of the U.S. research enterprise in the context of challenges and potential solutions driven by new partnerships and other opportunities related to economic security and competition. The workshop considered critical elements to promote and strengthen research commercialization, including new protection and security frameworks for intellectual property, advancement of U.S. leadership in international standards organizations, and best practices to promote technology transfer. This publication summarizes the presentations and discussion of the workshop.

Strategic Innovation and Commercialization: Supporting IP and Tech Transfer to Advance U.S. Research Competitiveness: Proceedings of a Workshop—in Brief

On March 6-8, 2023, at the Francis Crick Institute in London, the UK Royal Society and Academy of Medical Sciences, the U.S. National Academy of Sciences and National Academy of Medicine, and UNESCO-The World Academy of Sciences held the Third International Summit on Human Genome Editing. A follow-up to earlier international summits held in Washington, DC, in 2015 and in Hong Kong in 2018, the third summit examined scientific advances that have occurred since the previous summits and the need for global dialogue and collaboration on the safe and ethical application of human genome editing. The first two days of the summit focused largely on somatic human genome editing, where the cells being altered are non-reproductive cells - as a result genetic changes cannot be passed on to future generations. The third day of the summit broadened the discussion to include heritable human genome editing, in which genetic changes could be passed on to descendants. This publication highlights the presentations and discussion of the event.

Third International Summit on Human Genome Editing: Expanding Capabilities, Participation, and Access: Proceedings of a Workshop—in Brief

The mission of the Roundtable on Black Men and Black Women in Science, Engineering, and Medicine at the National Academies of Sciences, Engineering, and Medicine is to advance discussions on the barriers and opportunities encountered by Black students and professionals as they navigate the pathways from K-12 and postsecondary education to careers in science, engineering, and medicine (SEM) and highlight promising practices for increasing the representation, retention, and inclusiveness of Black men and Black women in SEM. This proceedings serves as a summary of the presentations and discussions from a capstone workshop, held virtually on December 6 and 7, 2021, to review how anti-Black racism had permeated across the topics covered by the Roundtable over the previous 2 years.

The State of Anti-Black Racism in the United States: Reflections and Solutions from the Roundtable on Black Men and Black Women in Science, Engineering, and Medicine: Proceedings of a Workshop

On November 14 and 15, 2022, the National Academies of Sciences, Engineering, and Medicine convened a two-day workshop under the auspices of the National Science, Technology, and Security Roundtable to assess the state of the U.S. research enterprise in a time of increasing global competition. The workshop also featured discussion of the challenges confronting researchers as they seek to ensure the vitality of research and innovation in America, foster increased international scientific research cooperation, and simultaneously counter illicit foreign interference that threatens national security interests. This publication summarizes the presentation and discussion of the workshop.

Openness, International Engagement, and the Federally Funded Science and Technology Research Enterprise: Proceedings of a Workshop—in Brief

Stakeholder Actions to Implement Open Scholarship: Proceedings of a Workshop—in Brief

More than 100 U.S. institutions of higher education hosted Confucius Institutes (CIs), Chinese government-funded language and culture centers, on campus during the late 2000s and 2010s. While CIs provided a source of funding and other resources that enabled U.S. colleges and universities to build capacity, offer supplemental programming, and engage with the local community, CIs presented an added, legitimate source of risk to host institutions with respect to academic freedom, freedom of expression, and national security.

By 2017, deteriorating U.S.-China relations led some U.S. colleges and universities to reconsider the value of having a CI on campus. Sustained interest by Congress and political pressure led numerous U.S.-based CIs to close, especially following the passage of the Fiscal Year 2019 National Defense Authorization Act, which contained a provision that ultimately barred institutions receiving Department of Defense (DOD) critical language flagship funding in Chinese from hosting a CI. While this provision allowed for a waiver process - and several affected colleges and universities applied for waivers in 2018 and 2019 - DOD did not issue any waivers. Today, seven CIs remain on U.S. university and college campuses. At the request of DOD, Confucius Institutes at U.S. Institutions of Higher Education presents a set of findings and recommendations for waiver criteria to potentially permit the continued presence of CIs on U.S. university campuses that also receive DOD funding.

Confucius Institutes at U.S. Institutions of Higher Education: Waiver Criteria for the Department of Defense

Domestic and international competition for STEM talent is driving institutions across the research and innovation landscape to consider new policies and partnerships for building and managing STEM knowledge and skills. New levels of investment in human capital to increase U.S. innovation capacity and competitiveness will require coordination and collaboration among academic institutions, workforce development programs, labor organizations, companies, and funders of research and education.

On October 18-19, 2022, the Government-University-Industry Research Roundtable convened a workshop for its membership and invited guests to consider the strategic priority of human capital investment in preparing the future workforce and supporting national innovation capacity. The workshop discussions examined the state of U.S. workforce preparation and considered trends in supply and demand for human capital to meet the needs of a transforming research enterprise. This publication summarizes the presentation and discussion of the workshop.

Developing Human Capital to Support U.S. Innovation Capacity: Proceedings of a Workshop—in Brief

The National Science Foundation (NSF) asked the National Academies of Sciences, Engineering, and Medicine to conduct a quadrennial review of the NSF Small Business Innovation Research (SBIR) and Small Business Technology Transfer (STTR) programs, in accordance with a legislative mandate. Drawing on published research plus existing data, Review of the SBIR and STTR Programs at the National Science Foundation analyzes the effectiveness of NSF award selection process and postaward assistance; the effectiveness of the STTR program in stimulating new collaborations; the economic and noneconomic impacts of the programs; effectiveness of the programs in stimulating technological innovation and supporting small, new firms across the technological spectrum; and the role of the programs in providing early capital to firms without other significant sources of support.

Review of the SBIR and STTR Programs at the National Science Foundation

Efforts over the last several decades to increase the participation and leadership of Black men and women in the scientific and medical workforce have had limited results. Despite many individual successes, the number of Black professionals in science, engineering, and medicine (SEM) fields has not reached a level that corresponds with African American representation in the country at large. Structural racism affects progress at all stages along the pathway - from young children through graduate and medical students through faculty and clinicians at all levels. Beyond entry into educational programs or recruitment into workplaces seeking to diversify, challenges persist to achieve equity and inclusion for Black males and females. Moreover, psychological barriers confound the engagement of Black men and women in SEM fields.

To explore these issues and suggest solutions, the National Academies of Sciences, Engineering, and Medicine launched the Roundtable on Black Men and Black Women in Science, Engineering, and Medicine. Over the past 2 years, the Roundtable has convened workshops on K-12 education, the impact of COVID-19, financial burdens to pursuing SEM careers, and other topics. Sessions during each of these workshops identified psychological factors related to those specific topics. To have a more targeted discussion, the Roundtable convened a virtual workshop on September 14-15, 2021. As summarized in this proceedings, panelists and participants identified policies and practices that perpetuate these factors and explored solutions toward achieving and maintaining wellness, especially among students and young professionals.

Psychological Factors That Contribute to the Dearth of Black Students in Science, Engineering, and Medicine: Proceedings of a Workshop

On September 22, 2022, the National Academy of Sciences held a symposium entitled Endless Frontier 2022: Research and Higher Education Institutions for the Next 75 Years. The event was a follow up to a February 2020 NAS symposium convened to celebrate the 75th anniversary of the landmark report Science, the Endless Frontier.

Building on the 2020 symposium and on lessons learned during the COVID-19 pandemic, the September 2022 symposium sought to generate tools, strategies, and actionable steps that people and institutions can implement to ensure that science and technology continue to serve the public good. The symposium was designed to progress from broad perspectives that encompass the entire science and technology enterprise to consideration of more specific issues. This proceedings summarizes the 2022 symposium.

Transforming Research and Higher Education Institutions in the Next 75 Years: Proceedings of the 2022 Endless Frontier Symposium

Brain-Machine and Related Neural Interface Technologies: Scientific, Technical, Ethical, and Regulatory Issues: Proceedings of a Workshop—in Brief

The Government-University-Industry Research Roundtable (GUIRR) of the National Academies of Sciences, Engineering, and Medicine was founded in 1984 to convene the senior-most representatives from government, universities, and industry to define and explore critical issues related to national and global science and technology issues. On June 28-29, 2022, GUIRR convened a workshop for its membership and invited guests to consider current and proposed cross-sector partnerships that enhance science and technology innovation, national security, and national prosperity, and to explore the motives, responsibilities, concerns, and objectives that bring institutions to the table to pursue and sustain partnerships. This publication highlights the presentation and discussion of the workshop.

Enhancing U.S. Science and Innovation with Novel Cross-Sector Partnerships: Proceedings of a Workshop—in Brief

U.S. leadership in technology innovation is central to our nation’s interests, including its security, economic prosperity, and quality of life. Our nation has created a science and technology ecosystem that fosters innovation, risk taking, and the discovery of new ideas that lead to new technologies through robust collaborations across and within academia, industry, and government, and our research and development enterprise has attracted the best and brightest scientists, engineers, and entrepreneurs from around the world. The quality and openness of our research enterprise have been the basis of our global leadership in technological innovation, which has brought enormous advantages to our national interests.

In today’s rapidly changing landscapes of technology and competition, however, the assumption that the United States will continue to hold a dominant competitive position by depending primarily on its historical approach of identifying specific and narrow technology areas requiring controls or restrictions is not valid. Further challenging that approach is the proliferation of highly integrated and globally shared platforms that power and enable most modern technology applications.

To review the protection of technologies that have strategic importance for national security in an era of openness and competition, Protecting U.S. Technological Advantage considers policies and practices related to the production and commercialization of research in domains critical to national security. This report makes recommendations for changes to technology protection policies and practices that reflect the current realities of how technologies are developed and incorporated into new products and processes.

Protecting U.S. Technological Advantage

The Action Collaborative on Preventing Sexual Harassment in Higher Education is an initiative where over 55 colleges, universities, and other research and training organizations are identifying, researching, developing, and implementing efforts that move beyond basic legal compliance to evidence-based policies and practices for addressing and preventing all forms of sexual harassment and promoting a campus climate of civility and respect. The Partner Network of the Action Collaborative provides an additional opportunity for collective sharing across colleges and universities, research entities, higher education associations, grassroots and non-profit organizations, federal agencies, national laboratories, industry, and other stakeholder organizations. This annual report for 2021-2022 highlights the overall progress of the Action Collaborative toward its goals and summarizes the work shared by member and partner network organizations during the third year of the collaborative.

Action Collaborative on Preventing Sexual Harassment in Higher Education: Year Three Annual Report of Activities

The needs and demands placed on science to address a range of urgent problems are growing. The world is faced with complex, interrelated challenges in which the way forward lies hidden or dispersed across disciplines and organizations. For centuries, scientific research has progressed through iteration of a workflow built on experimentation or observation and analysis of the resulting data. While computers and automation technologies have played a central role in research workflows for decades to acquire, process, and analyze data, these same computing and automation technologies can now also control the acquisition of data, for example, through the design of new experiments or decision making about new observations.

The term automated research workflow (ARW) describes scientific research processes that are emerging across a variety of disciplines and fields. ARWs integrate computation, laboratory automation, and tools from artificial intelligence in the performance of tasks that make up the research process, such as designing experiments, observations, and simulations; collecting and analyzing data; and learning from the results to inform further experiments, observations, and simulations. The common goal of researchers implementing ARWs is to accelerate scientific knowledge generation, potentially by orders of magnitude, while achieving greater control and reproducibility in the scientific process.

Automated Research Workflows for Accelerated Discovery: Closing the Knowledge Discovery Loop examines current efforts to develop advanced and automated workflows to accelerate research progress, including wider use of artificial intelligence. This report identifies research needs and priorities in the use of advanced and automated workflows for scientific research. Automated Research Workflows for Accelerated Discovery is intended to create awareness, momentum, and synergies to realize the potential of ARWs in scholarly discovery.

Automated Research Workflows for Accelerated Discovery: Closing the Knowledge Discovery Loop

According to the United Nations Food and Agriculture Organization's report, "State of Food Security and Nutrition in the World," between 702 and 828 million people were affected by hunger in 2021 - and projections indicate that by 2030, 670 million people will still be experiencing hunger. Gains in agricultural productivity over the past 60 years have increased the availability of food globally, but much more needs to be done. Even these gains were not made without expense; biodiversity loss, chemical runoff, water scarcity, soil degradation, and greenhouse gas emissions from food and agriculture industries, among other issues, have had extensive impacts on the health of natural and human systems during this time. While millions suffer from food insecurity, a large percentage of food is lost or wasted across the global supply chain. Addressing the multifaceted challenges of feeding a world under pressure from severe food insecurity, malnutrition, climate change, population growth, conflict, migration, and economic disruption will require transformative change to global food systems.

To discuss opportunities for supporting research and innovation to address global agricultural and human health challenges associated with the compounding pressures of producing more food, more nutritiously, and with less environmental impact, the Government-University-Industry Research Roundtable of the National Academies of Sciences, Engineering, and Medicine convened a workshop for its membership and invited guests on February 16, 2022. This publication summarizes the presentations and discussion of the workshop.

Supporting Cross-Sector Partnerships for Food Security and Sustainability: Proceedings of a Workshop–in Brief

In recent decades, production processes of intermediate and final products have been increasingly fragmented across countries in what are called global value chains (GVCs). GVCs may involve companies in one country outsourcing stages of production to unrelated entities in other countries, multinational enterprises (MNEs) offshoring stages of production to units of the MNE overseas, or both. GVCs can also involve completely independent companies merely sourcing their parts from whichever upstream company may be the most competitive, with no control arrangement necessarily involved. The changing global trade environment and the changes in firms' behavior have raised new and more complicated issues for policy makers and have made it difficult for them to understand the extent and operations of GVCs and their spillover effects on national and local economies.

To improve the understanding, measurement, and valuation of GVCs, the Innovation Policy Forum at the National Academies of Sciences, Engineering, and Medicine convened a workshop, "Innovation, Global Value Chains, and Globalization Measurement" May 5-7, 2021. This proceedings has been prepared by the workshop rapporteurs as a factual summary of what occurred at the workshop.

Innovation, Global Value Chains, and Globalization Measurement: Proceedings of a Workshop

On June 8-9, 2022, an ad hoc planning committee under the auspices of the National Academies of Sciences, Engineering, and Medicine's Committee on Science, Technology, and Law hosted a workshop, Location Data in the Context of Public Health, Research, and Law Enforcement: An Exploration of Governance Frameworks. The workshop examined the collection, interpretation, and use of location data by government, academia, and industry. This publication summarizes the presentations and discussion of the workshop.

Location Data in the Context of Public Health, Research, and Law Enforcement: An Exploration of Governance Frameworks: Proceedings of a Workshop—in Brief

The United States has long made substantial investments in clinical research with the goal of improving the health and well-being of our nation. There is no doubt that these investments have contributed significantly to treating and preventing disease and extending human life. Nevertheless, clinical research faces a critical shortcoming. Currently, large swaths of the U.S. population, and those that often face the greatest health challenges, are less able to benefit from these discoveries because they are not adequately represented in clinical research studies. While progress has been made with representation of white women in clinical trials and clinical research, there has been little progress in the last three decades to increase participation of racial and ethnic minority population groups. This underrepresentation is compounding health disparities, with serious consequences for underrepresented groups and for the nation.

At the request of Congress, Improving Representation in Clinical Trials and Research: Building Research Equity for Women and Underrepresented Groups identifies policies, procedures, programs, or projects aimed at increasing the inclusion of these groups in clinical research and the specific strategies used by those conducting clinical trials and clinical and translational research to improve diversity and inclusion. This report models the potential economic benefits of full inclusion of men, women, and racial and ethnic groups in clinical research and highlights new programs and interventions in medical centers and other clinical settings designed to increase participation.

Improving Representation in Clinical Trials and Research: Building Research Equity for Women and Underrepresented Groups

The number of Black students in science, engineering, and medicine in the United States has remained disproportionately low over the past several decades. A number of reasons have been identified as contributing to these low numbers, including those related to finances. Financial considerations range from the most immediate - the ability of students to pay for their education and associated costs - to more structural concerns, such as inequities that created and have perpetuated a wealth gap between races and ethnic groups.

On April 19-20, 2021, the Roundtable on Black Men and Black Women in Science, Engineering, and Medicine of the National Academies of Sciences, Engineering, and Medicine organized a virtual public workshop to examine financial barriers for Black students in science, engineering, and medicine, explore existing educational programs to them, and engage stakeholders in conversations about partnerships and policies that span academia, industry, and philanthropy. This publication summarizes the presentation and discussion of the workshop.

Understanding and Offsetting Financial Barriers for Black Students in Science, Engineering, and Medicine: Programs, Partnerships, and Pathways: Proceedings of a Workshop

The COVID-19 pandemic escalated supply chain vulnerabilities that affect almost every sector of the economy. Identifying key vulnerabilities and developing responses to them will require the mobilization of diverse actors across the U.S. research enterprise. In October and November 2021, the Government-University-Industry Research Roundtable (GUIRR) of the National Academies of Sciences, Engineering, and Medicine convened a number of virtual workshops for its membership and invited guests to discuss opportunities for enhancing U.S. approaches to addressing manufacturing and supply chain resilience, security, and sustainability challenges through partnerships and cross-sector collaboration. In convening experts and leaders from across sectors and disciplines, GUIRR provided a forum for dialogue on policy priorities for consideration by leaders across the research enterprise. This publication summarizes the presentation and discussion of the workshop series.

Addressing Supply Chain and Manufacturing Challenges and Opportunities: Proceedings of a Workshop Series–in Brief

The Roundtable on Aligning Incentives for Open Science of the National Academies of Sciences, Engineering, and Medicine brings together stakeholders to discuss the effectiveness of current incentives for adopting open science practices, barriers to adoption, and ways to move forward. According to the 2018 report Open Science by Design: Realizing a Vision for 21st Century Research, open science "aims to ensure the free availability and usability of scholarly publications, the data that result from scholarly research, and the methodologies, including code or algorithms that were used to generate those data." With the Roundtable coming to the end of its initial phase, a virtual workshop, held December 7, 2021, provided an opportunity to review lessons learned over the past 3 years and discuss next steps for Roundtable members, the National Academies, and others interested in advancing open science and open scholarship. This publication summarizes the presentations and discussion of the workshop.

Open Scholarship Priorities and Next Steps: Proceedings of a Workshop–in Brief

On December 7 and 8, 2020, the Roundtable on Black Men and Black Women in Science, Engineering, and Medicine of the National Academies of Sciences, Engineering, and Medicine convened a virtual workshop that examined how to strengthen mentoring and advising of Black students and professionals in science, engineering, and medicine. Presenters included faculty deans, social scientists who are experts in organizational and professional development, and program implementers. Throughout the workshop, individual presenters highlighted evaluation criteria used by successful pipeline programs, including statistics on recruitment, retention, and advancement; career and leadership accomplishments; and awards and publications. This publication summarizes the presentations and discussions of the workshop.

Mentoring of Black Graduate and Medical Students, Postdoctoral Scholars, and Early-Career Faculty in Science, Engineering, and Medicine: Proceedings of a Workshop

Demand for tech professionals is expected to increase substantially over the next decade, and increasing the number of women of color in tech will be critical to building and maintaining a competitive workforce. Despite years of efforts to increase the diversity of the tech workforce, women of color have remained underrepresented, and the numbers of some groups of women of color have even declined. Even in cases where some groups of women of color may have higher levels of representation, data show that they still face significant systemic challenges in advancing to positions of leadership. Research evidence suggests that structural and social barriers in tech education, the tech workforce, and in venture capital investment disproportionately and negatively affect women of color.

Transforming Trajectories for Women of Color in Tech uses current research as well as information obtained through four public information-gathering workshops to provide recommendations to a broad set of stakeholders within the tech ecosystem for increasing recruitment, retention, and advancement of women of color. This report identifies gaps in existing research that obscure the nature of challenges faced by women of color in tech, addresses systemic issues that negatively affect outcomes for women of color in tech, and provides guidance for transforming existing systems and implementing evidence-based policies and practices to increase the success of women of color in tech.

Transforming Trajectories for Women of Color in Tech

The National Institutes of Health (NIH) asked the National Academies of Sciences, Engineering, and Medicine to conduct a quadrennial review of its Small Business Innovation Research (SBIR) and Small Business Technology Transfer (STTR) programs, in accordance with a legislative mandate. Using quantitative and qualitative analyses of data, this report reviews the operations and outcomes stemming from NIH's SBIR/STTR awards.

Drawing on published research and conducting new analyses based on both publicly available data and applicant data provided by NIH, Assessment of the SBIR and STTR Programs at the National Institutes of Health analyzes (1) the effectiveness of NIH's processes and procedures for selecting SBIR and STTR awardees; (2) the effectiveness of NIH's outreach to increase SBIR and STTR applications from small businesses that are new to the programs, from underrepresented states, and from woman-owned and minority-owned businesses; (3) collaborations between small businesses and research institutions resulting from the programs; and (4) a range of direct economic and health care impacts attributable to the programs.

Assessment of the SBIR and STTR Programs at the National Institutes of Health

The COVID-19 pandemic upended nearly every aspect of academia, leading colleges and universities to reexamine how they instruct their students and how they reward their faculty. But the pandemic was not the only disruptive event that took place in 2020. Colleges and universities have been forced to address issues related to productivity, teaching,student learning, mentoring, service, and innovative research in the context of remote or hybrid work - all amplified by the increased attention to and discussion of systemic racism, widespread economic hardships, and extreme environmental events. To help leaders of higher education understand how one particular issue - the current faculty reward, advancement, and hiring system - has changed and continues to change in response to several of the events of 2020, the National Academies of Sciences, Engineering, and Medicine commissioned a set of papers. These papers formed the basis for a series of four virtual workshops held on September 20, 23, 27, and 30, 2021. They examined how the events of 2020 have affected existing hiring and advancement policies and practices of institutions of higher education, the differential effects on the promotion and advancement of faculty from different populations, new policies and programs to support faculty, and new approaches that support more equitable faculty advancement. This publication is a short summary of the three workshops on the Next Normal for Leadership and Culture; Recruitment,Retention, and Support; and Advancement of Tenure and Non-Tenure-Track Faculty, as well as the fourth workshop with closing discussions. The presentations were intended to point toward the pandemic and beyond, as well as contextualize how these issues have played out in the last two years.

Promotion, Tenure, and Advancement through the Lens of 2020: Proceedings of a Workshop–in Brief

Academic preparation is critical to increase Black representation in Science, Engineering, and Medicine, but so, too, are such interrelated factors as providing mentoring and role models in sufficient numbers, adequately funding school and community support services, and analyzing the intentional and unintentional consequences of a range of policies and practices. To address these issues, the Roundtable on Black Men and Black Women of the National Academies of Sciences, Engineering, and Medicine convened a virtual workshop on September 2 and 3, 2020. Titled "Educational Pathways for Blacks in Science, Engineering, and Medicine: Exploring Barriers and Possible Interventions," the workshop provided a platform to explore challenges and opportunities, beginning in the earliest years of life through K-12 schooling, undergraduate and postgraduate education, and into the workforce. Presenters throughout the workshop provided perspectives from research and from their own experiences to discuss the need for systemic solutions inside and outside of formal education institutions. This publication summarizes the presentation and discussion of the workshop.

Educational Pathways for Black Students in Science, Engineering, and Medicine: Exploring Barriers and Possible Interventions: Proceedings of a Workshop

The U.S. Food and Drug Administration (FDA) has responsibility for protecting public health by ensuring the safety, efficacy, and security of drugs, biological products, and medical devices. In certain declared emergencies, FDA has the option to authorize use of a new product or a new use of an approved product - an authority known as Emergency Use Authorization (EUA)- if it has reason to believe that the product may be effective and that its known benefits outweigh its known risks. By contrast, in non-emergency situations, applicants must demonstrate a product's safety and effectiveness through a lengthier, more extensive process.

On October 5-6, 2021, the Committee on Science, Technology, and Law of the National Academies of Sciences, Engineering, and Medicine convened a virtual workshop on the EUA process. At the workshop, presenters and participants examined FDA's recent and historic use of EUAs, discussed lessons learned during the COVID-19 pandemic, and considered how those lessons might inform future efforts. The workshop also highlighted emergency mechanisms used by other health regulators and considered how U.S. and global regulatory partners can strengthen cooperation in responding to global health emergencies. This publication summarizes the presentation and discussion of the workshop.

The Food and Drug Administration's Emergency Use Authorization: Lessons Learned from the Past to Guide the Future: Proceedings of a Workshop–in Brief

While there has been growth in the number of women entrepreneurs in the United States in recent years, the percentage of women - particularly women of color - who decide to pursue an entrepreneurial career continues to be significantly lower than that of men. Entrepreneurship is a crucial enterprise responsible for driving innovation and economic growth, and increasing the representation of women, especially in STEM and medical (STEMM) industries, is critical to ensuring the nation's overall health, economic well-being, and global competitiveness.

Overcoming Structural Barriers for Women in Entrepreneurship: Proceedings of a Workshop–in Brief

As global mean temperatures rise and extreme climate and weather events increase in frequency and intensity, the severity of the climate situation and its potential impacts on human well-being - particularly of the world's most vulnerable populations - is strikingly evident. Research and policy responses to address climate change are required to meet U.S. targets set for the coming decades, which include a reduction of 50 to 52 percent of greenhouse gas emissions by 2030 and net-zero emissions by 2050. In coordination with other activities on climate, energy, and environmental and human health across the National Academies of Sciences, Engineering, and Medicine, the Government-University-Industry Research Roundtable convened a series of five workshops in June and July of 2021 to discuss opportunities for enhancing U.S. scientific and technological approaches to climate research and policy through cross-sector collaboration and partnerships, and to examine the impacts of climate policy on economic development, inequality, and international competitiveness. This document summarizes the presentations and discussions at the five workshops.

Partnerships and Cross-Sector Collaboration Priorities to Support Climate Research and Policy: Proceedings of a Workshop–in Brief

The National Academies of Sciences, Engineering, and Medicine convened a virtual workshop on August 3-5, 2021 to explore effective, feasible, and secure ways to document and provide health information for safe international travel in a way that is ethical and does not exacerbate inequities. Experts considered the use of COVID-19 travel credentials, denoting the traveler’s vaccination, testing, and/or recovery status. This Proceedings of a Workshop-in Brief provides a high-level summary of the discussion on possibilities for employing COVID-19 travel credentials, including how to overcome practical and ethical challenges and their potential role in preventing the spread of disease.

The Utility, Feasibility, Security, and Ethics of Verifiable COVID-19 Credentials for International Travel: Proceedings of a Workshop—in Brief

The Action Collaborative's Year 2 Annual Report summarizes the Action Collaborative's collective progress in the 2020-2021 year, identifies areas for future focus, discusses trends in organizations' reported areas of work, and highlights work shared by member organizations and partner network organizations. A complete collection of "descriptions of work" authored by member and partner network organizations is available in the repository.

Action Collaborative on Preventing Sexual Harassment in Higher Education: Year Two Annual Report of Activities

The National Academies Roundtable on Aligning Incentives for Open Science, established in 2019, has taken on an important role in addressing issues with open science. The roundtable convenes critical stakeholders to discuss the effectiveness of current incentives for adopting open science practices, current barriers of all types, and ways to move forward in order to align reward structures and institutional values. The Roundtable convened a virtual public workshop on fostering open science practices on November 5, 2020. The broad goal of the workshop was to identify paths to growing the nascent coalition of stakeholders committed to reenvisioning credit/reward systems (e.g., academic hiring, tenure and promotion, and grants)to fully incentivize open science practices. The workshop explored the information and resource needs of researchers, research institutions, government agencies, philanthropies, professional societies, and other stakeholders interested in further supporting and implementing open science practices. This publication summarizes the presentations and discussion of the workshop.

Developing a Toolkit for Fostering Open Science Practices: Proceedings of a Workshop

Rising awareness of and increased attention to sexual harassment has resulted in momentum to implement sexual harassment prevention efforts in higher education institutions. Work on preventing sexual harassment is an area that has recently garnered a lot of attention, especially around education and programs that go beyond the standard anti-sexual harassment trainings often used to comply with legal requirements.

On April 20-21, 2021, the National Academies of Sciences, Engineering, and Medicine hosted the workshop Developing Evaluation Metrics for Sexual Harassment Prevention Efforts. The workshop explored approaches and strategies for evaluating and measuring the effectiveness of sexual harassment interventions being implemented at higher education institutions and research and training sites, in order to assist institutions in transforming promising ideas into evidence-based best practices. Workshop participants also addressed methods, metrics, and measures that could be used to evaluate sexual harassment prevention efforts that lead to change in the organizational climate and culture and/or a change in behavior among community members. This publication summarizes the presentations and discussion of the workshop.

Evaluating the Effectiveness of Interventions to Prevent and Address Sexual Harassment: Proceedings of a Workshop

The Action Collaborative's Year 1 Annual Report summarizes the Action Collaborative's collective progress in the 2019-2020 year, identifies areas for future focus, discusses trends in members' reported areas of work, and highlights work shared by some member organizations.

Action Collaborative on Preventing Sexual Harassment in Higher Education: Year One Annual Report of Member Activities

The U.S. innovation environment relies on complex and diverse cross-sector collaborations and multi-stakeholder coalitions, and international relationships are critical to this mix of partnerships. For decades, top students, researchers, and entrepreneurs from around the world have sought to come to the United States, drawn by a system that values innovation, creativity, and an open exchange of knowledge and talent. Prioritizing these values and partnerships has fostered U.S. science and technology leadership for decades. At the same time, countries are investing heavily in their own research and development capabilities, while U.S. federal spending has remained stagnant as a percent of gross domestic product. Economic and national security concerns have impacted some aspects of America's collaborative spirit and openness.

In February and March 2021, the Government-University-Industry Research Roundtable (GUIRR) of the National Academies of Sciences, Engineering, and Medicine convened its membership to examine the opportunities and challenges of global cooperation and partnerships in the pursuit of U.S. science and technology leadership. Over the two months, GUIRR hosted six virtual workshops on elements of U.S. science and technology policy related to international engagement and competition. Topics included challenges to U.S. science and technology leadership; the intersection of science, foreign policy, and development assistance; public-private partnerships to foster innovation; the value of international research collaboration; U.S. leadership in international standards bodies; and attracting and supporting international students and researchers in the United States. This document summarizes the presentations and discussions at the six workshops.

Strengthening U.S. Science and Technology Leadership through Global Cooperation and Partnerships: Proceedings of a Workshop Series–in Brief

Congress enacted Section 230 to foster the growth of the internet by providing certain immunities for internetbased technology companies. Section 230 contains two key immunity provisions. The first specifies that a provider of an interactive computer service shall not "be treated as the publisher or speaker of any information provided by another information content provider," effectively exempting internet social media and networking services from liability under laws that apply to publishers, authors, and speakers. The second provides "good Samaritan" protection for providers who, in good faith, remove or moderate content that is obscene, lewd, lascivious, filthy, excessively violent, harassing, or otherwise objectionable.

While Section 230 has played an important role in development of the internet as a platform for the global exchange of information and ideas, the internet has evolved in unanticipated ways since 1996. Today, a small number of large companies operate social media platforms that millions use for information and public discourse. Concentration of power, disinformation (including sophisticated disinformation campaigns), abuse on social media (hate speech, harassment, bullying, and discriminatory practices), use of algorithms to amplify and target content and advertising, and lack of transparency in content moderation have become issues of increasing concern. There are many opinions regarding potential solutions, including about whether (or by what means) Section 230 should be revised.

On April 22 and 27, 2021, the Committee on Science, Technology, and Law of the National Academies of Sciences, Engineering, and Medicine convened a virtual workshop titled Section 230 Protections: Can Legal Revisions or Novel Technologies Limit Online Misinformation and Abuse? Participants and presenters explored the legal, policy, and technological aspects of Section 230 and its relationship with such critical issues as free speech, privacy, and civil rights. The workshop also addressed concerns about internet immunity protections while preserving free speech and democratic norms. This publication summarizes the presentation and discussion of the workshop.

Section 230 Protections: Can Legal Revisions or Novel Technologies Limit Online Misinformation and Abuse?: Proceedings of a Workshop–in Brief

On February 24-25, 2021, an ad hoc planning committee under the auspices of the National Academies of Sciences, Engineering, and Medicine's Committee on Science, Technology, and Law hosted a workshop titled Emerging Areas of Science, Engineering, and Medicine for the Courts. The workshop was organized to explore emerging issues in science, technology, and medicine that might be the basis of new chapters in a fourth edition of the Reference Manual on Scientific Evidence. The Reference Manual, a primary resource for federal judges on questions of science in litigation, is a joint publication of the National Academies of Sciences, Engineering, and Medicine and the Federal Judicial Center, the research and education arm of the federal judiciary.

Over the course of the workshop, judges discussed how they evaluate scientific evidence in court and scientists and others spoke about emerging issues in science and technology that may come before the courts in coming years. This publication highlights the presentation and discussion of the workshop.

Emerging Areas of Science, Engineering, and Medicine for the Courts: Proceedings of a Workshop–in Brief

Each year, tens of millions of individuals in the U.S. suffer from neurological and psychiatric disorders including neurodegenerative diseases such as Alzheimer's Disease and Parkinson's Disease, and psychiatric disorders such as autism spectrum disorder, depression and schizophrenia. Treatments for these diseases are often completely lacking or only partially effective, due in large part to the difficulty of conducting brain research and the complexity of the brain itself.

Researchers in recent years have developed new models to better represent and study the human brain. The three models considered in this report, all of which generate and use pluripotent stem cells from healthy individuals and patients, are human neural organoids, human neural transplants, and human-animal neural chimeras. The Emerging Field of Human Neural Organoids, Transplants, and Chimeras: Science, Ethics, and Governance reviews the status of research, considers its benefits and risks, discusses associated ethical issues, and considers governance mechanisms for this type of research.

The Emerging Field of Human Neural Organoids, Transplants, and Chimeras: Science, Ethics, and Governance

The COVID-19 pandemic is transforming the global economy and significantly shifting workforce demand, requiring quick, adaptive responses. The pandemic has revealed the vulnerabilities of many organizations and regional economies, and it has accelerated trends that could lead to significant improvements in productivity, performance, and resilience, which will enable organizations and regions to thrive in the "next normal." To explore how communities around the United States are addressing workforce issues laid bare by the COVID-19 pandemic and how they are taking advantage of local opportunities to expand their science, technology, engineering, mathematics, and medicine (STEMM) workforces to position them for success going forward, the Board of Higher Education and Workforce of the National Academies of Sciences, Engineering, and Medicine convened a series of workshops to identify immediate and near-term regional STEMM workforce needs in the wake of the COVID-19 pandemic.

The workshop planning committee identified five U.S. cities and their associated metropolitan areas - Birmingham, Alabama; Boston, Massachusetts; Richmond, Virginia; Riverside, California; and Wichita, Kansas - to host workshops highlighting promising practices that communities can use to respond urgently and appropriately to their STEMM workforce needs. A sixth workshop discussed how the lessons learned during the five region-focused workshops could be applied in other communities to meet STEMM workforce needs.

This proceedings of a virtual workshop series summarizes the presentations and discussions from the six public workshops that made up the virtual workshop series and highlights the key points raised during the presentations, moderated panel discussions and deliberations, and open discussions among the workshop participants.

Meeting Regional STEMM Workforce Needs in the Wake of COVID-19: Proceedings of a Virtual Workshop Series

While the COVID-19 pandemic has had devastating health and economic impacts in the United States, communities of color, especially Black communities, have been disproportionately affected. On June 23, 2020, the Roundtable on Black Men and Black Women in Science, Engineering, and Medicine of the National Academies of Sciences, Engineering, and Medicine convened a virtual workshop to discuss the landscape of COVID-19, including how systemic racism contributes to the disproportionate effects related to infection rates and mortality of this virus and other health conditions. Presenters highlighted relevant research and creative responses from many perspectives, including how Black scientists, engineers, and doctors are contributing to solutions and are ready to do more. National Academies leaders and members also discussed the role of the National Academies in addressing the pandemic and underlying issues of systemic racism that have led to health disparities in the United States. This publication summarizes the presentation and discussion of the workshop.

COVID-19 and the Present and Future of Black Communities: The Role of Black Physicians, Engineers, and Scientists: Proceedings of a Workshop

On March 22-23, 2021, an ad hoc planning committee under the auspices of the National Academies of Sciences, Engineering, and Medicine's Committee on Science, Technology, and Law hosted a virtual workshop titled The Science of Implicit Bias: Implications for Law and Policy. Implicit bias has been commonly defined as any unconscious or unacknowledged preferences that can affect a person's beliefs or behaviors, and in particular, an unconscious favoritism toward or prejudice against people of a certain race, gender, or group that influences one's own actions or perceptions. The methods for identifying the presence and degree of an individual's implicit bias, the presence of implicit bias throughout society, and the successes or failures of attempts to mitigate implicit bias are topics of much scientific inquiry, with ramifications for law and policy as well as a multitude of organizational settings. The ways in which implicit bias reflects or contributes to structural and systemic racism in the U.S. remains an open and urgent question. The workshop, organized by the Committee on the Science of Implicit Bias: Implications for Law and Policy, was convened to better understand the state of the science on this topic in the context of critical and ongoing discussions about racism in the United States.

The Science of Implicit Bias: Implications for Law and Policy: Proceedings of a Workshop–in Brief

The movement toward open science, data sharing, and increased transparency is being propelled by the need to rapidly address critical scientific challenges, such as the global COVID-19 public health crisis. This movement has supported growth in fields, such as artificial intelligence (AI), which has demonstrated potential to accelerate science, engineering, and medicine in new and exciting ways. To further advance innovation around these new approaches, the National Academies of Sciences, Engineering, and Medicine's Board on Research Data and Information convened a public virtual workshop on October 14-15, 2020, to address how researchers in different domains are utilizing data that undergo repeated processing, often in real-time, to accelerate scientific discovery. Although these topics were not originally part of the workshop, the impact of COVID-19 prompted the planning committee to add sessions on early career researchers' perspectives, as well as rapid review and publishing activities as a result of the pandemic. Participants also explored the advances needed to enable future progress in areas such as AI, cyberinfrastructure, standards, and policies. This publication summarizes the presentations and discussion of the workshop.

Data in Motion: New Approaches to Advancing Scientific, Engineering and Medical Progress: Proceedings of a Workshop--in Brief

The spring of 2020 marked a change in how almost everyone conducted their personal and professional lives, both within science, technology, engineering, mathematics, and medicine (STEMM) and beyond. The COVID-19 pandemic disrupted global scientific conferences and individual laboratories and required people to find space in their homes from which to work. It blurred the boundaries between work and non-work, infusing ambiguity into everyday activities. While adaptations that allowed people to connect became more common, the evidence available at the end of 2020 suggests that the disruptions caused by the COVID-19 pandemic endangered the engagement, experience, and retention of women in academic STEMM, and may roll back some of the achievement gains made by women in the academy to date.

The Impact of COVID-19 on the Careers of Women in Academic Sciences, Engineering, and Medicine identifies, names, and documents how the COVID-19 pandemic disrupted the careers of women in academic STEMM during the initial 9-month period since March 2020 and considers how these disruptions - both positive and negative - might shape future progress for women. This publication builds on the 2020 report Promising Practices for Addressing the Underrepresentation of Women in Science, Engineering, and Medicine to develop a comprehensive understanding of the nuanced ways these disruptions have manifested. The Impact of COVID-19 on the Careers of Women in Academic Sciences, Engineering, and Medicine will inform the academic community as it emerges from the pandemic to mitigate any long-term negative consequences for the continued advancement of women in the academic STEMM workforce and build on the adaptations and opportunities that have emerged.

The Impact of COVID-19 on the Careers of Women in Academic Sciences, Engineering, and Medicine

Federal laboratories play a unique role in the U.S. economy. Research and development conducted at these labs has contributed to the advancement or improvement of such key general-purpose technologies as nuclear energy, computers, the Internet, genomics, satellite navigation, the Global Positioning System, artificial intelligence, and virtual reality. Digital output from federal laboratories includes data, metadata, images, software, code, tools, databases, algorithms, and statistical models. Importantly, these digital products are nonrivalrous, meaning that unlike physical products, they can be copied at little or no cost and used by many without limit or additional cost.

Advancing Commercialization of Digital Products from Federal Laboratories explores opportunities to add economic value to U.S. industry through enhanced utilization of intellectual property around digital products created at federal laboratories. This report examines the current state of commercialization of digital products developed at the federal labs and, to a limited extent, by extramural awardees, to help identify barriers to commercialization and technology transfer, taking into account differences between government-owned, contractor-operated (GOCO) and government-owned, government-operated (GOGO) federal labs.

Advancing Commercialization of Digital Products from Federal Laboratories

Over the last few decades, research, activity, and funding has been devoted to improving the recruitment, retention, and advancement of women in the fields of science, engineering, and medicine. In recent years the diversity of those participating in these fields, particularly the participation of women, has improved and there are significantly more women entering careers and studying science, engineering, and medicine than ever before. However, as women increasingly enter these fields they face biases and barriers and it is not surprising that sexual harassment is one of these barriers.

Over thirty years the incidence of sexual harassment in different industries has held steady, yet now more women are in the workforce and in academia, and in the fields of science, engineering, and medicine (as students and faculty) and so more women are experiencing sexual harassment as they work and learn. Over the last several years, revelations of the sexual harassment experienced by women in the workplace and in academic settings have raised urgent questions about the specific impact of this discriminatory behavior on women and the extent to which it is limiting their careers.

Sexual Harassment of Women explores the influence of sexual harassment in academia on the career advancement of women in the scientific, technical, and medical workforce. This report reviews the research on the extent to which women in the fields of science, engineering, and medicine are victimized by sexual harassment and examines the existing information on the extent to which sexual harassment in academia negatively impacts the recruitment, retention, and advancement of women pursuing scientific, engineering, technical, and medical careers. It also identifies and analyzes the policies, strategies and practices that have been the most successful in preventing and addressing sexual harassment in these settings.

Harcèlement sexuel des femmes: Climat, culture et conséquences dans les filières universitaires de sciences, d'ingénierie et de médecine

Student wellbeing is foundational to academic success. One recent survey of postsecondary educators found that nearly 80 percent believed emotional wellbeing is a "very" or "extremely" important factor in student success. Studies have found the dropout rates for students with a diagnosed mental health problem range from 43 percent to as high as 86 percent. While dealing with stress is a normal part of life, for some students, stress can adversely affect their physical, emotional, and psychological health, particularly given that adolescence and early adulthood are when most mental illnesses are first manifested. In addition to students who may develop mental health challenges during their time in postsecondary education, many students arrive on campus with a mental health problem or having experienced significant trauma in their lives, which can also negatively affect physical, emotional, and psychological wellbeing.

The nation's institutions of higher education are seeing increasing levels of mental illness, substance use and other forms of emotional distress among their students. Some of the problematic trends have been ongoing for decades. Some have been exacerbated by the COVID-19 pandemic and resulting economic consequences. Some are the result of long-festering systemic racism in almost every sphere of American life that are becoming more widely acknowledged throughout society and must, at last, be addressed.

Mental Health, Substance Use, and Wellbeing in Higher Education lays out a variety of possible strategies and approaches to meet increasing demand for mental health and substance use services, based on the available evidence on the nature of the issues and what works in various situations. The recommendations of this report will support the delivery of mental health and wellness services by the nation's institutions of higher education.

Mental Health, Substance Use, and Wellbeing in Higher Education: Supporting the Whole Student

Heritable human genome editing - making changes to the genetic material of eggs, sperm, or any cells that lead to their development, including the cells of early embryos, and establishing a pregnancy - raises not only scientific and medical considerations but also a host of ethical, moral, and societal issues. Human embryos whose genomes have been edited should not be used to create a pregnancy until it is established that precise genomic changes can be made reliably and without introducing undesired changes - criteria that have not yet been met, says Heritable Human Genome Editing .

From an international commission of the U.S. National Academy of Medicine, U.S. National Academy of Sciences, and the U.K.'s Royal Society, the report considers potential benefits, harms, and uncertainties associated with genome editing technologies and defines a translational pathway from rigorous preclinical research to initial clinical uses, should a country decide to permit such uses. The report specifies stringent preclinical and clinical requirements for establishing safety and efficacy, and for undertaking long-term monitoring of outcomes. Extensive national and international dialogue is needed before any country decides whether to permit clinical use of this technology, according to the report, which identifies essential elements of national and international scientific governance and oversight.

Heritable Human Genome Editing

Despite the changing demographics of the nation and a growing appreciation for diversity and inclusion as drivers of excellence in science, engineering, and medicine, Black Americans are severely underrepresented in these fields. Racism and bias are significant reasons for this disparity, with detrimental implications on individuals, health care organizations, and the nation as a whole. The Roundtable on Black Men and Black Women in Science, Engineering, and Medicine was launched at the National Academies of Sciences, Engineering, and Medicine in 2019 to identify key levers, drivers, and disruptors in government, industry, health care, and higher education where actions can have the most impact on increasing the participation of Black men and Black women in science, medicine, and engineering.

On April 16, 2020, the Roundtable convened a workshop to explore the context for their work; to surface key issues and questions that the Roundtable should address in its initial phase; and to reach key stakeholders and constituents. This proceedings provides a record of the workshop.

The Impacts of Racism and Bias on Black People Pursuing Careers in Science, Engineering, and Medicine: Proceedings of a Workshop

On May 29, 2019, the National Academies of Sciences, Engineering, and Medicine, in cooperation with the Laboratory for Innovation Science at Harvard (LISH), convened a workshop in Washington, D.C. on the role of inducement prizes to spur American innovation. Unlike prizes that recognize past achievements, these inducement prizes are designed to stimulate innovative activity, whether it be the creation of a desired technology, orienting research efforts toward designing products that are capable of being used at scale by customers, or developing products with wide societal benefits. Workshop participants explored how prizes fit into federal and non-federal support for innovation, the benefits and disadvantages of prizes, and the differences between cash and non-cash prizes. Other discussion topics included the conditions under which prizes are most effective, how to measure the effectiveness of prizes, and the characteristics of prize winners. This publication summarizes the presentations and discussions from the workshop.

The Role of Inducement Prizes: Proceedings of a Workshop–in Brief

Oceans have long been a frontier of opportunity for exploration, scientific understanding, commerce, and trade for the United States. The transformative technologies of the fourth industrial revolution - artificial intelligence, machine learning, robotics, Internet of Things, biotechnology, advanced materials - are expected to recondition traditional ocean-based industries and enable the growth of new markets prioritizing sustainability. At the nexus of unprecedented environmental change and rapid technology innovation, how should the United States position itself as a leader in the global ocean economy? On February 4 and 5, 2020, the Government-University-Industry Research Roundtable convened experts to discuss the importance of cross-sector collaboration and the opportunities for U.S. leadership in the context of a changing ocean and a changing ocean economy. This publication highlights the presentation and discussion of the workshop.

The Role of Research and Technology in the Changing Ocean Economy: Proceedings of a Workshop–in Brief

Since its founding in 1982, the Small Business Innovation Research (SBIR) program has become the largest and most comprehensive public research and development funding program of small business research in the United States. An underlying tenet of the SBIR program, and the related Small Business Technology Transfer (STTR) program, is that small and young firms are an important source of new ideas that provide the underlying basis for technological innovation, productivity increases, and subsequent economic growth. By involving qualified small businesses in the nation's research and development efforts, SBIR/STTR grants stimulate the development of innovative technologies and help federal agencies achieve their missions and objectives.

At the request of the Department of Energy (DOE), this report examines the SBIR and STTR programs at DOE, focusing on the effectiveness of DOE's SBIR/STTR processes and procedures on topic and awardee selection; DOE outreach efforts to SBIR and STTR applicants; collaborations created between small businesses and research institutions on account of the programs; a range of direct economic and non-economic impacts to awardees; and the role of SBIR/STTR programs in stimulating technological innovation and contributing to DOE's research and development needs, whether directly from awardees or indirectly through spillovers from other firms.

Review of the SBIR and STTR Programs at the Department of Energy

Careers in science, engineering, and medicine offer opportunities to advance knowledge, contribute to the well-being of communities, and support the security, prosperity, and health of the United States. But many women do not pursue or persist in these careers, or advance to leadership positions - not because they lack the talent or aspirations, but because they face barriers, including: implicit and explicit bias; sexual harassment; unequal access to funding and resources; pay inequity; higher teaching and advising loads; and fewer speaking invitations, among others.

There are consequences from this underrepresentation of women for the nation as well: a labor shortage in many science, engineering, and medical professions that cannot be filled unless institutions and organizations recruit from a broad and diverse talent pool; lost opportunities for innovation and economic gain; and lost talent as a result of discrimination, unconscious bias, and sexual harassment.

Promising Practices for Addressing the Underrepresentation of Women in Science, Engineering, and Medicine reviews and synthesizes existing research on policies, practices, programs, and other interventions for improving the recruitment, retention, and sustained advancement into leadership roles of women in these disciplines. This report makes actionable recommendations to leverage change and drive swift, coordinated improvements to the systems of education, research, and employment in order to improve both the representation and leadership of women.

Promising Practices for Addressing the Underrepresentation of Women in Science, Engineering, and Medicine: Opening Doors

Faculty in science, technology, engineering, and medicine (STEM) are expected to excel in their technical work, teaching, and professional service. Their career advancement is often determined by academic peers evaluating accomplishments in these three areas. Recently, however, there is a growing concern that the evaluation of those accomplishments and traditional incentive systems are misaligned with some of the values and missions of higher education institutions, such as student learning, public engagement, and innovative research. Debates about current advancement systems also point to a body of research on the negative effects of traditional advancement criteria on the academic environment and workforce, including the influence of systemic and individual biases on the promotion and advancement of women and individuals from underrepresented populations.

To examine and potentially re-envision advancement pathways for all STEM faculty, the National Academies of Sciences, Engineering, and Medicine convened academic leaders, scientists, engineers, medical professionals, scholars, philanthropists, and representatives from higher education associations and research funding agencies in October 2019. The Convocation on Re-envisioning Promotion and Advancement for STEM Faculty: Aligning Incentives with Values addressed challenges with the current system of faculty advancement and considered changes to promotion practices. Particular emphasis was placed on aligning faculty reward pathways to institutional and departmental missions and values. This publication highlights the presentation and discussion of the convocation.

Re-envisioning Promotion and Advancement for STEM Faculty: Proceedings of a Workshop–in Brief

A multi-sectoral, interdisciplinary approach to understanding the dimensions and determinants of social disparities and their intersections is necessary to work toward equity and equality of opportunity as rapid technology innovation changes the future of work. To examine the matter, the Government-University-Industry Research Roundtable held a workshop at the National Academy of Sciences on October 15-16, 2019 to consider how the research community of federal and state governments, academia, companies, and other actors can take institutional and collective action to identify and address disparities at the intersections that will make interventions most effective. The workshop explored how research institutions act as anchors in their communities to reach marginalized populations and considered best practices for community engagement. This publication summarizes the presentation and discussion of the workshop.

Using Research and Technology to Address Compounding Disparities: Proceedings of a Workshop–in Brief

The actual and potential benefits of open science include strengthened rigor and reliability, the ability to address new questions,faster and more inclusive dissemination of knowledge, broader participation in research, effective use of resources, improved performance of research tasks, and open publication for public benefit. As one effort to increase the contributions of open science among many, the Board on Research Data and Information of the National Academies of Sciences, Engineering,and Medicine established the Roundtable on Aligning Incentives for Open Science. On September 20, 2019, the Roundtable organized a public symposium in Washington, DC to consider some of the barriers and challenges to open science, as well as ways to overcome them. Key external stakeholders - including researchers, librarians, learned societies, publishers and infrastructure developers - shared their insights on the current state of the research ecosystem, as well as their visions for how open science can function at scale. This publication highlights the presentations of the event.

Advancing Open Science Practices: Stakeholder Perspectives on Incentives and Disincentives: Proceedings of a Workshop–in Brief

Teachers play a critical role in the success of their students, both academically and in regard to long term outcomes such as higher education participation and economic attainment. Expectations for teachers are increasing due to changing learning standards and a rapidly diversifying student population. At the same time, there are perceptions that the teaching workforce may be shifting toward a younger and less experienced demographic. These actual and perceived changes raise important questions about the ways teacher education may need to evolve in order to ensure that educators are able to meet the needs of students and provide them with classroom experiences that will put them on the path to future success.

Changing Expectations for the K-12 Teacher Workforce: Policies, Preservice Education, Professional Development, and the Workplace explores the impact of the changing landscape of K-12 education and the potential for expansion of effective models, programs, and practices for teacher education. This report explores factors that contribute to understanding the current teacher workforce, changing expectations for teaching and learning, trends and developments in the teacher labor market, preservice teacher education, and opportunities for learning in the workplace and in-service professional development.

Changing Expectations for the K-12 Teacher Workforce: Policies, Preservice Education, Professional Development, and the Workplace

Mentorship is a catalyst capable of unleashing one's potential for discovery, curiosity, and participation in STEMM and subsequently improving the training environment in which that STEMM potential is fostered. Mentoring relationships provide developmental spaces in which students' STEMM skills are honed and pathways into STEMM fields can be discovered. Because mentorship can be so influential in shaping the future STEMM workforce, its occurrence should not be left to chance or idiosyncratic implementation. There is a gap between what we know about effective mentoring and how it is practiced in higher education.

The Science of Effective Mentorship in STEMM studies mentoring programs and practices at the undergraduate and graduate levels. It explores the importance of mentorship, the science of mentoring relationships, mentorship of underrepresented students in STEMM, mentorship structures and behaviors, and institutional cultures that support mentorship. This report and its complementary interactive guide present insights on effective programs and practices that can be adopted and adapted by institutions, departments, and individual faculty members.

The Science of Effective Mentorship in STEMM

The fifth generation of wireless networks and technologies presents significant opportunity to transform connectivity. Improvements in bandwidth, latency, coverage, reliability, and security can enable an array of enhanced and new applications. On June 11-12, 2019, the National Academies of Sciences, Engineering, and Medicine convened experts to speak about the state of research, development, and deployment of 5G technologies; the challenges of securing 5G networked devices and infrastructure; and the global landscape of competition on 5G deployment. Priorities for cross sector collaboration and coordination between government, universities, and industry to advance the nation's leadership in wireless communication were also discussed. This publication summarizes the presentations and discussions from the workshop.

The Transformational Impact of 5G: Proceedings of a Workshop–in Brief

On May 24, 2019 the National Academies of Sciences, Engineering and Medicine, in partnership with the Royal Society, held a symposium entitled Artificial Intelligence: An International Dialogue in Washington, DC. The symposium addressed if and how artificial intelligence (AI) would benefit from further international cooperation. It primarily focused on the aspect of AI known as ‘machine learning’—or ‘deep learning’—as this is an area of increasing attention and rapid technological advancement. The event also addressed discussions at a day-and-a-half meeting on May 23-24, 2019* convened by the National Academies and Royal Society where 45 scientists, engineers, and other AI experts from the United States, United Kingdom, Canada, China, the European Commission, Germany, and Japan discussed key areas of national and international policy on AI where international collaboration would be most beneficial. This publication summarizes the presentations and discussions from the May 24 workshop.

Artificial Intelligence: An International Dialogue: Proceedings of a Workshop–in Brief

The Manufacturing USA initiative seeks to reinforce U.S.-based advanced manufacturing through partnerships among industry, academia, and government. Started in 2012 and established with bipartisan support by the Revitalize American Manufacturing and Innovation Act of 2014, the initiative envisages a nationwide network of research centers for manufacturing innovation. Some 14 manufacturing innovation institutes have been established to facilitate the movement of early-stage research into proven capabilities ready for adoption by U.S. manufacturers.

To better understand the role and experiences of the Manufacturing USA institutes, a committee of the National Academies of Sciences, Engineering, and Medicine convened a workshop on May 23, 2017 drawing together institute directors and manufacturing policy experts along with leaders from industry, academia, and government. Given the continued prominence of enhancing domestic manufacturing and international competitiveness in public policy discussions, the National Academies convened a second workshop on November 14, 2018, to monitor the progress of the Manufacturing USA institutes. This publication summarizes the presentations and discussions from this second workshop.

Revisiting the Manufacturing USA Institutes: Proceedings of a Workshop

Digital technology is incorporated into nearly every facet of American life, from commerce, community, healthcare, food systems, transportation, education, media, entertainment, and employment. Its increasingly integral nature raises critical questions moving forward, such as what emerging technologies are complicating the ability to verify authenticity and integrity in a digital world? Also, what emerging technologies enable the creation of systems of trust that enforce standards of authenticity, integrity, and security? How can partnerships between government, universities, and companies shape public policy to prioritize authenticity and integrity within systems—and who will be the stewards and custodians of such systems?

To address these and related questions, the National Academies of Sciences, Engineering, and Medicine held a workshop on February 19-20, 2019. Through presentations from experts and robust discussions, the workshop focused on technologies, processes, and governance that affect digital privacy and security now and into the coming decades. This publication summarizes the presentations and discussions from the workshop.

Authenticity, Integrity, and Security in a Digital World: Proceedings of a Workshop–in Brief

The innovation environment of the 21st century is characterized by disruption, accelerated technology development, and globalized access to information, which is a dramatic shift from the largely government-controlled Cold War innovation environment of the previous century. The shift from an industrial age to an information age has lowered the barriers to entry into the global economy and compelled institutions to respond with greater efficiency and speed. How are companies, universities, and the U.S. government adapting to the accelerated pace of innovation in the current open information age? How are partnerships among the three sectors being utilized to adapt to current innovation conditions, and how can partnerships be fostered more deliberately to increase U.S. competitiveness in the global economy? To address these and related questions, the National Academies of Sciences, Engineering, and Medicine held a workshop on October 16–17, 2018. This publication summarizes the presentations and discussions from the workshop.

Adapting to the 21st Century Innovation Environment: Proceedings of a Workshop–in Brief

There are over 20 million young people of color in the United States whose representation in STEM education pathways and in the STEM workforce is still far below their numbers in the general population. Their participation could help re-establish the United States' preeminence in STEM innovation and productivity, while also increasing the number of well-educated STEM workers.

There are nearly 700 minority-serving institutions (MSIs) that provide pathways to STEM educational success and workforce readiness for millions of students of color—and do so in a mission-driven and intentional manner. They vary substantially in their origins, missions, student demographics, and levels of institutional selectivity. But in general, their service to the nation provides a gateway to higher education and the workforce, particularly for underrepresented students of color and those from low-income and first-generation to college backgrounds. The challenge for the nation is how to capitalize on the unique strengths and attributes of these institutions and to equip them with the resources, exceptional faculty talent, and vital infrastructure needed to educate and train an increasingly critical portion of current and future generations of scientists, engineers, and health professionals.

Minority Serving Institutions examines the nation's MSIs and identifies promising programs and effective strategies that have the highest potential return on investment for the nation by increasing the quantity and quality MSI STEM graduates. This study also provides critical information and perspective about the importance of MSIs to other stakeholders in the nation's system of higher education and the organizations that support them.

Minority Serving Institutions: America's Underutilized Resource for Strengthening the STEM Workforce

On November 27-29, 2018, the U.S. National Academy of Sciences and U.S. National Academy of Medicine, the Royal Society of the United Kingdom, and the Academy of Sciences of Hong Kong convened the Second International Summit on Human Genome Editing at the University of Hong Kong. The summit brought together more than 500 researchers, ethicists, policymakers, representatives from scientific and medical academies, patient group representatives, and others from around the world. Over the two-and-a-half-day event, topics including the potential benefits and risks of human genome editing, ethical and cultural perspectives, regulatory and policy considerations, and public outreach and engagement efforts were explored. This publication summarizes the presentations and discussions from the event.

Second International Summit on Human Genome Editing: Continuing the Global Discussion: Proceedings of a Workshop–in Brief

In an increasingly interconnected world, perhaps it should come as no surprise that international collaboration in science and technology research is growing at a remarkable rate. As science and technology capabilities grow around the world, U.S.-based organizations are finding that international collaborations and partnerships provide unique opportunities to enhance research and training.

International research agreements can serve many purposes, but data are always involved in these collaborations. The kinds of data in play within international research agreements varies widely and may range from financial and consumer data, to Earth and space data, to population behavior and health data, to specific project-generated data—this is just a narrow set of examples of research data but illustrates the breadth of possibilities. The uses of these data are various and require accounting for the effects of data access, use, and sharing on many different parties. Cultural, legal, policy, and technical concerns are also important determinants of what can be done in the realms of maintaining privacy, confidentiality, and security, and ethics is a lens through which the issues of data, data sharing, and research agreements can be viewed as well.

A workshop held on March 14-16, 2018, in Washington, DC explored the changing opportunities and risks of data management and use across disciplinary domains. The third workshop in a series, participants gathered to examine advisory principles for consideration when developing international research agreements, in the pursuit of highlighting promising practices for sustaining and enabling international research collaborations at the highest ethical level possible. The intent of the workshop was to explore, through an ethical lens, the changing opportunities and risks associated with data management and use across disciplinary domains—all within the context of international research agreements. This publication summarizes the presentations and discussions from the workshop.

Data Matters: Ethics, Data, and International Research Collaboration in a Changing World: Proceedings of a Workshop

Micropolitan areas of the United States, cities with populations between 10,000 and 50,000 people, have received little attention from either the research or policy communities regarding their roles in the U.S. economy. Only recently has attention turned toward small-town America in order to try to characterize and understand the roles micropolitan areas play in productivity, growth, employment and, more broadly, in the innovation system of the nation. Likewise, little research has been undertaken to gain a better understanding of which policy instruments and frameworks can best support economic development and workforce readiness in micropolitan areas.

To explore this topic, the National Academies of Sciences, Engineering, and Medicine held a one-day workshop in Indianapolis in April 2018 on “Advancing Economic Development and Workforce Readiness in Micropolitan Areas” in coordination with the Indiana University Public Policy Institute’s “Thriving Communities, Thriving State” program. This event sought to connect theory to practice by bringing together policy makers, individuals from industry, and researchers across several disciplines to discuss lessons from current research and on-the-ground experience, explore how this knowledge may be utilized to shape policies to effectively support economic development and workforce readiness, and ultimately lay the groundwork for creating a research and knowledge agenda for future work. This publication summarizes the presentations and discussions from the workshop.

Advancing Economic Development and Workforce Readiness in Micropolitan Areas: Proceedings of a Workshop–in Brief

Partnerships between government, universities, and industry have formed the foundation of the national research enterprise for more than half a century, driving science and technology breakthroughs that have created significant social and economic benefits for the nation and the world. However, there is growing recognition that the partnerships supporting the U.S. innovation ecosystem extend beyond the “triple-helix actors” to include other stakeholders. Philanthropic institutions, angel and venture capital groups, political advocates, and nongovernmental organizations participate as partners in funding, influence, and support of basic science and research and development. How these partnerships are established and maintained, how effective they are in supporting science and innovation, and how to assess their overall impact on the U.S. research enterprise are increasingly relevant questions for leaders in both science and policy.

To address these and related questions, the National Academies of Sciences, Engineering, and Medicine held a workshop on June 12–13, 2018. Through presentations and robust discussion periods, workshop participants explored the engagement models, goals, incentives, and risk tolerance of some non-traditional research partners and discussed how their efforts intersect with those of the federal government, universities, and industry. This publication summarizes the presentations and discussions from the workshop.

Strategies for Engagement of Non-Traditional Partners in the Research Enterprise: Proceedings of a Workshop–in Brief

Securing the Vote: Protecting American Democracy

The U.S. system of graduate education in science, technology, engineering, and mathematics (STEM) has served the nation and its science and engineering enterprise extremely well. Over the course of their education, graduate students become involved in advancing the frontiers of discovery, as well as in making significant contributions to the growth of the U.S. economy, its national security, and the health and well-being of its people. However, continuous, dramatic innovations in research methods and technologies, changes in the nature and availability of work, shifts in demographics, and expansions in the scope of occupations needing STEM expertise raise questions about how well the current STEM graduate education system is meeting the full array of 21st century needs. Indeed, recent surveys of employers and graduates and studies of graduate education suggest that many graduate programs do not adequately prepare students to translate their knowledge into impact in multiple careers.

Graduate STEM Education for the 21st Century examines the current state of U.S. graduate STEM education. This report explores how the system might best respond to ongoing developments in the conduct of research on evidence-based teaching practices and in the needs and interests of its students and the broader society it seeks to serve. This will be an essential resource for the primary stakeholders in the U.S. STEM enterprise, including federal and state policymakers, public and private funders, institutions of higher education, their administrators and faculty, leaders in business and industry, and the students the system is intended to educate.

Graduate STEM Education for the 21st Century

Openness and sharing of information are fundamental to the progress of science and to the effective functioning of the research enterprise. The advent of scientific journals in the 17th century helped power the Scientific Revolution by allowing researchers to communicate across time and space, using the technologies of that era to generate reliable knowledge more quickly and efficiently. Harnessing today's stunning, ongoing advances in information technologies, the global research enterprise and its stakeholders are moving toward a new open science ecosystem. Open science aims to ensure the free availability and usability of scholarly publications, the data that result from scholarly research, and the methodologies, including code or algorithms, that were used to generate those data.

Open Science by Design is aimed at overcoming barriers and moving toward open science as the default approach across the research enterprise. This report explores specific examples of open science and discusses a range of challenges, focusing on stakeholder perspectives. It is meant to provide guidance to the research enterprise and its stakeholders as they build strategies for achieving open science and take the next steps.

Open Science by Design: Realizing a Vision for 21st Century Research

Sexual Harassment of Women: Climate, Culture, and Consequences in Academic Sciences, Engineering, and Medicine

Black men are increasingly underrepresented in medical schools and in the medical profession. A diverse workforce is a key attribute of quality healthcare and research suggests that a diverse workforce may help to advance cultural competency and increase access to high-quality health care, especially for underserved populations. Conversely, lack of diversity in the health workforce threatens health care quality and access and contributes to health disparities. In this way, the growing absence of Black men in medicine is especially troubling, because their absence in medicine may have adverse consequences for health care access, quality, and outcomes among Black Americans and Americans overall.

To better understand the factors that contribute to the low participation of Black men in the medical profession, facilitate discussion of current strategies used to increase their participation in medical education, and explore new strategies along the educational and professional pipeline that may have potential to increase participation in medicine, the National Academies of Sciences, Engineering, and Medicine and the Cobb Institute jointly convened a 2-day workshop in November 2017, in Washington, DC. This publication summarizes the presentations and discussions from the workshop.

An American Crisis: The Growing Absence of Black Men in Medicine and Science: Proceedings of a Joint Workshop

The big data revolution, accompanied by the development and deployment of wearable medical devices and mobile health applications, has enabled the biomedical community to apply artificial intelligence (AI) and machine learning algorithms to vast amounts of data. This shift has created new research opportunities in predictive analytics, precision medicine, virtual diagnosis, patient monitoring, and drug discovery and delivery, which has garnered the interests of government, academic, and industry researchers alike and is already putting new tools in the hands of practitioners.

This boom in digital health opportunities has also raised numerous questions concerning the future of biomedical research and healthcare practices. How reliable are deployed AI-driven diagnostic tools, and what is the impact of these tools on doctors and patients? How vulnerable are algorithms to bias and unfairness? How can research improve the process of detecting unfairness in machine learning algorithms? How are other fields simultaneously advancing AI applications? How will academia prepare scientists with the skills to meet the demands of the newly transformed industry? Informed answers to these and other questions require interdisciplinary discussion and collaboration. On February 13 and 14, 2018, the National Academies of Sciences, Engineering, and Medicine convened a workshop to explore these and other questions related to the emerging use of AI and machine learning technologies in translational research. This publication summarizes the presentations and discussions from the workshop.

Artificial Intelligence and Machine Learning to Accelerate Translational Research: Proceedings of a Workshop—in Brief

In the United States, broad study in an array of different disciplines —arts, humanities, science, mathematics, engineering— as well as an in-depth study within a special area of interest, have been defining characteristics of a higher education. But over time, in-depth study in a major discipline has come to dominate the curricula at many institutions. This evolution of the curriculum has been driven, in part, by increasing specialization in the academic disciplines. There is little doubt that disciplinary specialization has helped produce many of the achievement of the past century. Researchers in all academic disciplines have been able to delve more deeply into their areas of expertise, grappling with ever more specialized and fundamental problems.

Yet today, many leaders, scholars, parents, and students are asking whether higher education has moved too far from its integrative tradition towards an approach heavily rooted in disciplinary "silos". These "silos" represent what many see as an artificial separation of academic disciplines. This study reflects a growing concern that the approach to higher education that favors disciplinary specialization is poorly calibrated to the challenges and opportunities of our time.

The Integration of the Humanities and Arts with Sciences, Engineering, and Medicine in Higher Education examines the evidence behind the assertion that educational programs that mutually integrate learning experiences in the humanities and arts with science, technology, engineering, mathematics, and medicine (STEMM) lead to improved educational and career outcomes for undergraduate and graduate students. It explores evidence regarding the value of integrating more STEMM curricula and labs into the academic programs of students majoring in the humanities and arts and evidence regarding the value of integrating curricula and experiences in the arts and humanities into college and university STEMM education programs.

The Integration of the Humanities and Arts with Sciences, Engineering, and Medicine in Higher Education: Branches from the Same Tree

Since the end of the Second World War, the United States has developed the world's preeminent system for biomedical research, one that has given rise to revolutionary medical advances as well as a dynamic and innovative business sector generating high-quality jobs and powering economic output and exports for the U.S. economy. However, there is a growing concern that the biomedical research enterprise is beset by several core challenges that undercut its vitality, promise, and productivity and that could diminish its critical role in the nation's health and innovation in the biomedical industry.

Among the most salient of these challenges is the gulf between the burgeoning number of scientists qualified to participate in this system as academic researchers and the elusive opportunities to establish long-term research careers in academia. The patchwork of measures to address the challenges facing young scientists that has emerged over the years has allowed the U.S. biomedical enterprise to continue to make significant scientific and medical advances. These measures, however, have not resolved the structural vulnerabilities in the system, and in some cases come at a great opportunity cost for young scientists. These unresolved issues could diminish the nation's ability to recruit the best minds from all sectors of the U.S. population to careers in biomedical research and raise concerns about a system that may favor increasingly conservative research proposals over high-risk, innovative ideas.

The Next Generation of Biomedical and Behavioral Sciences Researchers: Breaking Through evaluates the factors that influence transitions into independent research careers in the biomedical and behavioral sciences and offers recommendations to improve those transitions. These recommendations chart a path to a biomedical research enterprise that is competitive, rigorous, fair, dynamic, and can attract the best minds from across the country.

The Next Generation of Biomedical and Behavioral Sciences Researchers: Breaking Through

Strengthening the long-standing university-industry-government partnership is necessary to safeguard continued American leadership in research and innovation to support America’s prosperity, security, and national goals. On November 15, 2017, more than 300 educators, researchers, business and government leaders, and others gathered for a day-long series of discussions on revitalizing this historical partnership. This publication briefly summarizes the presentations and discussions from the workshop.

Revitalizing the University-Industry-Government Partnership: Creating New Opportunities for the 21st Century: Proceedings of a Workshop–in Brief

The field of computer science (CS) is currently experiencing a surge in undergraduate degree production and course enrollments, which is straining program resources at many institutions and causing concern among faculty and administrators about how best to respond to the rapidly growing demand. There is also significant interest about what this growth will mean for the future of CS programs, the role of computer science in academic institutions, the field as a whole, and U.S. society more broadly.

Assessing and Responding to the Growth of Computer Science Undergraduate Enrollments seeks to provide a better understanding of the current trends in computing enrollments in the context of past trends. It examines drivers of the current enrollment surge, relationships between the surge and current and potential gains in diversity in the field, and the potential impacts of responses to the increased demand for computing in higher education, and it considers the likely effects of those responses on students, faculty, and institutions. This report provides recommendations for what institutions of higher education, government agencies, and the private sector can do to respond to the surge and plan for a strong and sustainable future for the field of CS in general, the health of the institutions of higher education, and the prosperity of the nation.

Assessing and Responding to the Growth of Computer Science Undergraduate Enrollments

Federal investments in research and development have historically supported the security of the nation, the protection of public health and the environment, the growth of new industries, and the employment of millions of Americans. However, proposed cuts to federal support and policy guidance could encourage more state governments to take on new or larger roles in developing innovation policy priorities. On October 17 and 18, 2017, the Government-University-Industry Research Roundtable held a meeting to consider how federal R&D policies affect states, and how state governments’ roles in shaping local and regional innovation ecosystems will affect national R&D competitiveness and economic growth. Speakers also discussed the ways in which economic development efforts in states and regions drive innovation and economic growth. This publication briefly summarizes the presentations and discussions from the meeting.

The Role of State Governments in Economic Development and R&D Competitiveness: Proceedings of a Workshop–in Brief

Advances in science and technology have led to the creation of large amounts of data—data that could be harnessed to improve productivity, cure disease, and address many other critical issues. Consensus in the scientific community is growing that the transition to truly data-driven and open science is best achieved by the establishment of a globally interoperable research infrastructure.

A number of projects are looking to establish this infrastructure and exploit data to its fullest potential. Several projects in the United States, Europe, and China have made significant strides toward this effort. The goal of these projects is to make research data findable, accessible, interoperable, and reusable, or FAIR. The expected impact and benefits of FAIR data are substantial. To realize these benefits, there is a need to examine critical success factors for implementation, including training of a new generation of data experts to provide the necessary capacity. On November 1, 2017, the National Academies of Sciences, Engineering, and Medicine organized a symposium to explore these issues. This publication briefly summarizes the presentations and discussions from the symposium.

International Coordination for Science Data Infrastructure: Proceedings of a Workshop–in Brief

The integrity of knowledge that emerges from research is based on individual and collective adherence to core values of objectivity, honesty, openness, fairness, accountability, and stewardship. Integrity in science means that the organizations in which research is conducted encourage those involved to exemplify these values in every step of the research process. Understanding the dynamics that support – or distort – practices that uphold the integrity of research by all participants ensures that the research enterprise advances knowledge.

The 1992 report Responsible Science: Ensuring the Integrity of the Research Process evaluated issues related to scientific responsibility and the conduct of research. It provided a valuable service in describing and analyzing a very complicated set of issues, and has served as a crucial basis for thinking about research integrity for more than two decades. However, as experience has accumulated with various forms of research misconduct, detrimental research practices, and other forms of misconduct, as subsequent empirical research has revealed more about the nature of scientific misconduct, and because technological and social changes have altered the environment in which science is conducted, it is clear that the framework established more than two decades ago needs to be updated.

Responsible Science served as a valuable benchmark to set the context for this most recent analysis and to help guide the committee's thought process. Fostering Integrity in Research identifies best practices in research and recommends practical options for discouraging and addressing research misconduct and detrimental research practices.

Fostering Integrity in Research

The Manufacturing USA initiative seeks to reinforce U.S.-based advanced manufacturing through partnerships among industry, academia, and government. Started in 2012 and established with bipartisan support by the Revitalize American Manufacturing and Innovation Act of 2014, the initiative envisages a nationwide network of research centers for manufacturing innovation. As of May 2017, 14 manufacturing innovation institutes had been established to facilitate the movement of early-stage research into proven capabilities ready for adoption by U.S. manufacturers.

To better understand the role and experiences of the Manufacturing USA institutes to date, a committee of the Innovation Policy Forum of the National Academies of Sciences, Engineering, and Medicine convened a workshop on May 23, 2017 drawing together institute directors and manufacturing policy experts along with leaders from industry, academia, and government. Participants addressed the role of the manufacturing institutes in increasing advanced manufacturing in the United States, examined selected foreign programs designed to support advanced manufacturing, and reviewed recent assessments of existing institutes. This publication summarizes the presentations and discussions from the workshop.

Securing Advanced Manufacturing in the United States: The Role of Manufacturing USA: Proceedings of a Workshop

Companies know exactly what they spend on research, but knowing exactly what they get for their money is a more difficult question to answer. Without a good way to measure value, research can be viewed as a cost that can be cut to boost near-term earnings. The number of patents is one metric for measuring the value of research, but that information is not necessarily useful without knowing the business value of those patents in the marketplace. What other metrics serve to highlight the value of research and innovation? On June 27 and 28, 2017, the Government-University-Industry Research Roundtable held a meeting to explore these issues. This publication briefly summarizes the presentations and discussions from the meeting.

Beyond Patents: Assessing the Value and Impact of Research Investments: Proceedings of a Workshop–in Brief

Through grants, contracts, and loans, the federal government invests over $140 billion annually in research and development. Most economists agree that public support for research and development is essential given that the private sector—unable to internalize the social benefits—will underinvest in the creation of knowledge that may lead to innovation. Critics have questioned the adequacy of the return on this public investment, arguing that government investment is ineffective, inefficient, and may simply crowd out private investment. Overlaying these issues are questions about the broader impacts of these federal investments on inequality—that is, the distribution of gains from the fruits of innovation.

On December 15, 2016, the Innovation Policy Forum of the Board on Science, Technology, and Economic Policy held a workshop designed to gather leading researchers to explore the returns to federal investments in the innovation system, which include economic growth, job creation, as well as improved health and quality of life. The workshop was structured into four panels investigating social returns to: (1) federal investments in basic research; (2) federal support to private-sector innovation (which include research and expenditure (R&E) tax credits, small business support, prizes, and the patent system, among other policies); (3) public investments in human capital and supporting infrastructure such as digital infrastructure; and (4) the relationship between innovation, inequality, and social mobility. This publication briefly summarizes the presentations and discussions from the workshop.

Returns to Federal Investments in the Innovation System: Proceedings of a Workshop–in Brief

The potential misuse of advances in life sciences research is raising concerns about national security threats. Dual Use Research of Concern in the Life Sciences: Current Issues and Controversies examines the U.S. strategy for reducing biosecurity risks in life sciences research and considers mechanisms that would allow researchers to manage the dissemination of the results of research while mitigating the potential for harm to national security.

Dual Use Research of Concern in the Life Sciences: Current Issues and Controversies

In 2005, the National Research Council report Rising Above the Gathering Storm recommended a new way for the federal government to spur technological breakthroughs in the energy sector. It recommended the creation of a new agency, the Advanced Research Projects Agency-Energy, or ARPA-E, as an adaptation of the Defense Advanced Research Projects Agency (DARPA) model—widely considered a successful experiment that has funded out-of-the-box, transformative research and engineering that made possible the Internet, GPS, and stealth aircraft. This new agency was envisioned as a means of tackling the nation's energy challenges in a way that could translate basic research into technological breakthroughs while also addressing economic, environmental, and security issues.

Congress authorized ARPA-E in the 2007 America COMPETES Act and requested an early assessment following 6 years of operation to examine the agency's progress toward achieving its statutory mission and goals. This report documents the results of that assessment. It includes both an operational assessment of the agency's funding programs and a technical assessment of its awards, to the extent possible.

An Assessment of ARPA-E

Genome editing is a powerful new tool for making precise alterations to an organism's genetic material. Recent scientific advances have made genome editing more efficient, precise, and flexible than ever before. These advances have spurred an explosion of interest from around the globe in the possible ways in which genome editing can improve human health. The speed at which these technologies are being developed and applied has led many policymakers and stakeholders to express concern about whether appropriate systems are in place to govern these technologies and how and when the public should be engaged in these decisions.

Human Genome Editing considers important questions about the human application of genome editing including: balancing potential benefits with unintended risks, governing the use of genome editing, incorporating societal values into clinical applications and policy decisions, and respecting the inevitable differences across nations and cultures that will shape how and whether to use these new technologies. This report proposes criteria for heritable germline editing, provides conclusions on the crucial need for public education and engagement, and presents 7 general principles for the governance of human genome editing.

Human Genome Editing: Science, Ethics, and Governance

The Government-University-Industry Research Roundtable held a meeting on February 28 and March 1, 2017, to explore trends in public opinion of science, examine potential sources of mistrust, and consider ways that cross-sector collaboration between government, universities, and industry may improve public trust in science and scientific institutions in the future. This publication briefly summarizes the presentations and discussions from the meeting.

Examining the Mistrust of Science: Proceedings of a Workshop–in Brief

Mentoring has long been understood as a beneficial component of academic and professional development. But investigations of the attributes of effective mentoring interactions in science, technology, engineering, mathematics, and medical (STEMM) education are only now starting to shed light on how exactly these complex and dynamic relationships form, evolve, and impact the lives and careers of the current and next generation of STEMM professionals.

To explore the conversation surrounding this highly interdisciplinary field, the Board on Higher Education and Workforce and the Committee on Women in Science, Engineering, and Medicine, in collaboration with the Board on Science Education and the Teacher Advisory Council, convened a workshop in Washington D.C. on February 9-10, 2017. Educators, scientists, engineers, industry leaders, and scholars from a wide range of career stages focused on identifying successful practices and metrics for mentoring students in STEMM career pathways. Workshop sessions spanned topics across the mentoring field: definitions, theories, practices, perspectives, evidence, research, identity, and reflection, with a particular emphasis on identifying the evidence supporting successful mentoring practices for women and students of color across high school and postsecondary education. This publication briefly summarizes the presentations and discussions from the workshop.

Effective Mentoring in STEMM: Practice, Research, and Future Directions: Proceedings of a Workshop–in Brief

Skilled technical occupations—defined as occupations that require a high level of knowledge in a technical domain but do not require a bachelor’s degree for entry—are a key component of the U.S. economy. In response to globalization and advances in science and technology, American firms are demanding workers with greater proficiency in literacy and numeracy, as well as strong interpersonal, technical, and problem-solving skills. However, employer surveys and industry and government reports have raised concerns that the nation may not have an adequate supply of skilled technical workers to achieve its competitiveness and economic growth objectives.

In response to the broader need for policy information and advice, Building America’s Skilled Technical Workforce examines the coverage, effectiveness, flexibility, and coordination of the policies and various programs that prepare Americans for skilled technical jobs. This report provides action-oriented recommendations for improving the American system of technical education, training, and certification.

Building America's Skilled Technical Workforce

How should we convey science—both its findings and its value to society—to the many members of the public who lack either scientific training or intense interest in scientific progress? In October 2016 the National Academies of Sciences, Engineering, and Medicine convened a workshop to explore ways of better presenting science—both specific findings and the processes of discovering and confirming—to the public. Participants discussed ways to develop data-enriched narratives that communicate to the public and policy makers in an engaging and rigorous way the work of basic research. They also explored the varied ways in which research provides the foundation for products, services, and activities that are of broad benefit to humanity. This publication briefly summarizes the presentations and discussions from the workshop.

Using Narrative and Data to Communicate the Value of Science: Proceedings of a Workshop—in Brief

In October 2016, the National Academies of Sciences, Engineering, and Medicine held a meeting to consider the Fourth Industrial Revolution and its implications for manufacturing, as well as its likely social and economic effects. The meeting also explored the cross-sector collaboration between government, universities, and industry needed to accommodate emerging developments in the key technologies of the Fourth Industrial Revolution, namely artificial intelligence, virtual and augmented reality, and the Internet of Things. This publication briefly summarizes the presentations and discussions from the meeting.

The Fourth Industrial Revolution: Proceedings of a Workshop–in Brief

On November 16, 2016, the Forum on Synthetic Biology of the National Academies of Sciences, Engineering, and Medicine's Committee on Science, Technology, and Law hosted a workshop titled Making the Living World Engineerable: Science, Practice, and Policy. The workshop was organized by an ad hoc planning committee to discuss current trends in synthetic biology, including international scientific and technical developments in synthetic biology; possible risks and benefits related to these developments; and legal, regulatory, and policy concerns. This Proceedings of a Workshop-in Brief highlights the presentation of the event.

Making the Living World Engineerable: Science, Practice, and Policy: Proceedings of a Workshop—in Brief

The Small Business Innovation Research (SBIR) program is one of the largest examples of U.S. public-private partnerships, and was established in 1982 to encourage small businesses to develop new processes and products and to provide quality research in support of the U.S. government’s many missions. The Small Business Technology Transfer (STTR) Program was created in 1992 by the Small Business Research and Development Enhancement Act to expand joint venture opportunities for small businesses and nonprofit research institutions by requiring small business recipients to collaborate formally with a research institution. The U.S. Congress tasked the National Research Council with undertaking a comprehensive study of how the SBIR and STTR programs have stimulated technological innovation and used small businesses to meet federal research and development needs, and with recommending further improvements to the programs. In the first round of this study, an ad hoc committee prepared a series of reports from 2004 to 2009 on the SBIR and STTR programs at the five agencies responsible for 96 percent of the programs’ operations -- including the Department of Energy (DoE). Building on the outcomes from the first round, this second round presents the committee’s second review of the DoE SBIR program’s operations.

Public-private partnerships like SBIR and STTR are particularly important since today's knowledge economy is driven in large part by the nation's capacity to innovate. One of the defining features of the U.S. economy is a high level of entrepreneurial activity. Entrepreneurs in the United States see opportunities and are willing and able to assume risk to bring new welfare-enhancing, wealth-generating technologies to the market. Yet, although discoveries in areas such as genomics, bioinformatics, and nanotechnology present new opportunities, converting these discoveries into innovations for the market involves substantial challenges. The American capacity for innovation can be strengthened by addressing the challenges faced by entrepreneurs.

SBIR/STTR at the Department of Energy

Over half the world’s population currently lives in urban areas, and the United Nations has projected that by 2050 that number will rise to 70 percent. Given the rapid urbanization trend happening around the globe, coupled with shifting demographics and disruptive technological change, many countries have started planning the development of smart cities and communities: urban centers that use intelligent, connected devices and automated systems that maximize the allocation of resources and the efficiency of services.

On June 21-22, 2016, the National Academies of Sciences, Engineering, and Medicine held a meeting to explore the role of connectedness and sustainability in developing smart communities, the challenges and opportunities associated with the roll-out of intelligent systems, and the partnerships among governments, universities, and industry that are integral to these advances. This publication summarizes the presentations and discussions from the meeting.

Building Smart Communities for the Future: Proceedings of a Workshop–in Brief

Recognizing the importance of eyewitness identifications in courts of law and motivated by data showing that at least one erroneous eyewitness identification was associated with almost 75% of cases where defendants were later exonerated by DNA evidence, in 2013 the Laura and John Arnold Foundation asked the National Academy of Sciences to undertake an assessment of the scientific research on eyewitness identification and offer recommendations to improve eyewitness performance. The appointed committee issued its report, Identifying the Culprit: Assessing Eyewitness Identification , in 2014.

In order to stimulate new and innovative research on statistical tools and the interrelationships between system and estimator variables, the Arnold Foundation in 2015 again called upon the National Academies. This report describes the development of the request for proposals, the processes followed by the committee as it evaluated the proposals, and the committee’s assessment of the scientific merit and research design of the proposals.

Review of Proposals for Research on Statistical Methodologies for Assessing Variables in Eyewitness Performance

Electricity, supplied reliably and affordably, is foundational to the U.S. economy and is utterly indispensable to modern society. However, emissions resulting from many forms of electricity generation create environmental risks that could have significant negative economic, security, and human health consequences. Large-scale installation of cleaner power generation has been generally hampered because greener technologies are more expensive than the technologies that currently produce most of our power. Rather than trade affordability and reliability for low emissions, is there a way to balance all three?

The Power of Change: Innovation for Development and Deployment of Increasingly Clean Energy Technologies considers how to speed up innovations that would dramatically improve the performance and lower the cost of currently available technologies while also developing new advanced cleaner energy technologies. According to this report, there is an opportunity for the United States to continue to lead in the pursuit of increasingly clean, more efficient electricity through innovation in advanced technologies. The Power of Change: Innovation for Development and Deployment of Increasingly Clean Energy Technologies makes the case that America's advantages—world-class universities and national laboratories, a vibrant private sector, and innovative states, cities, and regions that are free to experiment with a variety of public policy approaches—position the United States to create and lead a new clean energy revolution. This study focuses on five paths to accelerate the market adoption of increasing clean energy and efficiency technologies: (1) expanding the portfolio of cleaner energy technology options; (2) leveraging the advantages of energy efficiency; (3) facilitating the development of increasing clean technologies, including renewables, nuclear, and cleaner fossil; (4) improving the existing technologies, systems, and infrastructure; and (5) leveling the playing field for cleaner energy technologies.

The Power of Change: Innovation for Development and Deployment of Increasingly Clean Energy Technologies is a call for leadership to transform the United States energy sector in order to both mitigate the risks of greenhouse gas and other pollutants and to spur future economic growth. This study's focus on science, technology, and economic policy makes it a valuable resource to guide support that produces innovation to meet energy challenges now and for the future.

The Power of Change: Innovation for Development and Deployment of Increasingly Clean Electric Power Technologies

Research universities are critical contributors to our national research enterprise. They are the principal source of a world-class labor force and fundamental discoveries that enhance our lives and the lives of others around the world. These institutions help to create an educated citizenry capable of making informed and crucial choices as participants in a democratic society. However many are concerned that the unintended cumulative effect of federal regulations undercuts the productivity of the research enterprise and diminishes the return on the federal investment in research.

Optimizing the Nation's Investment in Academic Research reviews the regulatory framework as it currently exists, considers specific regulations that have placed undue and often unanticipated burdens on the research enterprise, and reassesses the process by which these regulations are created, reviewed, and retired. This review is critical to strengthen the partnership between the federal government and research institutions, to maximize the creation of new knowledge and products, to provide for the effective training and education of the next generation of scholars and workers, and to optimize the return on the federal investment in research for the benefit of the American people.

Optimizing the Nation's Investment in Academic Research: A New Regulatory Framework for the 21st Century

Students, parents, and government agencies need as much information as possible about the outcomes of the higher education experience and the extent to which they can expect a fair return on their investment in higher education.In order to better understand the concept of quality - enabling students to acquire knowledge in a variety of disciplines and deep knowledge in at least one discipline, as well as to develop a range of skills and habits of mind that prepare them for career success, engaged citizenship, intercultural competence, social responsibility, and continued intellectual growth - an ad hoc planning committee of the National Academies of Sciences, Engineering, and Medicine Board on Higher Education and Workforce, with funding from the Lumina Foundation, organized a workshop in Washington, D.C., on December 14-15, 2015.This report summarizes the presentations and discussion of that event.

Quality in the Undergraduate Experience: What Is It? How Is It Measured? Who Decides? Summary of a Workshop

The Small Business Innovation Research (SBIR) program is one of the largest examples of U.S. public-private partnerships, and was established in 1982 to encourage small businesses to develop new processes and products and to provide quality research in support of the U.S. government’s many missions. The U.S. Congress tasked the National Research Council with undertaking a comprehensive study of how the SBIR program has stimulated technological innovation and used small businesses to meet federal research and development needs, and with recommending further improvements to the program. In the first round of this study, an ad hoc committee prepared a series of reports from 2004 to 2009 on the SBIR program at the five agencies responsible for 96 percent of the program’s operations -- including NASA. In a follow-up to the first round, NASA requested from the Academies an assessment focused on operational questions in order to identify further improvements to the program.

Public-private partnerships like SBIR are particularly important since today's knowledge economy is driven in large part by the nation's capacity to innovate. One of the defining features of the U.S. economy is a high level of entrepreneurial activity. Entrepreneurs in the United States see opportunities and are willing and able to assume risk to bring new welfare-enhancing, wealth-generating technologies to the market. Yet, although discoveries in various fields present new opportunities, converting these discoveries into innovations for the market involves substantial challenges. The American capacity for innovation can be strengthened by addressing the challenges faced by entrepreneurs.

SBIR at NASA

The future competitiveness of the United States in an increasingly interconnected global economy depends on the nation fostering a workforce with strong capabilities and skills in science, technology, engineering, and mathematics (STEM). STEM knowledge and skills enable both individual opportunity and national competitiveness, and the nation needs to develop ways of ensuring access to high-quality education and training experiences for all students at all levels and for all workers at all career stages.

The National Science Foundation (NSF) holds a primary responsibility for overseeing the federal government’s efforts to foster the creation of a STEM-capable workforce. As part of its efforts in this endeavor, NSF’s Directorate on Education and Human Resources asked the National Academies of Sciences, Engineering, and Medicine to convene a workshop that would contribute to NSF’s preparation of a theoretical and evidence-based STEM Workforce Development R&D Core Framework. Participants discussed research themes, identified gaps and emerging research opportunities, and recommended refinements in the goals of the framework. This report summarizes the presentations and discussions from the workshop.

Developing a National STEM Workforce Strategy: A Workshop Summary

U.S. strength in science, technology, engineering, and mathematics (STEM) disciplines has formed the basis of innovations, technologies, and industries that have spurred the nation's economic growth throughout the last 150 years. Universities are essential to the creation and transfer of new knowledge that drives innovation. This knowledge moves out of the university and into broader society in several ways – through highly skilled graduates (i.e. human capital); academic publications; and the creation of new products, industries, and companies via the commercialization of scientific breakthroughs. Despite this, our understanding of how universities receive, interpret, and respond to industry signaling demands for STEM-trained workers is far from complete.

Promising Practices for Strengthening the Regional STEM Workforce Development Ecosystem reviews the extent to which universities and employers in five metropolitan communities (Phoenix, Arizona; Cleveland, Ohio; Montgomery, Alabama; Los Angeles, California; and Fargo, North Dakota) collaborate successfully to align curricula, labs, and other undergraduate educational experiences with current and prospective regional STEM workforce needs. This report focuses on how to create the kind of university-industry collaboration that promotes higher quality college and university course offerings, lab activities, applied learning experiences, work-based learning programs, and other activities that enable students to acquire knowledge, skills, and attributes they need to be successful in the STEM workforce. The recommendations and findings presented will be most relevant to educators, policy makers, and industry leaders.

Promising Practices for Strengthening the Regional STEM Workforce Development Ecosystem

New biochemical tools have made it possible to change the DNA sequences of living organisms with unprecedented ease and precision. These new tools have generated great excitement in the scientific and medical communities because of their potential to advance biological understanding, alter the genomes of microbes, plants, and animals, and treat human diseases. They also have raised profound questions about how people may choose to alter not only their own DNA but the genomes of future generations.

To explore the many questions surrounding the use of gene editing tools in humans, the U.S. National Academy of Sciences, the U.S. National Academy of Medicine, the Royal Society, and the Chinese Academy of Sciences hosted an international summit in December 2015 to present and deliberate on the scientific, ethical, legal, social, and governance issues associated with human gene editing.

International Summit on Human Gene Editing: A Global Discussion

Today's knowledge economy is driven in large part by the nation's capacity to innovate. One of the defining features of the U.S. economy is a high level of entrepreneurial activity. Entrepreneurs in the United States see opportunities and are willing and able to assume risk to bring new welfare-enhancing, wealth-generating technologies to the market. Yet, although discoveries in areas such as genomics, bioinformatics, and nanotechnology present new opportunities, converting these discoveries into innovations for the market involves substantial challenges. The American capacity for innovation can be strengthened by addressing the challenges faced by entrepreneurs. Public-private partnerships are one means to help entrepreneurs bring new ideas to market.

The Small Business Technology Transfer (STTR) and the Small Business Innovation Research (SBIR) program form one of the largest examples of U.S. public-private partnerships. In the SBIR Reauthorization Act of 2000, Congress tasked the National Research Council with undertaking a comprehensive study of how the SBIR program has stimulated technological innovation and used small businesses to meet federal research and development needs and with recommending further improvements to the program. When reauthorizing the SBIR and STTR programs in 2011, Congress expanded the study mandate to include a review of the STTR program. This report builds on the methodology and outcomes from the previous review of SBIR and assesses the STTR program.

STTR: An Assessment of the Small Business Technology Transfer Program

The market for high-skilled workers is becoming increasingly global, as are the markets for knowledge and ideas. While high-skilled immigrants in the United States represent a much smaller proportion of the workforce than they do in countries such as Australia, Canada, and the United Kingdom, these immigrants have an important role in spurring innovation and economic growth in all countries and filling shortages in the domestic labor supply.

This report summarizes the proceedings of a Fall 2014 workshop that focused on how immigration policy can be used to attract and retain foreign talent. Participants compared policies on encouraging migration and retention of skilled workers, attracting qualified foreign students and retaining them post-graduation, and input by states or provinces in immigration policies to add flexibility in countries with regional employment differences, among other topics. They also discussed how immigration policies have changed over time in response to undesired labor market outcomes and whether there was sufficient data to measure those outcomes.

Immigration Policy and the Search for Skilled Workers: Summary of a Workshop

The Small Business Innovation Research (SBIR) and Small Business Technology Transfer (STTR) programs provide federal research and development funding to small businesses. In 2008, the National Research Council completed a comprehensive assessment of the SBIR and STTR programs. The first-round study found that the programs were "sound in concept and effective in practice." Building on the outcomes from the Phase I study, this second phase examines both topics of general policy interest that emerged during the first phase and topics of specific interest to individual agencies, and provides a second snapshot to measure the program's progress against its legislative goals.

SBIR/STTR at the National Institutes of Health

The Small Business Innovation Research (SBIR) program is one of the largest examples of U.S. public-private partnerships, and was established in 1982 to encourage small businesses to develop new processes and products and to provide quality research in support of the U.S. government’s many missions. The U.S. Congress tasked the National Research Council with undertaking a comprehensive study of how the SBIR program has stimulated technological innovation and used small businesses to meet federal research and development needs, and with recommending further improvements to the program. In the first round of this study, an ad hoc committee prepared a series of reports from 2004 to 2009 on the SBIR program at the five agencies responsible for 96 percent of the program’s operations -- including the National Science Foundation (NSF). Building on the outcomes from the first round, this second round presents the committee’s second review of the NSF SBIR program’s operations.

Public-private partnerships like SBIR are particularly important since today's knowledge economy is driven in large part by the nation's capacity to innovate. One of the defining features of the U.S. economy is a high level of entrepreneurial activity. Entrepreneurs in the United States see opportunities and are willing and able to assume risk to bring new welfare-enhancing, wealth-generating technologies to the market. Yet, although discoveries in areas such as genomics, bioinformatics, and nanotechnology present new opportunities, converting these discoveries into innovations for the market involves substantial challenges. The American capacity for innovation can be strengthened by addressing the challenges faced by entrepreneurs.

SBIR at the National Science Foundation

The Small Business Innovation Research (SBIR) and Small Business Technology Transfer (STTR) programs provide federal research and development funding to small businesses. One of the the goals of these programs is to foster and encourage participation by minority and disadvantaged persons in technological innovation. Innovation, Diversity, and Success in the SBIR/STTR Programs is the summary of a workshop convened in February 2013 that focused on the participation of women, minorities, and both older and younger scientists, engineers, and entrepreneurs in the SBIR and STTR programs, with the goal of reviewing current efforts to expand the pool of SBIR/STTR-funded researchers and of identifying mechanisms for improving participation rates. This report is a record of the presentation and discussions of the event.

Innovation, Diversity, and the SBIR/STTR Programs: Summary of a Workshop

Identifying the Culprit: Assessing Eyewitness Identification makes the case that better data collection and research on eyewitness identification, new law enforcement training protocols, standardized procedures for administering line-ups, and improvements in the handling of eyewitness identification in court can increase the chances that accurate identifications are made. This report explains the science that has emerged during the past 30 years on eyewitness identifications and identifies best practices in eyewitness procedures for the law enforcement community and in the presentation of eyewitness evidence in the courtroom. In order to continue the advancement of eyewitness identification research, the report recommends a focused research agenda.

Identifying the Culprit: Assessing Eyewitness Identification

The Postdoctoral Experience Revisited builds on the 2000 report Enhancing the Postdoctoral Experience for Scientists and Engineers . That ground-breaking report assessed the postdoctoral experience and provided principles, action points, and recommendations to enhance that experience. Since the publication of the 2000 report, the postdoctoral landscape has changed considerably. The percentage of PhDs who pursue postdoctoral training is growing steadily and spreading from the biomedical and physical sciences to engineering and the social sciences. The average length of time spent in postdoctoral positions seems to be increasing. The Postdoctoral Experience Revisited reexamines postdoctoral programs in the United States, focusing on how postdocs are being guided and managed, how institutional practices have changed, and what happens to postdocs after they complete their programs. This book explores important changes that have occurred in postdoctoral practices and the research ecosystem and assesses how well current practices meet the needs of these fledgling scientists and engineers and of the research enterprise.

The Postdoctoral Experience Revisited takes a fresh look at current postdoctoral fellows - how many there are, where they are working, in what fields, and for how many years. This book makes recommendations to improve aspects of programs - postdoctoral period of service, title and role, career development, compensation and benefits, and mentoring. Current data on demographics, career aspirations, and career outcomes for postdocs are limited. This report makes the case for better data collection by research institution and data sharing.

A larger goal of this study is not only to propose ways to make the postdoctoral system better for the postdoctoral researchers themselves but also to better understand the role that postdoctoral training plays in the research enterprise. It is also to ask whether there are alternative ways to satisfy some of the research and career development needs of postdoctoral researchers that are now being met with several years of advanced training. Postdoctoral researchers are the future of the research enterprise. The discussion and recommendations of The Postdoctoral Experience Revisited will stimulate action toward clarifying the role of postdoctoral researchers and improving their status and experience.

The Postdoctoral Experience Revisited

Created in 1982 through the Small Business Innovation Development Act, the Small Business Innovation Research (SBIR) program remains the nation's single largest innovation program for small business. The SBIR program offers competitive awards to support the development and commercialization of innovative technologies by small private-sector businesses. At the same time, the program provides government agencies with technical and scientific solutions that address their different missions.

SBIR at the Department of Defense considers ways that the Department of Defense SBIR program could work better in addressing the congressional objectives for the SBIR program to stimulate technological innovation, use small businesses to meet federal research and development (R & D) needs, foster and encourage the participation of socially and economically disadvantaged small businesses, and increase the private sector commercialization of innovations derived from federal R&D. An earlier report, An Assessment of the Small Business Innovation Research Program at the Department of Defense , studied how the SBIR program has stimulated technological innovation and used small businesses to meet federal research and development needs. This report builds on the previous one, with a revised survey of SBIR companies. SBIR at the Department of Defense revisits some case studies from the 2009 study and develops new ones, and interviews agency managers and other stakeholders to provide a second snapshot of the program's progress toward achieving its legislative goals.

SBIR at the Department of Defense

In an increasingly interconnected world, science and technology research often transects international boundaries and involves researchers from multiple nations. This paradigm provides both new opportunities and new challenges. As science and technology capabilities grow around the world, United States-based organizations are finding that international collaborations and partnerships provide unique opportunities to enhance research and training. At the same time, enhancing international collaboration requires recognition of differences in culture, legitimate national security needs, and critical needs in education and training.

Culture Matters: International Research Collaboration in a Changing World: Summary of a Workshop

Research Universities and the Future of America presents critically important strategies for ensuring that our nation's research universities contribute strongly to America's prosperity, security, and national goals. Widely considered the best in the world, our nation's research universities today confront significant financial pressures, important advances in technology, a changing demographic landscape, and increased international competition. This report provides a course of action for ensuring our universities continue to produce the knowledge, ideas, and talent the United States needs to be a global leader in the 21st century.

Research Universities and the Future of America focuses on strengthening and expanding the partnership among universities and government, business, and philanthropy that has been central to American prosperity and security. The report focuses on the top 10 actions that Congress, the federal government, state governments, research universities, and others could take to strengthen the research and education missions of our research universities, their relationships with other parts of the national research enterprise, and their ability to transfer new knowledge and ideas to those who productively use them in our society and economy.

This report examines trends in university finance, prospects for improving university operations, opportunities for deploying technology, and improvement in the regulation of higher education institutions. It also explores ways to improve pathways to graduate education, take advantage of opportunities to increase student diversity, and realign doctoral education for the careers new doctorates will follow. Research Universities and the Future of America is an important resource for policy makers on the federal and state levels, university administrators, philanthropic organizations, faculty, technology transfer specialists, libraries, and researchers.

Research Universities and the Future of America: Ten Breakthrough Actions Vital to Our Nation's Prosperity and Security

The globalization of science, engineering, and medical research is proceeding rapidly. The globalization of research has important implications for the U.S. research enterprise, for the U.S. government agencies, academic institutions, and companies that support and perform research, and for the world at large. As science and technology capabilities grow around the world, U.S.-based organizations are finding that international collaborations and partnerships provide unique opportunities to enhance research and training. At the same time, significant obstacles exist to smooth collaboration across national borders. Enhancing international collaboration requires recognition of differences in culture, legitimate national security needs, and critical needs in education and training.

In response to these trends, the Government-University-Industry Research Roundtable (GUIRR) launched a Working Group on International Research Collaborations (I-Group) in 2008, following its meeting on New Partnerships on a Global Platform that June. As part of I-Group's continuing effort, a workshop on Examining Core Elements of International Research Collaboration was held July 26-27, 2010 in Washington, DC. One primary goal of the workshop is to better understand the risks involved in international research collaboration for organizations and individual participants, and the mechanisms that can be used to manage those risks. Issues to be addressed in the workshop include the following: (1) Cultural Differences and Nuances; (2) Legal Issues and Agreements; (3) Differences in Ethical Standards; (4) Research Integrity and the Responsible Conduct of Research; (5) Intellectual Property; (6) Risk Management; (7) Export Controls; and (8) Strategies for Developing Meaningful International Collaborations.

The goal for the workshop and the summary, Examining Core Elements of International Research Collaboration, is to serve as an information resource for participants and others interested in international research collaborations. It will also aid I-Group in setting its future goals and priorities.

Examining Core Elements of International Research Collaboration: Summary of a Workshop

The Reference Manual on Scientific Evidence, Third Edition , assists judges in managing cases involving complex scientific and technical evidence by describing the basic tenets of key scientific fields from which legal evidence is typically derived and by providing examples of cases in which that evidence has been used.

First published in 1994 by the Federal Judicial Center, the Reference Manual on Scientific Evidence has been relied upon in the legal and academic communities and is often cited by various courts and others. Judges faced with disputes over the admissibility of scientific and technical evidence refer to the manual to help them better understand and evaluate the relevance, reliability and usefulness of the evidence being proffered. The manual is not intended to tell judges what is good science and what is not. Instead, it serves to help judges identify issues on which experts are likely to differ and to guide the inquiry of the court in seeking an informed resolution of the conflict.

The core of the manual consists of a series of chapters (reference guides) on various scientific topics, each authored by an expert in that field. The topics have been chosen by an oversight committee because of their complexity and frequency in litigation. Each chapter is intended to provide a general overview of the topic in lay terms, identifying issues that will be useful to judges and others in the legal profession. They are written for a non-technical audience and are not intended as exhaustive presentations of the topic. Rather, the chapters seek to provide judges with the basic information in an area of science, to allow them to have an informed conversation with the experts and attorneys.

Reference Manual on Scientific Evidence: Third Edition

A Data-Based Assessment of Research-Doctorate Programs in the United States provides an unparalleled dataset that can be used to assess the quality and effectiveness of doctoral programs based on measures important to faculty, students, administrators, funders, and other stakeholders. The data, collected for the 2005-2006 academic year from more than 5,000 doctoral programs at 212 universities, covers 62 fields. Included for each program are such characteristics as faculty publications, grants, and awards; student GRE scores, financial support, and employment outcomes; and program size, time to degree, and faculty composition. Measures of faculty and student diversity are also included. The book features analysis of selected findings across six broad fields: agricultural sciences, biological and health sciences, engineering, physical and mathematical sciences, social and behavioral sciences, and humanities, as well as a discussion of trends in doctoral education since the last assessment in 1995, and suggested uses of the data . It also includes a detailed explanation of the methodology used to collect data and calculate ranges of illustrative rankings. Included with the book is a comprehensive CD-ROM with a data table in Microsoft Excel. In addition to data on the characteristics of individual programs, the data table contains illustrative ranges of rankings for each program, as well as ranges of rankings for three dimensions of program quality: (1) research activity, (2) student support and outcomes, and (3) diversity of the academic environment. As an aid to users, the data table is offered with demonstrations of some Microsoft Excel features that may enhance the usability of the spreadsheet, such as hiding and unhiding columns, copying and pasting columns to a new worksheet, and filtering and sorting data. Also provided with the data table are a set of scenarios that show how typical users may want to extract data from the spreadsheet. PhDs.org, an independent website not affiliated with the National Research Council, incorporated data from the research-doctorate assessment into its Graduate School Guide. Users of the Guide can choose the weights assigned to the program characteristics measured by the National Research Council and others, and rank graduate programs according to their own priorities.

A Data-Based Assessment of Research-Doctorate Programs in the United States (with CD)

In order for the United States to maintain the global leadership and competitiveness in science and technology that are critical to achieving national goals, we must invest in research, encourage innovation, and grow a strong and talented science and technology workforce. Expanding Underrepresented Minority Participation explores the role of diversity in the science, technology, engineering and mathematics (STEM) workforce and its value in keeping America innovative and competitive. According to the book, the U.S. labor market is projected to grow faster in science and engineering than in any other sector in the coming years, making minority participation in STEM education at all levels a national priority.

Expanding Underrepresented Minority Participation analyzes the rate of change and the challenges the nation currently faces in developing a strong and diverse workforce. Although minorities are the fastest growing segment of the population, they are underrepresented in the fields of science and engineering. Historically, there has been a strong connection between increasing educational attainment in the United States and the growth in and global leadership of the economy. Expanding Underrepresented Minority Participation suggests that the federal government, industry, and post-secondary institutions work collaboratively with K-12 schools and school systems to increase minority access to and demand for post-secondary STEM education and technical training.

The book also identifies best practices and offers a comprehensive road map for increasing involvement of underrepresented minorities and improving the quality of their education. It offers recommendations that focus on academic and social support, institutional roles, teacher preparation, affordability and program development.

Expanding Underrepresented Minority Participation: America's Science and Technology Talent at the Crossroads

Today, scores of companies, primarily in the United States and Europe, are offering whole genome scanning services directly to the public. The proliferation of these companies and the services they offer demonstrate a public appetite for this information and where the future of genetics may be headed; they also demonstrate the need for serious discussion about the regulatory environment, patient privacy, and other policy implications of direct-to-consumer (DTC) genetic testing. Rapid advances in genetic research already have begun to transform clinical practice and our understanding of disease progression. Existing research has revealed a genetic basis or component for numerous diseases, including Parkinson's disease, Alzheimer's disease, diabetes, heart disease, and several forms of cancer. The availability of the human genome sequence and the HapMap, plummeting costs of high-throughput screening, and increasingly sophisticated computational analyses have led to an explosion of discoveries of linkages between patterns of genetic variation and disease susceptibility. While this research is by no means a straight path toward better public health, improved knowledge of the genetic linkages has the potential to change fundamentally the way health professionals and public health practitioners approach the prevention and treatment of disease. Realizing this potential will require greater sophistication in the interpretation of genetic tests, new training for physicians and other diagnosticians, and new approaches to communicating findings to the public. As this rapidly growing field matures, all of these questions require attention from a variety of perspectives. To discuss some of the foregoing issues, several units of the National Academies held a workshop on August 31 and September 1, 2009, to bring together a still-developing community of professionals from a variety of relevant disciplines, to educate the public and policy-makers about this emerging field, and to identify issues for future study. The meeting featured several invited presentations and discussions on the many technical, legal, policy, and ethical questions that such DTC testing raises, including: (1) overview of the current state of knowledge and the future research trajectory; (2) shared genes and emerging issues in privacy; (3) the regulatory framework; and (4) education of the public and the medical community.

Direct-to-Consumer Genetic Testing: Summary of a Workshop

Gender Differences at Critical Transitions in the Careers of Science, Engineering, and Mathematics Faculty presents new and surprising findings about career differences between female and male full-time, tenure-track, and tenured faculty in science, engineering, and mathematics at the nation's top research universities. Much of this congressionally mandated book is based on two unique surveys of faculty and departments at major U.S. research universities in six fields: biology, chemistry, civil engineering, electrical engineering, mathematics, and physics. A departmental survey collected information on departmental policies, recent tenure and promotion cases, and recent hires in almost 500 departments. A faculty survey gathered information from a stratified, random sample of about 1,800 faculty on demographic characteristics, employment experiences, the allocation of institutional resources such as laboratory space, professional activities, and scholarly productivity.

This book paints a timely picture of the status of female faculty at top universities, clarifies whether male and female faculty have similar opportunities to advance and succeed in academia, challenges some commonly held views, and poses several questions still in need of answers. This book will be of special interest to university administrators and faculty, graduate students, policy makers, professional and academic societies, federal funding agencies, and others concerned with the vitality of the U.S. research base and economy.

Gender Differences at Critical Transitions in the Careers of Science, Engineering, and Mathematics Faculty

A Guide to the Methodology of the National Research Council Assessment of the Doctorate Programs describes the purpose, data and methods used to calculate ranges or rankings for research-doctorate programs that participated in the NRC Assessment of Research-Doctorate programs. It is intended for those at universities who will have to explain the NRC Assessment to others at their university, to potential students, and to the press. Although the main text is fairly non-technical, it includes a technical description of the statistical methods used to derive rankings of over 5000 doctoral programs in 61 fields.

A Guide to the Methodology of the National Research Council Assessment of Doctorate Programs

Scores of talented and dedicated people serve the forensic science community, performing vitally important work. However, they are often constrained by lack of adequate resources, sound policies, and national support. It is clear that change and advancements, both systematic and scientific, are needed in a number of forensic science disciplines to ensure the reliability of work, establish enforceable standards, and promote best practices with consistent application. Strengthening Forensic Science in the United States: A Path Forward provides a detailed plan for addressing these needs and suggests the creation of a new government entity, the National Institute of Forensic Science, to establish and enforce standards within the forensic science community.

The benefits of improving and regulating the forensic science disciplines are clear: assisting law enforcement officials, enhancing homeland security, and reducing the risk of wrongful conviction and exoneration. Strengthening Forensic Science in the United States gives a full account of what is needed to advance the forensic science disciplines, including upgrading of systems and organizational structures, better training, widespread adoption of uniform and enforceable best practices, and mandatory certification and accreditation programs.

While this book provides an essential call-to-action for congress and policy makers, it also serves as a vital tool for law enforcement agencies, criminal prosecutors and attorneys, and forensic science educators.

Strengthening Forensic Science in the United States: A Path Forward

The scientific research enterprise is built on a foundation of trust. Scientists trust that the results reported by others are valid. Society trusts that the results of research reflect an honest attempt by scientists to describe the world accurately and without bias. But this trust will endure only if the scientific community devotes itself to exemplifying and transmitting the values associated with ethical scientific conduct. On Being a Scientist was designed to supplement the informal lessons in ethics provided by research supervisors and mentors. The book describes the ethical foundations of scientific practices and some of the personal and professional issues that researchers encounter in their work. It applies to all forms of research—whether in academic, industrial, or governmental settings-and to all scientific disciplines. This third edition of On Being a Scientist reflects developments since the publication of the original edition in 1989 and a second edition in 1995. A continuing feature of this edition is the inclusion of a number of hypothetical scenarios offering guidance in thinking about and discussing these scenarios. On Being a Scientist is aimed primarily at graduate students and beginning researchers, but its lessons apply to all scientists at all stages of their scientific careers.

On Being a Scientist: A Guide to Responsible Conduct in Research: Third Edition

Rising Above the Gathering Storm: Energizing and Employing America for a Brighter Economic Future

The U.S. patent system is in an accelerating race with human ingenuity and investments in innovation. In many respects the system has responded with admirable flexibility, but the strain of continual technological change and the greater importance ascribed to patents in a knowledge economy are exposing weaknesses including questionable patent quality, rising transaction costs, impediments to the dissemination of information through patents, and international inconsistencies. A panel including a mix of legal expertise, economists, technologists, and university and corporate officials recommends significant changes in the way the patent system operates.

A Patent System for the 21st Century urges creation of a mechanism for post-grant challenges to newly issued patents, reinvigoration of the non-obviousness standard to quality for a patent, strengthening of the U.S. Patent and Trademark Office, simplified and less costly litigation, harmonization of the U.S., European, and Japanese examination process, and protection of some research from patent infringement liability.

A Patent System for the 21st Century

Assessing Research-Doctorate Programs: A Methodology Study

Doctoral programs at U.S. universities play a critical role in the development of human resources both in the United States and abroad. This volume reports the results of an extensive study of U.S. research-doctorate programs in five broad fields: physical sciences and mathematics, engineering, social and behavioral sciences, biological sciences, and the humanities.

Research-Doctorate Programs in the United States documents changes that have taken place in the size, structure, and quality of doctoral education since the widely used 1982 editions. This update provides selected information on nearly 4,000 doctoral programs in 41 subdisciplines at 274 doctorate-granting institutions.

This volume also reports the results of the National Survey of Graduate Faculty, which polled a sample of faculty for their views on the scholarly quality of program faculty and the effectiveness of doctoral programs in preparing research scholars/scientists.

This much-anticipated update of such an essential reference will be useful to education administrators, university faculty, and students seeking authoritative information on doctoral programs.

Research Doctorate Programs in the United States: Continuity and Change

Ten Types of Innovation: 30 new case studies for 2019

Ten Types of Innovation 30 new examples for 2019

If you’ve followed my work for a while, you’ll know that I’m a big fan of the Ten Types of Innovation, a framework developed by Doblin (now a part of Deloitte).

I previously listed it as the #2 innovation framework you should be using.

And with good reason. I have used it frequently with clients to get them to think beyond innovating their product , which becomes harder, more expensive and less differentiating over time.

However, what I have found in recent workshops is that since it was originally published in 2013, some of the case studies and examples in the book already come across as out of date. That’s how rapidly the world is changing.

So here, I present three new more recent case studies for each of the Ten Types of Innovation, along with an outline on what each of them represents. Try and see which of these examples you would also suggest touch on more than one of the Ten Types, and let me know in the comments below:

1) Profit Model: How you make money

Innovative profit models find a fresh way to convert a firm’s offerings and other sources of value into cash. Great ones reflect a deep understanding of what customers and users actually cherish and where new revenue or pricing opportunities might lie.

Innovative profit models often challenge an industry’s tired old assumptions about what to offer, what to charge, or how to collect revenues. This is a big part of their power: in most industries, the dominant profit model often goes unquestioned for decades.

Recent examples:

Fortnite – Pay to customise: This Free-to-Play video game by Epic Game Studios is currently one of the most popular and profitable games in the world. Unlike other “freemium” games which incentivise people to spend money to speed up progression, Fortnite is completely free to progress and people only need pay if they want to unlock cosmetic items which don’t affect gameplay but act to personalise their characters.
Deloitte – Value sharing: Professional Services firm Deloitte is the world’s largest Management Consulting firm and still growing. They noticed a desire from their clients for assurance that the advice they were being given and transformation projects which Deloitte was running would actually succeed. As a result, Deloitte has begun trialling projects where instead of their fee being based just on Time and Materials, they will also share in value delivery, where additional bonus payments are only activated if previously-agreed performance metrics are successfully met.
Supreme – Limiting supply: While most companies want to get their products in to the hands of as many people as possible, Supreme has built a cult following through deliberately forcing scarcity of its products. The streetwear clothing retailer announces limited items which will only be available from a specific day when they “drop”, and once they are sold out, that’s it, unless you want to pay huge markups for a second-hand item on eBay. Their red box logo is now so collectible and desirable that the company is able to sell almost anything by putting the logo on it for a limited time only. Case in point: you can find official Supreme Bricks (yes, like the ones used to build houses) which are still selling on eBay for $500.

Supreme’s limited quantity releases often lead to people queuing overnight

2) Network: How you connect with others to create value

In today’s hyper-connected world, no company can or should do everything alone. Network innovations provide a way for firms to take advantage of other companies’ processes, technologies, offerings, channels, and brands—pretty much any and every component of a business.

These innovations mean a firm can capitalize on its own strengths while harnessing the capabilities and assets of others. Network innovations also help executives to share risk in developing new offers and ventures. These collaborations can be brief or enduring, and they can be formed between close allies or even staunch competitors.

Recent Examples:

Ford & Volkswagen – Developing Self-driving cars: As two of the world’s largest car-makers, Ford and Volkswagen are competitors on the road. However, in 2019 they announced a partnership to work together to develop technology for self-driving cars and electric vehicles which would be used in both company’s fleets of the future. While Ford brings more advanced automated driving technology, Volkswagen was leading in electric vehicles. Through the combined venture called ARGO, both firms can spread their R&D spending across more cars, while both developing competing products.
Microsoft – launching on competitors platforms: Since new Microsoft CEO Satya Nadella has taken over, he has changed the innovation ethos of the company. Whereas previously Microsoft was a product-first company who tried to eliminate competing products and customers should stay within the company’s ecosystem, Nadella has shifted the mindset to a service company where their products should be accessible to customers should be able to access the products in whichever way they prefer. As a result, products such as Office 365 are now available in any web browser, as well as on the mobile marketplaces of Google’s Android and Apple’s IOS, previously seen as competitors.
Huawei – Leveraging celebrity endorsement: Until recently, “high-quality smartphone” made people think of companies like Apple (USA), Samsung and LG (South Korea). Brands from China were often seen as competing on price but suffering from lower build quality and a lack of innovation. So in order to raise their profile in Western markets, Huawei has invested heavily in celebrities to endorse their flagship phones, such as Scarlett Johanssen, Lionel Messi, Henry Cavill and Gal Gadot. This initial investment raised brand name recognition, to the stage where it is now focusing marketing more towards features and functionality.

Huawei has paid Lionel Messi millions to endorse their brand

3) Structure: How you organize and align your talent and assets

Structure innovations are focused on organizing company assets—hard, human, or intangible—in unique ways that create value. They can include everything from superior talent management systems to ingenious configurations of heavy capital equipment.

An enterprise’s fixed costs and corporate functions can also be improved through Structure innovations, including departments such as Human Resources, R&D, and IT. Ideally, such innovations also help attract talent to the organization by creating supremely productive working environments or fostering a level of performance that competitors can’t match.

Perpetual Guardian – Four-day working week: This small financial advisory firm in New Zealand trialed moving to a four-day working week, giving their staff an additional free day each week as long as they got their outputs done. As a result, they found people adjusted their working rhythm to achieve the same outcomes in 20% less time , while also resulting in more satisfied employees.
Netflix – Unlimited Vacations: In order to drive their breakneck growth, Netflix reviewed their formal HR policies to see what processes were getting in the way of people doing their best work. They discovered that most bureaucratic processes which slowed down high performing individuals were in place to only handle situations where a low-performance individual would do something wrong. As a result, they scrapped most formal HR policies to free people to work in their own ways to benefit the company, summarised in their “Freedom and Responsibility” culture document, including allowing staff to take as many vacation days as they felt they needed to produce their best work.
WeWork – Leveraging other companies’ hard assets: WeWork’s business model revolves around providing affordable office rentals for entrepreneurs and companies, fitting a lot of tenants into the same space by offering co-working areas. In order to rapidly deploy new working spaces and attract customers, WeWork started using a system called rental arbitrage, where they would rent commercial space, create a ready-to-use coworking setup, and then rent this space to customers. By not having to spend CAPEX on purchasing the buildings themselves, they were able to rapidly expand with lower overhead.

Netflix allows staff to take unlimited vacation days

4) Process: How you use signature or superior methods to do your work

Process innovations involve the activities and operations that produce an enterprise’s primary offerings. Innovating here requires a dramatic change from “business as usual” that enables the company to use unique capabilities, function efficiently, adapt quickly, and build market–leading margins.

Process innovations often form the core competency of an enterprise, and may include patented or proprietary approaches that yield advantage for years or even decades. Ideally, they are the “special sauce” you use that competitors simply can’t replicate.

Tesla – Vertically integrated supply chain: Tesla’s electric cars require huge packs of EV batteries, made of thousands of lithium-ion cells. Until recently, the lack of demand for electric vehicles meant that companies had not invested in battery technology development, resulting in prices remaining high and making the cost of cars prohibitively more expensive than their gasoline counterparts. Tesla invested in a massive gigafactory to produce the newest battery packs themselves, and the economies of scale, as well as not paying markups to manufacturers, are estimated to save them 30% of the cost of the batteries.
Amazon Web Services – opening internal technology to third parties: When Amazon Web Services initially launched in 2006 , it effectively launched the cloud computing market, allowing external companies to not just host webpages but run code and calculations at a fraction of the cost of building their own server network. Since then, Amazon has continued to develop new technology it would use for its own services, such as artificial intelligence, image recognition, machine learning, and natural-language processing, and later make this technology available to their customers.
AliExpress – Making everyone a Shop Owner: AliExpress is one of the world’s largest eCommerce sites, and serves as a commercial storefront for thousands of Chinese companies, allowing you to purchase everything to phone cases to forklifts. However, AliExpress also allows the platform to handle purchases as listed on external storefronts using a system called drop-shipping, where anyone can set up their own store, sell someone else’s products (but to customers it looks like they are coming from the seller) and then have those manufacturers send the product directly to the customer.

Tesla’s Gigafactory is the world’s largest building

5) Product Performance: How you develop distinguishing features and functionality

Product Performance innovations address the value, features, and quality of a company’s offering. This type of innovation involves both entirely new products as well as updates and line extensions that add substantial value. Too often, people mistake Product Performance for the sum of innovation. It’s certainly important, but it’s always worth remembering that it is only one of the Ten Types of Innovation, and it’s often the easiest for competitors to copy.

Think about any product or feature war you’ve witnessed—whether torque and toughness in trucks, toothbrushes that are easier to hold and use, even with baby strollers. Too quickly, it all devolves into an expensive mad dash to parity. Product Performance innovations that deliver long-term competitive advantage are the exception rather than the rule.

Gorilla Glass – Changing chemistry to improve smartphone durability: Gorilla Glass by Corning was listed as one of the original Ten Types by becoming scratch resistant. I have included it again for how it has changed the properties of its glass based on customer feedback each year. In 2016, version 5 of the glass was designed to resist shattering when dropped from 5+ feet, dubbed “selfie height” drops. However, after discussing what properties their customers wanted, by 2018 version 6 was no longer trying to resist shattering when dropped from a height once, instead the chemistry and manufacturing process had been changed to make it resistant to cracking after 15 drops from a lower height (1 meter, or a “fumble drop from your pocket”). I love this example of innovation as the product performance doesn’t just try to become “ better ” by resisting one drop from a higher height than last year, instead figuring out what really matters to customers and delivering that.
Raspberry Pi – full PC for $35: The original Rasperbby Pi was developed by a UK charity to make a simple yet expandable computer which was affordable enough for everyone. Their credit-card sized PC may look bare-bones (it comes without a case and is effectively an exposed circuit board), yet it contains everything which someone needs to run a Linux operating system, learn to program and even connect it with external sensors and peripherals to make all manner of machines. The latest version 4 is now powerful enough to serve as a dedicated PC, all for a price so low you can give it to a child to tinker with without fear of it being broken.
Lush Cosmetics – Removing what people don’t want anymore: As people become more aware of their impact on the environment, customers are demanding that customers do more to reduce the amount of plastic packaging their products use which could end up in landfill or the ocean. Lush Cosmetics was an early pioneer in bringing packaging-free cosmetics to scale, offering some of their packaging-free products like shampoo bars and soaps in dedicated packaging-free stores .

Giving children a cheap PC like the Raspberry Pi to learn and experiment on

6) Product System: How you create complementary products and services

Product System innovations are rooted in how individual products and services connect or bundle together to create a robust and scalable system. This is fostered through interoperability, modularity, integration, and other ways of creating valuable connections between otherwise distinct and disparate offerings. Product System innovations help you build ecosystems that captivate and delight customers and defend against competitors.

Ryobi – One battery to rule them all: While handheld tools have had rechargeable batteries for decades now, Ryobi’s innovation was designing the modular One+ battery which could be used with over 80 different tools. Not only was this convenient for customers who needed fewer batteries overall for multiple uses, it also encouraged someone to buy into the Ryobi tool ecosystem once they had previously purchased one tool and battery set.
Zapier – making APIs easy: Many web-based applications nowadays have an Application Programming Interface (API) which allows them to share data with other services. However, this often requires complex coding from the developers, and repeated effort to integrate with multiple different APIs. Zapier acts as a middleman for data, providing ready-made actions and API integrations between popular web services, allowing customers to automate certain activities every time a specific event happens.
Airbnb – Expanding into experiences: Airbnb built their business on allowing everyday people to sell accommodation in their homes to strangers. Now the company has begun offering complementary services to people visiting new places through Experiences . These experiences are also sold by local guides, and allow guests to try things they would otherwise not have known about in addition to staying somewhere new.

Ryobi One+ battery powers multiple different tools

7) Service: How you support and amplify the value of your offerings

Service innovations ensure and enhance the utility, performance, and apparent value of an offering. They make a product easier to try, use, and enjoy; they reveal features and functionality customers might otherwise overlook, and they fix problems and smooth rough patches in the customer journey. Done well, they elevate even bland and average products into compelling experiences that customers come back for again and again.

Kroger – Smartphone grocery scanning: US retail giant Kroger has been trialing a new smartphone app which allows shoppers to scan items as they shop, and then skip checking out altogether. Using the Scan, Bag, Go app, a customer will scan each item as they pick them up and place them into whatever bag they want, and once they are done, they can simply pay using the app and leave. This prevents shoppers having to wait in checkout lines and gives them an overview of their running total as they go, and also allows the supermarket to entice shoppers by sending coupons and offers directly to them.
PurpleBricks – bringing real estate online: Estate Agents have a poor reputation for treating both sellers and buyers, especially for the amount they charge relative to the service they provide. PurpleBricks was one of the first online-only estate agents , where they could charge a significantly lower fee if the seller chose to complete some of the service processes themselves, such as showing the home to potential buyers. The firm can provide additional services for additional charges.
Meituan Dianping – providing one app for all the services you want: As Fast Company’s 2019 Most Innovative company , Meituan Dianping provides a platform for Chinese consumers to purchase a variety of services. Known as a transactional super-app, you can use the app to book and pay for food delivery, travel, movie tickets and more from over 5 million Chinese small and large merchants.

Scan your own groceries with the Scan-Bag-Go app

8) Channel: How you deliver your offerings to customers and users

Channel innovations encompass all the ways that you connect your company’s offerings with your customers and users. While e-commerce has emerged as a dominant force in recent years, traditional channels such as physical stores are still important — particularly when it comes to creating immersive experiences.

Skilled innovators in this type often find multiple but complementary ways to bring their products and services to customers. Their goal is to ensure that users can buy what they want, when and how they want it, with minimal friction and cost and maximum delight.

Dollar Shave Club – Direct to your door: Razor Blades have always been high-margin products, and Gillette was one of the original innovators by giving away the razor handle to make money on the subsequent razor blade sales. Dollar Shave Club has taken a different approach, by reducing the cost of each set of blades, but having people join a subscription service where blades are delivered to them automatically. While the margin on each set of blades is lower than retail, the subscription model has provided steady, predictable revenue for the company, to the extend that subscription boxes can now be found for almost any consumable product.
Zipline – Blood Delivery for remote areas: In hospital settings, getting fresh blood can a matter of life and death. Unfortunately, many Sub-Sharan African countries don’t have road infrastructure suitable for quickly delivering blood between hospitals or storage locations. This is why Zipline has developed a simple, reliable drone network where hospitals in Rwanda and Ghana can order fresh blood from a central processing area and receive it within an average of 15 minutes, rather than the hours or days it would take using conventional transportation.
3D Printers – produce whatever you need at home: Instead of a single company, the industry of 3D printers is slowly beginning to change the way in which consumers get simple tools and parts. By downloading schematics from the internet (or designing their own), people owning a 3D printer now no longer to go to a retail location or order the parts they need. In commercial settings, this is also speeding up how quickly companies are able to prototype new ideas and designs, waiting hours rather than days or weeks.

zipline blood drone innovation

9) Brand: How you represent your offerings and business

Brand innovations help to ensure that customers and users recognize, remember, and prefer your offerings to those of competitors or substitutes. Great ones distill a “promise” that attracts buyers and conveys a distinct identity.

They are typically the result of carefully crafted strategies that are implemented across many touchpoints between your company and your customers, including communications, advertising, service interactions, channel environments, and employee and business partner conduct. Brand innovations can transform commodities into prized products, and confer meaning, intent, and value to your offerings and your enterprise.

Gillette / Nike – being willing to lose customers who don’t align with purpose: I have combined both Gillette and Nike into this example of brand innovation since they have both recently aligned their brands to a purpose (social and political), which has been positively welcomed by some people but has resulted in hatred from other groups. Nike began by making former NFL Quarterback Colin Kaepernick the face and voice of one of their advertising campaigns. Kaepernick rose in prominence when he refused to stand during the national anthem before his games, his way of protesting the police brutality and inequality towards his African American community. This led to some people claiming he was disrespecting the American Flag, and therefore what the flag stands for. When his advert launched, a vocal minority took to social media to upload videos of themselves saying that Nike no longer aligned with their values, and they burned their shoes, vowing to never buy Nike again. Similarily, Gillette came out with a commercial urging all men to be “The best a man can be”, by pushing aside previously ‘masculine’ traits like bullying, chauvinism or fighting, and showing children how a modern man should behave. As soon as the ad was released online, many media outlets praised its message, but it brought the wrath of angry men who claimed that the razor manufacturer shouldn’t tell them what to think or how to behave, how they would never buy the products again, and how the world was becoming too politically correct, with women and minorities getting preferential treatment over white men. The advert quickly became one of the most disliked videos on Youtube, and even my commentary about the innovative message (seen in the video below) had the comments section covered by hate-filled messages. What both Nike and Gillette realised was that if they wanted to align with positive, progressive messages and values (which align with their target demographic of the future), then they would risk upsetting and alienating the proportion of their current customer base who didn’t share those views. In both cases, these were decisions that would have been signed off by all levels in the company, through marketing, sales, legal and the board, and the brands will be stronger in the future because of it.
Burberry – modernising a classic brand: Burberry had built its luxury fashion reputation by aligning itself with the British Aristocracy, and its famous chequer patterned fabric was iconic. However, when trying to modernise and make the brand “sexy” in the early 2000s, a misstep happened when the luxury house began to license the chequered fabric, resulting in it becoming a status symbol and desired motif for a different social group: the British “Chavs” (rough, lower class and sometimes aggressive). This poisoned the once iconic brand in the eyes of their intended luxury clientele. In order to survive, the company and brand embraced innovation , by becoming one of the first fashion houses to redesign their website to be mobile-optimised, aligning their store layout to mirror the website, highlighting young British talent and livestreaming content and fashion shows. Most importantly, they moved away from the iconic chequer pattern in their fashion designs, where it is now limited to less than 10% of products.

10) Customer Engagement: How you foster compelling interactions

Customer Engagement innovations are all about understanding the deep-seated aspirations of customers and users, and using those insights to develop meaningful connections between them and your company.

Great Customer Engagement innovations provide broad avenues for exploration and help people find ways to make parts of their lives more memorable, fulfilling, delightful — even magical.

REI – closing their stores on the busiest shopping day: Outdoor equipment retailer REI had begun closing its doors on Black Friday , traditionally one of the busiest shopping days of the year. They claim they are doing this to Eddie their customers to actually get outdoors and use their equipment, rather than queuing for discounted material goods.
Peloton – bringing the gym into the home: Many people benefit from going to joint gym classes because the sense of a group working toward is goals together with a coach is more powerful than trying to exercise by yourself. Peloton makes exercise equipment with built-in screens, powered by a subscription to live and on-demand classes. It’s like being part of a workout group with the benefits of being at home.
NBA – bringing the fans into the action: The NBA had invested heavily in innovation to make their sport more immersive. From live analytics and player statistics, new ways to watch like VR video, and official video game players for each team, they are finding new ways to bring basketball to the next generation, while making it even more exciting for existing fans.

Peloton brings exercise classes into the home

There we go, a new set of 30 examples of the Ten Types of Innovation.

If you found some of these examples interesting, please share the article.

Can you think of any more good examples? Let me know in the comments below.

Did you know that scientific evidence shows your creativity decreases over time

Idea to Value Podcast: Listen and Subscribe now

Latest Posts

Nick Skillicorn

Latest posts by nick skillicorn ( see all ).

When your imagination is beyond your skill level - May 9, 2024
When does it make sense to play a character of yourself? - March 29, 2024
Focus on doing fewer things in order to make progress faster - March 14, 2024
Focus on what you can control - January 23, 2024

Is Design Thinking dead?

We could all benefit from more failures

Don’t experiment to prove. Do it to improve

Endowment effect: why people value what they currently have.

great examples! I now feel inspired to innovate in my entrepreneurial project. Thank you ?

Greetings from Mexico

Excellent work!

They’s very interesting. Do you have the solutions of some of recent examples?

My university has taken pretty much everything from here, poorly rephrased a few things and have delivered it to us, the student, as an entire weeks worth of content. Maybe i should be paying my fees here…

Bachelor of business student Australia

Very interesting. Which course was it being used for?

Toolkit Navigator

Exploring Policy Innovation: Tools, Techniques & Approaches

This resource is an introductory description of a paradigmatic shift in public policy-making from traditional methods to new, innovative approaches. This includes a whole-of-government approach, a refocus on the user, an embrace of complexity, pro-activeness, and collaboration. It describes 7 opportunity areas for innovation in the traditional policy-making process.

Brookfield Institute for Innovation and Entrepreneurship

Discipline or practice

Public Policy

Link to toolkit

https://brookfieldinstitute.ca/exploring-policy-innovation/

Source files

About this resource.

Guiding framework

Country/Territory

Date published.

PDF publication

See cases from others doing this in government

Go to case studies

Find experts and advisers who can assist me with this

Go to advice

Other toolkits related to Public Policy

EAST: Four Simple Ways to Apply Behavioural Insights

Behavioural insights team.

BASIC – the Behavioural Insights Toolkit and Ethical Guidelines for Policy Makers

Policy Methods Toolbox

New zealand government.

Open Policy Making Toolkit

United kingdom government.

Policy Quality Framework

Public Sector Innovation Toolkit

Australia government, view all toolkits related to public policy.

SUGGESTED TOPICS
The Magazine
Newsletters
Managing Yourself
Managing Teams
Work-life Balance
The Big Idea
Data & Visuals
Reading Lists
Case Selections
HBR Learning
Topic Feeds
Account Settings
Email Preferences

HBS Case Selections

OpenAI: Idealism Meets Capitalism

Shikhar Ghosh
Shweta Bagai

Generative AI and the Future of Work

Christopher Stanton
Matt Higgins

Copilot(s): Generative AI at Microsoft and GitHub

Frank Nagle
Shane Greenstein
Maria P. Roche
Nataliya Langburd Wright
Sarah Mehta

Innovation at Moog Inc.

Brian J. Hall
Ashley V. Whillans
Davis Heniford
Dominika Randle
Caroline Witten

Innovation at Google Ads: The Sales Acceleration and Innovation Labs (SAIL) (A)

Linda A. Hill
Emily Tedards

Juan Valdez: Innovation in Caffeination

Michael I. Norton
Jeremy Dann

UGG Steps into the Metaverse

Shunyuan Zhang
Sharon Joseph
Sunil Gupta
Julia Kelley

Metaverse Wars

David B. Yoffie

Roblox: Virtual Commerce in the Metaverse

Ayelet Israeli
Nicole Tempest Keller

Timnit Gebru: "SILENCED No More" on AI Bias and The Harms of Large Language Models

Tsedal Neeley
Stefani Ruper

Hugging Face: Serving AI on a Platform

Kerry Herman
Sarah Gulick

SmartOne: Building an AI Data Business

Karim R. Lakhani
Pippa Tubman Armerding
Gamze Yucaoglu
Fares Khrais

Honeywell and the Great Recession (A)

Sandra J. Sucher
Susan Winterberg

Target: Responding to the Recession

Ranjay Gulati
Catherine Ross
Richard S. Ruback
Royce Yudkoff

Hometown Foods: Changing Price Amid Inflation

Julian De Freitas
Jeremy Yang
Das Narayandas

Elon Musk's Big Bets

Eric Baldwin

Elon Musk: Balancing Purpose and Risk

Tesla's ceo compensation plan.

Krishna G. Palepu
John R. Wells
Gabriel Ellsworth

China Rapid Finance: The Collapse of China's P2P Lending Industry

William C. Kirby
Bonnie Yining Cao
John P. McHugh

Forbidden City: Launching a Craft Beer in China

Christopher A. Bartlett
Carole Carlson

Booking.com

Stefan Thomke
Daniela Beyersdorfer

Innovation at Uber: The Launch of Express POOL

Chiara Farronato
Alan MacCormack

Racial Discrimination on Airbnb (A)

Michael Luca
Scott Stern
Hyunjin Kim

Unilever's Response to the Future of Work

William R. Kerr
Emilie Billaud
Mette Fuglsang Hjortshoej

AT&T, Retraining, and the Workforce of Tomorrow

Joseph B. Fuller
Carl Kreitzberg

Leading Change in Talent at L'Oreal

Lakshmi Ramarajan
Vincent Dessain
Emer Moloney
William W. George
Andrew N. McLean

Eve Hall: The African American Investment Fund in Milwaukee

Steven S. Rogers
Alterrell Mills

United Housing - Otis Gates

Mercer Cook

The Home Depot: Leadership in Crisis Management

Herman B. Leonard
Marc J. Epstein
Melissa Tritter

The Great East Japan Earthquake (B): Fast Retailing Group's Response

Hirotaka Takeuchi
Kenichi Nonomura
Dena Neuenschwander
Meghan Ricci
Kate Schoch
Sergey Vartanov

Insurer of Last Resort?: The Federal Financial Response to September 11

David A. Moss
Sarah Brennan

Under Armour

Rory McDonald
Clayton M. Christensen
Daniel West
Jonathan E. Palmer
Tonia Junker

Hunley, Inc.: Casting for Growth

John A. Quelch
James T. Kindley

Bitfury: Blockchain for Government

Mitchell B. Weiss
Elena Corsi

Deutsche Bank: Pursuing Blockchain Opportunities (A)

Lynda M. Applegate
Christoph Muller-Bloch

Maersk: Betting on Blockchain

Scott Johnson

Yum! Brands

Jordan Siegel
Christopher Poliquin

Bharti Airtel in Africa

Tanya Bijlani

Li & Fung 2012

F. Warren McFarlan
Michael Shih-ta Chen
Keith Chi-ho Wong

Sony and the JK Wedding Dance

John Deighton
Leora Kornfeld

United Breaks Guitars

David dao on united airlines.

Benjamin Edelman
Jenny Sanford

Marketing Reading: Digital Marketing

Joseph Davin

Social Strategy at Nike

Mikolaj Jan Piskorski
Ryan Johnson

The Tate's Digital Transformation

Social strategy at american express, mellon financial and the bank of new york.

Carliss Y. Baldwin
Ryan D. Taliaferro

The Walt Disney Company and Pixar, Inc.: To Acquire or Not to Acquire?

Juan Alcacer
David J. Collis

Dow's Bid for Rohm and Haas

Benjamin C. Esty

Finance Reading: The Mergers and Acquisitions Process

John Coates

Apple: Privacy vs. Safety? (A)

Henry W. McGee
Nien-he Hsieh
Sarah McAra

Sidewalk Labs: Privacy in a City Built from the Internet Up

Leslie K. John

Data Breach at Equifax

Suraj Srinivasan
Quinn Pitcher
Jonah S. Goldberg

Apple's Core

Noam Wasserman

Design Thinking and Innovation at Apple

Barbara Feinberg

Apple Inc. in 2012

Penelope Rossano

Iz-Lynn Chan at Far East Organization (Abridged)

Anthony J. Mayo
Dana M. Teppert

Barbara Norris: Leading Change in the General Surgery Unit

Boris Groysberg
Nitin Nohria
Deborah Bell

Adobe Systems: Working Towards a "Suite" Release (A)

David A. Thomas
Lauren Barley

Home Nursing of North Carolina

Castronics, llc, gemini investors, angie's list: ratings pioneer turns 20.

Robert J. Dolan

Basecamp: Pricing

Frank V. Cespedes
Robb Fitzsimmons

J.C. Penney's "Fair and Square" Pricing Strategy

J.c. penney's 'fair and square' strategy (c): back to the future.

Jose B. Alvarez

Osaro: Picking the best path

James Palano
Bastiane Huang

HubSpot and Motion AI: Chatbot-Enabled CRM

Thomas Steenburgh

GROW: Using Artificial Intelligence to Screen Human Intelligence

Ethan S. Bernstein
Paul D. McKinnon
Paul Yarabe

Arup: Building the Water Cube

Robert G. Eccles
Amy C. Edmondson
Dilyana Karadzhova

(Re)Building a Global Team: Tariq Khan at Tek

Managing a global team: greg james at sun microsystems, inc. (a).

Thomas J. DeLong

Organizational Behavior Reading: Leading Global Teams

Ron ventura at mitchell memorial hospital.

Heide Abelli

Anthony Starks at InSiL Therapeutics (A)

Gary P. Pisano
Vicki L. Sato

Wolfgang Keller at Konigsbrau-TAK (A)

John J. Gabarro

Midland Energy Resources, Inc.: Cost of Capital

Timothy A. Luehrman
Joel L. Heilprin

Globalizing the Cost of Capital and Capital Budgeting at AES

Mihir A. Desai
Doug Schillinger

Cost of Capital at Ameritrade

Mark Mitchell
Erik Stafford

Finance Reading: Cost of Capital

David Neeleman: Flight Path of a Servant Leader (A)

Matthew D. Breitfelder

Coach Hurley at St. Anthony High School

Scott A. Snook
Bradley C. Lawrence

Shapiro Global

Michael Brookshire
Monica Haugen
Michelle Kravetz
Sarah Sommer

Kathryn McNeil (A)

Joseph L. Badaracco Jr.
Jerry Useem

Carol Fishman Cohen: Professional Career Reentry (A)

Myra M. Hart
Robin J. Ely
Susan Wojewoda

Alex Montana at ESH Manufacturing Co.

Michael Kernish

Michelle Levene (A)

Tiziana Casciaro
Victoria W. Winston

John and Andrea Rice: Entrepreneurship and Life

Howard H. Stevenson
Janet Kraus
Shirley M. Spence

Partner Center

All Headlines

Hertz CEO Kathryn Marinello with CFO Jamere Jackson and other members of the executive team in 2017

Top 40 Most Popular Case Studies of 2021

Two cases about Hertz claimed top spots in 2021's Top 40 Most Popular Case Studies

Two cases on the uses of debt and equity at Hertz claimed top spots in the CRDT’s (Case Research and Development Team) 2021 top 40 review of cases.

Hertz (A) took the top spot. The case details the financial structure of the rental car company through the end of 2019. Hertz (B), which ranked third in CRDT’s list, describes the company’s struggles during the early part of the COVID pandemic and its eventual need to enter Chapter 11 bankruptcy.

The success of the Hertz cases was unprecedented for the top 40 list. Usually, cases take a number of years to gain popularity, but the Hertz cases claimed top spots in their first year of release. Hertz (A) also became the first ‘cooked’ case to top the annual review, as all of the other winners had been web-based ‘raw’ cases.

Besides introducing students to the complicated financing required to maintain an enormous fleet of cars, the Hertz cases also expanded the diversity of case protagonists. Kathyrn Marinello was the CEO of Hertz during this period and the CFO, Jamere Jackson is black.

Sandwiched between the two Hertz cases, Coffee 2016, a perennial best seller, finished second. “Glory, Glory, Man United!” a case about an English football team’s IPO made a surprise move to number four. Cases on search fund boards, the future of malls, Norway’s Sovereign Wealth fund, Prodigy Finance, the Mayo Clinic, and Cadbury rounded out the top ten.

Other year-end data for 2021 showed:

Online “raw” case usage remained steady as compared to 2020 with over 35K users from 170 countries and all 50 U.S. states interacting with 196 cases.
Fifty four percent of raw case users came from outside the U.S..
The Yale School of Management (SOM) case study directory pages received over 160K page views from 177 countries with approximately a third originating in India followed by the U.S. and the Philippines.
Twenty-six of the cases in the list are raw cases.
A third of the cases feature a woman protagonist.
Orders for Yale SOM case studies increased by almost 50% compared to 2020.
The top 40 cases were supervised by 19 different Yale SOM faculty members, several supervising multiple cases.

CRDT compiled the Top 40 list by combining data from its case store, Google Analytics, and other measures of interest and adoption.

All of this year’s Top 40 cases are available for purchase from the Yale Management Media store .

And the Top 40 cases studies of 2021 are:

1. Hertz Global Holdings (A): Uses of Debt and Equity

2. Coffee 2016

3. Hertz Global Holdings (B): Uses of Debt and Equity 2020

4. Glory, Glory Man United!

5. Search Fund Company Boards: How CEOs Can Build Boards to Help Them Thrive

6. The Future of Malls: Was Decline Inevitable?

7. Strategy for Norway's Pension Fund Global

8. Prodigy Finance

9. Design at Mayo

10. Cadbury

11. City Hospital Emergency Room

13. Volkswagen

14. Marina Bay Sands

15. Shake Shack IPO

16. Mastercard

17. Netflix

18. Ant Financial

19. AXA: Creating the New CR Metrics

20. IBM Corporate Service Corps

21. Business Leadership in South Africa's 1994 Reforms

22. Alternative Meat Industry

23. Children's Premier

24. Khalil Tawil and Umi (A)

25. Palm Oil 2016

26. Teach For All: Designing a Global Network

27. What's Next? Search Fund Entrepreneurs Reflect on Life After Exit

28. Searching for a Search Fund Structure: A Student Takes a Tour of Various Options

30. Project Sammaan

31. Commonfund ESG

32. Polaroid

33. Connecticut Green Bank 2018: After the Raid

34. FieldFresh Foods

35. The Alibaba Group

36. 360 State Street: Real Options

37. Herman Miller

38. AgBiome

39. Nathan Cummings Foundation

40. Toyota 2010

Policy Reinvention and Diffusion: Evidence from Chinese Provincial Governments

Research Article
Published: 04 March 2021
Volume 26 , pages 723–741, ( 2021 )

Cite this article

Jing Chen ORCID: orcid.org/0000-0002-0237-8467 1 &
Cui Huang 2

1447 Accesses

15 Citations

1 Altmetric

Explore all metrics

Since the late 1950s, more than 1,000 academic articles related to policy innovation and diffusion have been published; most focus on how a policy spreads once it has been created, yet less attention is paid to how it is changed by those who adopt it. We extend the focus of policy innovation research from policy inventors to followers by focusing on the diffusion and adoption of the “One Visit at Most” policy in China. Fuzzy-set qualitative comparative analysis (fsQCA) is used to summarize the core factors affecting the reinvention of this policy by provincial governments in China. The results verify the characteristics of political systems are important factors in the provincial government’s policy reinvention. The vertical intergovernmental relation is the necessary condition to influence the policy reinvention. The combination of the factors for horizontal intergovernmental relations, the geographical leadership mobility of governors, and the characteristics of the government’s own economic environment also affect the provincial government’s policy reinvention. This study contributes the unique characteristics of the Chinese system and important theoretical developments to the study of policy reinvention within the broader policy innovation literature.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Policy Diffusion and European Public Policy Research

Policy Analysis and Innovation: Why the Humanities and the Social Sciences Matter for Social Transformation in Africa

Does Social Innovation Matter for Government? An Exploratory Study

Fifteen provinces have carried out “One Visit at Most” policy reinvention: Jiangsu, Jilin, Jiangxi, Chongqing, Tianjin, Guangxi, Yunnan, Guizhou, Shandong, Hunan, Shannxi, Beijing, Shanghai, Henan, and Guangdong.

Benoit, R., and C.C. Ragin. 2009. Configurational comparative methods: qualitative comparative analysis (QCA) and related techniques . London: Sage Publications Inc.

Google Scholar

Berry, F.S., and W.D. Berry. 1999. Innovation and diffusion models in policy research. In P.A. Sabatier (Ed.), Theories of the policy process . Boulder: Westview Press.

Berry, F.S., and W.D. Berry. 1990. State lottery adoptions as policy innovations: an event history analysis. American Political Science Review 84 (2): 395–415.

Article Google Scholar

Boehmke, F.J., and R. Witmer. 2004. Disentangling diffusion: the effects social learning and economic competition on state policy innovation and expansion. Political Research Quarterly 57 (1): 39–51.

Boushey, G. 2010. Policy Diffusion Dynamics in America . New York: Cambridge University Press.

Book Google Scholar

Cai, M.N., P.F. Liu, and H. Wang. 2020. Land commodification and hukou policy innovation in China: evidence from a survey experiment. Journal of Chinese Governance 5 (4): 419–438.

Canon, B.C., and L. Baum. 1981. Patterns of adoption of tort law innovations: an application of diffusion theory to judicial doctrines. The American Political Science Review 75 (4): 975.

Carley, S., S. Nicholson-Crotty, and C.J. Miller. 2017. Adoption, reinvention and amendment of renewable portfolio standards in the American States. Journal of Public Policy 37 (4): 431–458.

Carter, L.E., and J.T. LaPlant. 1997. Diffusion of health care policy innovation in the United States. State & Local Government Review 29 (1): 17–26.

Chan, H.S., and K.W. Chow. 2007. Public management policy and practice in Western China: metapolicy, tacit knowledge, and implications for management innovation transfer. American Journal of Public Administration 37 (4): 479–498.

Chen, Y., A.G.O. Yeh, and Y. Zhang. 2017. Political tournament and regional cooperation in China: a game theory approach. Annals of Regional Science 58 (3): 597–622.

Chien, S.S., and H. Bing. 2011. Globalization and the local government learning process in post-Mao China: a transnational perspective. Global Networks 11 (3): 315–333.

Clark, J. 1985. Policy diffusion and program scope: research directions. Publius 15 (4): 61–70.

Damanpour, F. 1991. Organizational innovation: a meta-analysis of effects of determinants and moderators. The Academy of Management Journal 34 (3): 555–590.

Dolowitz, D., and D. Marsh. 1996. Who learns what from whom? a review of the policy transfer literature. Political Studies 44 (2): 343–357.

Elkins, Z., and B. Simmons. 2005. On waves, clusters, and diffusion: a conceptual framework. The Annals of the American Academy of Political and Social Science 598 (1): 33–51.

Florini, A., H. Lai, and Y. Tan. 2012. China experiments: from local innovations to national reform . Washington: The Brookings Institution.

Gilardi, F. 2016. Four ways we can improve policy diffusion research. State Politics & Policy Quarterly 16 (1): 8–21.

Glick, H.R. 1992. Judicial innovation and policy reinvention: State supreme courts and the right to die. The Western Political Quarterly 45 (1): 71–92.

Glick, H.R., and S.P. Hays. 1991. Innovation and reinvention in State policy-making theory and evolution of living will laws. Journal of Politics 53 (3): 835–850.

Graham, E.R., C.R. Shipan, and C. Volden. 2013. The diffusion of policy diffusion research. British Journal of Political Science 43 (3): 673–701.

Guo, L., and Y. Ba. 2019. Adopt or not and innovation variation: a dynamic comparison study of policy innovation and diffusion mechanisms. Journal of Comparative Policy Analysis 22 (4): 1–22.

Hays, S.P. 1996. Influences on reinvention during the diffusion of innovations. Political Research Quarterly 49 (3): 631–650.

Hays, S.P. 1996. Patterns of reinvention : the nature of evolution during policy diffusion. Policy Studies Journal 24 (4): 551–566.

Heilmann, S. 2008. Policy experimentation in china’s economic rise. Studies in Comparative International Development 43 (1): 1–26.

Heilmann, S. 2018. Red swan: how unorthodox policy making facilitated China’s rise . Hong Kong: The Chinese University Press of Hong Kong.

Hsu, P.S. 2004. Deconstructing decentralization in China: Fiscal incentive versus local autonomy in policy implementation. Journal of Contemporary China 13 (40): 567–599.

Huang, B., and J.X. Yu. 2019. Leading digital technologies for coproduction: the case of ‘Visit Once’ administrative service reform in Zhejiang province China. Journal of Chinese Political Science 24 (3): 513–532.

Jacobs, D. 2014. Policy invention as evolutionary tinkering and codification: the emergence of feed-in tariffs for renewable electricity. Environmental Politics 23 (5): 755–773.

Jansa, J.M., E.R. Hansen, and V.H. Gray. 2018. Copy and paste lawmaking: legislative professionalism and policy reinvention in the states. American Politics Research 47 (4): 739–767.

Karch, A. 2007. Emerging issues and future directions in state policy diffusion research. State Politics & Policy Quarterly 7 (1): 54–80.

Lan, G.Z., and J. Galaskiewicz. 2012. Innovations in public and non-profit sector organizations in China. Management and Organization Review 8 (3): 491–506.

Landry, F.P. 2008. Decentralized authoritarianism in China: the Communist Party’s control of local elites in the post-Mao era . Cambridge: Cambridge University Press.

Li, H., and L.A. Zhou. 2005. Political turnover and economic performance: the incentive role of personnel control in China. Journal of Public Economics 89 (9): 1743–1762.

Lin, X.F., and X, X.L. . 2017. Structural restraints and institutional innovation in local governance: a case study of administrative examination and approval system reforms in Shunde, Ningbo, and Taizhou. Journal of Chinese Governance 2 (1): 20–49.

Liu, W., and W. Li. 2016. Divergence and convergence in the diffusion of performance management in China. Public Performance & Management Review 39 (3): 630–654.

Ma, L. 2012. Diffusion and assimilation of government microblogging: evidence from Chinese cities. Public Management Review 16 (2): 274–295.

Ma, L. 2017. Site visits, policy learning, and the diffusion of policy innovation: evidence from public bicycle programs in China. Journal of Chinese Political Science 22 (4): 581–599.

Mallinson, D.J. 2020. Policy innovation adoption across the diffusion life course. Policy Studies Journal, 1–24. https://doi.org/10.1111/psj.12406 .

Marsh, D., and J.C. Sharman. 2009. Policy diffusion and policy transfer. Policy Studies 30 (3): 269–288.

Mei, C.Q., and X. Wang. 2017. Political incentives and local policy innovations in china. Journal of Chinese Political Science 22 (4): 519–547.

Mei, C.Q., and X. Wang. 2019. Wire-walking: risk management and policy experiments in China from a comparative perspective. Journal of Comparative Policy Analysis: Research and Practice 22 (4): 360–382.

Mintrom, M., and S. Vergari. 1998. Policy networks and innovation diffusion: the case of state education reforms. The Journal of Politics 60 (1): 126–148.

Mooney, C.Z., and M.H. Lee. 1995. Legislative morality in the American States: the case of pre-roe abortion regulation reform. American Journal of Political Science 39 (3): 599–627.

Mooney, C.Z., and M.H. Lee. 1999. Morality policy reinvention: State death penalties. The Annals of the American Academy of Political and Social Science 566 (1): 80–92.

Neomi, F.A., C. Nissim, and B. Itai. 2020. Policy entrepreneurship in developing countries: a systematic review of the literature. Public Administration and Development 40 (1): 35–48.

Pacheco, J. 2012. The social contagion model: exploring the role of public opinion on the diffusion of antismoking legislation across the American States. The Journal of Politics 74 (1): 187–202.

Rice, R.E., and E.M. Rogers. 1980. Reinvention in the innovation process. Science Communication 1 (4): 499–514.

Rogers, E.M. 1978. Reinvention during the innovation process. In Radnor, M., Feller, I., and Rogers, E. (Eds). The diffusion of innovations: An assessment . Evanston: Northwestem University, Center for the Interdisciplinary Study of Science and Technology.

Rogers, E.M. 1983. Diffusion of innovations . New York: Free Press.

Schneider, C.Q., and I. Rohlfing. 2016. Case studies nested in fuzzy-set QCA on sufficiency: formalizing case selection and causal inference. Sociological Methods & Research 45 (3): 526–568.

Shin, D.H. 2010. Convergence and divergence: policy making about the convergence of technology in Korea. Government Information Quarterly 27 (2): 147–160.

Shin, G., and D.J. Webber. 2014. Red states, blue states: how well do the recent national election labels capture state political and policy differences? Social Science Journal 51 (3): 386–397.

Shipan, C., and C. Volden. 2008. The mechanisms of policy diffusion. American Journal of Political Science 52 (4): 840–857.

Stefaan, D.R., and D. Karolien. 2006. Policy convergence and divergence in Belgium: education and health care. West European Politics 29 (5): 1018–1033.

Taylor, J.K., D.C. Lewis, M.L. Jacobsmeier, and B.S. Disarro. 2012. Content and complexity in policy reinvention and diffusion: gay and transgender-inclusive laws against discrimination. State Politics & Policy Quarterly 12 (1): 75–98.

Teets, J.C., and W. Hurst. 2015. Introduction: the politics and patterns of policy diffusion in China. In J. C. Teets, and W. Hurst (Eds). Local governance innovation in China: experimentation, diffusion, and defiance . New York: Routledge.

Thomann, E., and M. Maggetti. 2020. Designing research with qualitative comparative analysis (QCA): approaches, challenges, and tools. Sociological Methods & Research 49 (2): 356–386.

Tolbert, C.J., K. Mossberger, and R. McNeal. 2008. Institutions, policy innovation, and e-government in the American States. Public Administration Review 68 (3): 549–563.

Tyran, J.R., and R. Sausgruber. 2005. The diffusion of policy innovations: an experimental investigation. Journal of Evolutionary Economics 15 (4): 423–442.

Vaisey, S. 2009. QCA 3.0: The “Ragin Revolution” continues. Contemporary Sociology 38 (4): 308–312.

Volden, C. 2006. States as policy laboratories: emulating success in the children’s health insurance program. American Journal of Political Science 50 (2): 294–312.

Walker, J.L. 1969. The diffusion of innovations among the American states. American Political Science Review 63 (3): 880–899.

Walker, R.M. 2008. An empirical evaluation of innovation types and organizational and environmental characteristics: towards a configuration framework. Journal of Public Administration Research and Theory 18 (4): 591–615.

Walker, R.M., C.N. Avellaneda, and F.S. Berry. 2011. Exploring the diffusion of innovation among high and low innovative localities. Public Management Review 13 (1): 95–125.

Wang, S. 2009. Adapting by learning: the evolution of China’s rural health care financing. Modern China 35 (4): 370–404.

Wang, C., X. Li, W. Ma, and X. Wang. 2020. Diffusion models over the life cycle of an innovation: a bottom-up and top-down synthesis approach. Public Administration and Development 40 (2): 105–118.

Wong, C. 2016. Budget reform in China :progress and prospects in the Xi Jinping era. OECD Journal on Budgeting 15 (3): 27–36.

Wu, J., and P. Zhang. 2018. Local government innovation diffusion in china: an event history analysis of a performance-based reform programme. International Review of Administrative Sciences 84 (1): 63–81.

Wu, J., L. Ma, and Y. Yang. 2013. Innovation in the Chinese public sector: typology and distribution. Public Administration 91 (2): 347–365.

Xu, C.G. 2011. The fundamental institutions of China’s reforms and development. Journal of Economic Literature 49 (4): 1076–1151.

Yang, K.F. 2016. Creating public value and institutional innovations across boundaries: an integrative process of participation, legitimation, and implementation. Public Administration Review 76 (6): 873–885.

Yi, H., F.S. Berry, and W. Chen. 2018. Management innovation and policy diffusion through leadership transfer networks: an agent network diffusion model. Journal of Public Administration Research and Theory 28 (4): 457–474.

Yu, J., E.T. Jennings, and J.S. Butler. 2020. Lobbying, learning and policy reinvention: an examination of the American states’ drunk driving laws. Journal of Public Policy 40 (2): 259–279.

Zhang, Y. 2012. Institutional sources of reform: the diffusion of land banking systems in China. Management and Organization Review 8 (3): 507–533.

Zhang, Y.L., and X.F. Zhu. 2019. Multiple mechanisms of policy diffusion in China. Public Management Review 21 (4): 495–514.

Zhu, X.F. 2017. Executive entrepreneurship, career mobility and the transfer of policy paradigms. Journal of Comparative Policy Analysis Research and Practice 20 (4): 354–369.

Zhu, X.F., and T.G. Meng. 2018. Geographical leadership mobility and policy isomorphism: narrowing the regional inequality of social spending in China. Policy Studies Journal 48 (3): 806–832.

Zhu, X.F., and Y.L. Zhang. 2016. Political mobility and dynamic diffusion of innovation: the spread of municipal pro-business administrative reform in china. Journal of Public Administration Research & Theory 26 (3): 535–551.

Zhu, X.F., and H. Zhao. 2018. Recognition of innovation and diffusion of welfare policy: alleviating urban poverty in Chinese cities during fiscal recentralization. Governance 31 (4): 721–739.

Download references

The funding information were submitted in the system before, specific information:

1. Ministry of Education of the People's Republic of China (18YJC870022)

2. National Natural Science Foundation of China (71722002)

3. National Natural Science Foundation of China (71673164)

Author information

Authors and affiliations.

School of Public Policy & Management, Tsinghua University, Beijing, 10084, People’s Republic of China

School of Public Affairs, Zhejiang University, Hangzhou, Zhejiang, 310000, People’s Republic of China

You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Cui Huang .

Rights and permissions

Reprints and permissions

About this article

Chen, J., Huang, C. Policy Reinvention and Diffusion: Evidence from Chinese Provincial Governments. J OF CHIN POLIT SCI 26 , 723–741 (2021). https://doi.org/10.1007/s11366-021-09725-8

Download citation

Accepted : 19 January 2021

Published : 04 March 2021

Issue Date : December 2021

DOI : https://doi.org/10.1007/s11366-021-09725-8

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Policy reinvention
Policy diffusion
Policy process
Find a journal
Publish with us
Track your research

Home > Journals > ELJ-COLLECTION > Emory Law Journal > Vol. 64 > Iss. 4 (2015)

The Fracking Revolution: Shale Gas as a Case Study in Innovation Policy

John M. Golden Hannah J. Wiseman

The early twenty-first century has witnessed a boom in oil and natural gas production that promises to turn the United States into a new form of petrostate. This boom raises various questions that scholars have begun to explore, including questions of risk governance, federalism, and export policy. Relatively neglected, however, have been questions of why the technological revolution behind the boom occurred and what this revolution teaches about innovation theory and policy. The boom in U.S. shale gas production reflected long-gestating infrastructure developments, a convergence of technological advances, government-sponsored research and development, the presence or absence of intellectual property rights, rights in tangible assets such as land and minerals, and tax and regulatory relief. Consequently, the story behind the boom reaches far beyond the risk-taking and persistence of George Mitchell, whose independent production company achieved pioneering success with hydraulic fracturing (fracking) in Texas¿ Barnett Shale. Indeed, the broader story demonstrates how a blend of distinct policy levers, reasonably adjusted over time, can combine to foster a diverse innovation ecosystem that provides a robust platform for game-changing innovation.

Recommended Citation

John M. Golden & Hannah J. Wiseman, The Fracking Revolution: Shale Gas as a Case Study in Innovation Policy , 64 Emory L. J. 955 (2015). Available at: https://scholarlycommons.law.emory.edu/elj/vol64/iss4/1

Since July 14, 2020

Journal Home
Copyright & Policies
Editorial Board
Symposia & Events
Emory Law Journal Online

Advanced Search

Search Help

Most Popular Papers
Receive Email Notices or RSS

ISSN: 0094-4076

Home | About | FAQ | My Account | Accessibility Statement

Privacy Copyright

Innovation Report 2024

Our 2024 Innovation Report, the Power of (Making) Futures, outlines nine innovative initiatives, categorised into three themes which are; rebalancing power dynamics in narratives and imagery; participatory funding models; and innovative approaches to development cooperation. Moreover, the report offers recommendations, drawn from shared success factors and lessons learned.

Rebalancing Power Dynamics in Narratives and Imagery

The lam sisterhood.

Click on the button to load the content from w.soundcloud.com.

Load content

The Ascend Programme - HEVA Fund

Africa Forward

By loading the video, you agree to YouTube's privacy policy. Learn more

Always unblock YouTube

Participatory Funding Models

Kenya community development foundation (kcdf), uhai, the east african sexual health and rights initiative (uhai-eashri).

The Change Fund - NEAR Network

Innovative approaches to development cooperation, a feminist south-south alliance: leading from the south.

The Grounded Accountability Model - Everyday Peace Indicators

Global Giving and YUM - Foundation for Noble Work

Interested in our previous Innovation Reports?

Innovation report 2022.

Our 2021/22 Innovation Report will be a timely look at how far our sector has come in advancing people-centred and nature-positive digitalisation.

Innovation Report 2020

The 2020 report highlights successful innovations in complex urban contexts to deliver inclusive solutions for marginalised communities in cities.

Innovation Report 2019

14 innovative civil society responses in the face of rising populism, one of the defining political features of our times.

Futures & Innovation
Collaboration
Trends & Ideas
Futures & Innovation
Trends & Ideas
Get Involved
Data & Privacy

By signing up to this newsletter, I agree to the International Civil Society Centre’s privacy policy and allow them to contact me via email with their latest news and events. You can unsubscribe at any time via the contact form here .

International Civil Society Centre gGmbH Agricolastraße 26 10555 Berlin Germany

Phone: +49 (0) 30 20 62 46 97 11 Fax: +49 (0) 30 20 62 46 97 19

Open access
Published: 15 May 2024

Learning together for better health using an evidence-based Learning Health System framework: a case study in stroke

Helena Teede 1 , 2 na1 ,
Dominique A. Cadilhac 3 , 4 na1 ,
Tara Purvis 3 ,
Monique F. Kilkenny 3 , 4 ,
Bruce C.V. Campbell 4 , 5 , 6 ,
Coralie English 7 ,
Alison Johnson 2 ,
Emily Callander 1 ,
Rohan S. Grimley 8 , 9 ,
Christopher Levi 10 ,
Sandy Middleton 11 , 12 ,
Kelvin Hill 13 &
Joanne Enticott ORCID: orcid.org/0000-0002-4480-5690 1

BMC Medicine volume 22 , Article number: 198 ( 2024 ) Cite this article

349 Accesses

2 Altmetric

Metrics details

In the context of expanding digital health tools, the health system is ready for Learning Health System (LHS) models. These models, with proper governance and stakeholder engagement, enable the integration of digital infrastructure to provide feedback to all relevant parties including clinicians and consumers on performance against best practice standards, as well as fostering innovation and aligning healthcare with patient needs. The LHS literature primarily includes opinion or consensus-based frameworks and lacks validation or evidence of benefit. Our aim was to outline a rigorously codesigned, evidence-based LHS framework and present a national case study of an LHS-aligned national stroke program that has delivered clinical benefit.

Current core components of a LHS involve capturing evidence from communities and stakeholders (quadrant 1), integrating evidence from research findings (quadrant 2), leveraging evidence from data and practice (quadrant 3), and generating evidence from implementation (quadrant 4) for iterative system-level improvement. The Australian Stroke program was selected as the case study as it provides an exemplar of how an iterative LHS works in practice at a national level encompassing and integrating evidence from all four LHS quadrants. Using this case study, we demonstrate how to apply evidence-based processes to healthcare improvement and embed real-world research for optimising healthcare improvement. We emphasize the transition from research as an endpoint, to research as an enabler and a solution for impact in healthcare improvement.

Conclusions

The Australian Stroke program has nationally improved stroke care since 2007, showcasing the value of integrated LHS-aligned approaches for tangible impact on outcomes. This LHS case study is a practical example for other health conditions and settings to follow suit.

Peer Review reports

Internationally, health systems are facing a crisis, driven by an ageing population, increasing complexity, multi-morbidity, rapidly advancing health technology and rising costs that threaten sustainability and mandate transformation and improvement [ 1 , 2 ]. Although research has generated solutions to healthcare challenges, and the advent of big data and digital health holds great promise, entrenched siloes and poor integration of knowledge generation, knowledge implementation and healthcare delivery between stakeholders, curtails momentum towards, and consistent attainment of, evidence-and value-based care [ 3 ]. This is compounded by the short supply of research and innovation leadership within the healthcare sector, and poorly integrated and often inaccessible health data systems, which have crippled the potential to deliver on digital-driven innovation [ 4 ]. Current approaches to healthcare improvement are also often isolated with limited sustainability, scale-up and impact [ 5 ].

Evidence suggests that integration and partnership across academic and healthcare delivery stakeholders are key to progress, including those with lived experience and their families (referred to here as consumers and community), diverse disciplines (both research and clinical), policy makers and funders. Utilization of evidence from research and evidence from practice including data from routine care, supported by implementation research, are key to sustainably embedding improvement and optimising health care and outcomes. A strategy to achieve this integration is through the Learning Health System (LHS) (Fig. 1 ) [ 2 , 6 , 7 , 8 ]. Although there are numerous publications on LHS approaches [ 9 , 10 , 11 , 12 ], many focus on research perspectives and data, most do not demonstrate tangible healthcare improvement or better health outcomes. [ 6 ]

Monash Learning Health System: The Learn Together for Better Health Framework developed by Monash Partners and Monash University (from Enticott et al. 2021 [ 7 ]). Four evidence quadrants: Q1 (orange) is evidence from stakeholders; Q2 (green) is evidence from research; Q3 (light blue) is evidence from data; and, Q4 (dark blue) is evidence from implementation and healthcare improvement

In developed nations, it has been estimated that 60% of care provided aligns with the evidence base, 30% is low value and 10% is potentially harmful [ 13 ]. In some areas, clinical advances have been rapid and research and evidence have paved the way for dramatic improvement in outcomes, mandating rapid implementation of evidence into healthcare (e.g. polio and COVID-19 vaccines). However, healthcare improvement is challenging and slow [ 5 ]. Health systems are highly complex in their design, networks and interacting components, and change is difficult to enact, sustain and scale up. [ 3 ] New effective strategies are needed to meet community needs and deliver evidence-based and value-based care, which reorients care from serving the provider, services and system, towards serving community needs, based on evidence and quality. It goes beyond cost to encompass patient and provider experience, quality care and outcomes, efficiency and sustainability [ 2 , 6 ].

The costs of stroke care are expected to rise rapidly in the next decades, unless improvements in stroke care to reduce the disabling effects of strokes can be successfully developed and implemented [ 14 ]. Here, we briefly describe the Monash LHS framework (Fig. 1 ) [ 2 , 6 , 7 ] and outline an exemplar case in order to demonstrate how to apply evidence-based processes to healthcare improvement and embed real-world research for optimising healthcare. The Australian LHS exemplar in stroke care has driven nationwide improvement in stroke care since 2007.

An evidence-based Learning Health System framework

In Australia, members of this author group (HT, AJ, JE) have rigorously co-developed an evidence-based LHS framework, known simply as the Monash LHS [ 7 ]. The Monash LHS was designed to support sustainable, iterative and continuous robust benefit of improved clinical outcomes. It was created with national engagement in order to be applicable to Australian settings. Through this rigorous approach, core LHS principles and components have been established (Fig. 1 ). Evidence shows that people/workforce, culture, standards, governance and resources were all key to an effective LHS [ 2 , 6 ]. Culture is vital including trust, transparency, partnership and co-design. Key processes include legally compliant data sharing, linkage and governance, resources, and infrastructure [ 4 ]. The Monash LHS integrates disparate and often siloed stakeholders, infrastructure and expertise to ‘Learn Together for Better Health’ [ 7 ] (Fig. 1 ). This integrates (i) evidence from community and stakeholders including priority areas and outcomes; (ii) evidence from research and guidelines; (iii) evidence from practice (from data) with advanced analytics and benchmarking; and (iv) evidence from implementation science and health economics. Importantly, it starts with the problem and priorities of key stakeholders including the community, health professionals and services and creates an iterative learning system to address these. The following case study was chosen as it is an exemplar of how a Monash LHS-aligned national stroke program has delivered clinical benefit.

Australian Stroke Learning Health System

Internationally, the application of LHS approaches in stroke has resulted in improved stroke care and outcomes [ 12 ]. For example, in Canada a sustained decrease in 30-day in-hospital mortality has been found commensurate with an increase in resources to establish the multifactorial stroke system intervention for stroke treatment and prevention [ 15 ]. Arguably, with rapid advances in evidence and in the context of an ageing population with high cost and care burden and substantive impacts on quality of life, stroke is an area with a need for rapid research translation into evidence-based and value-based healthcare improvement. However, a recent systematic review found that the existing literature had few comprehensive examples of LHS adoption [ 12 ]. Although healthcare improvement systems and approaches were described, less is known about patient-clinician and stakeholder engagement, governance and culture, or embedding of data informatics into everyday practice to inform and drive improvement [ 12 ]. For example, in a recent review of quality improvement collaborations, it was found that although clinical processes in stroke care are improved, their short-term nature means there is uncertainty about sustainability and impacts on patient outcomes [ 16 ]. Table 1 provides the main features of the Australian Stroke LHS based on the four core domains and eight elements of the Learning Together for Better Health Framework described in Fig. 1 . The features are further expanded on in the following sections.

Evidence from stakeholders (LHS quadrant 1, Fig. 1 )

Engagement, partners and priorities.

Within the stroke field, there have been various support mechanisms to facilitate an LHS approach including partnership and broad stakeholder engagement that includes clinical networks and policy makers from different jurisdictions. Since 2008, the Australian Stroke Coalition has been co-led by the Stroke Foundation, a charitable consumer advocacy organisation, and Stroke Society of Australasia a professional society with membership covering academics and multidisciplinary clinician networks, that are collectively working to improve stroke care ( https://australianstrokecoalition.org.au/ ). Surveys, focus groups and workshops have been used for identifying priorities from stakeholders. Recent agreed priorities have been to improve stroke care and strengthen the voice for stroke care at a national ( https://strokefoundation.org.au/ ) and international level ( https://www.world-stroke.org/news-and-blog/news/world-stroke-organization-tackle-gaps-in-access-to-quality-stroke-care ), as well as reduce duplication amongst stakeholders. This activity is built on a foundation and culture of research and innovation embedded within the stroke ‘community of practice’. Consumers, as people with lived experience of stroke are important members of the Australian Stroke Coalition, as well as representatives from different clinical colleges. Consumers also provide critical input to a range of LHS activities via the Stroke Foundation Consumer Council, Stroke Living Guidelines committees, and the Australian Stroke Clinical Registry (AuSCR) Steering Committee (described below).

Evidence from research (LHS quadrant 2, Fig. 1 )

Advancement of the evidence for stroke interventions and synthesis into clinical guidelines.

To implement best practice, it is crucial to distil the large volume of scientific and trial literature into actionable recommendations for clinicians to use in practice [ 24 ]. The first Australian clinical guidelines for acute stroke were produced in 2003 following the increasing evidence emerging for prevention interventions (e.g. carotid endarterectomy, blood pressure lowering), acute medical treatments (intravenous thrombolysis, aspirin within 48 h of ischemic stroke), and optimised hospital management (care in dedicated stroke units by a specialised and coordinated multidisciplinary team) [ 25 ]. Importantly, a number of the innovations were developed, researched and proven effective by key opinion leaders embedded in the Australian stroke care community. In 2005, the clinical guidelines for Stroke Rehabilitation and Recovery [ 26 ] were produced, with subsequent merged guidelines periodically updated. However, the traditional process of periodic guideline updates is challenging for end users when new research can render recommendations redundant and this lack of currency erodes stakeholder trust [ 27 ]. In response to this challenge the Stroke Foundation and Cochrane Australia entered a pioneering project to produce the first electronic ‘living’ guidelines globally [ 20 ]. Major shifts in the evidence for reperfusion therapies (e.g. extended time-window intravenous thrombolysis and endovascular clot retrieval), among other advances, were able to be converted into new recommendations, approved by the Australian National Health and Medical Research Council within a few months of publication. Feedback on this process confirmed the increased use and trust in the guidelines by clinicians. The process informed other living guidelines programs, including the successful COVID-19 clinical guidelines [ 28 ].

However, best practice clinical guideline recommendations are necessary but insufficient for healthcare improvement and nesting these within an LHS with stakeholder partnership, enables implementation via a range of proven methods, including audit and feedback strategies [ 29 ].

Evidence from data and practice (LHS quadrant 3, Fig. 1 )

Data systems and benchmarking : revealing the disparities in care between health services. A national system for standardized stroke data collection was established as the National Stroke Audit program in 2007 by the Stroke Foundation [ 30 ] following various state-level programs (e.g. New South Wales Audit) [ 31 ] to identify evidence-practice gaps and prioritise improvement efforts to increase access to stroke units and other acute treatments [ 32 ]. The Audit program alternates each year between acute (commencing in 2007) and rehabilitation in-patient services (commencing in 2008). The Audit program provides a ‘deep dive’ on the majority of recommendations in the clinical guidelines whereby participating hospitals provide audits of up to 40 consecutive patient medical records and respond to a survey about organizational resources to manage stroke. In 2009, the AuSCR was established to provide information on patients managed in acute hospitals based on a small subset of quality processes of care linked to benchmarked reports of performance (Fig. 2 ) [ 33 ]. In this way, the continuous collection of high-priority processes of stroke care could be regularly collected and reviewed to guide improvement to care [ 34 ]. Plus clinical quality registry programs within Australia have shown a meaningful return on investment attributed to enhanced survival, improvements in quality of life and avoided costs of treatment or hospital stay [ 35 ].

Example performance report from the Australian Stroke Clinical Registry: average door-to-needle time in providing intravenous thrombolysis by different hospitals in 2021 [ 36 ]. Each bar in the figure represents a single hospital

The Australian Stroke Coalition endorsed the creation of an integrated technological solution for collecting data through a single portal for multiple programs in 2013. In 2015, the Stroke Foundation, AuSCR consortium, and other relevant groups cooperated to design an integrated data management platform (the Australian Stroke Data Tool) to reduce duplication of effort for hospital staff in the collection of overlapping variables in the same patients [ 19 ]. Importantly, a national data dictionary then provided the common data definitions to facilitate standardized data capture. Another important feature of AuSCR is the collection of patient-reported outcome surveys between 90 and 180 days after stroke, and annual linkage with national death records to ascertain survival status [ 33 ]. To support a LHS approach, hospitals that participate in AuSCR have access to a range of real-time performance reports. In efforts to minimize the burden of data collection in the AuSCR, interoperability approaches to import data directly from hospital or state-level managed stroke databases have been established (Fig. 3 ); however, the application has been variable and 41% of hospitals still manually enter all their data.

Current status of automated data importing solutions in the Australian Stroke Clinical Registry, 2022, with ‘ n ’ representing the number of hospitals. AuSCR, Australian Stroke Clinical Registry; AuSDaT, Australian Stroke Data Tool; API, Application Programming Interface; ICD, International Classification of Diseases; RedCAP, Research Electronic Data Capture; eMR, electronic medical records

For acute stroke care, the Australian Commission on Quality and Safety in Health Care facilitated the co-design (clinicians, academics, consumers) and publication of the national Acute Stroke Clinical Care Standard in 2015 [ 17 ], and subsequent review [ 18 ]. The indicator set for the Acute Stroke Standard then informed the expansion of the minimum dataset for AuSCR so that hospitals could routinely track their performance. The national Audit program enabled hospitals not involved in the AuSCR to assess their performance every two years against the Acute Stroke Standard. Complementing these efforts, the Stroke Foundation, working with the sector, developed the Acute and Rehabilitation Stroke Services Frameworks to outline the principles, essential elements, models of care and staffing recommendations for stroke services ( https://informme.org.au/guidelines/national-stroke-services-frameworks ). The Frameworks are intended to guide where stroke services should be developed, and monitor their uptake with the organizational survey component of the Audit program.

Evidence from implementation and healthcare improvement (LHS quadrant 4, Fig. 1 )

Research to better utilize and augment data from registries through linkage [ 37 , 38 , 39 , 40 ] and to ensure presentation of hospital or service level data are understood by clinicians has ensured advancement in the field for the Australian Stroke LHS [ 41 ]. Importantly, greater insights into whole patient journeys, before and after a stroke, can now enable exploration of value-based care. The LHS and stroke data platform have enabled focused and time-limited projects to create a better understanding of the quality of care in acute or rehabilitation settings [ 22 , 42 , 43 ]. Within stroke, all the elements of an LHS culminate into the ready availability of benchmarked performance data and support for implementation of strategies to address gaps in care.

Implementation research to grow the evidence base for effective improvement interventions has also been a key pillar in the Australian context. These include multi-component implementation interventions to achieve behaviour change for particular aspects of stroke care, [ 22 , 23 , 44 , 45 ] and real-world approaches to augmenting access to hyperacute interventions in stroke through the use of technology and telehealth [ 46 , 47 , 48 , 49 ]. The evidence from these studies feeds into the living guidelines program and the data collection systems, such as the Audit program or AuSCR, which are then amended to ensure data aligns to recommended care. For example, the use of ‘hyperacute aspirin within the first 48 h of ischemic stroke’ was modified to be ‘hyperacute antiplatelet…’ to incorporate new evidence that other medications or combinations are appropriate to use. Additionally, new datasets have been developed to align with evidence such as the Fever, Sugar, and Swallow variables [ 42 ]. Evidence on improvements in access to best practice care from the acute Audit program [ 50 ] and AuSCR is emerging [ 36 ]. For example, between 2007 and 2017, the odds of receiving intravenous thrombolysis after ischemic stroke increased by 16% 9OR 1.06 95% CI 1.13–1.18) and being managed in a stroke unit by 18% (OR 1.18 95% CI 1.17–1.20). Over this period, the median length of hospital stay for all patients decreased from 6.3 days in 2007 to 5.0 days in 2017 [ 51 ]. When considering the number of additional patients who would receive treatment in 2017 in comparison to 2007 it was estimated that without this additional treatment, over 17,000 healthy years of life would be lost in 2017 (17,786 disability-adjusted life years) [ 51 ]. There is evidence on the cost-effectiveness of different system-focussed strategies to augment treatment access for acute ischemic stroke (e.g. Victorian Stroke Telemedicine program [ 52 ] and Melbourne Mobile Stroke Unit ambulance [ 53 ]). Reciprocally, evidence from the national Rehabilitation Audit, where the LHS approach has been less complete or embedded, has shown fewer areas of healthcare improvement over time [ 51 , 54 ].

Within the field of stroke in Australia, there is indirect evidence that the collective efforts that align to establishing the components of a LHS have had an impact. Overall, the age-standardised rate of stroke events has reduced by 27% between 2001 and 2020, from 169 to 124 events per 100,000 population. Substantial declines in mortality rates have been reported since 1980. Commensurate with national clinical guidelines being updated in 2007 and the first National Stroke Audit being undertaken in 2007, the mortality rates for men (37.4 deaths per 100,000) and women (36.1 deaths per 100,0000 has declined to 23.8 and 23.9 per 100,000, respectively in 2021 [ 55 ].

Underpinning the LHS with the integration of the four quadrants of evidence from stakeholders, research and guidelines, practice and implementation, and core LHS principles have been addressed. Leadership and governance have been important, and programs have been established to augment workforce training and capacity building in best practice professional development. Medical practitioners are able to undertake courses and mentoring through the Australasian Stroke Academy ( http://www.strokeacademy.com.au/ ) while nurses (and other health professionals) can access teaching modules in stroke care from the Acute Stroke Nurses Education Network ( https://asnen.org/ ). The Association of Neurovascular Clinicians offers distance-accessible education and certification to develop stroke expertise for interdisciplinary professionals, including advanced stroke co-ordinator certification ( www.anvc.org ). Consumer initiative interventions are also used in the design of the AuSCR Public Summary Annual reports (available at https://auscr.com.au/about/annual-reports/ ) and consumer-related resources related to the Living Guidelines ( https://enableme.org.au/resources ).

The important success factors and lessons from stroke as a national exemplar LHS in Australia include leadership, culture, workforce and resources integrated with (1) established and broad partnerships across the academic-clinical sector divide and stakeholder engagement; (2) the living guidelines program; (3) national data infrastructure, including a national data dictionary that provides the common data framework to support standardized data capture; (4) various implementation strategies including benchmarking and feedback as well as engagement strategies targeting different levels of the health system; and (5) implementation and improvement research to advance stroke systems of care and reduce unwarranted variation in practice (Fig. 1 ). Priority opportunities now include the advancement of interoperability with electronic medical records as an area all clinical quality registry’s programs needs to be addressed, as well as providing more dynamic and interactive data dashboards tailored to the need of clinicians and health service executives.

There is a clear mandate to optimise healthcare improvement with big data offering major opportunities for change. However, we have lacked the approaches to capture evidence from the community and stakeholders, to integrate evidence from research, to capture and leverage data or evidence from practice and to generate and build on evidence from implementation using iterative system-level improvement. The LHS provides this opportunity and is shown to deliver impact. Here, we have outlined the process applied to generate an evidence-based LHS and provide a leading exemplar in stroke care. This highlights the value of moving from single-focus isolated approaches/initiatives to healthcare improvement and the benefit of integration to deliver demonstrable outcomes for our funders and key stakeholders — our community. This work provides insight into strategies that can both apply evidence-based processes to healthcare improvement as well as implementing evidence-based practices into care, moving beyond research as an endpoint, to research as an enabler, underpinning delivery of better healthcare.

Availability of data and materials

Not applicable

Abbreviations

Australian Stroke Clinical Registry

Confidence interval

Learning Health System

World Health Organization. Delivering quality health services . OECD Publishing; 2018.

Enticott J, Braaf S, Johnson A, Jones A, Teede HJ. Leaders’ perspectives on learning health systems: A qualitative study. BMC Health Serv Res. 2020;20:1087.

Article PubMed PubMed Central Google Scholar

Melder A, Robinson T, McLoughlin I, Iedema R, Teede H. An overview of healthcare improvement: Unpacking the complexity for clinicians and managers in a learning health system. Intern Med J. 2020;50:1174–84.

Article PubMed Google Scholar

Alberto IRI, Alberto NRI, Ghosh AK, Jain B, Jayakumar S, Martinez-Martin N, et al. The impact of commercial health datasets on medical research and health-care algorithms. Lancet Digit Health. 2023;5:e288–94.

Article CAS PubMed PubMed Central Google Scholar

Dixon-Woods M. How to improve healthcare improvement—an essay by Mary Dixon-Woods. BMJ. 2019;367: l5514.

Enticott J, Johnson A, Teede H. Learning health systems using data to drive healthcare improvement and impact: A systematic review. BMC Health Serv Res. 2021;21:200.

Enticott JC, Melder A, Johnson A, Jones A, Shaw T, Keech W, et al. A learning health system framework to operationalize health data to improve quality care: An Australian perspective. Front Med (Lausanne). 2021;8:730021.

Dammery G, Ellis LA, Churruca K, Mahadeva J, Lopez F, Carrigan A, et al. The journey to a learning health system in primary care: A qualitative case study utilising an embedded research approach. BMC Prim Care. 2023;24:22.

Foley T, Horwitz L, Zahran R. The learning healthcare project: Realising the potential of learning health systems. 2021. Available from https://learninghealthcareproject.org/wp-content/uploads/2021/05/LHS2021report.pdf . Accessed Jan 2024.

Institute of Medicine. Best care at lower cost: The path to continuously learning health care in America. Washington: The National Academies Press; 2013.

Google Scholar

Zurynski Y, Smith CL, Vedovi A, Ellis LA, Knaggs G, Meulenbroeks I, et al. Mapping the learning health system: A scoping review of current evidence - a white paper. 2020:63

Cadilhac DA, Bravata DM, Bettger J, Mikulik R, Norrving B, Uvere E, et al. Stroke learning health systems: A topical narrative review with case examples. Stroke. 2023;54:1148–59.

Braithwaite J, Glasziou P, Westbrook J. The three numbers you need to know about healthcare: The 60–30-10 challenge. BMC Med. 2020;18:1–8.

Article Google Scholar

King D, Wittenberg R, Patel A, Quayyum Z, Berdunov V, Knapp M. The future incidence, prevalence and costs of stroke in the UK. Age Ageing. 2020;49:277–82.

Ganesh A, Lindsay P, Fang J, Kapral MK, Cote R, Joiner I, et al. Integrated systems of stroke care and reduction in 30-day mortality: A retrospective analysis. Neurology. 2016;86:898–904.

Lowther HJ, Harrison J, Hill JE, Gaskins NJ, Lazo KC, Clegg AJ, et al. The effectiveness of quality improvement collaboratives in improving stroke care and the facilitators and barriers to their implementation: A systematic review. Implement Sci. 2021;16:16.

Australian Commission on Safety and Quality in Health Care. Acute stroke clinical care standard. 2015. Available from https://www.safetyandquality.gov.au/our-work/clinical-care-standards/acute-stroke-clinical-care-standard . Accessed Jan 2024.

Australian Commission on Safety and Quality in Health Care. Acute stroke clinical care standard. Sydney: ACSQHC; 2019. Available from https://www.safetyandquality.gov.au/publications-and-resources/resource-library/acute-stroke-clinical-care-standard-evidence-sources . Accessed Jan 2024.

Ryan O, Ghuliani J, Grabsch B, Hill K, G CC, Breen S, et al. Development, implementation, and evaluation of the Australian Stroke Data Tool (AuSDaT): Comprehensive data capturing for multiple uses. Health Inf Manag. 2022:18333583221117184.

English C, Bayley M, Hill K, Langhorne P, Molag M, Ranta A, et al. Bringing stroke clinical guidelines to life. Int J Stroke. 2019;14:337–9.

English C, Hill K, Cadilhac DA, Hackett ML, Lannin NA, Middleton S, et al. Living clinical guidelines for stroke: Updates, challenges and opportunities. Med J Aust. 2022;216:510–4.

Cadilhac DA, Grimley R, Kilkenny MF, Andrew NE, Lannin NA, Hill K, et al. Multicenter, prospective, controlled, before-and-after, quality improvement study (Stroke123) of acute stroke care. Stroke. 2019;50:1525–30.

Cadilhac DA, Marion V, Andrew NE, Breen SJ, Grabsch B, Purvis T, et al. A stepped-wedge cluster-randomized trial to improve adherence to evidence-based practices for acute stroke management. Jt Comm J Qual Patient Saf. 2022.

Elliott J, Lawrence R, Minx JC, Oladapo OT, Ravaud P, Jeppesen BT, et al. Decision makers need constantly updated evidence synthesis. Nature. 2021;600:383–5.

Article CAS PubMed Google Scholar

National Stroke Foundation. National guidelines for acute stroke management. Melbourne: National Stroke Foundation; 2003.

National Stroke Foundation. Clinical guidelines for stroke rehabilitation and recovery. Melbourne: National Stroke Foundation; 2005.

Phan TG, Thrift A, Cadilhac D, Srikanth V. A plea for the use of systematic review methodology when writing guidelines and timely publication of guidelines. Intern Med J . 2012;42:1369–1371; author reply 1371–1362

Tendal B, Vogel JP, McDonald S, Norris S, Cumpston M, White H, et al. Weekly updates of national living evidence-based guidelines: Methods for the Australian living guidelines for care of people with COVID-19. J Clin Epidemiol. 2021;131:11–21.

Grimshaw JM, Eccles MP, Lavis JN, Hill SJ, Squires JE. Knowledge translation of research findings. Implement Sci. 2012;7:50.

Harris D, Cadilhac D, Hankey GJ, Hillier S, Kilkenny M, Lalor E. National stroke audit: The Australian experience. Clin Audit. 2010;2:25–31.

Cadilhac DA, Purvis T, Kilkenny MF, Longworth M, Mohr K, Pollack M, et al. Evaluation of rural stroke services: Does implementation of coordinators and pathways improve care in rural hospitals? Stroke. 2013;44:2848–53.

Cadilhac DA, Moss KM, Price CJ, Lannin NA, Lim JY, Anderson CS. Pathways to enhancing the quality of stroke care through national data monitoring systems for hospitals. Med J Aust. 2013;199:650–1.

Cadilhac DA, Lannin NA, Anderson CS, Levi CR, Faux S, Price C, et al. Protocol and pilot data for establishing the Australian Stroke Clinical Registry. Int J Stroke. 2010;5:217–26.

Ivers N, Jamtvedt G, Flottorp S, Young J, Odgaard-Jensen J, French S, et al. Audit and feedback: Effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev . 2012

Australian Commission on Safety and Quality in Health Care. Economic evaluation of clinical quality registries. Final report. . 2016:79

Cadilhac DA, Dalli LL, Morrison J, Lester M, Paice K, Moss K, et al. The Australian Stroke Clinical Registry annual report 2021. Melbourne; 2022. Available from https://auscr.com.au/about/annual-reports/ . Accessed 6 May 2024.

Kilkenny MF, Kim J, Andrew NE, Sundararajan V, Thrift AG, Katzenellenbogen JM, et al. Maximising data value and avoiding data waste: A validation study in stroke research. Med J Aust. 2019;210:27–31.

Eliakundu AL, Smith K, Kilkenny MF, Kim J, Bagot KL, Andrew E, et al. Linking data from the Australian Stroke Clinical Registry with ambulance and emergency administrative data in Victoria. Inquiry. 2022;59:469580221102200.

PubMed Google Scholar

Andrew NE, Kim J, Cadilhac DA, Sundararajan V, Thrift AG, Churilov L, et al. Protocol for evaluation of enhanced models of primary care in the management of stroke and other chronic disease (PRECISE): A data linkage healthcare evaluation study. Int J Popul Data Sci. 2019;4:1097.

CAS PubMed PubMed Central Google Scholar

Mosalski S, Shiner CT, Lannin NA, Cadilhac DA, Faux SG, Kim J, et al. Increased relative functional gain and improved stroke outcomes: A linked registry study of the impact of rehabilitation. J Stroke Cerebrovasc Dis. 2021;30: 106015.

Ryan OF, Hancock SL, Marion V, Kelly P, Kilkenny MF, Clissold B, et al. Feedback of aggregate patient-reported outcomes (PROs) data to clinicians and hospital end users: Findings from an Australian codesign workshop process. BMJ Open. 2022;12:e055999.

Grimley RS, Rosbergen IC, Gustafsson L, Horton E, Green T, Cadigan G, et al. Dose and setting of rehabilitation received after stroke in Queensland, Australia: A prospective cohort study. Clin Rehabil. 2020;34:812–23.

Purvis T, Middleton S, Craig LE, Kilkenny MF, Dale S, Hill K, et al. Inclusion of a care bundle for fever, hyperglycaemia and swallow management in a national audit for acute stroke: Evidence of upscale and spread. Implement Sci. 2019;14:87.

Middleton S, McElduff P, Ward J, Grimshaw JM, Dale S, D’Este C, et al. Implementation of evidence-based treatment protocols to manage fever, hyperglycaemia, and swallowing dysfunction in acute stroke (QASC): A cluster randomised controlled trial. Lancet. 2011;378:1699–706.

Middleton S, Dale S, Cheung NW, Cadilhac DA, Grimshaw JM, Levi C, et al. Nurse-initiated acute stroke care in emergency departments. Stroke. 2019:STROKEAHA118020701.

Hood RJ, Maltby S, Keynes A, Kluge MG, Nalivaiko E, Ryan A, et al. Development and pilot implementation of TACTICS VR: A virtual reality-based stroke management workflow training application and training framework. Front Neurol. 2021;12:665808.

Bladin CF, Kim J, Bagot KL, Vu M, Moloczij N, Denisenko S, et al. Improving acute stroke care in regional hospitals: Clinical evaluation of the Victorian Stroke Telemedicine program. Med J Aust. 2020;212:371–7.

Bladin CF, Bagot KL, Vu M, Kim J, Bernard S, Smith K, et al. Real-world, feasibility study to investigate the use of a multidisciplinary app (Pulsara) to improve prehospital communication and timelines for acute stroke/STEMI care. BMJ Open. 2022;12:e052332.

Zhao H, Coote S, Easton D, Langenberg F, Stephenson M, Smith K, et al. Melbourne mobile stroke unit and reperfusion therapy: Greater clinical impact of thrombectomy than thrombolysis. Stroke. 2020;51:922–30.

Purvis T, Cadilhac DA, Hill K, Reyneke M, Olaiya MT, Dalli LL, et al. Twenty years of monitoring acute stroke care in Australia from the national stroke audit program (1999–2019): Achievements and areas of future focus. J Health Serv Res Policy. 2023.

Cadilhac DA, Purvis T, Reyneke M, Dalli LL, Kim J, Kilkenny MF. Evaluation of the national stroke audit program: 20-year report. Melbourne; 2019.

Kim J, Tan E, Gao L, Moodie M, Dewey HM, Bagot KL, et al. Cost-effectiveness of the Victorian Stroke Telemedicine program. Aust Health Rev. 2022;46:294–301.

Kim J, Easton D, Zhao H, Coote S, Sookram G, Smith K, et al. Economic evaluation of the Melbourne mobile stroke unit. Int J Stroke. 2021;16:466–75.

Stroke Foundation. National stroke audit – rehabilitation services report 2020. Melbourne; 2020.

Australian Institute of Health and Welfare. Heart, stroke and vascular disease: Australian facts. 2023. Webpage https://www.aihw.gov.au/reports/heart-stroke-vascular-diseases/hsvd-facts/contents/about (accessed Jan 2024).

Download references

Acknowledgements

The following authors hold National Health and Medical Research Council Research Fellowships: HT (#2009326), DAC (#1154273), SM (#1196352), MFK Future Leader Research Fellowship (National Heart Foundation #105737). The Funders of this work did not have any direct role in the design of the study, its execution, analyses, interpretation of the data, or decision to submit results for publication.

Author information

Helena Teede and Dominique A. Cadilhac contributed equally.

Authors and Affiliations

Monash Centre for Health Research and Implementation, 43-51 Kanooka Grove, Clayton, VIC, Australia

Helena Teede, Emily Callander & Joanne Enticott

Monash Partners Academic Health Science Centre, 43-51 Kanooka Grove, Clayton, VIC, Australia

Helena Teede & Alison Johnson

Stroke and Ageing Research, Department of Medicine, School of Clinical Sciences at Monash Health, Monash University, Level 2 Monash University Research, Victorian Heart Hospital, 631 Blackburn Rd, Clayton, VIC, Australia

Dominique A. Cadilhac, Tara Purvis & Monique F. Kilkenny

Stroke Theme, The Florey Institute of Neuroscience and Mental Health, University of Melbourne, Heidelberg, VIC, Australia

Dominique A. Cadilhac, Monique F. Kilkenny & Bruce C.V. Campbell

Department of Neurology, Melbourne Brain Centre, Royal Melbourne Hospital, Parkville, VIC, Australia

Bruce C.V. Campbell

Department of Medicine, Faculty of Medicine, Dentistry and Health Sciences, University of Melbourne, Victoria, Australia

School of Health Sciences, Heart and Stroke Program, University of Newcastle, Hunter Medical Research Institute, University Drive, Callaghan, NSW, Australia

Coralie English

School of Medicine and Dentistry, Griffith University, Birtinya, QLD, Australia

Rohan S. Grimley

Clinical Excellence Division, Queensland Health, Brisbane, Australia

John Hunter Hospital, Hunter New England Local Health District and University of Newcastle, Sydney, NSW, Australia

Christopher Levi

School of Nursing, Midwifery and Paramedicine, Australian Catholic University, Sydney, NSW, Australia

Sandy Middleton

Nursing Research Institute, St Vincent’s Health Network Sydney and and Australian Catholic University, Sydney, NSW, Australia

Stroke Foundation, Level 7, 461 Bourke St, Melbourne, VIC, Australia

Kelvin Hill

You can also search for this author in PubMed Google Scholar

Contributions

HT: conception, design and initial draft, developed the theoretical formalism for learning health system framework, approved the submitted version. DAC: conception, design and initial draft, provided essential literature and case study examples, approved the submitted version. TP: revised the manuscript critically for important intellectual content, approved the submitted version. MFK: revised the manuscript critically for important intellectual content, provided essential literature and case study examples, approved the submitted version. BC: revised the manuscript critically for important intellectual content, provided essential literature and case study examples, approved the submitted version. CE: revised the manuscript critically for important intellectual content, provided essential literature and case study examples, approved the submitted version. AJ: conception, design and initial draft, developed the theoretical formalism for learning health system framework, approved the submitted version. EC: revised the manuscript critically for important intellectual content, approved the submitted version. RSG: revised the manuscript critically for important intellectual content, provided essential literature and case study examples, approved the submitted version. CL: revised the manuscript critically for important intellectual content, provided essential literature and case study examples, approved the submitted version. SM: revised the manuscript critically for important intellectual content, provided essential literature and case study examples, approved the submitted version. KH: revised the manuscript critically for important intellectual content, provided essential literature and case study examples, approved the submitted version. JE: conception, design and initial draft, developed the theoretical formalism for learning health system framework, approved the submitted version. All authors read and approved the final manuscript.

Authors’ Twitter handles

@HelenaTeede

@DominiqueCad

@Coralie_English

@EmilyCallander

@EnticottJo

Corresponding authors

Correspondence to Helena Teede or Dominique A. Cadilhac .

Ethics declarations

Ethics approval and consent to participate, consent for publication, competing interests, additional information, publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Teede, H., Cadilhac, D.A., Purvis, T. et al. Learning together for better health using an evidence-based Learning Health System framework: a case study in stroke. BMC Med 22 , 198 (2024). https://doi.org/10.1186/s12916-024-03416-w

Download citation

Received : 23 July 2023

Accepted : 30 April 2024

Published : 15 May 2024

DOI : https://doi.org/10.1186/s12916-024-03416-w

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Evidence-based medicine
Person-centred care
Models of care
Healthcare improvement

BMC Medicine

ISSN: 1741-7015

Submission enquiries: [email protected]
General enquiries: [email protected]

Innovation & Enterprise

For students
For businesses

Case studies

UCL startup ZNotes helps millions of students globally

Education technology and social impact startup ZNotes, founded by UCL alumnus Zubair Junjunia, has evolved into a global force, reaching over 5 million students.

21 May 2024

ZNotes addresses educational inequalities by giving millions of students across 190 countries unrestricted access to high-quality educational content. The platform has developed into a global community and empowers learners to become active contributors in the education system.

In 2021, Zubair received the Diana Legacy Award, in recognition of his exceptional contribution to education. Named after Diana, Princess of Wales, it’s recognised as the highest accolade a young person can receive for their social and humanitarian efforts.

Alongside this, in 2023 Zubair was appointed a Generation17 Young Leader by the United Nations Development Programme and Samsung for ZNotes' scale of impact on the UN Sustainable Development Goal 4 (quality education).

The business has been recognised by other global organisations too, including One Young World. And in 2023 Zubair became the world’s youngest recipient of the Freedom of the City of London.

Aged just 23, Zubair was also one of the youngest commencement speakers in the history of UCL at the 2023 Graduation Ceremony.

From informal beginnings to global movement

ZNotes traces its roots back to Zubair seeing huge education disparities himself in Saudi Arabia a decade ago, during his first set of high stakes exams, the International General Certificate of Secondary Education (IGCSEs).

A tech enthusiast, diligent student and copious notetaker, he decided to build a site to post his class notes so that anyone with an internet connection could freely access them. He describes the initial idea as simply being “the digital version of the age-old practice of group study”.

“I didn’t plan for ZNotes to be anything but a way to share my resources with other students. The pivotal moment really came when other students decided to help as well. I’m prouder of this more than anything else - young people dedicating their time by contributing notes and supporting each other through this growing ecosystem.”

Since those informal beginnings ten years ago, and becoming a registered organisation in 2020, ZNotes has now grown into a unique, community-powered learning platform. Over five million people have visited the site to access content since launch, and there are now more than 500,000 registered users.

Taking the offering global

Zubair describes his journey with ZNotes as “shifting up a gear” during the second year of his master’s in Mathematics at UCL. With exponential growth on the platform, he realised he needed some help evolving ZNotes from a passion project to a startup.

“I approached UCL Innovation & Enterprise and successfully applied for their UCL Launch programme (now called ‘Build your business or social enterprise’).

“I learned a whole host of things on that first course, from the legal side, to financing,” he explains. “But more than anything it gave me the confidence to speak about what I was doing and share our ambitions. From there, the startup and impact continued to scale.”

ZNotes was part of the Hatchery incubator at BaseKX, UCL’s dedicated entrepreneurship hub, managed by UCL Innovation & Enterprise. BaseKX plays host to a vibrant community of the university’s most promising startups by providing free, tailored support and dedicated office space.

Staff from UCL Innovation & Enterprise also helped Zubair in obtaining a Tier 1 Entrepreneur visa, to enable him to continue growing the company in the UK after graduating.

The future of education is youth-led

Zubair is keen to underline ZNotes’ community-led approach, which empowers learners to co-create resources, engage in peer-to-peer learning, and contribute to a robust content eco-system.

“Built with, and alongside, students we’ve developed a community-powered learning platform, where selected authors from all over the world can add content directly. We also integrated Discord to cultivate the community and enable peer learning.”

“We ran a survey of 23,000 students in 2023, to assess the impact of ZNotes on people’s education. 91% cited our platform as being crucial or highly useful for their academic success and 87% overshot or achieved their self-set target grades. Interns reported a 77% improvement in job candidacy, and contributors saw a 15% increase in importance serving marginalised communities.”

Supporting young people to make a difference

With over 100 contributors, 70 interns and 40 ambassadors, ZNotes has also become a catalyst for young people who want to make a difference.

“ZNotes ensures every student has a level playing field when it comes to high-stakes exams. But it does much more than that. It inspires and empowers young people to become leaders and changemakers. 82% percent of our contributors have engaged with more community initiatives since being involved with ZNotes. And one in five of our interns have launched their own initiative or joined another one tackling a social problem.

“Our vision now is to reach and impact 100 million learners in the most under-resourced education systems around the world. Given how rapidly ZNotes has grown, we’re determined to keep scaling, so that learners all over the planet have access to the high quality educational resources they need to thrive. And that young people themselves can get involved as agents of change and be part of this global movement.”

Watch a video about ZNotes

YouTube Widget Placeholder https://youtu.be/FKdE_-WuYP4

Find out more about:

Entrepreneurship support for UCL students and graduates
Zubair’s UCL Commencement speech: a call to action for people and our planet
Meet the Generation17 Young Leaders: The Story of Zubair Junjunia

Image © ZNotes

Email: [email protected]

Meet the team

Funnelback feed: https://cms-feed.ucl.ac.uk/s/search.json?collection=drupal-office-vice-p... Double click the feed URL above to edit

IMAGES

TIPC POLICY BRIEF: Formative Evaluation of Transformative Innovation
Nike case-study innovation-and-change-management
Section 5
innovation strategy case study
(PDF) Innovation policy: Rationales, lessons and challenges
(PDF) Introduction: Creative Industries and Innovation Policy

VIDEO

Young Producer Symposium: A Case Study: Innovation Through Diversity
MIPLM Case Study Innovation Management 1
Thesis & Independent Study Innovation Award 2024
Innovation Strategy
IDT- Module 1- Case study- Innovation and culture-Explore presentations designers across globe
Entrepreneurship Part 2 (Business Models & Business Innovation)

COMMENTS

Mechanisms and Impacts of Innovation Policy
Mechanisms and Impacts of Innovation Policy. The importance of innovation to job creation and economic growth — especially in young, high-growth firms — is widely accepted among economists as well as members of the business and policy communities. There is also a recognition that, at least at some times or in certain settings, the private ...
Transformative innovation policy: A systematic review
Smart specialization strategies as a case of mission-oriented policy-a case study on the emergence of new policy practices: Industrial and Corporate Change-Agenda-setting, policy formulation, implementation and monitoring and evaluation: Foray: 2018: On sector-non-neutral innovation policy: towards new design principles: Journal of Evolutionary ...
Innovation Strategy: Articles, Research, & Case Studies on Innovation
This study provides robust econometric evidence for how immigrant inventors shape the innovation dynamics of their receiving countries. Countries receiving inventors from other nations that specialize in patenting particular technologies are more likely to have a significant increase in patent applications of the same technology.
PDF Policy capacities for transformative innovation policy: A case study of
Policy capacities for transformative innovation policy: A case study of UK Research and Innovation. Julie McLaren and Rainer Kattel . Abstract . The 2008 financial collapse, climate crisis and COVID-19 have raised many questions about the sustainability of current systems and how to rebalance the impacts these have
Cities and regions in transformation: transformative innovation policy
Policy capacities for transformative innovation policy: A case study of UK Research and Innovation. UCL Institute for Innovation and Public Purpose, Working Paper Series (IIPP WP 2020-04). Moujaes, G. (2024). Moving to Smart Specialization for sustainability: the implications on the design of monitoring indicators, Science and Public Policy, 51 ...
Government‐led innovation acceleration: Case studies of US federal
It is difficult to make sweeping claims about innovation based on three case studies of government support for innovation for national security or domestic public policy. All of these programs are subject to lobbies asking government for support for their industries, often with a high degree of secrecy in military or intelligence fields.
Innovation Policy: Rationales, Lessons and Challenges
Innovation policy has emerged as a new field of economic policy during the last few decades. This paper explores the rationales for national innovation policies, as laid out in the existing literature on the subject, and considers what the lessons and challenges for theory and practice in this area are. Innovation policy attempts to influence ...
An Introduction to Research and Innovation Policy Design
It begins with a conceptual and operational definition of Research & Innovation (R&I) policy, which is defined as a generalized case of complex governance arrangements. Then it moves on to a review of the literature on the determinants of policy design choice. Here the author discusses the relevance of partisan politics, institutional layouts ...
Policy capacities for transformative innovation policy: A case study of
Rainer Kattel | UCL Institute for Innovation and Public Purpose; Reference: McLaren, J. and Kattel, R. (2022). Policy capacities for transformative innovation policy: A case study of UK Research and Innovation. UCL Institute for Innovation and Public Purpose, Working Paper Series (IIPP WP 2020-04).
Governance of Innovation Systems: Volume 2 : Case Studies in Innovation
This book presents case studies of governance of innovation policy in selected OECD countries. It reviews the ongoing changes in these countries with a focus on providing an analysis of governance challenges, institutional changes and policy learning practices. The book provides fresh insight into the emerging third-generation of innovation ...
U.S. Science and Innovation Policy
Established in 2019, the Roundtable on Black Men and Black Women in STEM convenes a broad array of stakeholders that focus on the barriers and opportunities encountered by Black men and Black women as they navigate the pathways from K-12 and postsecondary education to careers in science, engineering and medicine. U.S. Science and Innovation ...
Lessons from Tesla's Approach to Innovation
Lessons from Tesla's Approach to Innovation. Summary. Tesla has shifted the auto industry toward electric vehicles, achieved consistently growing revenues, and at the start of 2020 was the ...
The role of policy entrepreneurs in defining directions of innovation
4. Methodology and case study. To explore how and by whom directions are determined in innovation policy, we carried out qualitative research using the MS framework. We opted for a single in-depth case study, common in innovation and transition studies. It allows researchers to unfold complex causation in particular geographical and institutional contexts (Köhler et al., 2019).
Experimental Innovation Policy: Innovation Policy and the Economy: Vol 20
Experimental approaches are increasingly being adopted across many policy fields, but innovation policy has been lagging. This paper reviews the case for policy experimentation in this field, describes the different types of experiments that can be undertaken, discusses some of the unique challenges to the use of experimental approaches in innovation policy, and summarizes some of the emerging ...
Transformative Innovation Policy: A Case- Study from China
In this. paper, we will focus on this new policy approach - using one example from China as a main case-study. In a closely-related discussion, Schot and Steinmueller (2018) have described ...
Policy innovation lab scholarship: past, present, and the future
2. State of the policy innovation lab scholarship. Many theoretical policy frameworks have been employed to explain the rise of policy innovation labs and policy "labification," including design thinking, experimental government, and collaborative governance (Andersen, Kelemen, and Matzdorf Citation 2020).For example, it is argued that the role of design thinking in policymaking may lead ...
Ten Types of Innovation: 30 new case studies for 2019
3) Structure: How you organize and align your talent and assets. Structure innovations are focused on organizing company assets—hard, human, or intangible—in unique ways that create value. They can include everything from superior talent management systems to ingenious configurations of heavy capital equipment.
Exploring Policy Innovation: Tools, Techniques & Approaches
This resource is an introductory description of a paradigmatic shift in public policy-making from traditional methods to new, innovative approaches. This includes a whole-of-government approach, a refocus on the user, an embrace of complexity, pro-activeness, and collaboration. It describes 7 opportunity areas for innovation in the traditional policy-making process.
PDF Innovation in Policy-making in India: A Multi Case Study Analysis
The author has followed a multi-case study analysis following a Grounded Theory approach as posited by Barney Glaser to develop a preliminary model to create a ... Taking this as working definition of policy innovation, the study was undertaken to understand the phenomenon in India. The objective of the study was to find out if there had been any
Case Selections
Innovation happens at non-tech companies too. In this classic case from the early 2000s, Colombian coffee entrepreneurs attempt to revive Colombia's famous Juan Valdez brand in the age of Starbucks.
Innovation Ecosystems in the EU: Policy Evolution and Horizon ...
EU innovation policy is regarded as a cornerstone in the evolution of configuration of innovation ecosystems. Thus, the aim of the present document is to carry on the European policy case study, applying the proposed analytical framework, on the key innovation policy document, i.e., Horizon Europe FP proposal. Firstly, the general history of ...
Top 40 Most Popular Case Studies of 2021
Fifty four percent of raw case users came from outside the U.S.. The Yale School of Management (SOM) case study directory pages received over 160K page views from 177 countries with approximately a third originating in India followed by the U.S. and the Philippines. Twenty-six of the cases in the list are raw cases.
Policy Reinvention and Diffusion: Evidence from Chinese ...
In the past half century, the diffusion of policy innovation has been a major research topic in academia, especially in research on American politics and public policy [3, 5, 21].The study of diffusion of policy innovation as an objective political phenomenon or as a theory of policy process has been improved in recent decades, especially in the twenty-first century, and it has accumulated ...
The Fracking Revolution: Shale Gas as a Case Study in Innovation Policy
The early twenty-first century has witnessed a boom in oil and natural gas production that promises to turn the United States into a new form of petrostate. This boom raises various questions that scholars have begun to explore, including questions of risk governance, federalism, and export policy. Relatively neglected, however, have been questions of why the technological revolution behind ...
PDF Evidence Based Public Policy Making: A Comparative Case Study Analysis
The State of Washington, led by Governor Jay Inslee, implemented the Results Washington initiative in 2014, which emphasized five goals — "World Class Education," "Healthy and Safe Communities," "Effective," and "Accountable Government.". The initiative sought to improve the government operational system and public sector ...
Unleashing Innovation in Healthcare: How Automation is Transforming
Case Study: Transforming Cancer Care in West Yorkshire . e18 Innovation has been working in partnership with 52 NHS organisations across 17 Integrated Care Systems (ICS), from acute trusts, to community & mental health providers, and primary care networks to help bring intelligent automation solutions to the UK healthcare system.
Innovation Report 2024
Our 2024 Innovation Report, the Power of (Making) Futures, outlines nine innovative initiatives, categorised into three themes which are; rebalancing power dynamics in narratives and imagery; participatory funding models; and innovative approaches to development cooperation. Moreover, the report offers recommendations, drawn from shared success ...
Learning together for better health using an evidence-based Learning
In the context of expanding digital health tools, the health system is ready for Learning Health System (LHS) models. These models, with proper governance and stakeholder engagement, enable the integration of digital infrastructure to provide feedback to all relevant parties including clinicians and consumers on performance against best practice standards, as well as fostering innovation and ...
UCL startup ZNotes helps millions of students globally
A tech enthusiast, diligent student and copious notetaker, he decided to build a site to post his class notes so that anyone with an internet connection could freely access them. He describes the initial idea as simply being "the digital version of the age-old practice of group study".
AFRY advances innovation in mining with SolarWinds Hybrid Cloud
View Case Study Pine Labs Eliminates Tool Sprawl and Is On Track to Reduce Mean Time to Resolution by 60-70% SolarWinds Hybrid Cloud Observability enabled Pine Labs greater visibility into its IT environment and has eliminated tool sprawl, decreased operating costs, improved mean time to discovery, and reduced mean time to resolution of issues.