program verification in problem solving techniques

Introduction

Introduction #.

This course offers a hands-on introduction to the usage, construction, and theory of automated deductive program verifiers. Students should acquire the skills to apply and extend tool-supported methodologies for developing proven-correct software.

Why Program Verification? #

One of the most fundamental questions in computer science is whether our programs are correct. In fact, this concern is as old as computers: Alan Turing already asked in 1949 1

“how can one check a routine in the sense of making sure that it is right?”

Early on, we learn that we should test our programs to increase our confidence in their correctness. However, as adequately summarized by Turing Award winner Edsgar W. Dijkstra 2 ,

“program testing can be very effective to show the presence of bugs, but it is hopelessly inadequate for showing their absence.”

Dijkstra went on to note that

“the only effective way to raise the confidence level of a program is to give a convincing proof of its correctness.”

To judge whether a program is “correct”, we assume that there is (at least implicitly) a specification describing the properties a program should have.

The goal of program verification is then to prove that a program satisfies its specification.

Informal examples of specifications include:

the program produces a sorted copy of the given input array;

the program returns a binary search tree;

the program traverses a binary search tree in-order without changing it;

the program never attempts to dereference a null reference;

the program never attempts to evaluate ill-typed expressions (e.g. dividing a real number by a string);

the program eventually terminates;

the program terminates within one minute; and

the program never encounters a race condition.

For the purpose of proving that a program satisfies its specification, we typically need more detailed formal specifications.

Flavors of Program Verification #

There are many different program verification techniques with different characteristics such as:

what kinds of programs are targeted?

what kind of properties are considered and how are they specified?

how much interaction or help from humans is required to write a proof?

For example, the absence of typing errors can be viewed as an implicit part of a program’s specification. During compilation, a type checker verifies that a program satisfies the specification “is this program well-typed?” – the compiler then raises an error if the type checker fails to prove well-typedness. Type checking is a mostly automated verification technique but might require some interaction: programmers sometimes have to add type annotations to resolve tricky cases that the typer checker cannot deal with on its own. This is typical for verification techniques: proving more complex specifications often requires user-supplied hints.

Our Flavor of Program Verification #

This course is concerned with the principles and pragmatics of deductive program verification ­­­­- techniques for constructing proofs in a machine-checkable , compositional , and automated fashion. All of the highlighted aspects play an important role:

Deductive reasoning : we construct proofs by drawing sound conclusions from well-understood assumptions.

Machine-checkable : our proofs must be machine-checkable to rule out human errors and allow for changes to our programs without having to write a new proof (on a piece of paper) again.

Compositional : to scale to whole software projects, we derive proofs for large programs from existing proofs for smaller programs. Moreover, changing aspects of one software component should not affect the correctness of other components that do not depend on it.

Automated : while fully automated verification is, in general, not possible (due to known undecidability results), we want to require as little interaction with programmers as possible. Ideally, programmers should only specify the desirable properties a program and supply a few hints (like type annotations).

What will we do in the course? #

The course will cover the technology stack underlying modern deductive verification tools from the bottom up by introducing a verifier for a small core language and then progressively enriching the language with advanced features such as a mutable heap, objects, and concurrency. For each language extension, it will explain the necessary reasoning principles, specification techniques, pragmatics, and tool support. In particular, the course will introduce:

program logics for writing formal correctness proofs;

SMT solvers for automating reasoning about logical formulas and, thus, program proofs;

intermediate verification languages in which we can encode verification methodologies targeting, for example, recursive procedures, loops, termination, mathematical datatypes, mutable heap, object-oriented programs, and concurrency; and

source code verifiers for handling feature-rich programming languages.

Examples of verification technology in the wild #

Ada Spark is a programming language for safety-critical application domains, such as software for airplanes, that comes with a verification toolset.

The highest levels of the Automotive Safety Integrity Level (ASIL D) strongly recommend that formal verification is applied to software implementations.

Program Verification at Microsoft

Formal Verification in Amazon’s Automated Reasoning Group

Alan M. Turing. On checking a large routine. Report of a Conference on High Speed Automatic Calculating Machines, pages 67–69, 1949.

Edsger W. Dijkstra: The humble programmer, CACM 1972

Eight Disciplines of Problem Solving (8D)

– Eight Disciplines of Problem Solving –

⇓   Introduction to 8D

⇓   What is 8D

⇓   Why Apply 8D

⇓   When to Apply 8D

⇓   How to Apply 8D

Introduction to Eight Disciplines of Problem Solving (8D)

The Eight Disciplines of Problem Solving (8D) is a problem solving methodology designed to find the root cause of a problem, devise a short-term fix and implement a long-term solution to prevent recurring problems. When it’s clear that your product is defective or isn’t satisfying your customers, an 8D is an excellent first step to improving Quality and Reliability.

Ford Motor Company developed this problem solving methodology, then known as Team Oriented Problem Solving (TOPS), in the 1980s. The early usage of 8D proved so effective that it was adopted by Ford as the primary method of documenting problem solving efforts, and the company continues to use 8D today.

8D has become very popular among manufacturers because it is effective and reasonably easy to teach. Below you’ll find the benefits of an 8D, when it is appropriate to perform and how it is performed.

What is Eight Disciplines of Problem Solving (8D)

The 8D problem solving process is a detailed, team oriented approach to solving critical problems in the production process. The goals of this method are to find the root cause of a problem, develop containment actions to protect customers and take corrective action to prevent similar problems in the future.

The strength of the 8D process lies in its structure, discipline and methodology. 8D uses a composite methodology, utilizing best practices from various existing approaches. It is a problem solving method that drives systemic change, improving an entire process in order to avoid not only the problem at hand but also other issues that may stem from a systemic failure.

8D has grown to be one of the most popular problem solving methodologies used for Manufacturing, Assembly and Services around the globe. Read on to learn about the reasons why the Eight Disciplines of Problem Solving may be a good fit for your company.

Why Apply Eight Disciplines of Problem Solving (8D)

The 8D methodology is so popular in part because it offers your engineering team a consistent, easy-to-learn and thorough approach to solving whatever problems might arise at various stages in your production process. When properly applied, you can expect the following benefits:

• Improved team oriented problem solving skills rather than reliance on the individual
• Increased familiarity with a structure for problem solving
• Creation and expansion of a database of past failures and lessons learned to prevent problems in the future
• Better understanding of how to use basic statistical tools required for problem solving
• Improved effectiveness and efficiency at problem solving
• A practical understanding of Root Cause Analysis (RCA)
• Problem solving effort may be adopted into the processes and methods of the organization
• Improved skills for implementing corrective action
• Better ability to identify necessary systemic changes and subsequent inputs for change
• More candid and open communication in problem solving discussion, increasing effectiveness
• An improvement in management’s understanding of problems and problem resolution

8D was created to represent the best practices in problem solving. When performed correctly, this methodology not only improves the Quality and Reliability of your products but also prepares your engineering team for future problems.

When to Apply Eight Disciplines of Problem Solving (8D)

The 8D problem solving process is typically required when:

• Safety or Regulatory issues has been discovered
• Customer complaints are received
• Warranty Concerns have indicated greater-than-expected failure rates
• Internal rejects, waste, scrap, poor performance or test failures are present at unacceptable levels

How to Apply Eight Disciplines of Problem Solving (8D)

The 8D process alternates inductive and deductive problem solving tools to relentlessly move forward toward a solution. The Quality-One approach uses a core team of three individuals for inductive activities with data driven tools and then a larger Subject Matter Expert (SME) group for the deductive activities through brainstorming, data-gathering and experimentation.

D0: Prepare and Plan for the 8D

Proper planning will always translate to a better start. Thus, before 8D analysis begins, it is always a good idea to ask an expert first for their impressions. After receiving feedback, the following criterion should be applied prior to forming a team:

Collect information on the symptoms

Use a Symptoms Checklist to ask the correct questions

Identify the need for an Emergency Response Action (ERA), which protects the customer from further exposure to the undesired symptoms

D1: Form a Team

A Cross Functional Team (CFT) is made up of members from many disciplines. Quality-One takes this principle one step further by having two levels of CFT:

• The Core Team Structure should involve three people on the respective subjects: product, process and data
• Additional Subject Matter Experts are brought in at various times to assist with brainstorming, data collection and analysis

Teams require proper preparation. Setting the ground rules is paramount. Implementation of disciplines like checklists, forms and techniques will ensure steady progress.  8D must always have two key members: a Leader and a Champion / Sponsor:

• The Leader is the person who knows the 8D process and can lead the team through it (although not always the most knowledgeable about the problem being studied)
• The Champion or Sponsor is the one person who can affect change by agreeing with the findings and can provide final approval on such changes

D2: Describe the Problem

The 8D method’s initial focus is to properly describe the problem utilizing the known data and placing it into specific categories for future comparisons. The “Is” data supports the facts whereas the “Is Not” data does not. As the “Is Not” data is collected, many possible reasons for failure are able to be eliminated. This approach utilizes the following tools:

• Problem Statement
• Affinity Diagram (Deductive tool)
• Fishbone/Ishikawa Diagram (Deductive tool)
• Problem Description

D3: Interim Containment Action

In the interim, before the permanent corrective action has been determined, an action to protect the customer can be taken. The Interim Containment Action (ICA) is temporary and is typically removed after the Permanent Correct Action (PCA) is taken.

• Verification of effectiveness of the ICA is always recommended to prevent any additional customer dissatisfaction calls

D4: Root Cause Analysis (RCA) and Escape Point

The root cause must be identified to take permanent action to eliminate it. The root cause definition requires that it can be turned on or off, at will. Activities in D4 include:

• Comparative Analysis listing differences and changes between “Is” and “Is Not”
• Development of Root Cause Theories based on remaining items
• Verification of the Root Cause through data collection
• Review Process Flow Diagram for location of the root cause
• Determine Escape Point, which is the closest point in the process where the root cause could have been found but was not

D5: Permanent Corrective Action (PCA)

The PCA is directed toward the root cause and removes / changes the conditions of the product or process that was responsible for the problem. Activities in D5 include:

• Establish the Acceptance Criteria which include Mandatory Requirements and Wants
• Perform a Risk Assessment /  Failure Mode and Effects Analysis (FMEA) on the PCA choices
• Based on risk assessment, make a balanced choice for PCA
• Select control-point improvement for the Escape Point
• Verification of Effectiveness for both the PCA and the Escape Point are required

D6: Implement and Validate the Permanent Corrective Action

To successfully implement a permanent change, proper planning is essential. A project plan should encompass: communication, steps to complete, measurement of success and lessons learned. Activities in D6 include:

• Develop Project Plan for Implementation
• Communicate the plan to all stakeholders
• Validation of improvements using measurement

D7: Prevent Recurrence

D7 affords the opportunity to preserve and share the knowledge, preventing problems on similar products, processes, locations or families. Updating documents and procedures / work instructions are expected at this step to improve future use. Activities in D7 include:

• Review Similar Products and Processes for problem prevention
• Develop / Update Procedures and Work Instructions for Systems Prevention
• Capture Standard Work / Practice and reuse
• Assure FMEA updates have been completed
• Assure Control Plans have been updated

D8: Closure and Team Celebration

Teams require feedback to allow for satisfactory closure. Recognizing both team and individual efforts and allowing the team to see the previous and new state solidifies the value of the 8D process. Activities in D8 include:

• Archive the 8D Documents for future reference
• Document Lessons Learned on how to make problem solving better
• Before and After Comparison of issue
• Celebrate Successful Completion

8D and Root Cause Analysis (RCA)

The 8D process has Root Cause Analysis (RCA) imbedded within it. All problem solving techniques include RCA within their structure. The steps and techniques within 8D which correspond to Root Cause Analysis are as follows:

• Problem Symptom is quantified and converted to “Object and Defect”
• Problem Symptom is converted to Problem Statement using Repeated Whys
• Possible and Potential Causes are collected using deductive tools (i.e. Fishbone or Affinity Diagram)
• Problem Statement is converted into Problem Description using Is / Is Not
• Problem Description reduces the number of items on the deductive tool (from step 3)
• Comparative Analysis between the Is and Is Not items (note changes and time)
• Root Cause theories are developed from remaining possible causes on deductive tool and coupled with changes from Is / Is Not
• Compare theories with current data and develop experiments for Root Cause Verification
• Test and confirm the Root Causes

Example: Multiple Why Technique

The Multiple / Repeated Why (Similar to 5 Why) is an inductive tool, which means facts are required to proceed to a more detailed level. The steps required to determine problem statement are:

• Problem Symptom is defined as an Object and Defect i.e. “Passenger Injury”
• Why? In every case “SUV’s Roll Over”
• Why? In every case, it was preceded by a “Blown Tire”
• Why? Many explanations may be applied, therefore the team cannot continue with another repeated why past “Blown Tire”
• Therefore, the Problem Statement is “Blown Tire”
• Why? Low (Air) Pressure, Tire Defect (Degradation of an Interface) and High (Ambient) Temperature
• Counter measures assigned to low pressure and tire defect

This example uses only 4 of the 5 Whys to determine the root causes without going further into the systemic reasons that supported the failure. The Repeated Why is one way to depict this failure chain. Fault Tree Analysis (FTA) could also be used.

Learn More About Eight Disciplines of Problem Solving (8D)

Quality-One offers Quality and Reliability Support for Product and Process Development through Consulting, Training and Project Support. Quality-One provides Knowledge, Guidance and Direction in Quality and Reliability activities, tailored to your unique wants, needs and desires. Let us help you Discover the Value of 8D Consulting , 8D Training or 8D Project Support .

Contact Us | Discover the Value!

(248) 280-4800 | [email protected]

Remember Me

• Don't have an account? Register
• Lost your password? Click here
• Already have an account? Log in

What is Problem Solving? (Steps, Techniques, Examples)

By Status.net Editorial Team on May 7, 2023 — 5 minutes to read

What Is Problem Solving?

Definition and importance.

Problem solving is the process of finding solutions to obstacles or challenges you encounter in your life or work. It is a crucial skill that allows you to tackle complex situations, adapt to changes, and overcome difficulties with ease. Mastering this ability will contribute to both your personal and professional growth, leading to more successful outcomes and better decision-making.

Problem-Solving Steps

The problem-solving process typically includes the following steps:

• Identify the issue : Recognize the problem that needs to be solved.
• Analyze the situation : Examine the issue in depth, gather all relevant information, and consider any limitations or constraints that may be present.
• Generate potential solutions : Brainstorm a list of possible solutions to the issue, without immediately judging or evaluating them.
• Evaluate options : Weigh the pros and cons of each potential solution, considering factors such as feasibility, effectiveness, and potential risks.
• Select the best solution : Choose the option that best addresses the problem and aligns with your objectives.
• Implement the solution : Put the selected solution into action and monitor the results to ensure it resolves the issue.
• Review and learn : Reflect on the problem-solving process, identify any improvements or adjustments that can be made, and apply these learnings to future situations.

Defining the Problem

To start tackling a problem, first, identify and understand it. Analyzing the issue thoroughly helps to clarify its scope and nature. Ask questions to gather information and consider the problem from various angles. Some strategies to define the problem include:

• Brainstorming with others
• Asking the 5 Ws and 1 H (Who, What, When, Where, Why, and How)
• Analyzing cause and effect
• Creating a problem statement

Generating Solutions

Once the problem is clearly understood, brainstorm possible solutions. Think creatively and keep an open mind, as well as considering lessons from past experiences. Consider:

• Creating a list of potential ideas to solve the problem
• Grouping and categorizing similar solutions
• Prioritizing potential solutions based on feasibility, cost, and resources required
• Involving others to share diverse opinions and inputs

Evaluating and Selecting Solutions

Evaluate each potential solution, weighing its pros and cons. To facilitate decision-making, use techniques such as:

• SWOT analysis (Strengths, Weaknesses, Opportunities, Threats)
• Decision-making matrices
• Pros and cons lists
• Risk assessments

After evaluating, choose the most suitable solution based on effectiveness, cost, and time constraints.

Implementing and Monitoring the Solution

Implement the chosen solution and monitor its progress. Key actions include:

• Communicating the solution to relevant parties
• Setting timelines and milestones
• Assigning tasks and responsibilities
• Monitoring the solution and making adjustments as necessary
• Evaluating the effectiveness of the solution after implementation

Utilize feedback from stakeholders and consider potential improvements. Remember that problem-solving is an ongoing process that can always be refined and enhanced.

Problem-Solving Techniques

During each step, you may find it helpful to utilize various problem-solving techniques, such as:

• Brainstorming : A free-flowing, open-minded session where ideas are generated and listed without judgment, to encourage creativity and innovative thinking.
• Root cause analysis : A method that explores the underlying causes of a problem to find the most effective solution rather than addressing superficial symptoms.
• SWOT analysis : A tool used to evaluate the strengths, weaknesses, opportunities, and threats related to a problem or decision, providing a comprehensive view of the situation.
• Mind mapping : A visual technique that uses diagrams to organize and connect ideas, helping to identify patterns, relationships, and possible solutions.

Brainstorming

When facing a problem, start by conducting a brainstorming session. Gather your team and encourage an open discussion where everyone contributes ideas, no matter how outlandish they may seem. This helps you:

• Generate a diverse range of solutions
• Encourage all team members to participate
• Foster creative thinking

When brainstorming, remember to:

• Reserve judgment until the session is over
• Encourage wild ideas
• Combine and improve upon ideas

Root Cause Analysis

For effective problem-solving, identifying the root cause of the issue at hand is crucial. Try these methods:

• 5 Whys : Ask “why” five times to get to the underlying cause.
• Fishbone Diagram : Create a diagram representing the problem and break it down into categories of potential causes.
• Pareto Analysis : Determine the few most significant causes underlying the majority of problems.

SWOT Analysis

SWOT analysis helps you examine the Strengths, Weaknesses, Opportunities, and Threats related to your problem. To perform a SWOT analysis:

• List your problem’s strengths, such as relevant resources or strong partnerships.
• Identify its weaknesses, such as knowledge gaps or limited resources.
• Explore opportunities, like trends or new technologies, that could help solve the problem.
• Recognize potential threats, like competition or regulatory barriers.

SWOT analysis aids in understanding the internal and external factors affecting the problem, which can help guide your solution.

Mind Mapping

A mind map is a visual representation of your problem and potential solutions. It enables you to organize information in a structured and intuitive manner. To create a mind map:

• Write the problem in the center of a blank page.
• Draw branches from the central problem to related sub-problems or contributing factors.
• Add more branches to represent potential solutions or further ideas.

Mind mapping allows you to visually see connections between ideas and promotes creativity in problem-solving.

Examples of Problem Solving in Various Contexts

In the business world, you might encounter problems related to finances, operations, or communication. Applying problem-solving skills in these situations could look like:

• Identifying areas of improvement in your company’s financial performance and implementing cost-saving measures
• Resolving internal conflicts among team members by listening and understanding different perspectives, then proposing and negotiating solutions
• Streamlining a process for better productivity by removing redundancies, automating tasks, or re-allocating resources

In educational contexts, problem-solving can be seen in various aspects, such as:

• Addressing a gap in students’ understanding by employing diverse teaching methods to cater to different learning styles
• Developing a strategy for successful time management to balance academic responsibilities and extracurricular activities
• Seeking resources and support to provide equal opportunities for learners with special needs or disabilities

Everyday life is full of challenges that require problem-solving skills. Some examples include:

• Overcoming a personal obstacle, such as improving your fitness level, by establishing achievable goals, measuring progress, and adjusting your approach accordingly
• Navigating a new environment or city by researching your surroundings, asking for directions, or using technology like GPS to guide you
• Dealing with a sudden change, like a change in your work schedule, by assessing the situation, identifying potential impacts, and adapting your plans to accommodate the change.
• How to Resolve Employee Conflict at Work [Steps, Tips, Examples]
• How to Write Inspiring Core Values? 5 Steps with Examples
• 30 Employee Feedback Examples (Positive & Negative)

• Certification Test

Problem Solving (Basic) Skills Certification Test

Verify your problem solving skills. accelerate your job search..

Take the HackerRank Certification Test and showcase your knowledge as a HackerRank verified developer.

Skill over pedigree

Prove your skills.

The HackerRank Skills Certification Test is a standardized assessment to help developers prove their coding skills.

Get noticed by companies

Candidates who successfully clear the test will be specially highlighted to companies when they apply to relevant roles.

How does it work?

• Update Profile
• Take the Test
• Apply to Jobs
• Get highlighted to companies

No Worries. Zero risk.

If you fail to clear the test, no harm done. Your scores will remain private and will not be shared with any company. You will be allowed to retake the test (if available) after a stipulated number of days.

What can I expect during the test?

1 hr 30 mins timed test.

The test will be for a duration of 1 hr 30 mins.

Problem Solving Concepts

It covers basic topics of Data Structures (such as Arrays, Strings) and Algorithms (such as Sorting and Searching).

Do you have more questions? Check out our FAQ .

Cookie support is required to access HackerRank

Seems like cookies are disabled on this browser, please enable them to open this website

Connecting Program Synthesis and Reachability: Automatic Program Repair Using Test-Input Generation

• Conference paper
• First Online: 31 March 2017
• Cite this conference paper

• ThanhVu Nguyen 15 ,
• Westley Weimer 16 ,
• Deepak Kapur 17 &
• Stephanie Forrest 17  

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 10205))

Included in the following conference series:

• International Conference on Tools and Algorithms for the Construction and Analysis of Systems

1793 Accesses

12 Citations

We prove that certain formulations of program synthesis and reachability are equivalent. Specifically, our constructive proof shows the reductions between the template-based synthesis problem, which generates a program in a pre-specified form, and the reachability problem, which decides the reachability of a program location. This establishes a link between the two research fields and allows for the transfer of techniques and results between them.

To demonstrate the equivalence, we develop a program repair prototype using reachability tools. We transform a buggy program and its required specification into a specific program containing a location reachable only when the original program can be repaired, and then apply an off-the-shelf test-input generation tool on the transformed program to find test values to reach the desired location. Those test values correspond to repairs for the original program. Preliminary results suggest that our approach compares favorably to other repair methods.

You have full access to this open access chapter,  Download conference paper PDF

Similar content being viewed by others

Automatic Program Repair Using Formal Verification and Expression Templates

Deductive Program Repair

Pbcov: a property-based coverage criterion.

• Program synthesis
• Program verification
• Program reachability
• Reduction proof
• Automated program repair
• Test-input generation

1 Introduction

Synthesis is the task of generating a program that meets a required specification. Verification is the task of validating program correctness with respect to a given specification. Both are long-standing problems in computer science, although there has been extensive work on program verification and comparatively less on program synthesis until recently. Over the past several years, certain verification techniques have been adopted to create programs, e.g., applying symbolic execution to synthesize program repairs [ 25 , 26 , 29 , 32 ], suggesting the possibility that these two problems may be “two sides of the same coin”. Finding and formalizing this equivalence is valuable in both theory and practice: it allows comparisons between the complexities and underlying structures of the two problems, and it raises the possibility of additional cross-fertilization between two fields that are usually treated separately (e.g., it might enable approximations designed to solve one problem to be applied directly to the other).

This paper establishes a formal connection between certain formulations of program synthesis and verification. We focus on the template-based synthesis problem, which generates missing code for partially completed programs, and we view verification as a reachability problem, which checks if a program can reach an undesirable state. We then constructively prove that template-based synthesis and reachability are equivalent . We reduce a template-based synthesis problem, which consists of a program with parameterized templates to be synthesized and a test suite specification, to a program consisting of a specific location that is reachable only when those templates can be instantiated such that the program meets the given specification. To reduce reachability to synthesis, we transform a reachability instance consisting of a program and a given location into a synthesis instance that can be solved only when the location in the original problem is reachable. Thus, reachability solvers can be applied to synthesize code, and conversely, synthesis tools can be used to determine reachability.

To demonstrate the equivalence, we use the reduction to develop a new automatic program repair technique using an existing test-input generation tool. We view program repair as a special case of template-based synthesis in which “patch” code is generated so that it behaves correctly. We present a prototype tool called CETI that automatically repairs C programs that violate test-suite specifications. Given a test suite and a program failing at least one test in that suite, CETI first applies fault localization to obtain a list of ranked suspicious statements from the buggy program. For each suspicious statement, CETI transforms the buggy program and the information from its test suite into a program reachability instance. The reachability instance is a new program containing a special if branch, whose then branch is reachable only when the original program can be repaired by modifying the considered statement. By construction, any input value that allows the special location to be reached can map directly to a repair template instantiation that fixes the bug. To find a repair, CETI invokes an off-the-shelf automatic test-input generation tool on the transformed code to find test values that can reach the special branch location. These values correspond to changes that, when applied to the original program, cause it to pass the given test suite. This procedure is guaranteed to be sound, but it is not necessarily complete. That is, there may be bugs that the procedure cannot find repairs for, but all proposed repairs are guaranteed to be correct with respect to the given test suite. We evaluated CETI on the Tcas program [ 13 ], which has 41 seeded defects, and found that it repaired over 60%, which compares favorably with other state-of-the-art automated bug repair approaches.

To summarize, the main contributions of the paper include:

Equivalence Theorem: We constructively prove that the problems of template-based program synthesis and reachability in program verification are equivalent. Even though these two problems are shown to be undecidable in general, the constructions allow heuristics solving one problem to be applied to the other.

Automatic Program Repair: We present a new automatic program repair technique, which leverages the construction. The technique reduces the task of synthesizing program repairs to a reachability problem, where the results produced by a test-input generation tool correspond to a patch that repairs the original program.

Implementation and Evaluation: We implement the repair algorithm in a prototype tool that automatically repairs C programs, and we evaluate it on a benchmark that has been targeted by multiple program repair algorithms.

2 Motivating Example

We give a concrete example of how the reduction from template-based synthesis to reachability can be used to repair a buggy program. Consider the buggy code shown in Fig.  1 , a function excerpted from a traffic collision avoidance system [ 13 ]. The intended behavior of the function can be precisely described as: is_upward(in,up,down) = in*100 + up  > down . The table in Fig.  1 gives a test suite describing the intended behavior. The buggy program fails two of the tests, which we propose to repair by synthesizing a patch.

Example buggy program and test suite. CETI suggests replacing line 4 with the statement bias = up + 100; to repair the bug.

We solve this synthesis problem by restricting ourselves to generating patches under predefined templates, e.g., synthesizing expressions involving program variables and unknown parameters, and then transforming this template-based synthesis problem into a reachability problem instance. In this approach, a template such as

where bias , in , up , and down are the variables in scope at Line 4 and the value of each \(c_i\) must be found. We propose to find them by constructing a special program reachability instance and then solving that instance.

The reachability problem instance derived from the buggy program and test suite in Fig.  1 . Location L is reachable with values such as \(c_0=100,c_1=0,c_2=0,c_3=1,c_4=0\) . These values suggest using the statement bias = 100 + up; at Line 4 in the buggy program.

The new program, shown in Fig.  2 , contains a function is_upward \(_P\) that resembles the function is_upward in the original code but with Line 4 replaced by the template statement with each reference to a template parameter replaced by a reference to the corresponding new externally-defined program variable. The program also contains a starting function main , which encodes the inputs and expected outputs from the given test suite as the guards to the conditional statement leading to the target location L . Intuitively, the reachability problem instance asks if we can find values for each \(c_i\) that allow control flow to reach location L , which is only reachable iff all tests are satisfied.

This reachability instance can be given as input to any off-the-self test-input generation tool. Here, we use KLEE [ 8 ] to find value for each \(c_i\) . KLEE determines that the values \(c_0 = 100, c_1 =0, c_2 =0, c_3=1, c_4=0\) allow control flow to reach location L . Finally, we map this solution back to the original program repair problem by applying the \(c_i\) values to the template

generating the statement:

which reduces to bias = 100 + up . Replacing the statement bias = down in the original program with the new statement bias = 100 + up produces a program that passes all of the test cases.

To summarize, a specific question (i.e., can the bug be repaired by applying template X to line Y of program P while satisfying test suite T ?) is reduced to a single reachability instance, solvable using a reachability tool such as a test-input generator. This reduction is formally established in the next section.

3 Connecting Program Synthesis and Reachability

We establish the connection between the template-based formulation of program synthesis and the reachability problem in program verification. We first review these problems and then show their equivalence.

3.1 Preliminaries

We consider standard imperative programs in a language like C. The base language includes usual program constructs such as assignments, conditionals, loops, and functions. A function takes as input a (potentially empty) tuple of values and returns an output value. A function can call other functions, including itself. For simplicity, we equate a program P with its finite set of functions, including a special starting function \(\text {main}_P\) . For brevity, we write \(P(c_i,\dots ,c_n)=y\) to denote that evaluating the function \(\text {main}_P \in P\) on the input tuple \((c_i,\dots ,c_n)\) results in the value y . Program or function semantics are specified by a test suite consisting of a finite set of input/output pairs. When possible, we use \(c_i\) for concrete input values and \(v_i\) for formal parameters or variable names.

To simplify the presentation, we assume that the language also supports exceptions, admitting non-local control flow by raising and catching exceptions as in modern programming languages such as C++ and Java. We discuss how to remove this assumption in Sect.  3.3 .

Definition 1

Program Reachability. Program reachability is a classic problem which asks if a particular program state or location can be observed at run-time. It is not decidable in general, because it can encode the halting problem (cf. Rice’s Theorem [ 35 ]). However, reachability remains a popular and well-studied verification problem in practice. In model checking [ 10 ], for example, reachability is used to determine whether program states representing undesirable behaviors could occur in practice. Another application area is test-input generation [ 9 ], which aims to produce test values to explore all reachable program locations.

Definition 2

Program Reachability Problem. Given a program P , set of program variables \(\{ x_1,\dots ,x_n \}\) and target location L , do there exist input values \(c_i\) such that the execution of P with \(x_i\) initialized to \(c_i\) reaches L in a finite number of steps?

An instance of program reachability. Program P reaches location L using the solution \(\{x=-20,y=-40\}\) .

Reducing the reachability example in Fig.  3 to a template-based synthesis program (i.e., synthesize assignments to \(c_x\) and \(c_y\) ). The test suite of the reduced synthesis program is \(Q() = 1\) .

For example, the program in Fig.  3 has a reachable location L using the solution \(\{x=-20,y=-40\}\) . Similar to the synthesis problem, the decision problem formulation of reachability merely asks if the input values \(c_1,\dots ,c_n\) exist; in this presentation we require witnesses be produced.

3.2 Reducing Synthesis to Reachability

We present the constructive reduction from synthesis to reachability. The key to the reduction is a particular “gadget”, which constructs a reachability instance that can be satisfied iff the synthesis problem can be solved.

P also contains a starting function main \(_P\) that encodes the specification information from the test suite T as a conjunctive expression e :

where main \(_Q{_P}\) is a function in P corresponding to the starting function main \(_Q\) in Q . In addition, the body of main \(_P\) is one conditional statement leading to a fresh target location L if and only if e is true. Thus, main \(_P\) has the form

The derived program P consists of the declaration of the new variables (Step 1), the functions \(q_P\) ’s (Step 2), and the starting function main \(_P\) (Step 3).

Figure  2 illustrates the reduction using the example from Fig.  1 . The resulting reachability program can arrive at location L using the input \(\{c_0=100,c_1=0,c_2=0,c_3=1,c_4=0\}\) , which corresponds to a solution.

Reduction Correctness and Complexity: The correctness of \(\mathsf {GadgetS2R}\) , which transforms synthesis to reachability, relies on two key invariants Footnote 2 . First, function calls in the derived program P have the same behavior as template functions in the original program Q . Second, location L is reachable if and only if values \(c_i\) can be assigned to variables \(v_i\) such that Q passes all of the tests.

The complexity of \(\mathsf {GadgetS2R}\) is linear in both the program size and number of test cases of the input instance Q ,  S ,  T . The constructed program P consists of all functions in Q (with | S | extra variables) and a starting function main \(_P\) with an expression encoding the test suite T .

This reduction directly leads to the main result for this direction of the equivalence:

The template-based synthesis problem in Definition  1 is reducible to the reachability problem in Definition  2 .

3.3 Reducing Reachability to Synthesis

Here, we present the reduction from reachability to synthesis. The reduction also uses a particular gadget to construct a synthesis instance that can be solved iff the reachability instance can be determined.

Reduction: Let P be a program, L be a location in P , and \(V = \{v_1, \dots , v_n\}\) be global variables never directly assigned in P . We construct \(\mathsf {GadgetR2S}(P,L,V)\) , which returns a template program Q with template parameters S and a test suite T , as follows:

Raise a unique exception REACHED , at the location in Q corresponding to the location L in P . As usual, when an exception is raised, control immediately jumps to the most recently-executed try-catch block matching that exception. The exception REACHED will be caught iff the location in Q corresponding to \(L \in P\) would be reached.

Define a starting function main \(_Q\) that has no inputs and returns an integer value. Let main \(_P{_Q}\) be the function in Q corresponding to the starting function main \(_P\) in P .

Insert try-catch construct that calls \(p_Q\) and returns the value 1 if the exception REACHED is caught.

At the end of main \(_Q\) , return the value 0.

Thus, main \(_Q\) has the form

The test suite T for Q consists of exactly one test case \(Q() = 1\) , indicating the case when the exception REACHED is raised and caught.

Figure  4 illustrates the reduction using the example from Fig.  3 . The synthesized program can be satisfied by \(c_0=-20,c_1=-40\) , corresponding to the input ( \(x=-20,y=-40\) ) which reaches L in Fig.  3 .

The exception REACHED represents a unique signal to main \(_Q\) that the location L has been reached. Many modern languages support exceptions for handling special events, but they are not strictly necessary for the reduction to succeed. Other (potentially language-dependent) implementation techniques could also be employed. Or, we could use a tuple to represent the signal, e.g., returning \((v, \mathsf {false})\) from a function that normally returns v if the location corresponding L has not been reached and \((1, \mathsf {true})\) as soon as it has. BLAST [ 6 ], a model checker for C programs (which do not support exceptions), uses goto and labels to indicate when a desired location has been reached.

The complexity of \(\mathsf {GadgetR2S}\) is linear in the input instance \(P,L,v_i\) . The constructed program Q consists of all functions in P and a starting function main Q having n template variables, where \(n = |\{v_i\}|\) .

The reachability problem in Definition  2 is reducible to the template-based synthesis problem in Definition  1 .

3.4 Synthesis \(\equiv \) Reachability

Together, the above two theorems establish the equivalence between the reachability problem in program verification and the template-based program synthesis.

Corollary 1

The reachability problem in Definition  2 and the template-based synthesis problem in Definition  1 are linear-time reducible to each other.

This equivalence is perhaps unsurprising as researchers have long assumed certain relations between program synthesis and verification (e.g., see Sect.  5 ). However, we believe that a proof of the equivalence is valuable. First, our proof, although straightforward, formally shows that both problems inhabit the same complexity class (e.g., the restricted formulation of synthesis in Definition  1 is as hard as the reachability problem in Definition  2 ). Second, although both problems are undecidable in the general case, the linear-time transformations allow existing approximations and ideas developed for one problem to apply to the other one. Third, in term of practicality, the equivalence allows for direct application of off-the-shelf reachability and verification tools to synthesize and repair programs. Our approach is not so different from verification works that transform the interested problems into SAT/SMT formulas to be solved by existing efficient solvers. Finally, this work can be extended to more complex classes of synthesis and repair problems. While we demonstrate the approach using linear templates, more general templates can be handled. For example, combinations of nonlinear polynomials can be considered using a priority subset of terms (e.g., \(t_1=x^2, t_2=xy\) , as demonstrated in nonlinear invariant generation [ 33 ]).

We hope that these results help raise fruitful cross-fertilization among program verification and synthesis fields that are usually treated separately. Because our reductions produce reachability problem instances that are rarely encountered by current verification techniques (e.g., with large guards), they may help refine existing tools or motivate optimizations in new directions. As an example, our bug repair prototype CETI (discussed in the next Section) has produced reachability instances that hit a crashing bug in KLEE that was confirmed to be important by the developers Footnote 4 . These hard instances might be used to evaluate and improve verification and synthesis tools (similar to benchmarks used in annual SAT Footnote 5 and SMT Footnote 6 competitions).

4 Program Repair Using Test-Input Generation

We use the equivalence to develop CETI (Correcting Errors using Test Inputs), a tool for automated program repair (a synthesis problem) using test-input generation techniques (which solves reachability problems). We define problem of program repair in terms of template-based program synthesis:

Definition 3

Program Repair Problem. Given a program Q that fails at least one test in a finite test suite T and a finite set of parameterized templates S , does there exist a set of statements \(\{s_i\} \subseteq Q\) and parameter values \(c_1, \dots , c_n\) for the templates in S such that \(s_i\) can be replaced with \(S[c_1, \dots , c_n]\) and the resulting program passes all tests in T ?

This repair problem thus allows edits to multiple program statements (e.g., we can replace both lines 4 and 10 in Fig.  1 with parameterized templates). The single-edit repair problem restricts the edits to one statement.

CETI implements the key ideas from Theorem  1 in Sect.  3.2 to transform this repair problem into a reachability task solvable by existing verification tools. Given a test suite and a buggy program that fails some test in the suite, CETI employs the statistical fault localization technique Tarantula [ 23 ] to identify particular code regions for synthesis, i.e., program statements likely related to the defect. Next, for each suspicious statement and synthesis template, CETI transforms the buggy program, the test suite, the statement and the template into a new program containing a location reachable only when the original program can be repaired. Thus, by default CETI considers single-edit repairs, but it can be modified to repair multiple lines by using k top-ranked suspicious statements (cf. Angelix [ 29 ]). Such an approach increases the search space and thus the computational burden placed on the reachability solver.

Our current implementation employs CIL [ 31 ] to parse and modify C programs using repair templates similar to those given in [ 25 , 32 ]. These templates allow modifying constants, expressions (such as the linear template shown in Sect.  2 ), and logical, comparisons, and arithmetic operators (such as changing || to &&, \(\le \) to <, or + to −). Finally, we send the transformed program to the test-input generation tool KLEE, which produces test values that can reach the designated location. Such test input values, when combined with the synthesis template and the suspicious statement, correspond exactly to a patch that repairs the bug. CETI synthesizes correct-by-construction repairs, i.e., the repair, if found, is guaranteed to pass the test suite.

4.1 Evaluation

To evaluate CETI, we use the Tcas program from the SIR benchmark [ 13 ]. The program, which implements an aircraft traffic collision avoidance system, has 180 lines of code and 12 integer inputs. The program comes with a test suite of about 1608 tests and 41 faulty functions, consisting of seeded defects such as changed operators, incorrect constant values, missing code, and incorrect control flow. Among the programs in SIR, Tcas has the most introduced defects (41), and it has been used to benchmark modern bug repair techniques [ 12 , 26 , 32 ].

The behavior of CETI is controlled by customizable parameters. For the experiments described here, we consider the top \(n=80\) from the ranked list of suspicious statements and, then apply the predefined templates to these statements. For efficiency, we restrict synthesis parameters to be within certain value ranges: constant coefficients are confined to the integral range \([-100000, 100000]\) while the variable coefficients are drawn from the set \(\{-1,0,1\}\) .

Results. Table  1 shows the results with 41 buggy Tcas versions. These experiments were performed on a 32-core 2.60 GHz Intel Linux system with 128 GB of RAM. Column Bug Type describes the type of defect. Incorrect Const denotes a defect involving the use of the wrong constant, e.g., 700 instead of 600. Incorrect Op denotes a defect that uses the wrong operator for arithmetic, comparison, or logical calculations, e.g., \(\ge \) instead of >. Missing code denotes defects that entirely lack an expression or statement, e.g., \( a \& \& b\) instead of \( a \& \& b||c\) or return a instead of return a+b . Multiple denotes defects caused by several actions such as missing code at a location and using an incorrect operator at another location. Column T(s) shows the time taken in seconds. Column R-Prog lists the number of reachability program instances that were generated and processed by KLEE. Column Repair? indicates whether a repair was found.

We were not able to repair 15 of 41 defects, each of which requires edits at multiple locations or the addition of code that is beyond the scope of the current set of templates. As expected, CETI takes longer for these programs because it tries all generated template programs before giving up. One common pattern among these programs is that the bug occurs in a macro definition, e.g., #define C = 100 instead of #define C = 200 . Since the CIL front end automatically expands such macros, CETI would need to individually fix each use of the macro in order to succeed. This is an artifact of CIL, rather than a weakness inherent in our algorithm.

CETI, which repairs 26 of 41 Tcas defects, performs well compared to other reported results from repair tools on this benchmark program. GenProg, which finds edits by recombining existing code, can repair 11 of these defects [ 32 , Table 5]. The technique of Debroy and Wong, which uses random mutation, can repair 9 defects [ 12 , Table 2]. FoREnSiC, which uses the concolic execution in CREST, repairs 23 defects [ 26 , Table 1]. SemFix out-performs CETI, repairing 34 defects [ 32 , Table 5], but also uses fifty test cases instead of the entire suite of thousands Footnote 7 . Other repair techniques, including equivalence checking [ 26 ] and counterexample guided refinement [ 26 ], repair 15 and 16 defects, respectively.

Although CETI uses similar repair templates as both SemFix and FoREnSiC, the repair processes are different. SemFix directly uses and customizes the KLEE symbolic execution engine, and FoRenSiC integrates concolic execution to analyze programs and SMT solving to generate repairs. In contrast, CETI eschews heavyweight analyses, and it simply generates a reachability instance. Indeed, our work is inspired by, and generalizes, these works, observing that the whole synthesis task can be offloaded with strong success in practice.

However, there is a trade-off: customizing a reachability solver to the task of program repair may increase the performance or the number of repairs found, but may also reduce the generality or ease-of-adoption of the overall technique. We note that our unoptimized tool CETI already outperforms published results for GenProg, Debroy and Wong, and FoREnSiC on this benchmark, and is competitive with SemFix.

Limitations. We require that the program behaves deterministically on the test cases and that the defect be reproducible. This limitation can be mitigated by running the test cases multiple times, but ultimately our technique is not applicable if the program is non-deterministic. We assume that the test cases encode all relevant program requirements. If adequate test cases are not available then the repair may not retain required functionality. Our formulation also encodes the test cases as inputs to a starting function (e.g., main ) with a single expected output. This might not be feasible for certain types of specifications, such as liveness properties (“eventually” and “always”) in temporal logic. The efficiency of CETI depends on fault localization to reduce the search space. The reachability or test-input generation tool used affects both the efficiency and the efficacy of CETI. For example, if the reachability tool uses a constraint solver that does not support data types such as string or arrays then we will not be able to repair program defects involving those types. Finally, we assume that the repair can be constructed from the provided repair templates.

This paper concretely demonstrates the applicability of program reachability (test-input generation) to program synthesis (defect repair) but not the reverse direction of using program synthesis to solve reachability. Applying advances in automatic program repair to find test-inputs to reach nontrivial program locations remains future work.

5 Related Work

Program Synthesis and Verification. Researchers have long hypothesized about the relation between program synthesis and verification and proposed synthesis approaches using techniques or tools often used to verify programs such as constraint solving or model checking [ 1 , 43 ]. For example, Bodik and Solar-Lezama et al.’s work [ 39 , 40 ] on sketching defines the synthesis task as: \(\exists c~.~\forall (i,o)~.~ \in T ~.~ (P[c])(i) = o\) (similar to our template-based synthesis formulation in Definition  1 ) and solves the problem using a SAT solver. Other synthesis and program repair researches, e.g., [ 4 , 29 , 32 , 43 , 44 ], also use similar formulation to integrate verification tools, e.g., test-input generation, to synthesize desired programs. In general, such integrations are common in many ongoing synthesis works including the multi-disciplinary ExCAPE project [ 14 ] and the SyGuS competition [ 45 ], and have produced many practical and useful tools such as Sketch that generates low-level bit-stream programs [ 39 ], Autograder that provides feedback on programming homework [ 38 ], and FlashFill that constructs Excel macros [ 19 , 20 ].

The work presented in this paper is inspired by these works, and generalizes them by establishing a formal connection between synthesis and verification using the template-based synthesis and reachability formulations. We show that it is not just a coincident that the aforementioned synthesis works can exploit verification techniques, but that every template-based synthesis problem can be reduced to the reachability formulation in verification. Dually, we show the other direction that reduces reachability to template-based synthesis, so that every reachability problem can be solved using synthesis. Furthermore, our constructive proofs describe efficient algorithms to do such reductions.

Program Repair and Test-Input Generation. Due to the pressing demand for reliable software, automatic program repair has steadily gained research interests and produced many novel repair techniques. Constraint-based repair approaches, e.g., AFix [ 21 ], Angelix [ 29 ], SemFix [ 32 ], FoRenSiC [ 7 ], Gopinath et al. [ 18 ], Jobstmann et al. [ 22 ], generate constraints and solve them for patches that are correct by construction (i.e., guaranteed to adhere to a specification or pass a test suite). In contrast, generate-and-validate repair approaches, e.g., GenProg [ 46 ], Pachika [ 11 ], PAR [ 24 ], Debroy and Wong [ 12 ], Prophet [ 28 ], find multiple repair candidates (e.g., using stochastic search or invariant inferences) and verify them against given specifications.

The field of test-input generation has produced many practical techniques and tools to generate high coverage test data for complex software, e.g., fuzz testing [ 15 , 30 ], symbolic execution [ 8 , 9 ], concolic (combination of static and dynamic analyses) execution [ 16 , 37 ], and software model checking [ 5 , 6 ]. Companies and industrial research labs such as Microsoft, NASA, IBM, and Fujitsu have also developed test-input generation tools to test their own products [ 2 , 3 , 17 , 27 ]. Our work allows program repair and synthesis approaches directly apply these techniques and tools.

6 Conclusion

We constructively prove that the template-based program synthesis problem and the reachability problem in program verification are equivalent. This equivalence connects the two problems and enables the application of ideas, optimizations, and tools developed for one problem to the other. To demonstrate this, we develop CETI, a tool for automated program repair using test-input generation techniques that solve reachability problems. CETI transforms the task of synthesizing program repairs to a reachability problem, where the results produced by a test-input generation tool correspond to a patch that repairs the original program. Experimental case studies suggest that CETI has higher success rates than many other standard repair approaches.

More general templates (e.g., nonlinear polynomials) are also possible as shown in Sect.  3.4 .

The full proof is given in the Appendix of [ 34 ].

http://mailman.ic.ac.uk/pipermail/klee-dev/2016-February/001278.html .

SAT Competitions: http://www.satcompetition.org .

SMT competitions: http://smtcomp.sourceforge.net/2016 .

Thus CETI’s repairs, which pass the entire suite instead of just 50 selected tests, meet a higher standard. We were unable to obtain SemFix details, e.g., which 50 tests, online or from the authors.

Alur, R., Bodik, R., Juniwal, G., Martin, M.M., Raghothaman, M., Seshia, S.A., Singh, R., Solar-Lezama, A., Torlak, E., Udupa, A.: Syntax-guided synthesis. Dependable Softw. Syst. Eng. 40 , 1–25 (2015)

Google Scholar  

Anand, S., Păsăreanu, C.S., Visser, W.: JPF–SE: a symbolic execution extension to Java PathFinder. In: Grumberg, O., Huth, M. (eds.) TACAS 2007. LNCS, vol. 4424, pp. 134–138. Springer, Heidelberg (2007). doi: 10.1007/978-3-540-71209-1_12

Chapter   Google Scholar  

Artzi, S., Kiezun, A., Dolby, J., Tip, F., Dig, D., Paradkar, A., Ernst, M.D.: Finding bugs in dynamic web applications. In: ISSTA, pp. 261–272. ACM (2008)

Attie, P., Cherri, A., Al Bab, K.D., Sakr, M., Saklawi, J.: Model and program repair via sat solving. In: MEMOCODE, pp. 148–157. IEEE (2015)

Ball, T., Rajamani, S.K.: The SLAM project: debugging system software via static analysis. In: POPL, pp. 1–3. ACM (2002)

Beyer, D., Henzinger, T.A., Jhala, R., Majumdar, R.: The software model checker BLAST. Soft. Tools Technol. Transf. 9 (5–6), 505–525 (2007)

Article   Google Scholar  

Bloem, R., et al.: FoREnSiC – an automatic debugging environment for C programs. In: Biere, A., Nahir, A., Vos, T. (eds.) HVC 2012. LNCS, vol. 7857, pp. 260–265. Springer, Heidelberg (2013). doi: 10.1007/978-3-642-39611-3_24

Cadar, C., Dunbar, D., Engler, D.R.: KLEE: unassisted and automatic generation of high-coverage tests for complex systems programs. In: OSDI, vol. 8, pp. 209–224. USENIX Association (2008)

Cadar, C., Sen, K.: Symbolic execution for software testing: three decades later. Commun. ACM 56 (2), 82–90 (2013)

Clarke, E.M., Grumberg, O., Peled, D.: Model Checking. MIT Press, Cambridge (1999)

Dallmeier, V., Zeller, A., Meyer, B.: Generating fixes from object behavior anomalies. In: ASE, pp. 550–554. IEEE (2009)

Debroy, V., Wong, W.E.: Using mutation to automatically suggest fixes for faulty programs. In: Software Testing, Verification and Validation, pp. 65–74. IEEE (2010)

Do, H., Elbaum, S., Rothermel, G.: Supporting controlled experimentation with testing techniques: an infrastructure and its potential impact. Empir. Softw. Eng. 10 (4), 405–435 (2005)

ExCAPE: Expeditions in computer augmented program engineering. http://excape.cis.upenn.edu . Accessed 19 Oct 2016

Forrester, J.E., Miller, B.P.: An empirical study of the robustness of Windows NT applications using random testing. In: USENIX Windows System Symposium, pp. 59–68 (2000)

Godefroid, P., Klarlund, N., Sen, K.: DART: directed automated random testing. PLDI 40 (6), 213–223 (2005)

Godefroid, P., Levin, M.Y., Molnar, D.A., et al.: Automated whitebox fuzz testing. In: Network and Distributed System Security Symposium, pp. 151–166 (2008)

Gopinath, D., Malik, M.Z., Khurshid, S.: Specification-based program repair using SAT. In: Abdulla, P.A., Leino, K.R.M. (eds.) TACAS 2011. LNCS, vol. 6605, pp. 173–188. Springer, Heidelberg (2011). doi: 10.1007/978-3-642-19835-9_15

Gulwani, S.: Automating string processing in spreadsheets using input-output examples. In: POPL, pp. 317–330. ACM (2011)

Gulwani, S., Harris, W.R., Singh, R.: Spreadsheet data manipulation using examples. Commun. ACM 55 (8), 97–105 (2012)

Jin, G., Song, L., Zhang, W., Lu, S., Liblit, B.: Automated atomicity-violation fixing. In: PLDI, pp. 389–400. ACM (2011)

Jobstmann, B., Griesmayer, A., Bloem, R.: Program repair as a game. In: Etessami, K., Rajamani, S.K. (eds.) CAV 2005. LNCS, vol. 3576, pp. 226–238. Springer, Heidelberg (2005). doi: 10.1007/11513988_23

Jones, J.A., Harrold, M.J.: Empirical evaluation of the Tarantula automatic fault-localization technique. In: ICSE, pp. 273–282. IEEE (2005)

Kim, D., Nam, J., Song, J., Kim, S.: Automatic patch generation learned from human-written patches. In: ICSE, pp. 802–811. ACM (2013)

Könighofer, R., Bloem, R.: Automated error localization and correction for imperative programs. In: FMCAD. IEEE (2011)

Könighofer, R., Bloem, R.: Repair with on-the-fly program analysis. In: Biere, A., Nahir, A., Vos, T. (eds.) HVC 2012. LNCS, vol. 7857, pp. 56–71. Springer, Heidelberg (2013). doi: 10.1007/978-3-642-39611-3_11

Li, G., Ghosh, I., Rajan, S.P.: KLOVER: a symbolic execution and automatic test generation tool for C++ programs. In: Gopalakrishnan, G., Qadeer, S. (eds.) CAV 2011. LNCS, vol. 6806, pp. 609–615. Springer, Heidelberg (2011). doi: 10.1007/978-3-642-22110-1_49

Long, F., Rinard, M.: Automatic patch generation by learning correct code. In: POPL, vol. 51, pp. 298–312. ACM (2016)

Mechtaev, S., Yi, J., Roychoudhury, A.: Angelix: scalable multiline program patch synthesis via symbolic analysis. In ICSE, pp. 691–701. ACM (2016)

Miller, B.P., Fredriksen, L., So, B.: An empirical study of the reliability of UNIX utilities. Commun. ACM 33 (12), 32–44 (1990)

Necula, G.C., McPeak, S., Rahul, S.P., Weimer, W.: CIL: intermediate language and tools for analysis and transformation of C programs. In: Horspool, R.N. (ed.) CC 2002. LNCS, vol. 2304, pp. 213–228. Springer, Heidelberg (2002). doi: 10.1007/3-540-45937-5_16

Nguyen, H.D.T., Qi, D., Roychoudhury, A., Chandra, S., SemFix: program repair via semantic analysis. In: ICSE, pp. 772–781. ACM (2013)

Nguyen, T., Kapur, D., Weimer, W., Forrest, S.: Using dynamic analysis to discover polynomial and array invariants. In: ICSE, pp. 683–693. IEEE (2012)

Nguyen, T., Kapur, D., Weimer, W., Forrest, S.: Connecting program synthesis and reachability. Technical report, University of Nebraska, Lincoln, October 2016

Rice, H.: Classes of recursively enumerable sets and their decision problems. Trans. Am. Math. Soc. 74 (2), 358–366 (1953)

Article   MathSciNet   MATH   Google Scholar  

Saha, S., Garg, P., Madhusudan, P.: Alchemist: learning guarded affine functions. In: Kroening, D., Păsăreanu, C.S. (eds.) CAV 2015. LNCS, vol. 9206, pp. 440–446. Springer, Heidelberg (2015). doi: 10.1007/978-3-319-21690-4_26

Sen, K., Agha, G.: CUTE and jCUTE: concolic unit testing and explicit path model-checking tools. In: Ball, T., Jones, R.B. (eds.) CAV 2006. LNCS, vol. 4144, pp. 419–423. Springer, Heidelberg (2006). doi: 10.1007/11817963_38

Singh, R., Gulwani, S., Solar-Lezama, A.: Automated feedback generation for introductory programming assignments. In: PLDI, pp. 15–26. ACM (2013)

Solar-Lezama, A.: Program synthesis by sketching. Ph.D. thesis, University of California, Berkeley (2008)

Solar-Lezama, A., Arnold, G., Tancau, L., Bodík, R., Saraswat, V.A., Seshia, S.A.: Sketching stencils. In: PLDI, pp. 167–178. ACM (2007)

Solar-Lezama, A., Rabbah, R., Bodík, R., Ebcioğlu, K.: Programming by sketching for bit-streaming programs. PLDI 40 , 281–294 (2005)

Srivastava, S.: Satisfiability-based program reasoning and program synthesis. Ph.D. thesis, University of Maryland (2010)

Srivastava, S., Gulwani, S., Foster, J.S.: From program verification to program synthesis. In: POPL, pp. 313–326. ACM (2010)

Srivastava, S., Gulwani, S., Foster, J.S.: Template-based program verification and program synthesis. Soft. Tools Technol. Transf. 15 (5–6), 497–518 (2013)

SyGuS: Syntax-guided synthesis competition. www.sygus.org . Accessed 19 Oct 2016

Weimer, W., Nguyen, T., Le Goues, C., Forrest, S.: Automatically finding patches using genetic programming. In: ICSE, pp. 364–367. IEEE (2009)

Download references

Acknowledgments

This research was partially supported by NSF awards CCF 1248069, CNS 1619098, CNS 1619123, as well as AFOSR grant FA8750-11-2-0039 and DARPA grant FA8650-10-C-7089.

Author information

Authors and affiliations.

University of Nebraska, Lincoln, Nebraska, USA

ThanhVu Nguyen

University of Virginia, Charlottesville, Virginia, USA

Westley Weimer

University of New Mexico, Albuquerque, New Mexico, USA

Deepak Kapur & Stephanie Forrest

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to ThanhVu Nguyen .

Editor information

Editors and affiliations.

Inria, Rennes Cedex, France

University of Limerick and Lero - The Irish Software Research Center, Limerick, Ireland

Tiziana Margaria

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer-Verlag GmbH Germany

About this paper

Cite this paper.

Nguyen, T., Weimer, W., Kapur, D., Forrest, S. (2017). Connecting Program Synthesis and Reachability: Automatic Program Repair Using Test-Input Generation. In: Legay, A., Margaria, T. (eds) Tools and Algorithms for the Construction and Analysis of Systems. TACAS 2017. Lecture Notes in Computer Science(), vol 10205. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-54577-5_17

Download citation

DOI : https://doi.org/10.1007/978-3-662-54577-5_17

Published : 31 March 2017

Publisher Name : Springer, Berlin, Heidelberg

Print ISBN : 978-3-662-54576-8

Online ISBN : 978-3-662-54577-5

eBook Packages : Computer Science Computer Science (R0)

Share this paper

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

Societies and partnerships

• Find a journal
• Track your research

• Share full article

Advertisement

Supported by

The Algebra Problem: How Middle School Math Became a National Flashpoint

Top students can benefit greatly by being offered the subject early. But many districts offer few Black and Latino eighth graders a chance to study it.

By Troy Closson

From suburbs in the Northeast to major cities on the West Coast, a surprising subject is prompting ballot measures, lawsuits and bitter fights among parents: algebra.

Students have been required for decades to learn to solve for the variable x, and to find the slope of a line. Most complete the course in their first year of high school. But top-achievers are sometimes allowed to enroll earlier, typically in eighth grade.

The dual pathways inspire some of the most fiery debates over equity and academic opportunity in American education.

Do bias and inequality keep Black and Latino children off the fast track? Should middle schools eliminate algebra to level the playing field? What if standout pupils lose the chance to challenge themselves?

The questions are so fraught because algebra functions as a crucial crossroads in the education system. Students who fail it are far less likely to graduate. Those who take it early can take calculus by 12th grade, giving them a potential edge when applying to elite universities and lifting them toward society’s most high-status and lucrative professions.

But racial and economic gaps in math achievement are wide in the United States, and grew wider during the pandemic. In some states, nearly four in five poor children do not meet math standards.

To close those gaps, New York City’s previous mayor, Bill de Blasio, adopted a goal embraced by many districts elsewhere. Every middle school would offer algebra, and principals could opt to enroll all of their eighth graders in the class. San Francisco took an opposite approach: If some children could not reach algebra by middle school, no one would be allowed to take it.

The central mission in both cities was to help disadvantaged students. But solving the algebra dilemma can be more complex than solving the quadratic formula.

New York’s dream of “algebra for all” was never fully realized, and Mayor Eric Adams’s administration changed the goal to improving outcomes for ninth graders taking algebra. In San Francisco, dismantling middle-school algebra did little to end racial inequities among students in advanced math classes. After a huge public outcry, the district decided to reverse course.

“You wouldn’t think that there could be a more boring topic in the world,” said Thurston Domina, a professor at the University of North Carolina. “And yet, it’s this place of incredibly high passions.”

“Things run hot,” he said.

In some cities, disputes over algebra have been so intense that parents have sued school districts, protested outside mayors’ offices and campaigned for the ouster of school board members.

Teaching math in middle school is a challenge for educators in part because that is when the material becomes more complex, with students moving from multiplication tables to equations and abstract concepts. Students who have not mastered the basic skills can quickly become lost, and it can be difficult for them to catch up.

Many school districts have traditionally responded to divergent achievement levels by simply separating children into distinct pathways, placing some in general math classes while offering others algebra as an accelerated option. Such sorting, known as tracking, appeals to parents who want their children to reach advanced math as quickly as possible.

But tracking has cast an uncomfortable spotlight on inequality. Around a quarter of all students in the United States take algebra in middle school. But only about 12 percent of Black and Latino eighth graders do, compared with roughly 24 percent of white pupils, a federal report found .

“That’s why middle school math is this flashpoint,” said Joshua Goodman, an associate professor of education and economics at Boston University. “It’s the first moment where you potentially make it very obvious and explicit that there are knowledge gaps opening up.”

In the decades-long war over math, San Francisco has emerged as a prominent battleground.

California once required that all eighth graders take algebra. But lower-performing middle school students often struggle when forced to enroll in the class, research shows. San Francisco later stopped offering the class in eighth grade. But the ban did little to close achievement gaps in more advanced math classes, recent research has found.

As the pendulum swung, the only constant was anger. Leading Bay Area academics disparaged one another’s research . A group of parents even sued the district last spring. “Denying students the opportunity to skip ahead in math when their intellectual ability clearly allows for it greatly harms their potential for future achievement,” their lawsuit said.

The city is now back to where it began: Middle school algebra — for some, not necessarily for all — will return in August. The experience underscored how every approach carries risks.

“Schools really don’t know what to do,” said Jon R. Star, an educational psychologist at Harvard who has studied algebra education. “And it’s just leading to a lot of tension.”

In Cambridge, Mass., the school district phased out middle school algebra before the pandemic. But some argued that the move had backfired: Families who could afford to simply paid for their children to take accelerated math outside of school.

“It’s the worst of all possible worlds for equity,” Jacob Barandes, a Cambridge parent, said at a school board meeting.

Elsewhere, many students lack options to take the class early: One of Philadelphia’s most prestigious high schools requires students to pass algebra before enrolling, preventing many low-income children from applying because they attend middle schools that do not offer the class.

In New York, Mr. de Blasio sought to tackle the disparities when he announced a plan in 2015 to offer algebra — but not require it — in all of the city’s middle schools. More than 15,000 eighth graders did not have the class at their schools at the time.

Since then, the number of middle schools that offer algebra has risen to about 80 percent from 60 percent. But white and Asian American students still pass state algebra tests at higher rates than their peers.

The city’s current schools chancellor, David Banks, also shifted the system’s algebra focus to high schools, requiring the same ninth-grade curriculum at many schools in a move that has won both support and backlash from educators.

And some New York City families are still worried about middle school. A group of parent leaders in Manhattan recently asked the district to create more accelerated math options before high school, saying that many young students must seek out higher-level instruction outside the public school system.

In a vast district like New York — where some schools are filled with children from well-off families and others mainly educate homeless children — the challenge in math education can be that “incredible diversity,” said Pedro A. Noguera, the dean of the University of Southern California’s Rossier School of Education.

“You have some kids who are ready for algebra in fourth grade, and they should not be denied it,” Mr. Noguera said. “Others are still struggling with arithmetic in high school, and they need support.”

Many schools are unequipped to teach children with disparate math skills in a single classroom. Some educators lack the training they need to help students who have fallen behind, while also challenging those working at grade level or beyond.

Some schools have tried to find ways to tackle the issue on their own. KIPP charter schools in New York have added an additional half-hour of math time to many students’ schedules, to give children more time for practice and support so they can be ready for algebra by eighth grade.

At Middle School 50 in Brooklyn, where all eighth graders take algebra, teachers rewrote lesson plans for sixth- and seventh-grade students to lay the groundwork for the class.

The school’s principal, Ben Honoroff, said he expected that some students would have to retake the class in high school. But after starting a small algebra pilot program a few years ago, he came to believe that exposing children early could benefit everyone — as long as students came into it well prepared.

Looking around at the students who were not enrolling in the class, Mr. Honoroff said, “we asked, ‘Are there other kids that would excel in this?’”

“The answer was 100 percent, yes,” he added. “That was not something that I could live with.”

Troy Closson reports on K-12 schools in New York City for The Times. More about Troy Closson