International Business Machines Corporation

Itai Jaeger

Roy Emek

Yoav Avraham Katz

Yehuda Naveh

Tzach Schechner

Allon Adir

Laurent Fournier

Shady Copty

Sigal Asaf

Shai Lubliner

Sanjay Gupta

Avi Ziv

Todd Swanson

Eitan Marcus

Shai Fine

Ofer Peled

Ari Freund

Lawrence Allyn McConville

Alion Adir

Igor Dozorets

A method for solving a constraint satisfaction problem (CSP) comprises identifying a first illegal combination associated with at least one variable, wherein the first illegal combination contradicts at least one constraint; grouping the first illegal combination with a second illegal combination, in response to determining that the first and second illegal combinations contradict the same constraint; identifying at least one failure explanation for each group of illegal combinations, wherein the failure explanation is associated with at least one illegal combination in said group to provide a reason for a contradicted constraint associated with the illegal combination; assigning a value to each variable in the given domain to solve the CSP within the limitations defined by the constraints in a solution path; and generating a result, in response to determining that at least one assigned value to a variable contradicts a constraint.

Context-based failure reporting for a constraint satisfaction problem

A parametrically controlled model-based test generator automatically generates architectural compliance test suites for different implementations of a processor architecture, based on a set of architectural decisions chosen among optional behaviors for each implementation. Thus, different implementations of the same architecture can be easily supported by modifying the parameter values. In addition, ongoing changes to the architecture or comprehensive updates to the test suite can be easily handled by updating the architecture model or the coverage models, forgoing the need to review the whole, potentially huge, set of tests.

Automatic generation of test suite for processor architecture compliance

A computer software product is provided. The product includes a computer-readable medium in which computer program instructions are stored, which instructions, when read by a computer, cause the computer to perform a method for functional verification of a design, having an operational space comprising the steps of accepting as an input a test template that specifies test parameters directed to a function of the design, the template comprising constraints and variables covering a working space within the operational space, associating an exemption mode of operation with at least one of the constraints or at least one of the variables, wherein a domain of the at least one of the variables is modified or the at least one of the constraints is disabled in the exemption mode of operation, producing a test case that lies in the operational space of the design by enabling the exemption mode of operation, the test case comprising a series of transactions originating from the template, wherein the transactions exercise the function of the design, wherein the working space is modified in the exemption mode of operation and submitting the test case for execution by the design, wherein the exemption mode of operation is associated with only a portion of the template.

Highly specialized scenarios in random test generation

Systems and methods for solving a constraint satisfaction problem (CSP) defined by one or more variables, constraints and probabilistic requests (PRs) are provided. In one embodiment, each PR is mapped onto a set of constraints that implement the PR. The CSP solver then finds a series of solutions to the CSP. For each solution, if a constraint associated with a PR is satisfied, then a success counter for that PR is incremented; else a failure counter for that PR is incremented. If the constraint was not considered for the purpose of solving the CSP, then the failure counter for the PR associated with the constraint is incremented.

Determining compliance rates for probabilistic requests

A computer-implemented method for verification of a hardware design includes specifying requests to allocate regions in a memory of the hardware design, such that at least two of the requests are specified independently of one another. The requests indicate respective allocation types. Overlap restrictions are specified between at least some of the allocation types. The requests and the overlap restrictions are automatically converted to a constraint satisfaction problem (CSP), which includes CSP constraints based on the requests, the allocation types and the overlap restrictions. The CSP is solved to produce a random test program, which includes a memory map that allocates the regions in the memory while complying with the requests and the overlap restrictions. The test program is applied to the hardware design.

Random stimuli generation of memory maps and memory allocations

Methods and systems are presented for generation of a test suite in order to validate compliance of a process with its process specification. The methodology involves a formal description of the process using a flowchart, refinement of the flowchart to include misinterpretations of the process specification, defining compliance coverage models over the flowchart, and automatically generating test case scenarios that cover the models. Internal and external types of misinterpretation are distinguished. A compliance test suite is automatically generated and observations made of the details of the traversal through the flow chart when the tests are executed.

Systematic compliance checking of a process

The input for a test generator is a plurality of test templates, each of which typically aims at covering a specific verification task. Test templates direct the production of distinct transactions, which are the atomic functional building blocks of the design-under-verification. Test templates directed to different hardware functions of the scenario are dynamically interleaved. In this way several transactions are combined together in complex statements in order to achieve a complex test scenario. The transactions are submitted to the test generator, which generates test cases, in which the different hardware functions of the scenario are exercised in combinations. Variation among the test cases is achieved through a large number of random decisions made during the generation process.

Dynamically interleaving randomly generated test-cases for functional verification

Improvements in functional verification of a design are achieved by providing a test template that specifies test parameters directed to a function of the design. An exemption mode of operation is associated with a portion of the template, in which constraints and variables associated with the template are revised. The template is an input to a CSP engine, which, in cooperation with a test generator engine, produces test scenarios that lie in an expanded region of the generator's usual operational space. Provision is made for independently enabling and disabling a plurality of exemption modes of operation that are associated with the same or different areas of the template.

Methods, systems and apparatus for modeling a target system includes defining a constraint satisfaction problem (CSP) that characterizes the target system in terms of a set of variables, each having a respective input domain, and initial constraints applicable to the variables. The variables are partitioned into at least first and second sets. An abstract solution is found to the CSP, including a given assignment of the variables in the first set. A reduced domain of at least one of the variables in the second set is computed, so as to be compatible with the abstract solution. A redundant constraint on the abstract solution is determined responsively to the reduced domain. A concrete solution to the CSP is then found, using the abstract solution and the redundant constraint.

Accelerated solution of constraint satisfaction problems by partitioning of the variable space

Test generation is improved by learning the relationship between an initial state vector for a stimuli generator and generation success. A stimuli generator for a design-under-verification is provided with information about the success probabilities of potential assignments to an initial state bit vector. Selection of initial states according to the success probabilities ensures a higher success rate than would be achieved without this knowledge. The approach for obtaining an initial state bit vector employs a CSP solver. A learning system is directed to model the behavior of possible initial state assignments. The learning system develops the structure and parameters of a Bayesian network that describes the relation between the initial state and generation success.

Harnessing machine learning to improve the success rate of stimuli generation

A computer-implemented method for modeling a target system includes defining a cloned constraint satisfaction problem (CSP) that characterizes the target system in terms of a set of variables and constraints applicable to the variables. The cloned CSP includes a non-predetermined number of duplicate sub-problems corresponding to instances of a repeating feature of the target system. The variables are partitioned so as to define an abstract CSP containing a subset of the variables relating to the duplicate sub-problems. The abstract CSP is solved to generate an abstract solution indicating the number of duplicate sub-problems to use in the cloned CSP. A concrete solution to the cloned CSP is found using the abstract solution.

Growing community of inventors

Lavon, Israel

Itai Jaeger