Purdue Research Foundation

Jeffrey Mark Siskind

Barak Avrum Pearlmutter

Haonan Yu

Andrei Barbu

Siddharth Narayanaswamy

Daniel Paul Barrett

Scott Alan Bronikowski

A system and method which allows the basic checkpoint-reverse-mode AD strategy (of recursively decomposing the computation to reduce storage requirements of reverse-mode AD) to be applied to arbitrary programs: not just programs consisting of loops, but programs with arbitrarily complex control flow. The method comprises (a) transforming the program into a formalism that allows convenient manipulation by formal tools, and (b) introducing a set of operators to allow computations to be decomposed by running them for a given period of time then pausing them, while treating the paused program as a value subject to manipulation.

System and method for divide-and-conquer checkpointing

A system for directing the motion of a vehicle, comprising receiving commands in natural language using a processor, the commands specifying a relative path to be taken by the vehicle with respect to other objects in the environment; and determining an absolute path for the vehicle to follow based on the relative path using the processor, the absolute path comprising a series of coordinates in the environment; and directing the vehicle along the absolute path. Also provided is a system for training a lexicon of a natural language processing system, comprising receiving a data set containing a corpus of absolute paths driven by a vehicle annotated with natural language descriptions of the absolute paths using a processor, and determining parameters of the lexicon based on the data set.

System and method for controlling a self-guided vehicle

A system and method for determining the locations and types of objects in a plurality of videos. The method comprises pairing each video with one or more sentences describing the activity or activities in which those objects participate in the associated video, wherein no use is made of a pretrained object detector. The object locations are specified as rectangles, the object types are specified as nouns, and sentences describe the relative positions and motions of the objects in the videos referred to by the nouns in the sentences. The relative positions and motions of the objects in the video are described by a conjunction of predicates constructed to represent the activity described by the sentences associated with the videos.

System and method for sentence directed video object codetection

A method of testing a video against an aggregate query includes automatically receiving an aggregate query defining participant(s) and condition(s) on the participant(s). Candidate object(s) are detected in the frames of the video. A first lattice is constructed for each participant, the first-lattice nodes corresponding to the candidate object(s). A second lattice is constructed for each condition. An aggregate lattice is constructed using the respective first lattice(s) and the respective second lattice(s). Each aggregate-lattice node includes a scoring factor combining a first-lattice node factor and a second-lattice node factor. respective aggregate score(s) are determined of one or more path(s) through the aggregate lattice, each path including a respective plurality of the nodes in the aggregate lattice, to determine whether the video corresponds to the aggregate query. A method of providing a description of a video is also described and includes generating a candidate description with participant(s) and condition(s) selected from a linguistic model; constructing component lattices for the participant(s) or condition(s), producing an aggregate lattice having nodes combining component-lattice factors, and determining a score for the video with respect to the candidate description by determining an aggregate score for a path through the aggregate lattice. If the aggregate score does not satisfy a termination condition, participant(s) or condition(s) from the linguistic model are added to the condition, and the process is repeated. A method of testing a video against an aggregate query by mathematically optimizing a unified cost function is also described.

Correlating videos and sentences

The disclosed system provides a transformation-based implementation of forward-mode and reverse-mode automatic differentiation as a built-in, first-class function in a functional programming language. Each of these constructs imposes only a small constant factor of the computational burden (time) of the function itself, and the forward construct has the same properties in terms of space. The functions can be applied to any function, including those involving derivatives and nested closures.

Automatic derivative method for a computer programming language

The disclosed system provides a functional programming construct that allows convenient modular run-time nonstandard interpretation via reflection on closure environments. This construct encompasses both the ability to examine the contents of a closure environment and to construct a new closure with a modified environment. Examples of this powerful and useful construct support such tasks as tracing, security logging, sandboxing, error checking, profiling, code instrumentation and metering, run-time code patching, and resource monitoring. It is a non-referentially-transparent mechanism that reifies the closure environments that are only implicit in higher-order programs. A further example provides a novel functional-programming language that supports forward automatic differentiation (AD).

Growing community of inventors

West Lafayette, IN, United States of America

Jeffrey Mark Siskind