David Sarnoff Research Center, Inc.

Sarnoff Corporation

Sri International

Rca Thomson Licensing Corp.

Gooitzen Siemen Van Der Wal

Peter Jeffrey Burt

Michael Wade Hansen

Michael Raymond Piacentino

Rajesh Hingorani

Raymond J Kolczynski

David Alan Ackerman

Jeffrey Lubin

Stephen Charles Hsu

Robert G Bassman

John Robertson Tower

Michael Henry Brill

Lambert Ernest Wixson

Joseph O Sinniger

Bhavesh B Bhatt

Charles H Anderson

Frederic William Brehm

Mark S Lohmeyer

Bill J Call

The present invention provides a method and a system for high performance image signal processing of continuous images in real time. The system comprising a focal plane array for generating continuous source image frames in real time. The focal plane array divided logically into blocks of sub-frames. The system also comprising an analog to digital converter (ADC) layer having an array of ADC elements for converting the source image frames into a digital data. The system further comprising a digital processor layer having an array of processing elements for processing the digital data and an interconnecting layer for connecting each one of the ADC elements and the digital processing elements substantially vertically to the focal plane and substantially parallel to one another. The processing comprising reducing image motion blur, increasing image dynamic range, increasing image depth of field and obtaining features of the images.

System and method for high performance image processing

A real-time modular video processing system (VPS) which can be scaled smoothly from relatively small systems with modest amounts of hardware to very large, very powerful systems with significantly more hardware. The modular video processing system includes a processing module containing at least one general purpose microprocessor which controls hardware and software operation of the video processing system using control data and which also facilitates communications with external devices. One or more video processing modules are also provided, each containing parallel pipelined video hardware which is programmable by the control data to provide different video processing operations on an input stream of video data. Each video processing module also contains one or more connections for accepting one or more daughterboards which each perform a particular image processing task. A global video bus routes video data between the processing module and each video processing module and between respective processing modules, while a global control bus provides the control data to/from the processing module from/to the video processing modules separate from the video data on the global video bus. A hardware control library loaded on the processing module provides an application programming interface including high level C-callable functions which allow programming of the video hardware as components are added and subtracted from the video processing system for different applications.

Modular parallel-pipelined vision system for real-time video processing

Digital signal processing circuitry implemented in ASICs or FPGAs is built by combining multi-component constructs (e.g. macrocells). These circuits may be modified to include a timing channel by augmenting selected ones of the constructs to include a path which propagates a timing signal with a delay that compensates for the signal processing delay through the construct. The selected constructs are those that are used in a critical processing path in the digital signal processing circuitry. A timing compensation circuit may also be defined as a construct. This block receives two digital data signals having accompanying timing signals and delays the first signal that provides valid data until the second signal also provides valid data, as determined by their timing signals. A configurable arithmetic and logic unit (ALU) made using these techniques includes a timing compensation circuit, a look-up table and an accumulator. The configurable ALU may also include a timing signal selection circuit which selects between the each of two input timing signals, the logical AND of the input timing signals and the logical OR of the two input timing signals to produce an output timing signal. A programmable multiply-accumulator includes a matrix of multipliers, each of which may receive one of a plurality of input signals. The input signals are delayed through a pipeline and a portion of this pipeline is reserved for delaying one or more timing signals associated with the plurality of input signals. The delayed timing signals form the timing signals that are associated with the output signal of the programmable multiply accumulator.

Digital signal processing circuitry having integrated timing information

A digital image warper system produces a warped output image from an input image. The warper system increases the sampling rate of a sampled image above the Nyquist rate or samples an analog image at a higher rate than the Nyquist rate and prefilters the upsampled image to counteract filtering which may occur during the warping operation. The upsampled image is warped using an interpolator for resampling to produce a warped upsampled image. The interpolator is, for example, a low quality interpolator such as a bilinear interpolator. The warped image is then down-sampled to the same resolution as the input image to produce the warped image. Down-sampling and warping can be combined into one step by modifying the geometric transformation function implemented when warping the upsampled image.

Quality image warper

Apparatus for image processing a sequence of images containing a parallel-pipelined image processor comprised of image memories, a pyramid processing circuit, an arithmetic logic unit, a crossbar switch for video routing through the various components of the processor, signal processors to provide hardware programming through a global bus and also perform image processing operations. Images can be passed directly from the crossbar switch to internal static RAM of the signal processors through a first-in, first-out interface at full video rates.

Parallel-pipelined image processing system

An apparatus for assessing the visibility of differences between two input image sequences. The apparatus comprises a pair of pre-filtered processors, a pair of pyramid constructors and a JND processor. Each pre-filter processor receives a separate input image sequence and performs temporal filtering to produce a low-pass temporal response and a bandpass temporal response. In turn, the pyramid constructors receive the temporal responses and generate contrast pyramids for the temporal responses. The JND processor evaluates the contrast pyramids to produce an image metric which is used to assess the visibility of differences between the two input image sequences.

Apparatus for assessing the visiblity of differences between two image

Multiresolution (pyramid) filtering is useful in image processing. An IC for implementing a variety of multiresolution filters includes a programmable, symmetric, and separable two-dimensional filter. The input signal to the filter can be an input signal applied to the IC or a combination of two such signals. Circuitry in the IC may be programmed to imply pixel values around the edges of the two-dimensional image signals processed by the filter. The filter provides an output signal as well as each of the unfiltered signals from a tapped delay line of a filter that combines successive lines of the image. The IC also includes an arithmetic and logic unit in which the filtered output signal may be combined with an unfiltered input signal or one of the unfiltered tap signals. If the filter is programmed to produce a Gaussian low-pass filtered image, this image, combined with image data from a center one of the filter taps produces a Laplacian function of the original image.

Pyramid processor integrated circuit

A method for fusing two or more source images to form a composite image with extended information content which may be color augmented and apparatus for forming the composite image from the source images is disclosed. Each source image is decomposed into a number of source images of varying resolution. The decomposed source images are analyzed using directionally sensitive operators to generate a set of oriented basis functions characteristic of the information content of the original images. The oriented basis functions for the composite image are then selected from those of the different source images and the inverse of the decomposition performed to construct the composite image. Color augmentation provides information as to the relative weighting of the contribution of each source to the composite image.

Method for fusing images and apparatus therefor

A method for fusing two or more source images to form a composite image with extended information content and apparatus for forming the composite image from the source images is disclosed. Each source image is decomposed into a number of source images of varying resolution. The decomposed source images are analyzed using directionally sensitive operators to generate a set of oriented basis functions characteristic of the information content of the original images. The oriented basis functions for the composite image are then selected from those of the different source images and the inverse of the decomposition performed to construct the composite image. Apparatus for fusing two or more source images to form a composite image is also disclosed.

First circuit apparatus, comprising a given number of prior-art image-pyramid stages, together with second circuit apparatus, comprising the same given number of novel motion-vector stages, perform cost-effective hierarchical motion analysis (HMA) in real time, with minimum system processing delay and/or employing minimum hardware structure. Specifically, the first and second circuit apparatus, in response to relatively high-resolution image data from an ongoing input series of successive given pixel-density image-data frames that occur at a relatively high frame rate (e.g., 30 frames per second), derives, after a certain processing-system delay, an ongoing output series of successive given pixel-density vector-data frames that occur at the same given frame rate. Each vector-data frame is indicative of image motion occurring between each pair of successive image frames.

Growing community of inventors

Hopewell, NJ, United States of America

Gooitzen Siemen Van Der Wal