Sarnoff Corporation

David Sarnoff Research Center, Inc.

Sri International

General Electric Corporation Gmbh

Peter Jeffrey Burt

Keith James Hanna

Michael Wade Hansen

Stephen Charles Hsu

Michal Irani

Gooitzen Siemen Van Der Wal

Padmanabhan Anandan

Harpreet Singh Sawhney

Rakesh Kumar

Gooitzen Sieman Van Der Wal

Shmuel Peleg

Rajesh Hingorani

John Robertson Tower

Chao Zhang

Douglas Fremont Dixon

Raymond J Kolczynski

Peter Alan Levine

David Alan Ackerman

James Russell Bergen

Vincent Paragano

Manoj Aggarwal

Sek Meng Chai

Michael Henry Brill

Peter James Coyle

Lambert Ernest Wixson

David Chao Zhang

Michael Raymond Piacentino

Jayan Eledath

Gary A Gendel

Thomas Lee Vogelsong

Deepam Mishra

Robert Mandlebaum

Joshua Randy Herman, Deceased

Mark S Lohmeyer

James R Bergen

Joshua R Herman

Supun Samarasakera

Norman Winarsky

A computer implemented method for fusing images taken by a plurality of cameras is disclosed, comprising the steps of: receiving a plurality of images of the same scene taken by the plurality of cameras; generating Laplacian pyramid images for each source image of the plurality of images; applying contrast normalization to the Laplacian pyramids images; performing pixel-level fusion on the Laplacian pyramid images based on a local salience measure that reduces aliasing artifacts to produce one salience-selected Laplacian pyramid image for each pyramid level; and combining the salience-selected Laplacian pyramid images into a fused image. Applying contrast normalization further comprises, for each Laplacian image at a given level: obtaining an energy image from the Laplacian image; determining a gain factor that is based on at least the energy image and a target contrast; and multiplying the Laplacian image by a gain factor to produce a normalized Laplacian image.

Multi-scale adaptive fusion with contrast normalization

A method and apparatus for optimizing image quality based on scene content comprising a sensor for generating a sequence of frames where each frame in the sequence of frames comprises content representing a scene and a digital processor, coupled to the sensor, for performing scene content analysis and for establishing a window defining a number of input frames from the sensor and processed output frames, and for aligning and combining the number of frames in the window to form an output frame, wherein sensor parameters and frame combination parameters are adjusted based on scene content.

Method and apparatus for optimizing image quality based on scene content

The present invention provides a method and a system for high performance image signal processing of continuous images in real time. The system comprising a focal plane array for generating continuous source image frames in real time. The focal plane array divided logically into blocks of sub-frames. The system also comprising an analog to digital converter (ADC) layer having an array of ADC elements for converting the source image frames into a digital data. The system further comprising a digital processor layer having an array of processing elements for processing the digital data and an interconnecting layer for connecting each one of the ADC elements and the digital processing elements substantially vertically to the focal plane and substantially parallel to one another. The processing comprising reducing image motion blur, increasing image dynamic range, increasing image depth of field and obtaining features of the images.

System and method for high performance image processing

Multi-scale multi-camera adaptive fusion with contrast normalization

A method and system for detecting moving objects and controlling a surveillance system includes a processing module adapted to receive image information from at least one imaging sensor. The system performs motion detection analysis upon captured images and controls the camera in a specific manner upon detection of a moving object. The image processing uses the camera's physical orientation to a surveillance area to facilitate mapping images captured by the camera to a reference map of the surveillance area. Using the camera orientation, a moving object's position (e.g., latitude, longitude and altitude) within a scene can be derived.

Method and system for performing surveillance

A scaleable video encoder has one or more encoding modes in which at least some, and possibly all, of the motion information used during motion-based predictive encoding of a video stream is excluded from the resulting encoded video bitstream, where a corresponding video decoder is capable of performing its own motion computation to generate its own version of the motion information used to perform motion-based predictive decoding in order to decode the bitstream to generate a decoded video stream. All motion computation, whether at the encoder or the decoder, is preferably performed on decoded data. For example, frames may be encoded as either H, L, or B frames, where H frames are intra-coded at full resolution and L frames are intra-coded at low resolution. The motion information is generated by applying motion computation to decoded L and H frames and used to generate synthesized L frames. L-frame residual errors are generated by performing inter-frame differencing between the synthesized and original L frames and are encoded into the bitstream. In addition, synthesized B frames are generated by tweening between the decoded H and L frames and B-frame residual errors are generated by performing inter-frame differencing between the synthesized B frames and, depending on the implementation, either the original B frames or sub-sampled B frames. These B-frame residual errors are also encoded into the bitstream. The ability of the decoder to perform motion computation enables motion-based predictive encoding to be used to generate an encoded bitstream without having to expend bits for explicitly encoding any motion information.

Tweening-based codec for scaleable encoders and decoders with varying motion computation capability

A recognition system which obtains and analyzes images of at least one object in a scene comprising a wide field of view (WFOV) imager which is used to capture an image of the scene and to locate the object and a narrow field of view (NFOV) imager which is responsive to the location information provided by the WFOV imager and which is used to capture an image of the object, the image of the object having a higher resolution than the image captured by the WFOV imager is disclosed. In one embodiment, a system that obtains and analyzes images of the irises of eyes of a human or animal in an image with little or no active involvement by the human or animal is disclosed. A method for obtaining and analyzing images of at least one object in a scene comprising capturing a wide field of view image of the object to locate the object in the scene; and then using a narrow field of view imager responsive to the location information provided in the capturing step to obtain higher resolution image of the object is also disclosed.

Fully automated iris recognition system utilizing wide and narrow fields of view

Apparatus and a concomitant method of identifying the direction of motion within a scene that is represented by a sequence of images, e.g., a series of frames in a video sequence. The apparatus contains a flow field generator, a flow field segmentor, and alarm detector and an alarm processor. The flow field generator processes the sequence of images and generates a substantially distortionless flow field representing the motion within a scene. The flow field is a vector representation of the motion of the scene that represents both the magnitude and the direction of the motion. The flow field is generated by correlating at least two frames in the sequence of images. This flow field is analyzed by a flow field segmentor to determine the magnitude and direction of motion with the scene and segment the motion information from the static portions of the scene. An alarm detector then processes the motion information to determine if an alarm should be generated based upon the motion information. The alarm processor activates an appropriate alarm response, e.g., illuminate warning lamps, sound a warning alarm, lock doors, alert security personnel, and the like.

Apparatus and a method for detecting motion within an image sequence

A system for automatically generating a mosaic from a plurality of input images. The system sequentially executes an image alignment process, an editing process, and a combining process such that, from a sequence of images, the system automatically produces a seamless mosaic for various applications. Illustrative applications for the mosaic include: (1) a mosaic based display system including an image printing system, (2) a surveillance system and (3) a mosaic based compression system. The mosaic based display system permits a system user to display, manipulate and alter a mosaic. The mosaic based compression system exploits the temporal and spatial redundancy in image sequences and efficiently compresses the image information. The compression system can be used for compressing image information for storage in a storage device or can be used for compressing image information for transmission through a band-limited transmission channel. The surveillance system combines the mosaic construction system, the compression system and the display system into a comprehensive system for remote motion detection.

Mosaic based image processing system

Apparatus and a concomitant method of identifying the direction of motion within a scene that is represented by a sequence of images, e.g., a series of frames in a video sequence. The apparatus contains a flow field generator, a flow field segmentor, and alarm detector and an alarm processor. The flow field generator processes the sequence of images and generates a substantially distortionless flow field representing the motion within a scene. The flow field is a vector representation of the motion of the scene that represents both the magnitude and the direction of the motion. The flow field is generated by correlating at least two frames in the sequence of images. This flow field is analyzed by a flow field segmentor to determine the magnitude and direction of motion with the scene and segment the motion information from the static portions of the scene. An alarm detector then processes the motion information to determine if an alarm should be generated based upon the motion information. The alarm processor activates an appropriate alarm response, e.g., illuminate warning lamps, sound a warning alarm, lock doors, and alert security personnel.

A method of constructing an image mosaic comprising the steps of selecting source images, aligning the source images, selecting source segments, enhancing the images, and merging the images to form the image mosaic is disclosed. An apparatus for constructing an image mosaic comprising means for selecting source images, means for aligning the source images, means for selecting source image segments, means for enhancing the images, and means for merging the images to form the image mosaic is also disclosed. The process may be performed automatically by the system or may be guided interactively by a human operator. Applications include the construction of photographic quality prints form video and digital camera images.

Method and apparatus for mosaic image construction

A digital image warper system produces a warped output image from an input image. The warper system increases the sampling rate of a sampled image above the Nyquist rate or samples an analog image at a higher rate than the Nyquist rate and prefilters the upsampled image to counteract filtering which may occur during the warping operation. The upsampled image is warped using an interpolator for resampling to produce a warped upsampled image. The interpolator is, for example, a low quality interpolator such as a bilinear interpolator. The warped image is then down-sampled to the same resolution as the input image to produce the warped image. Down-sampling and warping can be combined into one step by modifying the geometric transformation function implemented when warping the upsampled image.

Quality image warper

System for automatically aligning images to form a mosaic image

Method and apparatus for performing mosaic based image compression

Disclosed is a technique for deriving a composite video image by merging foreground and background video image data supplied from a plurality of separate video signal sources employing pattern-key insertion, rather than prior-art color-key insertion, for this purpose. Pattern-key insertion involves replacing a first pattern in a video scene with a second pattern. This is accomplished by first detecting the first pattern in the video scene and estimating the pose of this detected first pattern with respect to a reference object in the video scene. The second pattern is then geometrically transformed using the pose estimate of the detected first pattern. Finally, the detected first pattern is replaced with the geometrically-transformed second pattern.

Video merging employing pattern-key insertion

Growing community of inventors

Princeton, NJ, United States of America

Peter Jeffrey Burt