Yeda Research and Development Co. Ltd.

Sarnoff Corporation

The Weizmann Institute of Science

Other

David Sarnoff Research Center, Inc.

Michal Irani

Padmanabhan Anandan

Peter Jeffrey Burt

Stephen Charles Hsu

Michael Wade Hansen

Yaron Caspi

Oren Boiman

Eli Shechtman

Keith James Hanna

Rakesh Kumar

James Russell Bergen

Padmananbhan Anandan

Shai Bagon

Denis Simakov

Daphna Weinshall

Tomer Michaeli

Eli Schechtman

Alon Faktor

Assaf Shocher

Daniel Glasner

Oded Shahar

Given an input image, an image enhancement task, and no external examples available to train on, an Image-Specific Deep Network is constructed tailored to solve the task for this specific image. Since there are no external examples available to train on, the network is trained on examples extracted directly from the input image itself. The current solution solves the problem of Super-Resolution (SR), whereas the framework is more general and is not restricted to SR.

Signal enhancement and manipulation using a signal-specific deep network

Devices, systems, and methods of blind deblurring and blind super-resolution utilizing internal patch recurrence. Small signal patches tend to repeat 'as is' across multiple scales of a natural signal. This fractal-like behavior is utilized for signal processing tasks, including 'Blind Deblurring' or &#x201c;Blind Super-Resolution&#x201d;, namely, removing signal blur or increasing signal resolution without a-priori knowledge of the underlying blur kernel. While the cross-scale patch recurrence is strong in signals taken under ideal conditions, the cross-scale patch recurrence significantly diminishes when the acquisition blur deviates from an ideal blur. These deviations from ideal patch recurrences are used for recovering the underlying (unknown) blur kernel. The correct blur kernel is recovered by seeking the kernel which maximizes the patch similarity across scales of a related &#x201c;reference&#x201d; signal. For example, this reference signal may be the low-resolution input signal, the sharp deblurred-version of a blurry input signal, or the like. Quantitative and qualitative experiments indicate that this approach yields improved or superior results, in &#x201c;Blind Deblurring&#x201d; and in &#x201c;Blind Super-Resolution&#x201d;.

Device, system, and method of blind deblurring and blind super-resolution utilizing internal patch recurrence

Device, system, and method of visual inference by collaborative composition. A method of searching for similar target content-items for a plurality of source content-items, the method implementable on a computing device, comprises: for each of said plurality of source content-items, searching for similar target content-items in a dataset, while taking into account collaborative feedback exchanged among at least some of currently-searched said plurality of source content-items.

Device, system, and method of visual inference by collaborative composition

A method implementable on a computing device includes exploiting data redundancy to combine high frequency information from at least two different scales of an input signal to generate a super resolution version of said input signal. An alternative method includes exploiting recurrence of data from an input signal in at least two different scales of at least one reference signal to extract and to combine high frequency information from a plurality of scales of said at least one reference signal to generate a super resolution version of said input signal. An alternative method includes generating a super resolution version of a single input video sequence in at least the temporal dimension by exploiting data recurrence within the input video sequence or with respect to an external database of example video sequences. A signal may be an image, a video sequence, an audio signal, etc.

Super resolution from a single signal

A method for measuring bi-directional similarity between a first signal of a first size and a second signal of a second size includes matching at least some patches of the first signal with patches of the second signal for data completeness, matching at least some patches of the second signal with patches of the first signal for data coherence, calculating the bi-directional similarity measure as a function of the matched patches for coherence and the matched patches for completeness and indicating the similarity between the first signal and the second signal. Another method generates a second signal from a first signal where the second signal is different than the first signal by at least one parameter. The method includes attempting to maximize a bi-directional similarity measure between the second signal and the first signal.

Bidirectional similarity of signals

Method and apparatus for inferring irregularities in query data relative to referential data includes attempting to compose the query data, like a puzzle, from large chunks of the referential data, and inferring irregularities in the query data based on at least the size of the matching chunks. The larger the size of a matching chunk, the more likely it is that its corresponding region in the query data is valid and not irregular. Regions in the query data which cannot be composed from the referential data, or can only be composed using small fragmented pieces and not large chunks of the referential data, are considered irregular. The method and apparatus is applicable to all types of signals, including images, video data, medical data, one-dimensional signals and multi-dimensional signals, and can be used to identify, inter alia, suspicious behaviors, suspicious objects, irregular patterns and defects in goods.

Detecting irregularities

A method includes measuring the likelihood that two different space-time video segments could have resulted from a similar underlying motion field without computing the field. The method may be employed to identify locations in a video sequence where at least one behavioral phenomenon similar to that demonstrated in the video segment occurs. For example, the phenomenon might be a dynamic behavior, an action, a rigid motion, and/or a non-rigid motion.

Space-time behavior based correlation

A method includes finding regions of a reference signal which provide at least one of: local evidence scores and a global evidence score. The local evidence scores indicate local similarity of the regions of the reference signal to regions of a query signal and the global evidence score defines the extent of a global similarity of the query signal to the reference signal. A media exploring device is also included which includes an importance encoder and a media explorer. The importance encoder generates importance scores of at least portions of digital media as a function of at least one of local evidence scores and global evidence scores. The media explorer enables exploring through the digital media according to (i) the importance scores, (ii) data associations/links induced by the evidence scores between different portions of the digital media. The device may also include a media player to play the digital media with adaptive speeds as a function of the importance scores. The device may also include a labeling/annotation module which inherits labels/annotations/markings according to the abovementioned data associations.

Data similarity and importance using local and global evidence scores

A method and system for computing at least one of spatial and temporal relationships between at least first and second sequences of representations having respective first and second temporal progressions and including employing the first and second temporal progressions to obtain at least one of the spatial and temporal relationships.

Apparatus and method for alignment of spatial or temporal non-overlapping images sequences

A method and system for aligning in at least one of time and space temporally ordered sequences of images comprising receiving a plurality of sequences of images, each sequence including a multiplicity of images, each plurality of sequences defining a space-time volume, and providing an output indication relating at least one point in a space-time volume corresponding to one of the plurality of sequences to at least one point in a space-time volume corresponding to at least another one of the plurality of sequences.

Apparatus and method for spatio-temporal alignment of image sequences

A system and method for increasing space or time resolution of an image sequence by combination of a plurality input sources with different space-time resolutions such that the single output displays increased accuracy and clarity without the calculation of motion vectors. This system of enhancement may be used on any optical recording device, including but not limited to digital video, analog video, still pictures of any format and so forth. The present invention includes support for example for such features as single frame resolution increase, combination of a number of still pictures, the option to calibrate spatial or temporal enhancement or any combination thereof, increased video resolution by using high-resolution still cameras as enhancement additions and may optionally be implemented using a camera synchronization method.

System and method for increasing space or time resolution in video

A system for automatically generating a mosaic from a plurality of input images. The system sequentially executes an image alignment process, an editing process, and a combining process such that, from a sequence of images, the system automatically produces a seamless mosaic for various applications. Illustrative applications for the mosaic include: (1) a mosaic based display system including an image printing system, (2) a surveillance system and (3) a mosaic based compression system. The mosaic based display system permits a system user to display, manipulate and alter a mosaic. The mosaic based compression system exploits the temporal and spatial redundancy in image sequences and efficiently compresses the image information. The compression system can be used for compressing image information for storage in a storage device or can be used for compressing image information for transmission through a band-limited transmission channel. The surveillance system combines the mosaic construction system, the compression system and the display system into a comprehensive system for remote motion detection.

Mosaic based image processing system

A method for performing parametric image alignment, which when given any local match-measure, applies global estimation directly to the local match-measure data, without first going through an intermediate step of local flow estimation. The method can utilize any local match-measure, such as, correlation, normalized-correlation, squared or absolute brightness differences, statistical measures such as mutual information, and the like.

Method and apparatus for performing global image alignment using any local match measure

View synthesis from plural images using a trifocal tensor data structure in a multi-view parallax geometry

A method and apparatus for processing a geometric relationship between the image motion of pairs of points over multiple image frames representing a three-dimensional scene. This relationship is based on the parallax motion of points with respect to an arbitrary planar surface, and does not involve epipolar geometry. A constraint is derived over two frames for any pair of points, relating their projective structure (with respect to the plane) based only on their image coordinates and their parallax displacements. Similarly, a 3D-rigidity constraint between pairs of points over multiple frames is derived. Also disclosed are applications of these parallax-based constraints to solving three important problems in 3D scene analysis: (i) the recovery of 3D scene structure, (ii) the detection of moving objects in the presence of camera induced motion, and (iii) the synthesis of new camera views based on a given set of views. Moreover, this approach can handle difficult situations for 3D scene analysis, e.g., where there is only a small set of parallax vectors, and in the presence of independently moving objects.

Growing community of inventors

Rehovot, Israel

Michal Irani