Qualcomm Technologies, Inc.

Intel Corporation

Csr Technology Inc.

Qualcomm Incorporated

Intel Corportation

Noam Levy

Meir Tzur

Noam Elron

Guy Ben-Artzi

Shahar S Yuval

Jarno Nikkanen

German Voronov

Amichay Amitay

Guy Rapaport

Edmond Chalom

Dmitry Rudoy

Liron Lvov

Oskar Pelc

Harish Essaky Sankaran

Amir Goren

Alexander Itskovich

A method, system, and article is directed to real-time super-resolution image processing using neural networks.

Method and system of real-time super-resolution image processing

Techniques are provided for generation of secure video and tamper detection of the secure video. A methodology implementing the techniques according to an embodiment includes selecting a subset of macroblocks from a video frame to be transmitted and calculating a low frequency metric on each of the selected macroblocks. The method also includes performing a hash calculation on the low frequency metrics to generate a frame signature, encrypting the frame signature (using a private key) to generate an encrypted watermark, and modifying pixels of each of the selected macroblocks to generate the secured video frame, the modifications based on bits of the encrypted watermark that are associated with the selected macroblock. The method further includes authenticating a received video frame by comparing a calculated frame signature to an authenticated frame signature, the authenticated frame signature decrypted (using a public key) from an extracted watermark of the received video frame.

Detection of video tampering

Techniques are provided for generation of secure video and tamper detection of the secure video. A methodology implementing the techniques according to an embodiment includes selecting a subset of macroblocks from a video frame to be transmitted and calculating a low frequency metric on each of the selected macroblocks. The method also includes performing a hash calculation on the low frequency metrics to generate a frame signature; encrypting the frame signature (using a private key) to generate an encrypted watermark; and modifying pixels of each of the selected macroblocks to generate the secured video frame, the modifications based on bits of the encrypted watermark that are associated with the selected macroblock. The method further includes authenticating a received video frame by comparing a calculated frame signature to an authenticated frame signature, the authenticated frame signature decrypted (using a public key) from an extracted watermark of the received video frame.

An example apparatus for video frame segmentation includes a receiver to receive a current video frame to be segmented. The apparatus also includes a segmenting neural network to receive a previous mask including a segmentation mask corresponding to a previous frame and generate a segmentation mask for the current frame based on the previous mask and the video frame.

Video frame segmentation using reduced resolution neural network and masks from previous frames

Techniques related to temporal noise reduction in captured video are discussed. Such techniques include performing motion estimation on a portion of a downsampled current frame performed during the downsampling of the current frame, replacing one or more of the resultant motion vectors based on confidence scores of the resultant motion vector, and blending the current frame and a temporally previous frame to generate a temporally filtered current frame. The temporally filtered current frame may be displayed to a user and/or encoded to generate a bitstream.

Local motion compensated temporal noise reduction with sub-frame latency

Embodiments are directed towards enabling digital cameras to create a 3D view, which can be re-rendered onto any object within a scene, so that it is both in focus and a center of perspective, based on capturing a single set of multiple 2D images of the scene. From capturing a single set of 2D images for a scene, a depth map of the scene may be generated, and used to calculate principal depths, which are then used to capture an image focused at each of the principal depths. A correspondence between a 2D image of the scene and the principal depths are determined that corresponds to a specific principal depth. For different coordinates of the 2D image, different 3D views of the scene are created that are each focused at a principal dept that corresponds to the given coordinate.

System and method for real time 2D to 3D conversion of a video in a digital camera

Methods, apparatuses and systems may provide for operating a machine learning device by obtaining training image data, conducting an offline prediction analysis of the training image data with respect to one or more real-time parameters of an image capture device, and generating one or more parameter detection models based on the offline prediction analysis. Additionally, methods, apparatuses and systems may provide for operating the image capture device by obtaining a candidate image associated with the image capture device, determining that the candidate image corresponds to a particular type of scene represented in a parameter prediction model, and adjusting one or more real-time parameters of the image capture device based at least in part on one or more parameter values associated with the particular type of scene.

Machine learning of real-time image capture parameters

Embodiments are directed towards enabling digital cameras to digitally process a captured a Low Dynamic Range image sequence at a real time video rate, and to convert the image sequence into an High Dynamic Range (HDR) image sequence using a pipelined architecture. Two or more image frames are captured using different exposure settings and then combined to form a single HDR output frame in a video sequence. The pipelined architecture operate on adjacent image frames by performing an image alignment, an image mixing, and a tone mapping on the adjacent image frames to generate the HDR image sequence.

Device and algorithm for capturing high dynamic range (HDR) video

A camera that provides for a panorama mode of operation that stitches together two or more images to create a single image with a wide format. In panorama mode, a live view of a scene is transformed from rectangular coordinates to cylindrical coordinates and displayed by the camera. Also, an overlap portion between the previous image and the next image to be captured is characterized. In real time, after the previous image is captured, the overlap portion of the previous image is also transformed from rectangular coordinates into cylindrical coordinates. Next, the camera displays an overlay of at least a portion of the overlap portion onto the live view of the next image to be captured. And this overlay can assist the user in aligning the live view of the next image to be captured with the overlap portion of the previously captured image.

In-camera panorama image stitching assistance

A digital camera system for super resolution image processing constructed to receive a plurality of input frames and output at least one digitally zoomed frame is provided. The digital camera system includes a motion registration module configured to generate motion information associated with the plurality of input frames, an interpolation module configured to generate a plurality of interpolated input frames based at least in part on the plurality of input frames and the motion information, a weights calculation module configured to calculate one or more weights associated with the plurality of input frames based on at least the motion information, and a weighted merging module configured to merge the plurality interpolated input frames consistent with the one or more weights to generate the at least one digitally zoomed frame.

System and method for multiple-frame based super resolution interpolation for digital cameras

Embodiments are directed towards determining within a digital camera whether a pixel belongs to a foreground or background segment within a given image by evaluating a ratio of derivative and deviation metrics in an area around each pixel in the image, or ratios of derivative metrics across a plurality of images. For each pixel within the image, a block of pixels are examined to determine an aggregate relative derivative (ARD) in the block. The ARD is compared to a threshold value to determine whether the pixel is to be assigned in the foreground segment or the background segment. In one embodiment, a single image is used to determine the ARD and the pixel segmentation for that image. Multiple images may also be used to obtain ratios of a numerator of the ARD, useable to determine an extent of the foreground.

Method for determining the extent of a foreground object in an image

Embodiments are directed towards enabling digital cameras to digitally process captured two dimensional image sequences at a real time video rate, to convert the two dimensional image sequences into stereoscopic three dimensional image sequences. In one embodiment, using a pipelining architecture, various statistics are obtained for a captured two dimensional sequence. The statistics are used to estimate depth within each frame image within the sequence, in real time. Using the depth data a disparity map is generated that is provided to a warping component to generate a second perspective of the two dimensional image. The two images for the frame provide a three dimensional perspective for the frame within the sequence. Together the two perspective images for the frame are provided to a video encoder component to encode the stereoscopic three dimensional frame for the sequence.

System and method for real time 2D to 3D conversion of video in a digital camera

A camera that provides for a panorama mode of operation that employs internal software and internal acceleration hardware to stitch together two or more captured images to create a single panorama image with a wide format. Captured images are projected from rectilinear coordinates into cylindrical coordinates with the aid of image interpolation acceleration hardware. Matches are quickly determined between each pair of images with a block based search that employs motion estimation acceleration hardware. Transformation are found, utilizing regression and robust statistics techniques, to align the captured images with each other, which are applied to the images using the interpolation acceleration hardware. A determination is made for an optimal seam to stitch images together in the overlap region by finding a path which cuts through relatively non-noticeable regions so that the images can be stitched together into a single image with a wide panoramic effect.

Robust fast panorama stitching in mobile phones or cameras

A device and methods are provided for low-light imaging enhancement by a digital imaging device. In one embodiment, a method includes detecting an image associated with ambient lighting of a scene, detecting an image associated with artificial lighting of the scene, aligning the image associated with ambient lighting relative to the image associated with artificial lighting based on a motion parameter of ambient lighting image data and artificial lighting image data, and calculating data for a combined image based on aligned ambient lighting image data and artificial lighting image data, wherein image data for the combined image is selected to maximize an objective quality criterion of the combined image. The method may further include determining an image parameter based on ambient lighting image data, blending image data associated with the artificial lighting and the combined image based on the image parameter to generate a tone rendered image.

Growing community of inventors

Karmiel, Israel

Noam Levy