Qualcomm Incorporated

Qualcomm Technologies, Inc.

Csr Technology Inc.

Zoran Corporation

Eran Pinhasov

Victor Pinto

Yuri Dolgin

Meir Tzur

Ron Gaizman

Yevgeni Litvin

Hee Jun Park

Tauseef Kazi

Young Hoon Kang

Nir Strauss

Scott Cheng

Costia Parfenyev

Maksym Aslianskyi

Mark Gintsburg

Eran Scharam

Anatoly Gurevich

Micha Galor

Hilla Ben-Yaacov

The present disclosure generally relates to image processing. For example, aspects of the present disclosure include systems and techniques for performing spatial and temporal processing of image data. Certain aspects provide an apparatus for processing frame data. The apparatus generally includes a memory, and one or more processors coupled to the memory, the one or more processors configured to: perform a first noise reduction operation based on first frame data via a machine learning component to generate first processed frame data; generate first feedback data based on the first processed frame data; and perform, via the machine learning component, a second noise reduction operation based on second frame data and the first feedback data.

Image signal processor

Techniques and systems are provided for processing one or more frames or images. For instance, a process for determining exposure for one or more frames includes obtaining a motion map for one or more frames. The process includes determining, based on the motion map, motion associated with the one or more frames of a scene. The motion corresponds to movement of one or more objects in the scene relative to a camera used to capture the one or more frames. The process includes determining, based on the determined motion, a number of frames and an exposure for capturing the number of frames. The process further includes sending a request to capture the number of frames using the determined exposure duration.

Frame processing and/or capture instruction systems and techniques

Examples are described for applying different settings for image capture to different portions of image data. For example, an image sensor can capture image data of a scene and can send the image data to an image signal processor (ISP) and a classification engine for processing. The classification engine can determine that a first object image region depicts a first category of object, and a second object image region depicts a second category of object. Different confidence regions of the image data can identify different degrees of confidence in the classifications. The ISP can generate an image by applying a different settings to the different portions of the image data. The different portions of the image data can be identified based on the object image regions and confidence regions.

Image processing based on object categorization

Techniques and systems are provided for machine-learning based image stabilization. In some examples, a system obtains a sequence of frames captured by an image capture device during a period of time, and collects motion sensor measurements calculated by a motion sensor associated with the image capture device based on movement of the image capture device during the period of time. The system generates, using a deep learning network and the motion sensor measurements, parameters for counteracting motions in one or more frames in the sequence of frames, the motions resulting from the movement of the image capture device during the period of time. The system then adjusts the one or more frames in the sequence of frames according to the parameters to generate one or more adjusted frames having a reduction in at least some of the motions in the one or more frames.

Image stabilization using machine learning

Embodiments may be directed to lens cameras which may be cameras arranged as a sensor in a lens cap. A lens camera may comprise a printed circuit board with a digital image sensor and associated components enclosed in a cylindrical body that may be constructed of metal, plastic, or the like, or combination thereof. Lens cameras may be fitted with lens mounts for attaching host devices, cameras, interchangeable lens, or the like. Lens mounts on a lens camera may be arranged to be compatible with one or more standard lens mounts. Accordingly, a lens camera may be attached to cameras that have compatible lens mounts. Also, interchangeable lens having lens mounts compatible with the lens camera may be attached to the lens camera. Further, lens cameras may communicate with host devices using wired or wireless communication facilities.

Camera having additional functionality based on connectivity with a host device

A method, apparatus, and manufacture for encoding a video sequence is provided. Encoding the video sequence includes performing a weighted prediction estimation between a reference frame of the video sequence and a target frame of the video sequence. Performing the weighted prediction includes performing an initial weighted prediction estimation for each block of the reference frame. Next, blocks are clustered according to their initial weighted prediction estimates. Then, each block of the target image is assigned to a corresponding region based on the clustering. During the video encoding, weighted-prediction is employed for each block according to its corresponding region.

Method, apparatus, and manufacture for local weighted prediction coefficients estimation for video encoding

A video encoder with an integrated temporal filter, which has an output carrying a sequence of encoded frames, includes: an input receiving input frame data; a temporal filter receiving the input frame data from the input and producing filtered frame data; motion processing modules receiving reconstructed frames derived from the sequence of encoded frames and the filtered frame data from the temporal filter and producing a residual error signal output; and a coder module which receives the input frame data and the residual error signal and produces the sequence of encoded frames. A method of encoding video includes: receiving a current frame to be encoded into a motion estimator and a temporal filter substantially at one time; receiving a previously encoded reference frame; generating a reconstructed motion-compensated reference frame from the previously encoded reference frame; and determining from the motion-compensated reference frame and the current frame whether and how much blending to perform between the motion compensated reference frame and the current frame.

Video encoder with an integrated temporal filter

Embodiments are directed towards modifying a quality of an image without affecting a bit rate associated with a video sequence of images. For each macro-block (MB) or sub-block within a MB for a target image to be encoded, various statistics are determined for a luminance component and chrominance components that provide at least average values for the chrominance components that may then be used to identify saturation thresholds. The saturation thresholds are used to determine a Qlevel. An average Qlevel from at least a previous frame is used to generate a Q modulation factor that may be combined with an activity based modulation factor and/or variance based modulation factor, or used singly. The final quantizer is calculated by bit-rate controller base quantizer multiplied by the Q modulation factor, and may be used to encode a MB or sub-block within a MB.

Quantization based on statistics and threshold of luminanceand chrominance

A system, method, and computer program product for digital stabilization of video data from cameras producing multiple simultaneous views, typically from rolling shutter type sensors, and without requiring a motion sensor. A first embodiment performs an estimation of the global transformation on a single view and uses this transformation for correcting other views. A second embodiment selects a distance at which a maximal number of scene points is located and considers only the motion vectors from these image areas for the global transformation. The global transformation estimate is improved by averaging images from several views and reducing stabilization when image conditions may cause incorrect stabilization. Intentional motion is identified confidently in multiple views. Local object distortion may be corrected using depth information. A third embodiment analyzes the depth of the scene and uses the depth information to perform stabilization for each of multiple depth layers separately.

Digital video stabilization for multi-view systems

A system, method, and computer program product for saturation insensitive weighted prediction coefficients estimation for video coding. The saturated pixels in frames of an input video are excluded from consideration in estimating weighted prediction coefficients that are then used to encode the input video. The saturated pixels include those having luma and/or chroma values above or below limits, and may be identified by use of a histogram. The saturated pixels may also be found by analyzing a lower resolution version of the frames, and/or sampling pixels in the frames. Embodiments enable improved video quality for a given bit rate, and increased coding efficiency, particularly for input video including frames with rapid pixel value changes. Separate portions of the frames may be processed separately, and the embodiments may execute within an optimization loop that minimizes a predetermined error function. The encoding format may include the H.264/AVC standard.

Saturation insensitive H.264 weighted prediction coefficients estimation

A method, apparatus, and manufacture for encoding a video sequence is provided. During a first exploitation phase of an encoding pass of the video encoding, macro-blocks are encoded employing at least one encoding parameter. At least one encoder statistic is evaluated based on the encoding of macro-blocks during the first exploitation phase. Next, during a first exploration phase of the encoding pass, macro-blocks are encoded employing at least one encoding parameter that is different than the encoding parameter(s) used during the first exploitation phase. At least one encoder statistic is evaluated based on the encoding of the macro-blocks during the first exploitation phase. The encoder statistic(s) based on the first exploration stage is compared when the encoder statistic(s) based on the first exploitation phase, and a steady-state value of the encoding parameter(s) is updated based on the comparison. The process may then continue to alternate between exploration and exploitation phases.

Method, apparatus, and manufacture for adaptation of video encoder tuning parameters

A method and device are provided for method for stabilization of image data by an imaging device. In one embodiment, a method includes detecting image data for a first frame and a second frame, performing motion estimation to determine one or more motion vectors associated with global frame motion for image data of the first frame, performing an outlier rejection function to select at least one of the one or more motion vectors, and determining a global transformation for image data of the first frame based, at least in part, on motion vectors selected by the outlier rejection function. The method may further include determining a stabilization transformation for image data of the first frame by refining the global transformation to correct for unintentional motion and applying the stabilization transformation to image data of the first frame to stabilize the image data of the first frame.

Method and apparatus for image stabilization

A system and method for automatically removing polarized light in an environment having polarized light and unpolarized light. Light is processed by a polarizer and measured by a sensor. Multiple measurements of a light characteristic are taken, each measurement corresponding to a polarity setting of the polarizer. The polarizer is automatically adjusted to enable the multiple measurements. The measurements may be of average light intensity, contrast, saturation, or another characteristic. Based on the multiple measurements and the corresponding polarity settings, an optimal polarity setting is determined, such that the amount of polarized light is minimized. A curve fitting calculation may be used to make the determination. The polarizer is adjusted to the determined polarity setting.

Growing community of inventors

Zichron Yaakov, Israel

Eran Pinhasov