Apple Inc.

Albert E Keinath

Hsi-Jung Wu

Xiaosong Zhou

Yunfei Zheng

Chris Yoochang Chung

Ke Zhang

Shujie Liu

Jiefu Zhai

Dazhong Zhang

Douglas Scott Price

David R Conrad

Alexandros Michael Tourapis

Beibei Wang

James O Normile

Christopher M Garrido

Jae Hoon Kim

Jun Xin

Mei Guo

Karthick Santhanam

Ming Jin

Hang Yuan

Alican Nalci

Hilmi Enes Egilmez

Eric L Chien

Xiaohua Yang

Peikang Song

Yeqing Wu

Vikrant Kasarabada

Yixin Du

Gurtej Singh Chandok

Davide Concion

Bess C Chan

Giribalan Gopalan

An encoder or decoder can perform enhanced motion vector prediction by receiving an input block of data for encoding or decoding and accessing stored motion information for at least one other block of data. Based on the stored motion information, the encoder or decoder can generate a list of one or more motion vector predictor candidates for the input block in accordance with an adaptive list construction order. The encoder or decoder can predict a motion vector for the input block based on at least one of the one or more motion vector predictor candidates.

Enhanced motion vector prediction

Techniques are disclosed for generating virtual reference frames that may be used for prediction of input video frames. The virtual reference frames may be derived from already-coded reference frames and thereby incur reduced signaling overhead. Moreover, signaling of virtual reference frames may be avoided until an encoder selects the virtual reference frame as a prediction reference for a current frame. In this manner, the techniques proposed herein contribute to improved coding efficiencies.

Predictive video coding employing virtual reference frames generated by direct MV projection (DMVP)

Techniques are disclosed for coding video data in which frames from a video source are partitioned into a plurality of tiles of common size, and the tiles are coded as a virtual video sequence according to motion-compensated prediction, each tile treated as having respective temporal location of the virtual video sequence. The coding scheme permits relative allocation of coding resources to tiles that are likely to have greater significance in a video coding session, which may lead to certain tiles that have low complexity or low motion content to be skipped during coding of the tiles for select source frames. Moreover, coding of the tiles may be ordered to achieve low coding latencies during a coding session.

Efficient coding of source video sequences partitioned into tiles

Systems and processes for improved video editing, summarization and navigation based on generation and analysis of metadata are described. The metadata may be content-based (e.g., differences between neighboring frames, exposure data, key frame identification data, motion data, or face detection data) or non-content-based (e.g., exposure, focus, location, time) and used to prioritize and/or classify portions of video. The metadata may be generated at the time of image capture or during post-processing. Prioritization information, such as a score for various portions of the image data may be based on the metadata and/or image data. Classification information such as the type or quality of a scene may be determined based on the metadata and/or image data. The classification and prioritization information may be metadata and may be used to automatically remove undesirable portions of the video, generate suggestions during editing or automatically generate summary video.

Video analysis techniques for improved editing, navigation, and summarization

Techniques are disclosed for estimating quality of images in an automated fashion. According to these techniques, a source image may be downsampled to generate at least two downsampled images at different levels of downsampling. Blurriness of the images may be estimated starting with a most-heavily downsampled image. Blocks of a given image may be evaluated for blurriness and, when a block of a given image is estimated to be blurry, the block of the image and co-located blocks of higher resolution image(s) may be designated as blurry. Thereafter, a blurriness score may be calculated for the source image from the number of blocks of the source image designated as blurry.

Hierarchical sharpness evaluation

A system for processing media on a resource restricted device, the system including a memory to store data representing media assets and associated descriptors, and program instructions representing an application and a media processing system, and a processor to execute the program instructions, wherein the program instructions represent the media processing system, in response to a call from an application defining a plurality of services to be performed on an asset, determine a tiered schedule of processing operations to be performed upon the asset based on a processing budget associated therewith, and iteratively execute the processing operations on a tier-by-tier basis, unless interrupted.

Media analysis and processing framework on a resource restricted device

Techniques for cropping images containing an occlusion are presented. A method for image editing is presented comprising, when an occlusion is detected in an original digital image, determining an area occupied by the occlusion, assigning importance scores to different content elements of the original digital image, defining a cropping window around an area of the original digital image that does not include the area occupied by the occlusion and that is based on the importance scores, and cropping the original digital image to the cropping window.

Image/video editor with automatic occlusion detection and cropping

A method for processing media assets includes, given a first media asset, deriving characteristics from the first media asset, searching for other media assets having characteristics that correlate to the characteristics of the first media asset, when a match is found, deriving content corrections for the first media asset or a matching media asset from the other of the first media asset or the matching media asset, and correcting content of the first media asset or the matching media asset based on the content corrections.

Cross-asset media analysis and processing

Methods for organizing media data by automatically segmenting media data into hierarchical layers of scenes are described. The media data may include metadata and content having still image, video or audio data. The metadata may be content-based (e.g., differences between neighboring frames, exposure data, key frame identification data, motion data, or face detection data) or non-content-based (e.g., exposure, focus, location, time) and used to prioritize and/or classify portions of video. The metadata may be generated at the time of image capture or during post-processing. Prioritization information, such as a score for various portions of the image data may be based on the metadata and/or image data. Classification information such as the type or quality of a scene may be determined based on the metadata and/or image data. The classification and prioritization information may be metadata and may be used to organize the media data.

Hierarchical segmentation and quality measurement for video editing

An adaptive scaler switching system may implement multiple scalers including both a software scaler and a hardware scaler, and a controller that may manage the switch between scalers by considering the real-time constraints of the system and the available system resources. Information about the availability of system resources may be received in real-time, for example the controller may receive information about the system thermal status, the timing requirements for processing the video data, the quality of the scaled data, and any other relevant system statistics that may affect the scaler switch decision. According to an embodiment, the system may maintain statistics in a table, and update the table information as necessary.

Adaptive scaler switching

The invention is directed to an efficient way for encoding and decoding video. Embodiments include identifying different coding units that share a similar characteristic. The characteristic can be, for example: quantization values, modes, block sizes, color space, motion vectors, depth, facial and non-facial regions, and filter values. An encoder may then group the units together as a coherence group. An encoder may similarly create a table or other data structure of the coding units. An encoder may then extract the commonly repeating characteristic or attribute from the coding units. The encoder may transmit the coherence groups along with the data structure, and other coding units which were not part of a coherence group. The decoder may receive the data, and utilize the shared characteristic by storing locally in cache, for faster repeated decoding, and decode the coherence group together.

Coherence groups: region descriptors for low bit rate encoding

Growing community of inventors

Sunnyvale, CA, United States of America

Albert E Keinath