Qualcomm Incorporated

Yen-Chi Lee

Khaled Helmi El-Maleh

Ming-Chang Tsai

Gokce Dane

Vikram Makam Gupta

Peerapol Tinnakornsrisuphap

Christopher G Lott

Yiliang L Bao

Yan Ye

Chia-Yuan Teng

Kai Wang

Min Dai

Vikram Gupta

Nikolai Konrad Nepomuceno Leung

Nien-Chung Feng

Karl Ni

Jill Olson

Christopher Gerard Lott

Fan Ling

The disclosure is directed to techniques for encoder-assisted adaptive interpolation of video frames. According to the disclosed techniques, an encoder generates information to assist a decoder in interpolation of a skipped video frame, i.e., an S frame. The information permits the decoder to reduce visual artifacts in the interpolated frame and thereby achieve improved visual quality. The information may include interpolation equation labels that identify selected interpolation equations to be used by the decoder for individual video blocks. As an option, to conserve bandwidth, the equation labels may be transmitted for only selected video blocks that meet a criterion for encoder-assisted interpolation. Other video blocks without equation labels may be interpolated according to a default interpolation technique.

Encoder-assisted adaptive video frame interpolation

A video decoder performs a sequential error handling process to detect and conceal errors within a corrupted data segment of video data units. The decoder sequentially decodes a current data unit. Upon detecting an error, the decoder sets an error flag and resynchronizes decoding at the start of the next unit. If the error flag is set, the video decoder identifies the end of the corrupted data segment based on the start of the later unit. The decoder conceals data between the start of the current unit and the end of the corrupted data segment. If the error flag is not set, the decoder may decode the remainder of the current unit and proceed to decode the next available unit without performing error handling and concealment for the current unit. The decoder also may address reference unit mismatches caused by lost video data units.

Video decoder error handling

The disclosure is directed to techniques for picture-in-picture (PIP) processing for video telephony (VT). According to the disclosed techniques, a local video communication device transmits PIP information to a remote video communication device. Using the PIP information, the remote video communication device applies preferential encoding to non-PIP regions of video transmitted to the local video communication device.

Picture-in-picture processing for video telephony

The disclosure is directed to techniques for region-of-interest (ROI) processing for video telephony (VT) applications. According to the disclosed techniques, a recipient device defines ROI information for video information transmitted by a sender device, i.e., far-end video information. The recipient device transmits the ROI information to the sender device. Using the ROI information transmitted by the recipient device, the sender device applies preferential encoding to an ROI within a video scene. ROI extraction may be applied to process a user description of a region of interest (ROI) to generate information specifying the ROI based on the description. The user description may be textual, graphical, or speech-based. An extraction module applies appropriate processing to generated the ROI information from the user description. The extraction module may locally reside with a video communication device, or reside in a distinct intermediate server configured for ROI extraction.

Region-of-interest extraction for video telephony

Methods and systems for processing video data are described. A set of candidate motion vectors is selected from motion vectors associated with macroblocks in a first frame of video data and from motion vectors associated with macroblocks in a second frame of the video data. A statistical measure of the set is determined. The statistical measure defines a motion vector for a macroblock of interest in the second frame.

Temporal error concealment for video communications

One technique for implementing error control in response to a video data error comprises receiving an indication of a video data error from a video decoder, determining whether the video data error occurred on a reverse link of a wireless network between a video encoder and a network device, and applying error control in response to the video data error if the video data error did not occur on the reverse link. For example, the indication of the video data error may include a first sequence number (SN) of a packet comprising lost data, and determining whether the video data error occurred on the reverse link may include comparing first SN with a second SN of a packet associated with a most recent RL error.

Error filter to differentiate between reverse link and forward link video data errors

An apparatus performs efficient coding techniques to more efficiently locate motion vector data within neighboring video data units. The apparatus comprises a motion vector (MV) location unit that includes a look-up table (LUT), where the MV location unit obtains video data defining a plurality of video data units and processes the plurality of video data units. The apparatus further includes a geometric resolution unit that determines, while processing a current one of the plurality of video data units, which of the plurality of video data units neighbor the current video data unit. The MV location unit then accesses, for each of the neighboring video data units, the LUT to determine a location of a motion vector within a section of the video data to which the neighboring video data unit is associated.

Locating motion vectors for video data units

An apparatus performs efficient coding techniques to more efficiently resolve geometric relationships between video data units and thereby determine neighboring video data units for a current video data unit. The apparatus comprises a geometric resolution unit that obtains video data defining a plurality of video data units, and determines, for the current one of the plurality of video data units to be processed, a partition width and a video unit number of the current video data unit. The geometric resolution unit accesses, using the determined partition width and video unit number, a plurality of look-up tables (LUTs) to output one or more indices identifying one or more of the plurality of video data units that neighbor the current video data unit.

Resolving geometric relationships among video data units

An apparatus performs efficient coding techniques to more efficiently determine an availability of neighboring video data units determined for the current video data unit. The apparatus comprises an availability determination unit that maintains an availability counter. The availability determination unit determination unit increments the availability counter after each of the plurality of video data units are processed and determines, based on the availability counter, whether one or more of the plurality of video data units (i) have been previously processed and (ii) are available for use as neighboring video data units in the processing of a current one of the plurality of video data units video data units. The apparatus further processes the current video data unit based on the determination of whether one or more of the neighboring video data units are available.

Determining availability of video data units

The disclosure relates to techniques for video source rate control for video telephony (VT) applications. The source video encoding rate may controlled using a dual-buffer based estimation of a frame budget that defines a number of encoding bits available for a frame of the video. The dual-buffer based estimation technique may track the fullness of a physical video buffer and the fullness of the virtual video buffer. The source video encoding rate is then controlled based on the resulting frame budget. The contents of the virtual buffer depend on constraints imposed by a target encoding rate, while the contents of the physical buffer depend on constraints imposed by varying channel conditions. Consideration of physical video buffer fullness permits the video source rate control technique to be channel-adaptive. Consideration of virtual video buffer fullness permits the video source rate control technique to avoid encoding excessive video that could overwhelm the channel.

Video source rate control for video telephony

The disclosure relates to reverse link lower layer assisted video error control. A method may encode video data, form a packet with the encoded video data, and transmit the packet across a wireless channel to an access network. A medium access control (MAC) layer may receive a negative acknowledgement (NAK) from the access network. The method may determine whether the received NAK is associated with a packet that contains video data. If the received NAK is associated with a packet that contains video data, the method may perform error control.

Reverse link lower layer assisted video error control

The disclosure relates to video rate adaptation techniques that may use information from a medium access control (MAC) layer and radio link protocol (RLP) layer. The techniques may greatly reduce video delay by adjusting video encoding rate. For real-time video telephony (VT) applications, these techniques may provide graceful quality degradation and improve user experience, especially when the channel conditions degrade.

Growing community of inventors

San Diego, CA, United States of America

Yen-Chi Lee