Google Inc.

Stefan Holmer

Henrik Lundin

James Bankoski

Patrik Westin

Danil Chapovalov

In an embodiment, a computing system can include one or more processors and one or more non-transitory computer-readable media that store instructions that, when executed by the one or more processors, cause the computing system to perform operations. The operations can include: receiving an internal encoder state of an encoder running on a first computing device being used to participate in a video conference currently in progress; receiving data indicative of a second computing device being used to join the video conference; compressing, based at least in part on receipt of the data, the internal encoder state to generate a compressed internal encoder state of the encoder; and/or transmitting the compressed internal encoder state to the second computing device to synchronize the internal encoder state of the encoder running on the first computing device with an internal decoder state of a decoder running on the second computing device.

Videoconferencing with reduced quality interruptions upon participant join

Provided are methods and systems for estimating bandwidth in a communications network in which time slotting is occurring. A pre-filter is used to alter the distribution of transit deltas between data packets transmitted over the network to a receiver. By calculating the differences between arrival times of consecutive data packets received at a receiving end, and comparing the differences between the arrival times of the packets to the differences between corresponding send times for the packets, a determination is made as to whether such packets should be logically grouped together so as to constitute a single arrival event. The sum of the payload sizes of data packets within a group, together with timestamps for the arrival time and send time of the last packet included in the group is used as input to a bandwidth estimator.

Methods and systems for improved bandwidth estimation

Methods and systems are provided for transmitting multiple video streams to clients based on individual preferences of the clients and network conditions. A cost-function minimization problem is formulated based on various network parameters. The network parameters (e.g., round-trip time (RTT), available bandwidth, packet loss rate, etc.) may be inputs to this optimization problem, which attempts to find optimal resolutions, bitrates, etc., given the network parameters and the video content. Additionally, by incorporating quality prediction, the cost-function may be based on actual content of video sequences.

Multi-stream optimization

Disclosed herein is a method of transmitting a video bitstream to a first and at least a second endpoint with varying bandwidth capacities including identifying bandwidth capacities of the first and second endpoints, the second endpoint having less bandwidth capacity than the first endpoint, encoding at least a portion of the video bitstream to generate at least one version of a first data partition and a plurality of versions of at least a second data partition, the plurality of versions of the second data partition including at least one high quality version and at least one low quality version of the second data partition, transmitting the at least one version of the first partition and the at least one high quality version of the second partition to the first endpoint and transmitting the at least one low quality version of the second partition to the second endpoint.

System, method and apparatus for multipoint video transmission

A method and apparatus for performing motion estimation aided noise reduction encoding and decoding are provided. Motion estimation aided noise reduction encoding can include identifying a motion vector for encoding a current block in a video frame based on a reference block in a reference frame, identifying a noise reduction block for denoising the current block, aligning the noise reduction block with the current block, denoising the current block, identifying a motion vector for encoding the denoised block, generating a residual block for encoding the denoised block, and encoding the denoised block. Denoising the current block can include identifying a filter coefficient for a pixel in the current block based on a corresponding pixel in the noise reduction block, producing a denoised pixel based on the coefficient and the corresponding pixel, and determining whether to use the denoised pixel or the current pixel for encoding the block.

Motion estimation aided noise reduction

A system, apparatus, and method for estimating available bandwidth for transmitting a media stream over a network, the media stream having a plurality of frames. One method includes receiving some of the plurality of frames, each frame of the plurality of frames having a inter-frame size differential and an inter-arrival time differential, detecting whether at least some of the inter-arrival time differentials are outside of a steady-state range using at least some of the inter-frame size differentials, and estimating an available bandwidth based on the detected change and using a processor.

Apparatus and method for media transmission bandwidth control using bandwidth estimation

A buffer level for jitter data buffer is determined. A frame payload size difference is determined for a plurality of video frames encoded into data packets sequentially received from a network. The difference is a difference in a payload size of a current frame and a previous frame. A frame network transit delay is determined as a difference in a transport time between the current frame and the previous frame and an expected transport time between the current frame and the previous frame. A slope and a variance of a linear relationship between the frame payload size difference and the frame network transit delay are determined for the plurality of video frames. Finally, a buffer level of a jitter data buffer is determined using a maximum frame payload size, an average frame payload size, the slope and the variance.

Method and device for determining a jitter buffer level

Taught herein are techniques to incorporate temporal motion vector prediction in encoding and decoding a video stream. Motion vectors are temporally predicted for blocks of a frame. An extrapolated position for at least one block of a first frame is predicted in a second frame using a respective motion vector of the at least one block. A temporal motion vector for a block of the second frame is calculated as a combination of respective motion vectors of blocks overlapping the block of the second frame if at least one of the overlapping blocks is an extrapolated block at the extrapolated position. The temporal motion vector is used in a motion estimation algorithm to select a motion vector for the block, and the block is encoded using the selected motion vector.

Temporal motion vector prediction

Error concealment guided robustness may include identifying a current portion of a current video stream. Identifying the current portion may include identifying a feature, or a vector of features, for the current portion. An estimated vulnerability metric may be identified based on the feature and an associated learned feature weight. An error correction code for the current portion may be generated based on the estimated vulnerability metric. Error concealment guided robustness may include generating learned feature weights based on one or more training videos by generating vulnerability metrics for the training videos and identifying relationships between features of the training videos and the vulnerability metrics generated for the training videos.

Error concealment guided robustness

In one method for communicating, a server can receive a first encoded media stream from a first sending device and a second encoded media stream from a second sending device from a plurality of co-located sending devices. The server can determine spatial relationships between the plurality of co-located sending devices and a plurality of co-located receiving devices. The server can select a first receiving device as a destination for the first encoded media stream and a second receiving device as a destination for the second encoded media stream based on the sending and receiving spatial relationships. The server can send the first encoded media stream to the first receiving device and the second encoded media stream to the second receiving device to be decoded and rendered. The spatial relationships can be based on physical locations for the plurality of co-located sending and receiving devices.

Distributed communications

A system, apparatus, and method for rendering a video stream, the video stream having a plurality of frames. One method includes receiving the plurality of frames via a network, each frame having an estimated arrival time, determining a retransmission rendering delay, based on a round-trip delay and a round-trip delay constant, for a current frame of the plurality of frames using a processor, determining a render time for the current frame using a sum of the current frame's estimated arrival time and the retransmission rendering delay, and rendering the current frame at the render time.

Apparatus and method for rendering video with retransmission delay

It is disclosed a method for determining a buffer level of a jitter data buffer () comprised in a receiver () adapted to sequentially receive data packets from a communications network (), wherein frames are encoded into the data packets. Each frame may comprise timestamp information and payload size information. The buffer level is determined on the basis of a first part of frame arrival delay related to payload size variation between frames and a second part related to the amount of crosstraffic in the communications network (). It is further disclosed a receiver () adapted to sequentially receive data packets from a communications network (), wherein frames are encoded into the data packets, the receiver () comprising a data jitter buffer () and being adapted to perform the method.

Growing community of inventors

Stockholm, Sweden

Stefan Holmer