Twitch Interactive, Inc.

Amazon Technologies, Inc.

Yueshi Shen

Abhinav Kapoor

Jorge Arturo Villatoro

Eran Ambar

Yuechuan Li

Spencer Nelson

Andrew Francis

Cyrus Hall

Shawn Hsu

Martin Hess

Chih-Chiang Lu

Jyotindra Vasudeo

Jeffrey Garneau

Ivan Marcin

Yongjun Wu

Ilya Vladimirovich Brailovskiy

Qi Keith Wang

Tarek Amara

Matthew Louis Szatmary

Naushirwan Navroze Patuck

Josh Tabak

Jeff Li

Rohit Puri

Luke Curley

Selvanayagam Sendurpandian

Nikhil Purushe

Alex Converse

Benjamin Hirsch

Xiangbo Li

Renganathan Veerasubramanian

Graham Keefe Tremper

Sangmook Jung

Alex Battaglia

A video streaming service may include a segmentation component that provides segment metadata that indicates starting locations for a plurality of segments of video content that are produced during transcoding of the video content. The segment metadata may be sent to a first transcode server and a second transcode server to allow alignment of segmentations between the first transcode server and the second transcode server. This may allow transcoding of the video content to be seamlessly switched from the first transcode server to the second transcode server, during transmission of the video content, without causing interruption of playback. The first transcode server and the second transcode server may also operate in parallel to produce different versions of the video content, such as versions encoded using different codecs. A video player may seamlessly switch between these different codec versions without causing interruption of playback.

Seamless transcode server switching

Techniques for low latency streaming, for example in a broadcasting environment, are described herein. In some examples, a playlist may include both currently encoded segments, which are segments that are fully encoded at or before playlist generation, and also future encoded segments, which are segments that have not yet been fully encoded at playlist generation. In some cases, the inclusion of future encoded segments in a playlist may result in a player requesting a segment that has not yet been fully encoded at the time that the request is received by the server. In some examples, even though the segment is not yet fully encoded, the server may nevertheless save and process the request, for example by transmitting encoded portions of the requested segment as those portions are made available by the encoder.

Streaming playlist including future encoded segments

Techniques for low latency streaming, for example in a broadcasting environment, are described herein. In some examples, one or more individual renditions may be encoded into multiple rendition versions associated with different respective latencies. Also, in some examples, one or more individual renditions may be encoded into multiple rendition versions having different respective amounts of forward error correction (FEC), for example by an edge node of a video streaming service. Also, in some examples, video may be broadcast using a protocol that does not require retransmission of lost packets, such as Web Real-Time Communication (WebRTC), which is commonly used for point-to-point transmissions. Also, in some examples, one or more servers may receive quality of service feedback information from each player to which video content is transmitted. The one or more servers may use this feedback information to select and switch between appropriate renditions and rendition versions for each player.

Server-based encoded version selection

Methods to switch between renditions of a video stream are generally described. In some examples, the methods may include encoding a video stream at a first image quality in a first rendition and a second, lower image quality in a second rendition. The methods may further include sending the first rendition to a recipient computing device. The methods may include receiving a request to switch from the first rendition to the second rendition. The methods may include determining that first indicator data of a first inter-coded frame indicates that the video stream can be switched to a lower image quality rendition at the first inter-coded frame. In some examples, the methods may further include sending the second rendition to the recipient computing device.

Rendition switch indicator

Methods of video streaming are generally described. In some examples, the methods may be performed by a server device and may include generating first intra-coded frame (I-frame) data and a plurality of enhancement layer data corresponding to the first I-frame data. The methods may further include sending the first I-frame data to a client device. The methods may further include transmitting first enhancement layer data to the client device. The methods may include sending first inter-coded frame data to the client device. In some examples, the first inter-coded frame data can be decoded by referencing first reference frame data generated by combining the first I-frame data and the first enhancement layer data.

Enhanced encoding and decoding of video reference frames

Requesting transmission of future encoded segments

Intra-rendition latency variation

Techniques for estimation of bandwidth for transfer of video content in chunked transfer mode are described herein. A number of chunk samples may be collected during video content transmission. For each chunk sample, its respective delivery speed may be compared to its respective encoding bitrate. If the delivery speed is greater than or equal to the encoding bitrate, then the chunk sample may be included in a primary set of chunk samples. The primary set of chunk samples may be clustered into a lower speed cluster and a higher speed cluster. A representative speed, such as an average or weighted average speed, of the delivery speeds of the chunk samples in the lower speed cluster may be determined, and the bandwidth may be estimated based on the representative speed.

Chunked transfer mode bandwidth estimation

Forward error correction for low latency streaming

Techniques for output frame correction for unstable video streams are described herein. A video item may be transmitted via an input video stream from a first entity to a second entity over one or more electronic communications networks. The incoming frames from the input video stream may then be used, by the second entity, to generate an output video stream for presentation to one or more viewers. The transmission of the input video stream may temporarily become unstable and may be interrupted such that one or more frames of the input video stream are delayed and/or lost. When a transmission interruption is detected, the output video stream may be adjusted by inserting one or more correction frames into the output video stream. The inserted correction frames may include one or more repetitions of one or more prior frames in the output video stream and/or one or more interpolated frames.

Output frame correction for unstable video streams

The techniques described herein may, for example, allow a quantity of buffered video frames to be reduced, for example for use in adaptive bitrate (ABR) streaming and other streaming transmissions. The quantity of buffered video frames may be reduced by dropping certain selected buffered frames from a resulting video output that is displayed to the viewer. This may reduce end-to-end latency without requiring a change in frame rate or fast forwarding of the video output, which may sometimes not be available or permitted by a resulting video player. These buffer reduction techniques may sometimes be employed upon a determination that transmission conditions have stabilized and that holding large quantities of buffered video may be unnecessary. In some examples, the disclosed techniques may be particularly advantageous for live streaming scenarios, in which end-to-end latency may be particularly problematic for the viewer.

Buffer reduction using frame dropping

Growing community of inventors

Cupertino, CA, United States of America

Yueshi Shen