Twitch Interactive, Inc.

Adobe, Inc.

Abhinav Kapoor

Jorge Arturo Villatoro

Yueshi Shen

Spencer Nelson

Cyrus Hall

Eran Ambar

Yuechuan Li

Shawn Hsu

Andrew Francis

Martin Hess

Jeffrey Garneau

Jyotindra Vasudeo

Chih-Chiang Lu

Matthew Louis Szatmary

Daniel Lin

Ivan Marcin

Luke Curley

Scott Wang

Maciej Laszcz

Daniel Arthur McClain

Diana Lu

Phillip Conover

Scott Rabin

Kevin Michael Streeter

Josh Tabak

Jeff Li

Asa Whillock

Stephen Cheng

Tarek Amara

Nikhil Purushe

Jonas Bengtson

Graham Keefe Tremper

Techniques for low latency streaming, for example in a broadcasting environment, are described herein. In some examples, a playlist may include both currently encoded segments, which are segments that are fully encoded at or before playlist generation, and also future encoded segments, which are segments that have not yet been fully encoded at playlist generation. In some cases, the inclusion of future encoded segments in a playlist may result in a player requesting a segment that has not yet been fully encoded at the time that the request is received by the server. In some examples, even though the segment is not yet fully encoded, the server may nevertheless save and process the request, for example by transmitting encoded portions of the requested segment as those portions are made available by the encoder.

Streaming playlist including future encoded segments

Techniques for low latency streaming, for example in a broadcasting environment, are described herein. In some examples, one or more individual renditions may be encoded into multiple rendition versions associated with different respective latencies. Also, in some examples, one or more individual renditions may be encoded into multiple rendition versions having different respective amounts of forward error correction (FEC), for example by an edge node of a video streaming service. Also, in some examples, video may be broadcast using a protocol that does not require retransmission of lost packets, such as Web Real-Time Communication (WebRTC), which is commonly used for point-to-point transmissions. Also, in some examples, one or more servers may receive quality of service feedback information from each player to which video content is transmitted. The one or more servers may use this feedback information to select and switch between appropriate renditions and rendition versions for each player.

Server-based encoded version selection

Various techniques for providing video feed redundancy are described herein. Instructions may be provided for switching input to an output video feed between two or more redundant input video feeds. In some examples, the redundant input video feeds may not be duplicates, may not be frame synchronized, may not be transmitted from the same location, may not be transmitted using the same network types or transmission protocols, and/or may not be initiated at the same time. In some examples, the instructions for video feed redundancy may be associated with respective authorization keys for the redundant input video feeds.

Video feed redundancy

Requesting transmission of future encoded segments

Intra-rendition latency variation

Techniques for estimation of bandwidth for transfer of video content in chunked transfer mode are described herein. A number of chunk samples may be collected during video content transmission. For each chunk sample, its respective delivery speed may be compared to its respective encoding bitrate. If the delivery speed is greater than or equal to the encoding bitrate, then the chunk sample may be included in a primary set of chunk samples. The primary set of chunk samples may be clustered into a lower speed cluster and a higher speed cluster. A representative speed, such as an average or weighted average speed, of the delivery speeds of the chunk samples in the lower speed cluster may be determined, and the bandwidth may be estimated based on the representative speed.

Chunked transfer mode bandwidth estimation

First streaming video content may be received from a provider. The provider may issue a request to replace part of the first streaming video content with replacement content, such as advertisements. Second streaming video content may be selected as a replacement for a first video player, while third streaming video content may be selected as a replacement for a second video player. Instructions may be sent to the first and the second video player to play a first portion of the first streaming video content, followed by their respective selected replacement content, followed by a third portion of the first streaming video content, with the respective selected replacement content replacing a second portion of the first streaming video content. Replacement of the second portion of the first streaming video content may sometimes introduce latency, which may be reduced using various techniques.

Session-specific streaming content replacement

Forward error correction for low latency streaming

Provider-requested streaming content replacement

Latency reduction for streaming content replacement

The techniques described herein may, for example, allow a quantity of buffered video frames to be reduced, for example for use in adaptive bitrate (ABR) streaming and other streaming transmissions. The quantity of buffered video frames may be reduced by dropping certain selected buffered frames from a resulting video output that is displayed to the viewer. This may reduce end-to-end latency without requiring a change in frame rate or fast forwarding of the video output, which may sometimes not be available or permitted by a resulting video player. These buffer reduction techniques may sometimes be employed upon a determination that transmission conditions have stabilized and that holding large quantities of buffered video may be unnecessary. In some examples, the disclosed techniques may be particularly advantageous for live streaming scenarios, in which end-to-end latency may be particularly problematic for the viewer.

Buffer reduction using frame dropping

Various embodiments herein include one or more of systems, methods, and software for multi-pane display capture, aggregation, and sharing. Some such embodiments include generating an image on a computing device to share with participants in a web conference meeting by capturing a first image of a first graphic display output layer and a second graphic display output layer and removing a portion of the first image. Some embodiments may also include capturing a second image of the second graphic display output layer and combining the first and second images to form a third image. The third image may then be sent to one or more participants of the web conference meeting. The first graphic display output layer may include an overlay pane graphical output of a computing device and the second graphic display output layer may include a standard pane graphical output of the computing device.

Multi-pane display capture, aggregation, and sharing

In general, in one aspect, streams associated with a playlist are received from a server. The playlist includes an ordering of streams that are queued for client-side playback from the server according to the ordering. Information associated with the receipt and playback of at least one of the streams is accessed. The client-side application requests to alter the streams of the playlist in response to the accessed information. The request identifies a desired playlist alteration and includes information that controls the playlist alteration. The streams of the playlist are displayed according to a playlist alteration made in response to the request.

Dynamically altering playlists

Growing community of inventors

Fremont, CA, United States of America

Abhinav Kapoor