Facebook, Inc.

Amit Puntambekar

Michael Hamilton Coward

Mathieu R Henaire

Minchuan Chen

Sonal Gandhi

Wurzel David Parsons-Keir

Melanie Sclar

Shankar Lakshmi Regunathan

Yaming He

Evgeny V Kuzyakov

David Young Joon Pio

Thomas George

Sabyasachi Roy

David Brandon Miller

Randall Eugene Rizun

Systems and methods for creating automatic digital representations of events may include (1) collecting data indicative of one or more emotions experienced by viewers of an event, (2) generating a timeline of the emotions based on the collected data, (3) creating a dynamic digital representation of the timeline, and (4) transmitting the dynamic digital representation to a device for display via a display element of the device. Various other methods, systems, and computer-readable media are also disclosed.

Automatic digital representations of events

An online system receives live stream content to be provided to one or more client devices as the live stream is ongoing. The online system generates a variety of transcoded live stream content to ensure that client devices can readily playback the appropriately encoded content. Once the live stream ends, the online system determines whether to use a transcoded live stream content or to use the original content received by the online system in order to generate a video on demand. To do so, the online system considers the quality of the original content and issues associated with the original content such as missing video frames or missing audio excerpts. The online system can choose to normalize the original content by eliminating the issues to improve the quality of the normalized original content and subsequently generate the video on demand from the normalized original content.

Live to video on demand normalization

An online system receives a media stream from a host of a live broadcasting stream and a second media stream from a guest of the host, and generates a single composite stream that includes the multiple live streams. The online system decodes and aligns the video and audio frames of each live stream to ensure that the composite stream displays the multiple live streams in a synchronized fashion. Additionally, the composite stream can display the multiple live streams in a particular format (e.g. side-by-side, or picture-in-picture), and therefore, the online system adjusts the video and audio frames of each live stream to fit that format. For each composite stream, the online system employs a decision engine that identifies the optimal set of encoded composite streams based on a variety of characteristics. The online system encodes the composite stream and distributes the appropriately encoded composite stream to client devices for display to viewers.

Presentation of composite streams to users

An online system receives content (e.g. video content and/or audio content) from a content provider device to be appropriately transcoded and distributed to client devices for consumption. During the transcoding process, the online system decodes the content into an intermediate format, and. considers a variety of characteristics to decide the optimal set of content encodings for encoding the decoded content. Characteristics include source characteristics associated with the content provider device, social graph characteristics of the user of the online system that is using the content provider device, viewer characteristics associated with the client devices that request to access the content, and content characteristics. The online system encodes the content for each client device with appropriate encoders selected based on the optimal set of content encodings. For each client device that requests to access the content, the online system provides encoded content that the client device can appropriately playback.

Decision engine for dynamically selecting media streams

Various of the disclosed embodiments relate to multiple video encoders that are used to simultaneously encode a video using encoders configured using different encoding parameters. A segment selector selects an encoded version of the encoded video segment using operational criteria such as video quality and bandwidth. A configuration determination module may analyze the video segment to make a decision about which encoding parameter configurations may be suitable for encoding the video segment. The configuration determination module may be trainable, based on past encoding results.

Ultra-high video compression

A method and system for allocating a variable number of bits per frame in a distributed video encoding using a complexity analyzer is disclosed. The method includes receiving an input video for allocating the number of bits for each segment based on the complexity of the segment. Further, the method includes splitting the input video into plurality of segments. Further, the method includes determining the number of bits to be allocated to the plurality of segments based on the complexity measurement of the input video. In an embodiment, single complexity analyzer can be used to determine the complexity of the plurality of segments. In another embodiment, separate complexity analyzer can be used for each segment to determine the complexity. Further, the method includes allocating the bits to the plurality of segments of the input video. Further, the method includes combining the plurality of segments to form a single output video; thereby, obtaining the encoded output video.

Variable bitrate control for distributed video encoding

In one embodiment, a method includes receiving multimedia content information associated with at least one segment of a multimedia content, receiving a request to view the at least one segment of the multimedia content from a client device, logging playback information associated with the viewing of the at least one segment of the multimedia content, determining a multimedia quality metric associated with the at least one segment of multimedia content based in part upon a portion of the received multimedia content information and a portion of the logged playback information, and classifying the at least one segment of the multimedia content with the multimedia quality metric.

Visual quality metrics

A neural network learning algorithm obtains a fingerprint of a video sequence. Using the fingerprint, selection of an appropriate video stabilization algorithm, and its parameter settings, is performed. The video thus stabilized is made more appealing for viewing and easier for subsequent video compression. The neural network may be trained during a training phase to select a correct fingerprint of each video sequence based on the effectiveness of resulting video stabilization and/or compression.

Neural network to optimize video stabilization parameters

Systems, methods, and non-transitory computer-readable media can present one or more base segments of a first stream of a content item in a viewport interface, the content item being composed using a set of streams that each capture at least one scene from a particular direction, wherein the viewport interface is provided through a display screen of the computing device. A determination is made that a direction of the viewport interface has changed to a different direction during playback of a first base segment of the first stream. One or more offset segments of a second stream that correspond to the different direction are presented in the viewport interface, the offset segments being offset from the set of base segments of the first stream.

Systems and methods for providing content

Video encoding operation is performed in a distributed video encoding system to meet a service layer agreement such as a target time for completion of the video encoding operation. Each of multiple videos being encoded is split into smaller jobs corresponding to video segments of the videos. The jobs are processed according to a job queue. The sequence of jobs in the job queue is periodically updated such that jobs corresponding to each video are processed within the corresponding target encoding time.

Distributed control of video encoding speeds

Video foreground and background detection is performed on a received video sequence on a macroblock by macroblock basis using motion vector encoding information associated with each macroblock. Some of the macroblocks in each frame, for example, edge or corner macroblocks, may be exempt from the foreground and background detection. The detection information, along with the video data, is passed to an image stabilization algorithm that prepares the video for subsequent video compression.

Foreground detection for video stabilization

A method and system for encoding a segment of an input video with accurate placement of key frames, e.g., I-Frames, using Group of Pictures (GOPs) length for output in a distributed video encoding system with two or more encoders is disclosed. The method includes creating overlapping segments in order to allow precise key frame placement. Further, the method includes using desired segment length and the desired key frame placement to create the overlapping segments. Further, the method includes skipping a certain number of frames at the beginning of the segment, and a certain number of frames at the end of the segment, thereby achieving accurate placement of key frames.

Key frame placement for distributed video encoding

An online system more efficiently streams multimedia content over the Internet for play back on client devices with varying computing power and network bandwidths by generating enhanced manifest files that more efficiently identify suitable media representations of the multimedia content. Each media representation has multiple media segments according to predefined byte ranges and a manifest file, which identifies location of the media file, bitrates, resolution, byte range, total duration, and other metadata. The online system customizes a manifest file for a user based on various factors including device capacity, network connectivity type and geolocation of the user. The online system also generates manifest fetch commands, which more efficiently fetch media segments for streaming. In response to changes of streaming server and media file (e.g., increased popularity), the online system dynamically updates corresponding manifest files.

Customizing manifest file for enhancing media streaming

Effectively fetch media content for enhancing media streaming

In one embodiment, a computer server machine generates a time map associated with the video stream. The video stream comprises one or more time slices. The computer server machine identifies a non-relevant time frame. The computer machine receives a comment and a localized comment time stamp, and associates the comment with one or more time slices. One or more client devices communicates a request for one or more time slices to the computer server machine, and the computer server machine determines whether the localized comment time stamp is within the requested one or more time slices. When the localized comment time stamp is within the requested one or more time slices, computer server machine transforms the localized time stamp to a synchronized comment time stamp. Computer server machine sends information regarding the synchronized comment time stamp to the one or more client devices.

Growing community of inventors

Fremont, CA, United States of America

Amit Puntambekar