Second Spectrum, Inc.

Genius Sports Ss, LLC

Jeffrey Wayne Su

Yu-Han Chang

Rajiv Tharmeswaran Maheswaran

Noel Hollingsworth

Edward Shek Chan

Emil Dotchevski

Jason Kent Simon

Data processing systems and methods are disclosed for combining video content with one or more augmentations to produce augmented video. Objects within video content may have associated bounding boxes that may each be associated with respective RGB values. Upon user selection of a pixel, the RGBA value of the pixel may be used to determine a bounding box associated with the RGBA value. The client may transmit an indicator of the determined bounding box to an augmentation system to request augmentation data for the object associated with the bounding box. The system then uses the indicator to determine the augmentation data and transmits the augmentation data to the client device.

Methods and systems of combining video content with one or more augmentations to produce augmented video

In various embodiments, a Data Processing System for Generating Interactive User Interfaces and Interactive Game Systems Based on Spatiotemporal Analysis of Video Content may be configured to: (1) enable a user to select one or more players participating in a substantially live (e.g., live) sporting or other event; (2) determine scoring data for each of the one or more selected players during the sporting or other event; (3) track the determined scoring data; (4) generate a custom (e.g., to the user) user interface that includes the scoring data; and (5) display the custom user interface over at least a portion of a display screen (e.g., on a mobile computing device) displaying one or more video feeds of the sporting or other event. In this way, the system may be configured to convert a video feed of a sporting event into an interactive game.

Data processing systems and methods for generating interactive user interfaces and interactive game systems based on spatiotemporal analysis of video content

A location in a video may be specified and content related to the location may be accessed. A method for accessing the related content may include receiving a reference to a pixel location in a frame of a video feed of a filmed occurrence and accessing a spatio-temporal index corresponding to the filmed occurrence. The spatio-temporal index may index information relating to events or objects of the filmed occurrence and corresponding pixel locations at which the events or the objects are detected in the video feed. The method may further include querying the spatio-temporal index using the pixel location to determine particular information of an indexed event or an indexed object and receiving the particular information wherein the particular information indicates at least one of spatial and temporal alignment parameters for aligning the indexed event with a corresponding event in at least one other video feed of the filmed occurrence.

Methods and systems of spatiotemporal pattern recognition for video content development

Presenting event-specific video content that conforms to a user selection of an event type includes processing at least one video feed through at least one spatiotemporal pattern recognition algorithm that uses machine learning to develop an understanding of at least one event within the at least one video feed to determine at least one event type, wherein the at least one event type includes an entry in a relationship library at least detailing a relationship between two visible features of the at least one video feed, extracting the video content displaying the at least one event and associating the understanding with the video content in a video content data structure. A user interface is configured to permit a user to indicate a preference for at least one event type that is used to retrieve and provide corresponding extracted video content with the data structure in a new video feed.

A media system generally includes a memory device that stores an event datastore that stores a plurality of event records, each event record corresponding to a respective event and event metadata describing at least one feature of the event. The media system (a) receives a request to generate an aggregated clip comprised of one or more media segments, where each media segment depicts a respective event; (b) for each event record from at least a subset of the plurality of event records, determines an interest level of the event corresponding to the event record; (c) determines one or more events to depict in the aggregated clip based on the respective interest levels of the one or more events; (d) generates the aggregated clip based on the respective media segments that depict the one or more events; and (e) transmits the aggregated clip to a user device.

Methods and systems of combining video content with one or more augmentations

Producing an event related video content data structure includes processing a video feed through a spatiotemporal pattern recognition algorithm that uses machine learning to develop an understanding of an event within the video feed. Developing the understanding includes identifying context information relating to the event and identifying an entry in a relationship library at least detailing a relationship between two visible features of the video feed. Content of the video feed that displays the event is automatically extracted by a computer and associated with the context information. A video content data structure that includes the context information is produced.

Interacting with a broadcast video content stream is performed with a machine learning facility that processes a video feed of a video broadcast through a spatiotemporal pattern recognition algorithm that applies machine learning on at least one event in the video feed in order to develop an understanding of the at least one event. Developing the understanding includes identifying context information relating to the at least one event and identifying an entry in a relationship library detailing a relationship between two visible features of the video feed. Interacting is further enabled with a touch screen user interface configured to permit at least one broadcaster to control a portion of the content of the video feed through interaction options that are based on the identified context information. Interacting is further enhanced through an interface configured to permit remote viewers to control the portion of the content.

Providing enhanced video content includes processing at least one video feed through at least one spatiotemporal pattern recognition algorithm that uses machine learning to develop an understanding of a plurality of events and to determine at least one event type for each of the plurality of events. The event type includes an entry in a relationship library detailing a relationship between two visible features. Extracting and indexing a plurality of video cuts from the video feed is performed based on the at least one event type determined by the understanding that corresponds to an event in the plurality of events detectable in the video cuts. Lastly, automatically and under computer control, an enhanced video content data structure is generated using the extracted plurality of video cuts based on the indexing of the extracted plurality of video cuts.

Data processing systems and methods for generating and interactive user interfaces and interactive game systems based on spatiotemporal analysis of video content

An enhanced video of an event in a first video feed, which is identified by a spatiotemporal pattern recognition algorithm that uses machine learning for understanding the event, is produced by including in the enhanced video an animation that characterizes a person's motions that are derived from a machine learning-based understanding of an event in a second video.

Growing community of inventors

South Pasadena, CA, United States of America

Jeffrey Wayne Su