Nvidia Corporation

Texas Instruments Corporation

Nvidia Corporaton

Prabindh Sundareson

Prakash Yadav

Shyam Raikar

Himanshu Bhat

Sachin Pandhare

Jonathan B White

Travis Muhlestein

Nathan Otterness

Jim Van Welzen

Jack Van Welzen

Dave Clark

Krishnakumar Gopalakrishnan

Anand Balagopalakrishnan

Shanmuga Sundaram Mahendran

Toshant Sharma

Apparatuses, systems, and techniques to predict events within a gameplay session and modify a game broadcast based at least in part on the predicted events. In at least one embodiment, events within the gameplay session are predicted, which once predicted, cause assets to be generated and once the event is detected the assets are included in the game broadcast.

Augmenting multimedia streaming by anticipating events using artificial intelligence

In various examples, potentially highlight-worthy video clips are identified from a gameplay session that a gamer might then selectively share or store for later viewing. The video clips may be identified in an unsupervised manner based on analyzing game data for durations of predicted interest. A classification model may be trained in an unsupervised manner to classify those video clips without requiring manual labeling of game-specific image or audio data. The gamer can select the video clips as highlights (e.g., to share on social media, store in a highlight reel, etc.). The classification model may be updated and improved based on new video clips, such as by creating new video-clip classes.

Video clip classification using feature vectors of a trained image classifier

In various examples, the decoding and upscaling capabilities of a client device are analyzed to determine encoding parameters and operations used by a content streaming server to generate encoded video streams. The quality of the upscaled content of the client device may be monitored by the streaming servers such that the encoding parameters may be updated based on the monitored quality. In this way, the encoding operations of one or more streaming servers may be more effectively matched to the decoding and upscaling abilities of one or more client devise such that an increased number of client devices may be served by the streaming servers.

Encoding output for streaming applications based on client upscaling capabilities

In various examples, durations of relatively high user activity within a gameplay session may be determined from user input events using a running user activity measurement. Once a duration is identified, it may be further analyzed to merge the duration with one or more other durations and/or to determine or predict whether the duration would be of sufficient interest for further action. A user interest score for an identified duration may be computed based on a set of the user input events that occur in the duration and used to determine and/or predict whether the duration would be of sufficient interest for further action. In some cases, an action may be performed based on determining the user interest score is greater than a statistical value that is computed from user interest scores of multiple identified durations.

Determining high-interest durations of gameplay sessions from user inputs

A game-agnostic event detector can be used to automatically identify game events. Game-specific configuration data can be used to specify types of pre-processing to be performed on media for a game session, as well as types of detectors to be used to detect events for the game. Event data for detected events can be written to an event log in a form that is both human- and process-readable. The event data can be used for various purposes, such as to generate highlight videos or provide player performance feedback.

Game event recognition

In various examples, recordings of gameplay sessions are enhanced by the application of special effects to relatively high(er) and/or low(er) interest durations of the gameplay sessions. Durations of relatively high(er) or low(er) predicted interest in a gameplay session are identified, for instance, based upon level of activity engaged in by a gamer during a particular gameplay session duration. Once identified, different variations of video characteristic(s) are applied to at least a portion of the identified durations for implementation during playback. The recordings may be generated and/or played back in real-time with a live gameplay session, or after completion of the gameplay session. Further, video data of the recordings themselves may be modified to include the special effects and/or indications of the durations and/or variations may be included in metadata and used for playback.

Automatic generation of video playback effects

Unsupervised classification of gameplay video using machine learning models

This patent application focuses on partitioning the computation to perform the image processing on the DSP, communicate the results to the GPU in an optimized manner, and use the GPU to render 3D information to the display. This invention solves the problem of seamlessly merging live streams from plural automotive cameras into a 3D display that can be rotated to see different view points. This purpose is achieved by partitioning the computation task between a digital signal processor (DSP) for image processing and a graphics processing unit (GPU) for rendering and display. This invention also includes meta data structures for communication of results from the DSP to the GPU.

Image synthesis method with DSP and GPU

This invention generates object-focused thumbnails from input images reflecting the mood and intention of the user, based on the original high-resolution picture. The invention includes edge detection, clustering detected edges into regions, ranking the regions and forming the thumbnail from a portion of the input image having a predetermined thumbnail size centered at a center of the highest ranking region. With this invention, the thumbnail accurately captures the focus of the image.

Method to generate thumbnails for digital images

An audio file is divided into frames in the time domain and each frame is compressed, according to a psycho-acoustic algorithm, into file in the frequency domain. Each frame is divided into sub-bands and each sub-band is further divided into split sub-bands. The spectral energy over each split sub-band is averaged for all frames. The resulting quantity for each split sub-band provides a parameter. The set of parameters can be compared to a corresponding set of parameters generated from a different audio file to determine whether the audio files are similar. In order to provide for the higher sensitivity of the auditory response, the comparison of individual split sub-bands of the lower order sub-bands can be performed. Selected constants can be used in the comparison process to improve further the sensitivity of the comparison. In the side-information generated by the psycho-acoustic compression, data related to the rhythm, i.e., related percussive effects, is present. The data known as attack flags can also be used as part of the audio frame comparison.

Growing community of inventors

Bengaluru, India

Prabindh Sundareson