Disney Enterprises, Inc.

Brown University

Qualcomm Incorporated

Carnegie Mellon University

Royal Bank of Canada

Leonid Sigal

Michael Julian Black

Jessica Kate Hodgins

Matthew M Loper

Alexander W Weiss

Yanwei Fu

Alexandru O Balan

Timothy S St Clair

Vincent Douglas Park

Alan William O'Neill

Richard John Dynarski

David R Mazik

Zuxuan Wu

Sung Ju Hwang

Michail Raptis

Georgios Tsirtsis

Anthony M Accardo

Marc Junyent Martin

Aljoscha Smolic

Nimesh C Narayan

Tian Lan

Boyang Li

Jordi Pont-Tuset

Arik Shamir

Pelin Dogan

Atousa Torabi

Guy Rosenthal

Gunhee Kim

Pablo Beltran

Andreas Lehrmann

Ivan Poupyrev

Miquel Angel Farre Guiu

M Scott Corson

George Tsirtsis

Kenneth John Mitchell

Philip John Jackson

Yaser Sheikh

Takaaki Shiratori

Mathew Scott Corson

Ariel Shamir

Gregory Mori

Moshe Benjamin Mahler

Andreas Steffen Michael Lehrmann

Daniel Leithinger

Sheldon Paul Andrews

Erika Elizabeth Varis Doggett

Megha Nawhal

Adrien Treuille

Shugao Ma

Ivan Huerta Casado

Paul Hongsuck Seo

Oz Radiano

Mengyao Zhai

Miguel Angel Farre Guiu

Seungwhan Moon

Alina Kuznetsova

Jonghyun Choi

Edilson De Aguiar

Nataliya Shapovalova

Marek Vondrak

Peng Guan

Anna M C Wolak

Mykhaylo Andriluka

Matthew Scott Corson

Hyun Soo Park

Alan O'neill

Scott M Corson

George Flarion Technologies Inc Tsirtsis

Techniques are disclosed for recognizing aspect ratio errors in image frames of a video and reporting the same. An aspect ratio checker application receives a video that includes multiple image frames and identifies aspect ratio changes in those image frames using a first differential of a time series that includes determined positions of the top, bottom, left, and right of content regions within the image frames. In particular, the aspect ratio checker may identify aspect ratio changes based on non-zero points of the first differential, and the aspect ratio checker further determines aspect ratios of content regions within image frames corresponding to the non-zero points of the first differential. In addition, the aspect ratio checker may generate and display a report indicating the determined aspect ratios and image frame ranges associated with those aspect ratios.

Aspect ratio error recognition

Systems, methods and articles of manufacture for modeling a joint language-visual space. A textual query to be evaluated relative to a video library is received from a requesting entity. The video library contains a plurality of instances of video content. One or more instances of video content from the video library that correspond to the textual query are determined, by analyzing the textual query using a data model that includes a soft-attention neural network module that is jointly trained with a language Long Short-term Memory (LSTM) neural network module and a video LSTM neural network module. At least an indication of the one or more instances of video content is returned to the requesting entity.

Joint heterogeneous language-vision embeddings for video tagging and search

A computer implemented system for generating one or more data structures is described, the one or more data structures representing an unseen composition based on a first category and a second category observed individually in a training data set. During training of a generator, a proposed framework utilizes at least one of the following discriminators&#x2014;three pixel-centric discriminators, namely, frame discriminator, gradient discriminator, video discriminator; and one object-centric relational discriminator. The three pixel-centric discriminators ensure spatial and temporal consistency across the frames, and the relational discriminator leverages spatio-temporal scene graphs to reason over the object layouts in videos ensuring the right interactions among objects.

System and method for generation of unseen composite data objects

Methods and apparatus for storing, manipulating, retrieving, and forwarding state, e.g., context and other information, used to support communications sessions with one or more end nodes, e.g., mobile devices, are described. Various features are directed to a mobile node controlling the transfer of state from a first access node to a second access node during a handoff operation thereby eliminating any need for state transfer messages to be transmitted between the second access node and the first access node during handoff. Other features of the invention are directed to the use of a core network node to store state information. State information stored in the core node can be accessed and used by access nodes in cases where a mobile node does not send a state transfer message during a handoff, e.g., because communication with the first access node is lost or because such messages are not supported.

Enhanced techniques for using core based nodes for state transfer

There is provided a system comprising a label database including a plurality of label, a non-transitory memory storing an executable code, and a hardware processor executing the executable code to receive a media content including a plurality of segments, each segment including a plurality of frames, extract a first plurality of features from a segment, extract a second plurality of features from each frame of the segment, determine an attention weight for each frame of the segment based on the first plurality of features extracted from the segment and the second plurality of features extracted from the segment, and determine that the segment depicts one of the plurality of labels in a label database based on the first plurality of features, the second plurality of features, and the attention weight of each frame of the plurality of frames of the segment.

Systems and methods for determining actions depicted in media contents based on attention weights of media content frames

There is provided a system including a computing platform having a hardware processor and a memory, and a metadata extraction and management unit stored in the memory. The hardware processor is configured to execute the metadata extraction and management unit to extract a plurality of metadata types from a media asset sequentially and in accordance with a prioritized order of extraction based on metadata type, aggregate the plurality of metadata types to produce an aggregated metadata describing the media asset, use the aggregated metadata to include at least one database entry in a graphical database, wherein the at least one database entry describes the media asset, display a user interface for a user to view tags of metadata associated with the media asset, and correcting presence of one of the tags of metadata associated with the media asset, in response to an input from the user via the user interface.

Metadata extraction and management

Systems, methods and computer program products related to aligning heterogeneous sequential data are disclosed. Video data in a media presentation and textual data corresponding to content of the media presentation are received. An action related to aligning the video data and the textual data is determined using an alignment neural network, such that the video data and the textual data are at least partially aligned following the action. The alignment neural network includes a first fully connected layer that receives as input the video data, the textual data, and data relating to a previously determined action by the alignment neural network related to aligning the video data and the textual data. The determined action related to aligning the video data and the textual data is performed.

Alignment of video and textual sequences for metadata analysis

Training data from multiple types of sensors and captured in previous capture sessions can be fused within a physics-based tracking framework to train motion priors using different deep learning techniques, such as convolutional neural networks (CNN) and Recurrent Temporal Restricted Boltzmann Machines (RTRBMs). In embodiments employing one or more CNNs, two streams of filters can be used. In those embodiments, one stream of the filters can be used to learn the temporal information and the other stream of the filters can be used to learn spatial information. In embodiments employing one or more RTRBMs, all visible nodes of the RTRBMs can be clamped with values obtained from the training data or data synthesized from the training data. In cases where sensor data is unavailable, the input nodes may be unclamped and the one or more RTRBMs can generate the missing sensor data.

Deep-learning motion priors for full-body performance capture in real-time

Systems, methods and articles of manufacture for training a convolutional neural network for feature recognition within digital images. A spatial context neural network is trained using a plurality of patches cropped from a plurality of digital images, the spatial context neural network comprising a first convolutional neural network configured to predict a feature representation for a first specified portion of a first digital image, a second convolutional neural network configured to compute a feature representation for a second specified portion of a second digital image, and a spatial context module that accepts output of the first and second convolutional neural networks as input. The second convolutional neural network is refined by regressing features of the second convolutional neural network to features of the first convolutional neural network. The refined second convolutional neural network is used to recognize one or more features within a third digital image.

Weakly-supervised spatial context networks to recognize features within an image

A dialog engine configured to answer a sequence of questions related to an image. An attention module residing in the dialog engine includes an attention retrieval process and a tentative attention process. The attention retrieval process retrieves a relevant attention map that may have previously been used to answer a previous query. The tentative attention process generates a tentative attention map based on an image and other input parameters. The attention module combines the relevant attention map and the tentative attention map to generate a fused attention map. Based on the fused attention map, the dialog engine generates a response to the query. Finally, the dialog engine stores the fused attention map in an attention memory for use in answering future queries.

Visual reference resolution using attention memory for visual dialog

Techniques are disclosed for identifying and filtering outliers from a sample set of data prior to training a classifier on an object using the sample set. A data set including a plurality of samples used to train a classification model is retrieved. The samples in the data set have a feature dimensionality. A graph of the data set is built. Each node in the graph corresponds to a sample in the data set and edges connecting the nodes correspond to a measure of similarity between the nodes. The feature dimensionality of the sample data set is reduced based on a topology of the graph. One or more outliers in the data set are identified based on the reduced feature dimensionality.

Robust classification by pre-conditioned lasso and transductive diffusion component analysis

Systems, methods and computer program products related to aligning heterogeneous sequential data. A first sequential data stream and a second sequential data stream are received. An action related to aligning the first sequential data stream and the second sequential data stream is determined using an alignment neural network. The alignment neural network includes a fully connected layer that receives as input: data from the first sequential data stream, data from the second sequential data stream, and data relating to a previously determined action by the alignment neural network related to aligning the first sequential data stream and the second sequential data stream.

A system and method of estimating the body shape of an individual from input data such as images or range maps. The body may appear in one or more poses captured at different times and a consistent body shape is computed for all poses. The body may appearin minimal tight-fitting clothing or in normal clothing wherein the described method produces an estimate of the body shape under the clothing. Clothed or bare regions of the body are detected via image classification and the fitting method is adapted to treat each region differently. Body shapes are represented parametrically and are matched to other bodies based on shape similarity and other features. Standard measurements are extracted using parametric or non-parametric functions of body shape. The system components support many applications in body scanning, advertising, social networking, collaborative filtering and Internet clothing shopping.

Method and apparatus for estimating body shape

There is provided a system including a non-transitory memory storing an executable code and a hardware processor executing the executable code to receive a plurality of training contents depicting a plurality of activities, extract training object data from the plurality of training contents including a first training object data corresponding to a first activity, extract training scene data from the plurality of training contents including a first training scene data corresponding to the first activity, determine that a probability of the first activity is maximized when the first training object data and the first training scene data both exist in a sample media content.

Systems and methods for identifying activities and/or events in media contents based on object data and scene data

A system and method of estimating the body shape of an individual from input data such as images or range maps. The body may appear in one or more poses captured at different times and a consistent body shape is computed for all poses. The body may appear in minimal tight-fitting clothing or in normal clothing wherein the described method produces an estimate of the body shape under the clothing. Clothed or bare regions of the body are detected via image classification and the fitting method is adapted to treat each region differently. Body shapes are represented parametrically and are matched to other bodies based on shape similarity and other features. Standard measurements are extracted using parametric or non-parametric functions of body shape. The system components support many applications in body scanning, advertising, social networking, collaborative filtering and Internet clothing shopping.

Growing community of inventors

Pittsburgh, PA, United States of America

Leonid Sigal