Other

Amazon Technologies, Inc.

Videoken, Inc.

Arijit Biswas

Ankit Gandhi

Om Dadaji Deshmukh

Sumit Negi

Vineet Shashikant Chaoji

Anil Raghavendrachar Yelundur

Subhajit Sanyal

Kuldeep Yadav

Harish Arsikere

Kundan Shrivastava

Amrith Rajagopal Setlur

Sanket Sanjay Barhate

Sahil Loomba

Sonal S Patil

Sovan Biswas

Shiv Surya

Ranjeet Kumar

Systems and methods are disclosed to implement a neural network training system to train a multitask neural network (MNN) to generate a low-dimensional entity representation based on a sequence of events associated with the entity. In embodiments, an encoder is combined with a group of decoders to form a MNN to perform different machine learning tasks on entities. During training, the encoder takes a sequence of events in and generates a low-dimensional representation of the entity. The decoders then take the representation and perform different tasks to predict various attributes of the entity. As the MNN is trained to perform the different tasks, the encoder is also trained to generate entity representations that capture different attribute signals of the entities. The trained encoder may then be used to generate semantically meaningful entity representations for use with other machine learning systems.

Low-dimensional neural-network-based entity representation

Respective initial feature sets are obtained for the nodes of a graph in which the nodes represent instances of entity types and edges represent relationships. Using the initial feature sets and the graph, a graph convolutional model is trained to generate one or more types of predictions. In the model, a representation of a particular node at a particular hidden layer is based on aggregated representations of neighbor nodes, and an embedding produced at a final hidden layer is used as input to a prediction layer. The trained model is stored.

Artificial intelligence system employing graph convolutional networks for analyzing multi-entity-type multi-relational data

Techniques are generally described for context aware text-to-image synthesis. First text data comprising a description of an object may be received. A recurrent neural network may determine a first semantic representation data representing the first text data. A generator trained using a first generative adversarial network (GAN) may determine first image data representing the object using the first semantic representation. An encoder of a second GAN may generate a first feature representation of the first image data. The first feature representation may be combined with a projection of the first semantic representation data. A decoder of the second GAN may generate second image data representing the first text data.

Context aware text-to-image synthesis

A method and a system are provided for segmenting a multimedia content. The method estimates a count of a plurality of multimedia segments in the multimedia content, and a duration of each of the plurality of multimedia segments in the multimedia content. The method determines a cost function associated with a multimedia segment from the plurality of multimedia segments, based on the count of the plurality of multimedia segments, and the duration of each of the plurality of multimedia segments. The method further determines an updated count of the plurality of multimedia segments, and an updated duration of each of the plurality of multimedia segments until the cost function satisfies a pre-defined criteria. Based on the updated count of the plurality of multimedia segments, and the updated duration of each of the plurality of multimedia segments, the method segments the multimedia content into the plurality of multimedia segments.

Methods and systems for segmenting multimedia content

The disclosed embodiments illustrate methods of generation of a table of content by processing multimedia content. The method includes identifying a set of key-phrases from the multimedia content based on one or more external data sources. The method further includes determining one or more segments of the multimedia content, based on the identified set of key-phrases, wherein a segment of the determined one or more segments comprises a subset of key-phrases from the set of key-phrases. The method further includes selecting at least a key-phrase from the subset of key-phrases of each of the corresponding one or more segments. The method further includes generating the table of content based on the selected key-phrase from each of the one or more segments, wherein the selected key-phrase from each of the one or more segments in the generated table of content is utilized to navigate through the multimedia content.

Method and system for generation of a table of content by processing multimedia content

Features are extracted from visual and audio modalities of a video to infer the location of figures/tables/equations/graphs/flow-charts determined as video anchor points which are highlighted on the video timeline to enable quick navigation and provide a quick summary of the video. A voice-based mechanism navigates to a point-of-interest in the video. In case of bandwidth-constrained settings, videos are often played at a very low resolution (quality), and often users need to increase video resolution manually to understand content presented in the figures. Using the automatic identification of these aforementioned anchored points, the resolution can be changed dynamically during streaming a video, which will provide a better viewing experience.

System and methods to create multi-faceted index instructional videos

The disclosed embodiments illustrate method and system of processing multimedia content to generate a text transcript. The method includes segmenting each of a set of text frames to determine spatial regions. The method further includes extracting one or more keywords from each of the determined spatial regions. The method further includes determining the first set of keywords from the extracted one or more keywords based on filtering of one or more off-topic keywords from the extracted one or more keywords. The method further includes extracting a second set of keywords based on the determined first set of keywords. The method further includes generating a graph between each of a first set of keywords and one or more of a second set of keywords. The method further includes dynamically generating the text transcript of audio content in the multimedia content based on the generated graph.

Method and system for processing multimedia content to dynamically generate text transcript

A method for detecting one or more topic transitions in a multimedia content includes identifying, one or more frames from a plurality of frames of the multimedia content based on a comparison between one or more content items in a first frame of the plurality of frames, and the one or more content items in a first set of frames of the plurality of frames. The method further includes determining at least a first score, and a second score for each of the one or more frames. Additionally, the method includes determining a likelihood for each of the one or more frames based at least on the first score, and the second score, wherein the likelihood is indicative of a topic transition among the one or more frames.

Methods and systems for detecting topic transitions in a multimedia content

Embodiments disclose methods, systems and non-transitory computer readable medium for joint temporal segmentation and classification of user activities in an egocentric video. The method includes extracting low-level features from a live dataset based on predefined feature categories; determining at least one activity change frame from the egocentric video frames based on the extracted features; dividing the live dataset into partitions based on the activity change frame, each partition begins with a candidate frame; computing a recursive cost function at the candidate frame of each partition based on dynamic programming; determining a beginning time instant of the candidate frame based on the computation; segmenting the live dataset into multiple segments based on the determined time instant; identifying at least one activity segment that corresponds a user activity using a trained activity model based on multiple instance learning approach; and simultaneously associating a predefined activity label with the identified activity segment.

Joint temporal segmentation and classification of user activities in egocentric videos

According to embodiments illustrated herein, a method and system is provided for indexing a multimedia content. The method includes extracting, by one or more processors, a set of frames from the multimedia content, wherein the set of frames comprises at least one of a human object and an inanimate object. Thereafter, a body language information pertaining to the human object is determined from the set of frames by utilizing one or more image processing techniques. Further, an interaction information is determined from the set of frames. The interaction information is indicative of an action performed by the human object on the inanimate object. Thereafter, the multimedia content is indexed in a content database based at least on the body language information and the interaction information.

Growing community of inventors

Kolkata, India

Arijit Biswas