International Business Machines Corporation

Columbia University

Koninklijke Philips Corporation N.v.

John Ronald Kender

Matthew Leon Hill

John Richard Smith

Apostol Ivanov Natsev

Lexing Xie

Bishwaranjan Bhattacharjee

Parijat Dube

Patrick D Watson

Tiecheng Liu

Michael Robert Glass

Noel C Codella

Siyu Huo

Brian Michael Belgodere

Rong Yan

Nevenka Dimitrova

Milind R Naphade

Lalitha Agnihotri

Radu Florian

Michele Merler

Hassan Malik

Hui Wan

Jelena Tesic

Quoc-Bao Nguyen

Yan Liu

Samuel D Fenster

Earl M Smith

Parul Awasthy

In an embodiment, a method for fine-tuning a pre-trained neural network for transfer learning, the method comprising obtaining a first target feature vector from a first layer of a pre-trained neural network responsive to a first target data element of a target dataset passing therethrough, obtaining a first source feature vector associated with the first layer of the pre-trained neural network, calculating a first divergence value for the first layer of the pre-trained neural network based at least in part on the first target feature vector and the first source feature vector, and setting a learning rate for the first layer of the pre-trained neural network based at least in part on the first divergence value.

Automated fine-tuning of a pre-trained neural network for transfer learning

A computer system, product, and method are provided. The computer system includes an artificial intelligence (AI) platform operatively coupled to a processor. The AI platform includes tools in the form of a machine learning model (MLM) manager, a metric manager, and a training manager. The MLM manager accesses a plurality of pre-trained source MLMs, and inputs a plurality of data objects of a test dataset into each of the source MLMs. The test dataset includes the plurality of data objects associated with respective labels. For each source MLM, associated labels are generated from the inputted data objects and a similarity metric is calculated. The MLM manager selects a base MLM to be used for transfer learning from the plurality of source MLMs based upon the calculated similarity metric. The training manager trains the selected base MLM with a target dataset for the target domain.

Predictive data and model selection for transfer learning in natural language processing

Whether to train a new neural network model can be determined based on similarity estimates between a sample data set and a plurality of source data sets associated with a plurality of prior-trained neural network models. A cluster among the plurality of prior-trained neural network models can be determined. A set of training data based on the cluster can be determined. The new neural network model can be trained based on the set of training data.

Training transfer-focused models for deep learning

A computer-implemented method for data labeling is provided. The computer-implemented method assigns pseudo-labels to unlabeled examples of data using a similarity metric on an embedding space to produce pseudo-labeled examples. A curriculum learning model is trained using the pseudo-labeled examples. The curriculum learning model trained with the pseudo-labeled examples is employed in in a fine-tuning task to enhance classification accuracy of the data.

Generating and augmenting transfer learning datasets with pseudo-labeled images

A system and method for analyzing visual memes includes identifying visual memes associated with at least one topic in a data source. The visual memes propagated over time are tracked to extract information associated with identified visual memes. The information associated with the visual memes is analyzed to determine at least one of generation, propagation, and use of the identified memes.

Visual meme tracking for social media analysis

A system and method for constructing a hierarchical multi-faceted classification structure includes organizing a plurality of visual categories into a multi-relational reference ontology that accounts for a plurality of different types of relationships. Media artifacts are categorized into the plurality of visual categories. The categories of artifacts are refined based on faceted ontology relationships or constraints from the multi-relational reference ontology. The multi-relational reference ontology and the one or more media artifacts with relationships are stored as the hierarchical multi-faceted classification structure in computer readable memory storage.

Multi-facet classification scheme for cataloging of information artifacts

Techniques for tracking one or more objects at each position in an interval in a video input with the use of a Kalman filter, including obtaining a first location estimate of an object with an object detector, obtaining a second location estimate and a movement estimate of the object with an object tracker, determining a final estimate of a position and/or a velocity of the object with the Kalman filter.

Kalman filter approach to augment object tracking

A system and method for analyzing video include segmenting video stored in computer readable storage media into keyframes. Near-duplicate keyframes are represented as a sequence of indices. The near-duplicate keyframes are rendered in a graphical representation to determine relationships between video content.

Using near-duplicate video frames to analyze, classify, track, and visualize evolution and fitness of videos

Apparatus, systems, and methods can operate to provide efficient data clustering, data classification, and data compression. A method comprises training set of training instances can be processed to select a subset of size-1 patterns, initialize a weight of each size-1 pattern, include the size-1 patterns in classes in a model associated with the training set, and then include a set of top-k size-2 patterns in a way that provides an effective balance between local, class, and global significance patterns. A method comprises processing a dataset to compute an overall significance value of each size-2 pattern in each instance in the dataset, sort the size-2 patterns, and select the top-k size-2 patterns to be represented in clusters, which can be refined into a clustered hierarchy. A method comprises creating an uncompressed bitmap, reordering the bitmap, and compressing the bitmap. Additional apparatus, systems, and methods are disclosed.

Data classification and hierarchical clustering

A technique for semantic video compression is shown in block (). Uncompressed video data (), including a plurality of video data segments (S, S, . . . Sn), are organized into two or more buffer slots (), such that each of the two or more buffer slots is filled with one or more of the received video data segments, thereby forming two or more buffered video portions corresponding to the two or more buffer slots. The buffered video data is then processed by a leaking rule, to extract one or more buffered video portions, while outputting one or more non-extracted buffered video portions, as compressed video data (). The leaking rule data is stored in a histogram () and later used to organize and index data according to a users request.

Method and device for online dynamic semantic video compression and video indexing

Each video segment in a plurality of video segments is annotated with an indicator of the likelihood that the respective video segment shows a particular feature. The plurality of video segments forms an episode of interest from a given video domain. Initial feature probabilities are calculated for respective ones of the plurality of video segments using a machine learning algorithm. Each initial feature probability indicates the likelihood that its respective video segment shows the particular feature. Refined feature probabilities are determined for respective ones of the plurality of video segments by finding the most probable state sequence in a finite state machine. This is accomplished at least in part using the determined initial feature probabilities. Finally, each of the video segments in the plurality of vides segments is annotated with its respective refined feature probability.

Annotating video segments using feature rhythm models

A method for selecting a subsequence of video frames (-) from a sequence of video frames () comprising defining a distance function between video frames (-) in the sequence of video frames (). An optimization criterion is defined to express a feature of a plurality of subsequences of video frames (-) selected from the sequence of video frames (). A method is disclosed for displaying key frames for browsing and streaming.

Methods for selecting a subsequence of video frames from a sequence of video frames

A method and apparatus are provided for segmenting and summarizing a music video () in a multimedia stream () using content analysis. A music video () is segmented in a multimedia stream () by evaluating a plurality of content features that are related to the multimedia stream. The plurality of content features includes at least two of a face presence feature; a videotext presence feature; a color histogram feature; an audio feature, a camera cut feature; and an analysis of key words obtained from a transcript of the at least one music video. The plurality of content features are processed using a pattern recognition engine (), such as a Bayesian Belief Network, or one or more video segmentation rules () to identify the music video () in the multimedia stream (). A chorus is detected in at least one music video () using a transcript (T) of the music video () based upon a repetition of words in the transcript. The extracted chorus may be employed for the automatic generation of a summary of the music video ().

Growing community of inventors

Leonia, NJ, United States of America

John Ronald Kender