International Business Machines Corporation

Udi Barzelay

Ophir Azulai

Daniel Nechemia Rotman

Rami Ben-Ari

Dror Porat

Tal Hakim

Elad Amrani

Mattias Marder

Yochay Tzur

Yevgeny Yaroker

Oshri Pesah Naparstek

Oded Margalit

Joseph Shtok

Slava Shechtman

Inbar Shapira

Yevgeny Burshtein

Mor Shpigel Nacson

Approaches presented herein enable action recognition. More specifically, a plurality of video segments having one or more action representations is received. One or more sub-action representations in the plurality of video segments are learned. An embedding in a space of a distance metric learning (DML) network for each of the one or more sub-action representations is determined. A set of respective trajectory distances between each of the one or more sub-action representations and one or more class representatives in the space of the DML network based on the embedding is computed, and the one or more action representations based on the set of respective trajectory distances are classified.

Action recognition using limited data

Described are techniques for font attribute detection. The techniques include receiving a document having different font attributes amongst a plurality of words respectively comprised of at least one character. The techniques further include generating a dense image document from the document by setting the plurality of words to a predefined size, removing blank spaces from the document, and altering an order of characters relative to the document. The techniques further include determining characteristics of the characters in the dense image document and aggregating the characteristics for at least one word. The techniques further include annotating the at least one word with a font attribute based on the aggregated characteristics.

Font attribute detection

A method, computer system, and a computer program product for text detection is provided. The present invention may include training a text detection model. The present invention may include performing text detection on an inputted image using the trained text detection model. The present invention may include determining whether at least one of a plurality of bounding boxes generated using the inputted image has an aspect ratio above a threshold. The present invention may include based upon determining that at least one of the plurality of bounding boxes generated using the inputted image has the aspect ratio above the threshold, upscaling any text within the at least one bounding box and performing text detection on a new image using the trained text detection model. The present invention may include outputting an output image.

Text detection algorithm for separating words detected as one text bounding box

A system and method for generating hard training data from easy training data. Training data including visual data with synthetic semantic implants ('VSSI') having at least one cue is received. An annotator identifies at least one cue in the VSSI and annotates the VSSI to indicate the cue to create a modified training data set. A data scrambler removes at least one cue from the VSSI to create the tagged training data, which can then be used to train a classifier to identify transitions between segments when the cues are not present.

Automatic creation of difficult annotated data leveraging cues

A method for synthesizing negative training data associated with training models to detect text within documents and images. The method includes one or more computer processors receiving a set of dictates associated with generating one or more negative training datasets for training a set of models to classify a plurality of features found within a data source. The method further includes identifying a set of rules related to generating negative training data to detect text based on the received set of dictates. The method further includes compiling one or more arrays of elements of hard-negative training data into a negative training data dataset based on the identified set of rules and one or more dictates. The method further includes determining metadata corresponding an array of elements of hard-negative training data.

Synthesizing hard-negative text training data

An example system includes a processor to receive an image containing an object to be detected. The processor is to detect the object in the image via a binary object detector trained via a self-supervised training on raw and unlabeled videos.

Object detector trained via self-supervised training on raw and unlabeled videos

An approach to identifying text within an image may be presented. The approach can receive an image. The approach can classify an image on a pixel-by-pixel basis whether the pixel is text. The approach can generate bounding boxes around groups of pixels that are classified as text. The approach can mask sections of an image that where pixels are not classified as text. The approach may be used as a pre-processing technique for optical character recognition in documents, scanned images, or still images.

Document segmentation for optical character recognition

In some examples, a system for detecting text in an image includes a memory device to store a text detection model trained using images of up-scaled text, and a processor configured to perform text detection on an image to generate original bounding boxes that identify potential text in the image. The processor is also configured to generate a secondary image that includes up-scaled portions of the image associated with bounding boxes below a threshold size, and perform text detection on the secondary image to generate secondary bounding boxes that identify potential text in the secondary image. The processor is also configured to compare the original bounding boxes with the secondary bounding boxes to identify original bounding boxes that are false positives, and generate an image file that includes the original bounding boxes, wherein those original bounding boxes that are identified as false positives are removed.

Techniques for detecting text

An example system includes a processor to receive raw and unlabeled videos. The processor is to extract speech from the raw and unlabeled videos. The processor is to extract positive frames and negative frames from the raw and unlabeled videos based on the extracted speech for each object to be detected. The processor is to extract region proposals from the positive frames and negative frames. The processor is to extract features based on the extracted region proposals. The processor is to cluster the region proposals and assign a potential score to each cluster. The processor is to train a binary object detector to detect objects based on positive samples randomly selected based on the potential score.

Training an object detector using raw and unlabeled videos and extracted speech

A video scene detection machine learning model is provided. A computer device receives feature vectors corresponding to audio and video components of a video. The computing device provides the feature vectors as input to a trained neural network. The computing device receives from the trained neural network, a plurality of output feature vectors that correspond to shots of the video. The computing device applies optimal sequence grouping to the output feature vectors. The computing device further trains the trained neural network based, at least in part, on the applied optimal sequence grouping.

Deterministic learning video scene detection

An example system includes a processor to receive input data comprising noisy positive data and clean negative data. The processor is to cluster the input data. The processor is to compute a potential score for each cluster of the clustered input data. The processor is to iteratively refine cluster quality of the clusters using the potential scores of the clusters as weights. The processor is to train a classifier by sampling the negative dataset uniformly and the positive set in a non-uniform manner based on the potential score.

Classifier training using noisy samples

Embodiments may provide techniques that provide identification of images that can provide reduced resource utilization due to reduced sampling of video frames for visual recognition. For example, in an embodiment, a method of visual recognition processing may be implemented in a computer system comprising a processor, memory accessible by the processor, and computer program instructions stored in the memory and executable by the processor, the method comprising: coarsely segmenting video frames of video stream into a plurality of clusters based on scenes of the video stream, sampling a plurality of video frames from each cluster; determining a quality of each cluster, re-clustering the video frames of video stream to improve the quality of at least some of the clusters.

System and method for reducing resources costs in visual recognition of video based on static scene summary

Automated detection and approximation of objects in a video, including: (a) sampling a provided digital video, to obtain a set of sampled frames; (b) applying an object detection algorithm to the sampled frames, to detect objects appearing in the sampled frames; (c) based on the detections in the sampled frames, applying an object approximation algorithm to each sequence of frames that lie between the sampled frames, to approximately detect objects appearing in each of the sequences; (d) applying a trained regression model to each of the sequences, to estimate a quality of the approximate detection of objects in the respective sequence; (e) applying the object detection algorithm to one or more frames in those of the sequences whose quality of the approximate detection is below a threshold, to detect objects appearing in those frames.

Automated detection and approximation of objects in video

Methods and systems for annotating an object in a video frame. An annotation of an object in a video frame is obtained and a plurality of computerized object trackers are run on the annotated object to automatically label the annotated object in a plurality of subsequent video frames. A computerized user interface aggregating results of the automatic labeling by the plurality of computerized object trackers is presented and, based on the computerized user interface, an identification of a best one of the plurality of computerized object trackers is obtained. The selected computerized object tracker is used to annotate the object in the plurality of subsequent video frames.

Creating sparsely labeled video annotations

Technology for matching images (for example, video images, still images) of an identical infrastructure object (for example, a tower component of a tower supporting power lines) for purposes of comparing the infrastructure object to itself at different points in time to detect a potential anomaly and the potential need for maintenance of the infrastructure object. In some embodiments, this matching of images is done using creation of a three dimensional (#D) computer model of the infrastructure object and by tagging captured images with location on the 3D model across multiple videos taken at different points in time.

Growing community of inventors

Haifa, Israel

Udi Barzelay