International Business Machines Corporation

Daniel Nechemia Rotman

Dror Porat

Udi Barzelay

Gal Ashour

Yevgeny Burshtein

Tal Hakim

Peter Bak

Joseph Shtok

Ophir Azulai

Tal Drory

Rami Ben-Ari

Elad Amrani

Yevgeny Yaroker

Described are techniques for font attribute detection. The techniques include receiving a document having different font attributes amongst a plurality of words respectively comprised of at least one character. The techniques further include generating a dense image document from the document by setting the plurality of words to a predefined size, removing blank spaces from the document, and altering an order of characters relative to the document. The techniques further include determining characteristics of the characters in the dense image document and aggregating the characteristics for at least one word. The techniques further include annotating the at least one word with a font attribute based on the aggregated characteristics.

Font attribute detection

A system and method for generating hard training data from easy training data. Training data including visual data with synthetic semantic implants ('VSSI') having at least one cue is received. An annotator identifies at least one cue in the VSSI and annotates the VSSI to indicate the cue to create a modified training data set. A data scrambler removes at least one cue from the VSSI to create the tagged training data, which can then be used to train a classifier to identify transitions between segments when the cues are not present.

Automatic creation of difficult annotated data leveraging cues

A processor may receive a query from a user. The processor may identify one or more subjects in the query. The one or more subjects may include a particular subject. The processor may identify one or more sections of a recording. The processor may group the one or more sections into one or more subdivisions. The processor may determine that the particular subject is in at least one of the one or more sections. The processor may display the one or more subdivisions of the one or more sections that include the particular subject to the user.

Query-based granularity selection for partitioning recordings

A video scene detection machine learning model is provided. A computer device receives feature vectors corresponding to audio and video components of a video. The computing device provides the feature vectors as input to a trained neural network. The computing device receives from the trained neural network, a plurality of output feature vectors that correspond to shots of the video. The computing device applies optimal sequence grouping to the output feature vectors. The computing device further trains the trained neural network based, at least in part, on the applied optimal sequence grouping.

Deterministic learning video scene detection

An example system includes a processor to receive a number of videos. The processor is to generate a number of image clouds based on the videos. The processor is to present an interactive image cloud representation including the image clouds. Each image cloud includes a representative image and a number of associated images.

Interactive image cloud visualization of videos

Embodiments may provide techniques that provide identification of images that can provide reduced resource utilization due to reduced sampling of video frames for visual recognition. For example, in an embodiment, a method of visual recognition processing may be implemented in a computer system comprising a processor, memory accessible by the processor, and computer program instructions stored in the memory and executable by the processor, the method comprising: coarsely segmenting video frames of video stream into a plurality of clusters based on scenes of the video stream, sampling a plurality of video frames from each cluster; determining a quality of each cluster, re-clustering the video frames of video stream to improve the quality of at least some of the clusters.

System and method for reducing resources costs in visual recognition of video based on static scene summary

Automated detection and approximation of objects in a video, including: (a) sampling a provided digital video, to obtain a set of sampled frames; (b) applying an object detection algorithm to the sampled frames, to detect objects appearing in the sampled frames; (c) based on the detections in the sampled frames, applying an object approximation algorithm to each sequence of frames that lie between the sampled frames, to approximately detect objects appearing in each of the sequences; (d) applying a trained regression model to each of the sequences, to estimate a quality of the approximate detection of objects in the respective sequence; (e) applying the object detection algorithm to one or more frames in those of the sequences whose quality of the approximate detection is below a threshold, to detect objects appearing in those frames.

Automated detection and approximation of objects in video

An example system includes a processor to receive a number of video frames. The processor is to apply a grid to each video frame of the number of video frames and generate features for each cell in the grid. The processor is to calculate distances between matching regions of two consecutive frames. The processor is to apply max pooling followed by min pooling in horizontal regions of the number of video frames. The processor is to apply max pooling followed by min pooling in vertical regions of the number of video frames. The processor is to detect an application switch in response to detecting both a horizontal change and a vertical change between the two consecutive video frames exceed a threshold.

Detecting application switches in video frames using min and max pooling

A computer-implemented method for providing a graphical user interface (GUI) for video navigation by a user, including generating a GUI comprising a representation of partitioned segments of video content, the representation depicting a dimension of time, a dimension of scale, and a dimension of relevance for the partitioned segments of video content, wherein the GUI includes a first axis that depicts the dimension of time and a second axis that depicts the dimension of scale. The method further entails determining the dimension of relevance for the partitioned segments of video content based on receiving a text query from a user dynamically, or from a pre-defined source, and comparing one or more attributes of the partitioned segments of video content to the text query, or to the pre-defined source. The method further includes assigning a color to the partitioned segments of video content according to a color scheme for easy navigation.

Generating a graphical user interface to navigate video content

A system comprising a non-transient computer-readable storage medium having stored thereon instructions and at least one hardware processor configured to execute the instructions, to receive a video sequence; divide the video sequence into one or more scenes based on scene boundaries, wherein each scene comprises a plurality of temporally-contiguous image frames, and wherein said scene boundaries are being determined based on a similarity metric between two temporally-contiguous image frames meeting a dissimilarity threshold; and, for each scene of the one or more scenes, (i) generate a plurality of preliminary classifications of an object appearing in at least some of said image frames in the scene, wherein each of said plurality of preliminary classifications has a confidence score, and (ii) calculate a combined classification of the object based on said plurality of preliminary classifications, wherein each of said preliminary classifications is weighted in accordance with its confidence score.

Object recognition in video

Segmenting serial data by processing multiple candidate segmentation point sets associated with a serial data set, each candidate set including a different number of candidate segmentation points representing the serial data set as segments in accordance with segmentation criteria, where the processing includes determining, for each of the candidate points, a count of the candidate sets that include the candidate point, and creating, for each of the candidate sets, a corresponding alternative segmentation point set associated with the serial data set, where the alternative set includes n alternative segmentation points representing the serial data set as n+1 segments, where n equals the number of candidate points in the candidate set to which the alternative set corresponds, and where the n alternative points in the alternative set correspond to n of the candidate points having the greatest counts, and arranging the alternative sets in order of their numbers of segmentation points.

Growing community of inventors

Haifa, Israel

Daniel Nechemia Rotman