International Business Machines Corporation

Sivan Harary

Mattias Marder

Joseph Shtok

Leonid Karlinsky

Eliyahu Schwartz

Asaf Tzadok

Charles Francis Wiecha

Noel S Kropf

Peter Bak

Amir Geva

Avi Yaeli

Ophir Azulai

Harold Jeffrey Ship

Assaf Arbelle

A computing system, computer program product, and computer-implemented method for incremental learning without forgetting for a classification/detection model are provided. The method includes receiving, at a computing system, a classification/detection model including a base embedding space and corresponding base embedding vectors that are based on a base training dataset including base classes. The method also includes expanding the classification/detection model to account for a new training dataset including new classes by lifting the base embedding space to add an orthogonal subspace for the new classes, producing an expanded embedding space and corresponding expanded embedding vectors that are of a higher dimension than the base embedding vectors. In some embodiments, the method also includes further expanding the expanded classification/detection model to account for another new training dataset including additional new classes by lifting the expanded embedding space to add another orthogonal subspace for the additional new classes.

Incremental learning without forgetting for classification and detection models

An example system includes a processor that can randomly mask tokens using different masks to generate different subsets of masked tokens. The processor can process the different sets of masked tokens via a pretrained masked auto-encoder (MAE) encoder to output intermediate representations. The processor can process the intermediate representations via a pretrained MAE decoder to output reconstructed images. The processor can further compare input image with the output reconstructed images to generate an anomaly score.

Anomaly detection using masked auto-encoder

An example system includes a processor to receive detected chart regions in a page of a document. The processor is to produce, via a graphical elements detector, predicted heatmaps and bounding boxes for graphical objects in the detected chart regions. The processor is also to apply chart type specific analysis algorithm to the predicted heatmaps and bounding boxes, to extract tabular chart data. The processor can then generate an output data file and a visualization based on the predicted heatmap and the extracted tabular chart data.

Detecting graphical elements in charts using predicted heatmaps

A method can include learning a common embedding space and a set of parameters for each one of a plurality of sets of mixture models, wherein one mixture model is associated with one class of objects within a set of object categories. The method can also include adding new mixture models to the set of mixture models to support novel categories based on a set of example embedding vectors computed for each one of the novel categories. Additionally, the method includes detecting in images a plurality of boxes with associated labels and corresponding confidence scores, wherein the boxes correspond to image regions comprising objects of both known categories and the novel categories. Furthermore, the method includes, given a query image, executing an instruction based on the common embedding space and the set of mixture models, the instruction comprising identifying objects from both categories in the query image.

Representative-based metric learning for classification and few-shot object detection

Embodiments of the present disclosure include training a model using a plurality of pairs of feature vectors related to a first class. Embodiments include providing a sample feature vector related to a second class as an input to the model. Embodiments include receiving at least one synthesized feature vector as an output from the model. Embodiments include training a classifier to recognize the second class using a training data set comprising the sample feature vector related to the second class and the at least one synthesized feature vector. Embodiments include providing a query feature vector as an input to the classifier. Embodiments include receiving output from the classifier that identifies the query feature vector as being related to the second class, wherein the output is used to perform an action.

Out-of-sample generating few-shot classification networks

There is provided a method of identifying objects in an image, comprising: extracting query descriptors from the image, comparing each query descriptor with training descriptors for identifying matching training descriptors, each training descriptor is associated with a reference object identifier and with relative location data (distance and direction from a center point of a reference object indicated by the reference object identifier), computing object-regions of the digital image by clustering the query descriptors having common center points defined by the matching training descriptors, each object-region approximately bounding one target object and associated with a center point and a scale relative to a reference object size, wherein the object-regions are computed independently of the identifier of the reference object associated with the object-regions, wherein members of each cluster point toward a common center point, and classifying the target object of each object-region according to the reference object identifier of the cluster.

Systems and methods for identifying a target object in an image

A method comprising: training a price tag detector, comprising a gross feature detector and a classifier, to automatically detect a price tag in an image, by: a) training the gross feature detector using supervised learning with labeled images, and b) training the classifier using a two-phase hybrid learning process comprising: c) applying an initial supervised learning using the labeled images, yielding a semi-trained version of the classifier, and d) applying a subsequent unsupervised learning using unlabeled images, yielding a fully trained version of the classifier, wherein applying the unsupervised learning comprises: for each unlabeled image: i) detecting multiple price tag hypotheses using the gross feature detector, ii) classifying each price tag hypothesis using the semi-trained classifier, ii) rating each classification based contextual data extracted from the unlabeled image, iv) retraining the semi-trained classifier with the rated classifications, and repeating steps ii) through iv) until the reclassification converges.

Semi-supervised price tag detection

Machines, systems and methods for recognizing visual change in a graphical user interface (GUI) environment, the method comprising determining position of an active GUI object in the GUI environment based on known attributes of the active GUI object; monitoring a focus area in the active GUI object to detect visual changes in attributes of the focus area, without regard to any visual changes outside the focus area; determining whether the active GUI object has moved or has been resized, in response to determining a visual change in the attributes of the focus area; and determining position of a new active GUI object in the GUI environment, in response to determining that the active GUI object has not been moved or has not been resized.

Active and efficient monitoring of a graphical user interface

A method comprising using at least one hardware processor for: applying an edge detection algorithm to an image of a document to receive a map of edges from which multiple optional contours of the document in the image are identified; splitting the multiple optional contours into line segments; sorting the line segments into equivalence classes of collinearity representing distinct line segments of the line segments, wherein each one of the classes of collinearity represents a distinct line segment of the distinct line segments; deriving a connectivity graph based on the equivalence classes of collinearity; identifying four vertex polygons in said connectivity graph; evaluating each one of the identified four vertex polygons according to one or more segmentation criterions; and segmenting the document in the image according to the most highly evaluated four vertex polygon of the four vertex polygons.

Image segmentation

Methods, system and computer program products for spatiotemporal encounters detection of a plurality of moving objects are disclosed. The method includes receiving a dataset of a plurality of objects moving in a domain, structuring the dataset in a data structure to detect a plurality of spatiotemporal encounters among the plurality of objects, outputting a list of the detected spatiotemporal encounters. The plurality of spatiotemporal encounters may be detected in a single sweep over the received dataset.

Growing community of inventors

Haifa, Israel

Sivan Harary