International Business Machines Corporation

Guy Lev

Ofer Lavi

Boaz Carmeli

Einat Kermany

Guy Hadash

David Konopnicki

Boris Rozenberg

Michal Shmueli-Scheuer

Oren Sar-Shalom

Jonathan Herzig

George Kour

Yehoshua Sagron

Elad Mezuman

Achiya Jerbi

Yftah Ziser

A specification of a property required to be upheld by a computerized machine learning system is obtained. A training data set corresponding to the property and inputs and outputs of the system is built. The system is trained on the training data set. Activity of the system is monitored before, during, and after the training. Based on the monitoring, performance of the system is evaluated to determine whether the system, once trained on the training data set, upholds the property.

Deep learning testing

Detecting sequences of computer-executed operations, including training a BLSTM to determine forward and backward probabilities of encountering each computer-executed operations within a training set of consecutive computer-executed operations in forward and backward execution directions of the operations, and identifying reference sequences of operations within the training set where for each given one of the sequences the forward probability of encountering a first computer-executed operation in the given sequence is below a predefined lower threshold, the forward probability of encountering a last computer-executed operation in the given sequence is above a predefined upper threshold, the backward probability of encountering the last computer-executed operation in the given sequence is below the predefined lower threshold, and the backward probability of encountering the first computer-executed operation in the given sequence is above the predefined upper threshold, and where the predefined lower threshold is below the predefined upper threshold.

Detecting non-anomalous and anomalous sequences of computer-executed operations

During end-to-end training of a Deep Neural Network (DNN), a differentiable estimator subnetwork is operated to estimate a functionality of an external software application. Then, during inference by the trained DNN, the differentiable estimator subnetwork is replaced with the functionality of the external software application, by enabling API communication between the DNN and the external software application.

Integration of external applications into deep neural networks

Embodiments may provide techniques to generate training data for summarization of complex documents, such as scientific papers, articles, etc., that are scalable to provide large scale training data. For example, in an embodiment, a method may be implemented in a computer system and may comprise collecting a plurality of video and audio recordings of presentations of documents, collecting a plurality of documents corresponding to the video and audio recordings, converting the plurality of video and audio recordings of presentations of documents into transcripts of the plurality of presentations, generating a summary of each document by selecting a plurality of sentences from each document using the transcript of the that document, generating a dataset comprising a plurality of the generated summaries, and training a machine learning model using the generated dataset.

Automatic generation of training data for scientific paper summarization using videos

An example system includes a processor to receive a plurality of object aspects of an object to be evaluated using a process, a structure of the process, a plurality of extracted facts from documents, a tree related to the plurality of object aspects and the structure, and a thesis for each leaf in the tree. The processor is also to relate the extracted facts to the theses in the tree. The processor is to generate a score for each leaf corresponding to a fact in the tree. The processor is to generate a thesis score and a thesis summary for each thesis based on the scores and the summaries of related facts for each thesis. The processor is to further generate a final score for the object based on the thesis scores.

Evaluating theses using tree structures

Growing community of inventors

Tel Aviv, Israel

Guy Lev