International Business Machines Corporation

Nuance Communications, Inc.

Jiri Navratil

Ganesh Nachiappa Ramaswamy

Upendra V Chaudhari

Stephane Herman Maes

Ran D Zilca

Jason William Pelecanos

Bhuvana Ramabhadran

Matthew R Arnold

Salim Estephan Roukos

Rishav Chakravarti

Benjamin Tyler Elder

Bowen Zhou

Robert T Ward

Ruhi Sarikaya

James H Nealand

Mahesh Viswanathan

David M Lubensky

Evelyn Duesterwald

Tsuyoshi Ide

Amit Dhurandhar

Osamuyimen Stewart

Naoki Abe

Michael James Hind

James William Murdock, Iv

Moninder Singh

Ganesh Venkataraman

Anupama Murthi

Douglas George Heintzman

Ryan L Osborn

Jagon Pelecanos

Janice J Kim

David Eubensky

Begum Taskazan

A computer implemented method of performing large-scale machine learning experiments includes expanding on one or more input datasets by systematically generating several data set drift splits. A set of experimental jobs corresponding to the generated data set drift splits are executed to generate experimental results. The experimental results are processed, consolidated, and clustered according to the generated data set drift splits.

Assistant for large-scale machine learning experiments

A technique for generating a performance prediction of a machine learning model with uncertainty intervals includes obtaining a first model configured to perform a task and a production dataset. At least one metric predicting a performance of the first model at performing the task on the production dataset is generated using a second model. The second model is a meta-model associated with the first model. At least one value predicting an uncertainty of the at least one metric predicting the performance of the first model at performing the task on the production dataset is generated using a third model. The third model is a meta-meta-model associated with the second model. An indication of the at least one metric and the at least one value is provided.

Generating performance predictions with uncertainty intervals

A computer-implemented method, a computer program product, and a system for reducing labeled sample quantities required to update test sets. The computer-implemented method includes inputting a portion of unlabeled production data into a base model and generating labeled output relating to the unlabeled production data. The computer-implemented method also includes inputting the labeled output into a performance predictor. The performance predictor is a meta model of the base model that is trained with another portion of the unlabeled production data, a training set used to train the base model, and a test set portioned from the training set. The computer-implemented method further includes outputting, by the performance predictor, a performance metric relating to the labeled output produced by the trained base model. The performance metric can be any metric capable of measuring the output performance of the base model.

Label reduction in maintaining test sets

A computer-implemented method, a computer program product, and a computer system for diagnosing anomalies detected by a black-box machine learning model. A computer determines a local variance of a test sample in a test dataset, where the local variance represents uncertainty of a prediction by the black-box machine learning model. The computer initializes optimal compensations for the test sample, where the optimal compensations are optimal perturbations to test sample values of respective components of a multivariate input variable. The computer determines local gradients for the test sample. Based on the local variance and the local gradients, the computer updates the optimal compensations until convergences of the optimal compensations are reached. Using the optimal compensations, the computer diagnoses the anomalies detected by the black-box machine learning model.

Diagnosing anomalies detected by black-box machine learning models

A method is provided that includes generating answer-passage pairs, each associated with a respective one of multiple feature vectors. The method includes, for each answer in the pairs, merging the feature vectors associated with any of the pairs that include the answer to obtain a merged answer vector. The method includes, for each passage in the pairs, (i) merging the feature vectors associated with any of the pairs that includes the passage to obtain a merged passage vector, (ii) merging the feature vectors from the merged answer vector for each answer that is associated with the passage via at least one of the pairs to obtain a merged passage-answer vector, and (iii) concatenating the merged passage vector and the merged passage-answer vector to obtain a concatenated passage vector. The method includes ranking the concatenated passage vector for each passage to obtain a ranked list of passages with associated confidence scores.

Ranking passages by merging features from factoid answers

According to one embodiment, a method, computer system, and computer program product for continuously ranking components in a live information is provided. The present embodiment may include receiving search feedback derived from interactions between users and information retrieval systems; assigning weights to each of the ranking components; adjusting the assigned weights based on search feedback; modifying the current set of ranking components based on the search feedback by adding new ranking components and deleting old ranking components; transmitting a query from the users to the current set of ranking components; aggregating ranking results from the transmitted query into a single ranking based on the weights; and transmitting the single ranking to the users.

Continuous augmentation method for ranking components in information retrieval

Techniques are disclosed for authentication and identification of a user by use of an electroencephalographic (EEG) signal. For example, a method for authenticating a user includes the following steps. At least one electroencephalographic response is obtained from a user in accordance with perceptory stimuli presented to the user. The user is authenticated based on the obtained electroencephalographic response. The authenticating step may be based on detection of an event-related potential in the obtained electroencephalographic response. The event-related potential may be a P300 event-related potential. The method may also include the step of enrolling the user prior to authenticating the user. The enrolling step may include a supervised enrollment procedure or an unsupervised enrollment procedure.

User authentication via evoked potential in electroencephalographic signals

According to an aspect, a first word in a first language and a second word in a second language in a bilingual corpus are stemmed. A probability for aligning the first stem and the second stem and a distance metric between the normalized first stem and the normalized second stem are calculated. The first word and the second word are identified as a cognate pair when the probability and the distance metric meet a threshold criterion and stored as a cognate pair in a set of cognates. A candidate sentence in the second language is retrieved from a corpus. The candidate sentence is filtered by the active vocabulary of a user in the second language and the set of cognates. A sentence quality score is calculated for the candidate sentence; and the candidate sentence is ranked for presentation to the user based on the sentence quality scorer.

Automatic cognate detection in a computer-assisted language learning system

Methods and apparatus are provided for sequential authentication of a user that employ one or more error rates characterizing each security challenge. According to one aspect of the invention, a user is challenged with at least one knowledge challenge to obtain an intermediate authentication result; and the user challenges continue until a cumulative authentication result satisfies one or more criteria. The intermediate authentication result is based, for example, on one or more of false accept and false reject error probabilities for each knowledge challenge. A false accept error probability describes a probability of a different user answering the knowledge challenge correctly. A false reject error probability describes a probability of a genuine user not answering the knowledge challenge correctly. The false accept and false reject error probabilities can be adapted based on field data or known information about a given challenge.

Method and apparatus for sequential authentication using one or more error rates characterizing each security challenge

A method of identifying and using side information available to statistical machine translation systems within an enterprise setting, the method including extracting user-specific interaction and non-interaction-based information from at least one corresponding database within the enterprise for each of a plurality of users, aggregating the user-specific interaction and non-interaction based information from a plurality of users, by using a processor on a computer, to tune and adapt background translation and language models, and updating all relevant models within the enterprise after user activity based on the tuned and adapted translation and language models.

Machine translation with side information

A method of generating a statistical machine translation database through a game in which a monolingual structure is provided to a plurality of players. A first translation attempt is received from each of the plurality of players. The first translation attempt from each of the plurality of players is compared. Feedback is provided to each of the plurality of players and the attempts are received and compared to provide feedback to iteratively converge subsequent translations from each of the plurality of players into a final translated structure.

Game based method for translation data acquisition and evaluation

Growing community of inventors

White Plains, NY, United States of America

Jiri Navratil