International Business Machines Corporation

Jalal U Mahmud

Rama Kalyani T Akkiraju

Vibha Singhal Sinha

Anbang Xu

Pritam S Gundecha

Shereen M Oraby

Zhe Liu

Mansurul Bhuiyan

Haibin Liu

Sabina Tomkins

Jeffrey William Nichols

Tessa Ann Lau

Clemens Drews

Michelle Xue Zhou

Liang Gou

Yufan Guo

Rama K Akkiraju

Amita Misra

Nimrod Megiddo

Mark D Vickstrom

Jilin Chen

Joel Alexander Bullard

Arnaldo Carreno-Fuentes

Md Mansurul A Bhuiyan

Margaret Cho

Michael G Collins

John J Ponzo

Eser Kandogan

Barton Allen Smith

Yunwu Huang

Eben Merriam Haber

Aditya Pal

Roger Alan Pollak

Pablo Pedemonte

Mengdi Zhang

Fei Wang

Bin Xu

Thomas Guthrie Zimmerman

David Blake Werts

Richard Walton Ragan, Jr

Peifeng Yin

David Amid

Saleema Amin Amershi

Ateret Anaby-Tavor

Jerald Thomas Schoudt

Huahai Yang

Hernan Badenes

Xiaotong Liu

Maria Nichole Schwenger

Luna Xu

Mengdie Hu

Yi Wang

Rupaningal Sarasi Sarangi Lalithsena

Alexander Brooks

Gaurav Kumbhat

Kyumin Lee

Saurabh Tripathy

Yash Bhalgat

Gabriel Goodhart

Sukriti Sharma

Nhan Hoang

Zachary Branson

Richard P Gabriel

Huiji Gao

A system includes a memory having instructions therein and at least one processor in communication with the memory. The at least one processor is configured to execute the instructions to determine a global-level importance magnitude value for a global-level importance of an explainable feature of a machine learning base model based on a first prediction of the machine learning base model. The at least one processor is also configured to execute the instructions to determine a global-level importance direction label for the global-level importance of the explainable feature based on the first prediction. The at least one processor is also configured to execute the instructions to generate a communication for presentation to a user based on a second prediction of the machine learning base model, based on the global-level importance magnitude value, and based on the global-level importance direction label.

Machine learning model error detection

A computer-implemented method according to one embodiment includes receiving, utilizing a processor, textual data associated with a conversation between a first participant and a second participant; receiving, utilizing the processor, an objective of the first participant for the conversation between the first participant and the second participant, where the objective is separate from the conversation; determining, utilizing the processor, a dialog act to be entered by the first participant that meets the objective, utilizing a model, including scoring a plurality of proposed dialog acts based on an amount that each proposed dialog act will change a probability of the objective being achieved during the conversation, and determining the dialog act to be entered, based on the scoring; and returning, utilizing the processor, the dialog act to the first participant.

Recommending a dialog act using model-based textual analysis

A cognitive system (artificial intelligence) is optimized by assessing different data augmentation methods used to augment training data, and then training the system using a training set augmented by the best identified method. The augmentation methods are assessed by applying them to the same set of training data to generate different augmented training data sets. Respective instances of the cognitive system are trained with the augmented sets, and each instance is subjected to validation testing to assess its goodness. The validation testing can include multiple validation tests leading to component scores, and a combined validation score is computed as a weighted average of the component scores using respective weights for each validation test. The augmentation method corresponding to the instance having the highest combined validation score is selected as the optimum augmentation method for the particular cognitive system at hand.

Data augmentation for text-based AI applications

An input text that is also transmitted to a text processing service (e.g., a cloud based text processing service) is received. Characterizing information (e.g., contiguous parts of speech, terms used per part of speech, payload length, etc.) is extracted from the input text. A text payload is generated using the characterizing information. A performance test is run on the text payload. The performance test can include performing at least one selected from a group consisting of: sentiment analysis on the text payload, entity analysis on the text payload, content classification on the text payload, and syntax analysis on the text payload. The performance test can yield a processing time required to perform the performance test. Memory and processing power resource allocation to the text processing service can be altered based on the processing time of the performance test.

Natural language processing payload generation

A method and a system for model training are provided. The method can include training a first classifier, a second classifier, and a third classifier with subsets of a labeled dataset. The method can also include predicting a pseudo labeled dataset from an unlabeled dataset using the first classifier, the second classifier, and the third classifier. The method further includes assigning a role to the first classifier, to the second classifier, and to the third classifier. The method can further include selecting a teaching sample dataset from the pseudo labeled dataset based on the role assigned to the third classifier, wherein the third classifier is assigned a role of a student. The method can also include retraining the third classifier with the teaching sample dataset in conjunction with a subset of the labeled dataset.

Model training using a teacher-student learning paradigm

Embodiments provide a computer implemented method of evaluating one or more IR systems, the method including: providing, by a processor, a pre-indexed knowledge-based document to a pre-trained sentence identification model; identifying, by the sentence identification model, a predetermined number of query-worthy sentences from the pre-indexed knowledge-based document, wherein the query-worthy sentences are ranked based on a prediction probability value of each query-worthy sentence; providing, by the sentence identification model, the query-worthy sentences to a pre-trained query generation model; generating, by the query generation model, a query for each query-worthy sentence; and evaluating, by the processor, the one or more IR systems using the generated queries, wherein one or more searches are performed via the one or more IR systems, and the one or more searches are performed in a set of knowledge-based documents including the pre-indexed knowledge-based document.

Knowledge-based information retrieval system evaluation

A method, system, and a computer program product automatically select training data for updating a model by applying human-annotated training data to a model to generate results that are evaluated to identify correct case results and false case results that are categorized into error type categories for use in building error models corresponding to the error type categories, where each error model is built from at least failed case results belonging to a corresponding error type, and where unlabeled data samples are applied to each error model to compute an error likelihood for each unlabeled data sample with respect to each error type category, thereby enabling the selection and display of unlabeled data samples for annotation by a subject matter expert based on a computed error likelihood for the one or more unlabeled data samples in a specified error type category meeting or exceeding an error threshold requirement.

Method and system for unlabeled data selection using failed case analysis

An enhanced information retrieval system takes a customer utterance and constructs a contextually-enriched content-based query allowing the system to retrieve the most relevant documents to assist an agent in a real-time conversation with the customer. Phrases in the utterance are classified as informational or non-informational using a machine learning system trained with phrases from prior conversations of multiple users. Content phrases are extracted from the informational phrases using keyword extraction (ranking noun phrases), intent/action extraction (semantic role labeling), and topic label extraction (clustering of historical logs). Emotional content is identified using a sequence tagging model and removed. Contextual information from prior conversations with this user is combined with the updated content phrases to create the contextually-enhanced content-based query, which can then be submitted to the information retrieval system.

Relevant document retrieval to assist agent in real time customer care conversations

A method, system and computer-usable medium are disclosed for sentiment detection based on applying an antonym dictionary to a natural language processing (NLP) system. A binary classifier is trained to predict negation cues, where a constituency parse tree is used to create rules for scope detection. The trained binary classifier, a list of conversational negation terms, and a list of antonyms are used to annotate content that considers negation cues and scope for the created rules.

System and method for negation aware sentiment detection

In an approach for training customer service agents using persona-based chatbots, a processor retrieves customer service interaction information. A processor analyzes, using natural language processing, the customer service interaction information, wherein the analyzing includes preprocessing and aggregating the customer service interaction information. A processor interacts with a user. A processor provides feedback to the user, based on the user's style and performance during training.

Customer care training with situational feedback generation

Mechanisms are provided to implement a brand personality inference engine. The mechanisms receive crowdsource information and extract features associated with a brand from the crowdsource information. The crowdsource information comprises natural language content submitted by a plurality of providers to a crowdsource information source. The mechanisms analyze features associated with the brand in accordance with a brand personality model configured to predict a brand personality for the brand based on the features associated with the brand. The mechanisms generate an inferred brand personality data structure, representing a perceived brand personality of providers providing the crowdsource information, and output an output indicating aspects of the perceived brand personality based on the inferred brand personality data structure.

Brand personality inference and recommendation system

A system, computer program product, and method are disclosed. In an approach to train customer service agent using chatbots. The method includes training a chatbot for a customer chat simulation based on a customer service conversation data, a task scenario, and a customer persona. The method also includes monitoring an interaction between a customer service agent and the chatbot. The method further includes determining an assessment of the performance of the customer service agent based on the interaction between the customer service agent and the chatbot. The method additionally includes generating feedback for the customer service agent based on the assessment of the performance of the customer service agent.

Customer care training using chatbots

Growing community of inventors

San Jose, CA, United States of America

Jalal U Mahmud