International Business Machines Corporation

Ayush Gupta

Neil Rohit Mallinar

Rajendra G Ugrani

Tin Kam Ho

Abhishek Shah

Robert Leslie Yates

Blake McGregor

A combination of propagation operations and learning algorithms is applied, using a selected set of labeled conversational logs retrieved from a subset of a plurality of conversational logs, to a remaining corpus of the plurality of conversational logs to train an automated response system according to an intent associated with each of the conversational logs. The combination of propagation operations and learning algorithms may include defining the labels by a user for the selected set of the subset of the plurality of conversational logs; training a probabilistic classifier using the defined labels of features of the selected set, wherein the probabilistic classifier produces labeling decisions for the subset of conversational logs; weighting the features of the selected set in a model optimization process; and/or training an additional classifier using the weighted features of the selected set and applying the additional classifier to the remaining corpus.

Intent authoring using weak supervision and co-training for automated response systems

Expansion of intent classification data utilizing batch utterance scheduling, by a processor in a computing environment. A set of unlabeled examples for intent processing is received by an intent builder iteratively defining an intent. The set of examples are separated into a first subset processed according to a first model and a second subset processed according to a second model. The first subset is incorporated into the intent builder during a building iteration and scheduling a first batch processing of the second subset processed according to the second model based on a scheduling criteria. The first batch processing of the second subset is initiated once the scheduling criteria is satisfied. Upon completion of the first batch processing, results of the completion are used to influence additional examples retrieved from the first subset and the second subset during a subsequent building iteration by the intent builder.

Iterative intent building utilizing dynamic scheduling of batch utterance expansion methods

Evaluating intent authoring processes, by a processor in a computing environment. A dataset comprising utterances of interactive dialog sessions between agents and clients for a given product or service is received. A classification of at least a portion of the utterances is performed for a target intent according to at least one of a plurality of recommendation algorithms, where the classification is performed by an automatic driver invoking the recommendation algorithm and simulating a manual confirmation of the algorithm's decision by a user. A classifier trained with the utterances recommended and confirmed by the automatic driver is automatically evaluated according to at least one of the plurality of evaluation criteria. A report tracking the evaluation results is generated.

Evaluation framework for intent authoring processes

Evaluating intent authoring processes, by a processor in a computing environment. Results are received of a simulated intent labeling effort of a dataset comprising utterances of interactive dialog sessions between agents and clients for a given product or service. Figures of merits for respective algorithms used to perform the simulated intent labeling effort are computed. Each of the respective algorithms are evaluated according to the computed figures of merits; and one of the respective algorithms is implemented for labeling intents of a remaining corpus of the synthesized dataset according to parameters evaluated in the computed figures of merits.

Embodiments for training an automated response system using weak supervision and co-training in a computing environment are provided. A plurality of conversational logs comprising interactive dialog sessions between agents and clients for a given product or service are received. A subset of the plurality of conversational logs are retrieved according to a defined criterion, and a selected set of the subset of the plurality of retrieved conversational logs are labeled by a user. The labeling is associated with a semantic scope of intent considered by the clients. A combination of propagation operations and learning algorithms using the selected set of labeled conversational logs are applied to a remaining corpus of the plurality of conversational logs to train the automated response system according to the semantic scope of intent.

Growing community of inventors

Morrisville, NC, United States of America

Ayush Gupta