Intuit, Inc.

Talia Tron

Adi Shalev

Yair Horesh

Yehezkel Shraga Resheff

Rami Cohen

Alexander Zhicharevich

Elik Sror

Noa Haas

Tzvika Barenholz

Nitzan Gado

Osnat Haj Yahia

Kfir Aharon

Oren Dar

A method classifies feedback from transcripts. The method includes receiving an utterance from a transcript from a communication session and processing the utterance with a classifier model to identify a topic label for the utterance. The classifier model is trained to identify topic labels for training utterances. The topic labels correspond to topics of clusters of the training utterances. The training utterances are selected using attention values for the training utterances and clustered using encoder values for the utterances. The method further includes routing the communication session using the topic label for the utterance.

Classifying feedback from transcripts

A method of score prediction uses hierarchical attention. Word features, positioning features, participant embedding features, and metadata are extracted from a transcript of a conversation. A word encoder vector is formed by multiplying weights of a word encoder layer to one or more word features. A sentence vector is formed by multiplying weights of a word attention layer to word encoder vectors. An utterance encoder vector is formed by multiplying weights of an utterance encoder layer to the sentence vector. A conversation vector is formed by multiplying weights of an utterance attention layer to utterance encoder vectors. The utterance encoder vector is combined with one or more positioning features and one or more participant embedding features. A predicted net promoter score is generated by multiplying weights of an output layer to the conversation vector combined with the metadata. The predicted net promoter score is presented in a list of conversations.

Score prediction using hierarchical attention

Systems and methods are disclosed for managing a generative artificial intelligence (AI) model to improve performance. Managing the generative AI model includes using a second generative AI model to generate outputs from similar inputs and comparing the outputs of the generative AI models to determine their similarity. The second generative AI model is trained using fresher training data but may not be manually supervised and adjusted to the level of the generative AI model being managed. As such, an output of the second generative AI model is compared to an output of the managed generative AI model by a classification model to determine a relevance of the output from the managed generative AI model. An output of the classification model is used to perform various suitable policies to optimize the performance of the managed generative AI model, such as providing alternate outputs, preventing providing the output, or retraining the model.

Supervisory systems for generative artificial intelligence models

Systems and methods for automatically assessing a quality of service for agents of a customer support system are disclosed. An example method may include retrieving historical conversations between the agents and users of the customer support system, receiving user comments for one or more of the historical conversations, identifying which of the received user comments includes keywords indicative of one or more quality of service attributes, generating transcripts of historical conversations associated with the identified user comments, training a machine learning model based at least in part on the generated transcripts and the user comments of the historical conversations associated with the identified user comments, providing a plurality of current conversations between agents and users of the customer support system to the trained machine learning model, and generating a behavioral score for each of the agents using the trained machine learning model.

Combining behavioral quality of service assessment with machine learning prediction

One or more embodiments includes storing a group email message in multiple email mailboxes to obtain multiple stored email messages, receiving, for a first stored email message in the stored email messages, an updated label from a first user computing device, detecting, in response to receiving the updated label, that the stored email messages matches the first stored email message, and storing the updated label with at least a subset of the stored email messages. The method further includes transmitting, with the updated label and to a second user computing system, a second stored email message in the stored email messages.

Growing community of inventors

Shefayim, Israel

Talia Tron