Intuit, Inc.

Adi Shalev

Yair Horesh

Alexander Zhicharevich

Oren Sar Shalom

Elik Sror

Yehezkel Shraga Resheff

Talia Tron

Shlomi Medalion

Rami Cohen

Noa Haas

Miriam Hanna Manevitz

Sigalit Bechler

Natalie Bar Eliyahu

Shiomi Medalion

Ido Farhi

Alexander Zicharevich

Nitzan Gado

Osnat Haj Yahia

Kfir Aharon

Oren Dar

A method classifies feedback from transcripts. The method includes receiving an utterance from a transcript from a communication session and processing the utterance with a classifier model to identify a topic label for the utterance. The classifier model is trained to identify topic labels for training utterances. The topic labels correspond to topics of clusters of the training utterances. The training utterances are selected using attention values for the training utterances and clustered using encoder values for the utterances. The method further includes routing the communication session using the topic label for the utterance.

Classifying feedback from transcripts

A method of score prediction uses hierarchical attention. Word features, positioning features, participant embedding features, and metadata are extracted from a transcript of a conversation. A word encoder vector is formed by multiplying weights of a word encoder layer to one or more word features. A sentence vector is formed by multiplying weights of a word attention layer to word encoder vectors. An utterance encoder vector is formed by multiplying weights of an utterance encoder layer to the sentence vector. A conversation vector is formed by multiplying weights of an utterance attention layer to utterance encoder vectors. The utterance encoder vector is combined with one or more positioning features and one or more participant embedding features. A predicted net promoter score is generated by multiplying weights of an output layer to the conversation vector combined with the metadata. The predicted net promoter score is presented in a list of conversations.

Score prediction using hierarchical attention

A method and computer system are provided for generating a text summary. An input text is processed by a model that comprises a set of attention heads. The model is trained to generate abstractive summaries of text documents. A subset of the attention heads are identified. For each attention head in the subset, a portion is identified from the input text that is used to generate the abstractive summary. For each sentence of the input text, a fractional size is calculated for an intersection of the portion and the respective sentence relative to the respective sentence. A subset of the sentences is then determined, where the respective fractional size of each sentence in the subset meets a first threshold. An extractive summary of the input text is then generated from the subset of sentences.

Extractive summary generation by abstractive trained model

Systems and methods are disclosed for managing a generative artificial intelligence (AI) model to improve performance. Managing the generative AI model includes using a second generative AI model to generate outputs from similar inputs and comparing the outputs of the generative AI models to determine their similarity. The second generative AI model is trained using fresher training data but may not be manually supervised and adjusted to the level of the generative AI model being managed. As such, an output of the second generative AI model is compared to an output of the managed generative AI model by a classification model to determine a relevance of the output from the managed generative AI model. An output of the classification model is used to perform various suitable policies to optimize the performance of the managed generative AI model, such as providing alternate outputs, preventing providing the output, or retraining the model.

Supervisory systems for generative artificial intelligence models

A method including inputting, into a phrase recognition model comprising a neural network, a vector comprising a plurality of ngrams of text. The method also includes applying, using the phrase recognition model, a filter to the plurality of ngrams during execution. The filter has a skip word setting of at least one. The method also includes determining, based on the skip word setting, at least one ngram in the vector to be skipped to form at least one skip word. The method also includes outputting an intermediate score for a set of ngrams that match the filter. The method also includes calculating a scalar number representing a semantic meaning of the at least one skip word. The method also includes generating based on the scalar number and the intermediate score, a final score for the set of ngrams. A computer action is performed using the final score.

Machine learning models with improved semantic awareness

A method may include receiving, for a package, shipment details including attributes, obtaining, for a subset of the attributes, logistic preferences, applying the logistic preferences to the shipment details to obtain modified shipment details, training a classifier using shipment transactions each including values for the attributes and labeled with a vendor logistic service, generating, by applying the classifier to the modified shipment details, scores for vendor logistic services, and recommending a vendor logistic service from the vendor logistic services using the scores.

Logistic recommendation engine

A method including embedding, by a trained issue MLM (machine learning model), a new natural language issue statement into an issue vector. An inner product of the issue vector with an actions matrix is calculated. The actions matrix includes centroid-vectors calculated using a clustering method from a second output of a trained action MLM which embedded prior actions expressed in natural language action statements taken as a result of prior natural issue statements. Calculating the inner product results in probabilities associated with the prior actions. Each of the probabilities represents a corresponding estimate that a corresponding prior action is relevant to the issue vector. A list of proposed actions relevant to the issue vector is generated by comparing the probabilities to a threshold value and selecting a subset of the prior actions with corresponding probabilities above the threshold. The list of proposed actions is transmitted to a user device.

Machine learning to propose actions in response to natural language questions

Systems and methods may be used to generate and use a merchant community graph generated based on merchant financial transaction data. Connections between merchants and other data within the merchant community graph can be used to detect fraud, target product offerings and or other advertisements, detect similar communities, generate dynamic attributes that may be used to develop machine learning models, and develop new user interfaces (UIs) and other features of an information service.

Methods and systems using and constructing merchant communities based on financial transaction data

Systems and methods for automatically identifying problem-relevant sentences in a transcript are disclosed. In an example method, a transcript may be received of a first support call. A region of the first support call transcript may be identified, and first customer utterances may be detected in the region using a trained classification model. A trained regression model may estimate a relevancy to the problem statement of each of the first customer utterances, and one or more most problem-relevant statements may be selected from the first customer utterances, based on the estimated relevancies.

Extracting customer problem description from call transcripts

Systems and methods for automatically assessing a quality of service for agents of a customer support system are disclosed. An example method may include retrieving historical conversations between the agents and users of the customer support system, receiving user comments for one or more of the historical conversations, identifying which of the received user comments includes keywords indicative of one or more quality of service attributes, generating transcripts of historical conversations associated with the identified user comments, training a machine learning model based at least in part on the generated transcripts and the user comments of the historical conversations associated with the identified user comments, providing a plurality of current conversations between agents and users of the customer support system to the trained machine learning model, and generating a behavioral score for each of the agents using the trained machine learning model.

Combining behavioral quality of service assessment with machine learning prediction

A method of training machine learning models (MLMs). An issue vector is generated using an issue MLM to generate a first output including first embedded natural language issue statements. An action vector is generated using an action MLM to generate a second output including related embedded natural language action statements. The issue and action MLMs are of a same type. An inner product of the first and second output is calculated, forming a third output. The third output is processed according to a sigmoid gate process to predict whether a given issue statement and corresponding action statement relate to a same call, resulting in a fourth output. A loss function is calculated from the fourth output by comparing the fourth output to a known result. The issue MLM and the action MLM are modified using the loss function to obtain a trained issue MLM and a trained action MLM.

A method for improving a transcription may include identifying, in the transcription, reliable channel tokens of an utterance of a reliable channel and an unreliable channel token of an utterance of an unreliable channel, and generating, using a machine learning model, a vector embedding for the unreliable channel token and vector embeddings for the reliable channel tokens. The method may further include calculating vector distances between the vector embedding and the vector embeddings, and generating, for the unreliable channel token and using the vector distances, a score corresponding to a reliable channel token. The method may further include determining that the score is within a threshold score, and in response to determining that the score is within the threshold score, replacing, in the transcription, the unreliable channel token with the reliable channel token.

Contextual multi-channel speech to text

A processor may generate a plurality of vectors from an original text by processing the original text with at least one unsupervised learning algorithm. Each of the plurality of vectors may correspond to a separate portion of a plurality of portions of the original text. The processor may determine respective segments to which respective vectors belong. The processor may minimize a distance between at least one vector belonging to the segment and a known vector from among one or more known vectors and applying a label of the known vector to the segment.

Growing community of inventors

Herzlia, Israel

Adi Shalev