Intuit, Inc.

Aviv Ben Arie

Yair Horesh

Sheer Dangoor

Miriam Hanna Manevitz

Omer Zalmanson

Itay Margolin

Ido Meir Mintz

Elhanan Mishraky

Or Basson

Liat Ben Porat Roda

Natalie Grace De Shetler

Hagai Fine

Meghan Mergui

Liran Dreval

Shir Meir Lador

Aspects of the present disclosure provide techniques for detecting latent bias in machine learning models. Embodiments include receiving a data set comprising features of a plurality of individuals. Embodiments include receiving identifying information for each individual of the plurality of individuals. Embodiments include predicting, for each respective individual of the plurality of individuals, a probability that the respective individual belongs to a given class based on the identifying information for the given individual. Embodiments include providing, as inputs to a machine learning model, the features of the plurality of individuals from the data set. Embodiments include receiving outputs from the machine learning model in response to the inputs. Embodiments include determining whether the machine learning model is biased against the given class based on the outputs and the probability that each respective individual of the plurality of individuals belongs to the given class.

Machine learning model bias detection

Aspects of the present disclosure relate to detecting spam messages. Embodiments include creating condensed vector representations of messages; calculating similarity scores for each message relative to other messages using the vector representations; associating messages of the plurality of messages with a grouping based on the calculated similarity score for the messages within the grouping exceeding a threshold; determining that a grouping of messages are spam messages by comparing an amount of messages of the grouping of messages sent within a first time window to an amount of messages of the grouping of messages sent within a second time window, wherein the second time window comprises a time period preceding the first time window; providing the identified spam message to a machine learning model; and training the machine learning model by iteratively adjusting parameters of the model based on tracking the identified spam message through multiple layers of the model.

Spam prevention using vector-based text similarity comparisons across multiple time windows

A method implements efficient real time serving of ensemble models. The method includes receiving an input and processing the input with an abridged model to generate a set of component scores and an abridged score. The method further includes processing the set of component scores with a deviation threshold to select one of the abridged score and an ensemble score as an output and presenting the output.

Efficient real time serving of ensemble models

A plurality of graph snapshots for a plurality of consecutive periodic time samples maps between connected components in consecutive graph snapshots and describes at least one feature of each connected component. A recursively-built tree tracks an evolution of one of the connected components through the plurality of graph snapshots, the tree including a root node representing the connected component at a final one of the consecutive periodic time samples and a plurality of leaf nodes branching from the root node. A plurality of paths is extracted from the tree by traversing the tree from the root node to respective ones of the plurality of leaf nodes. Each path contains data describing an evolution of a respective one of the connected components through time as indicated by evolution of the at least one feature of the respective one of the connected components. Each of the plurality of paths is converted into a respective numerical vector of a plurality of numerical vectors that may be used as inputs to a time series anomaly detection algorithm.

Feature extraction and time series anomaly detection over dynamic graphs

A method includes processing a set of query texts to identify a set of expressions, where each expression references a set of columns of datetime data in a datastore. The method also includes training a statistical model to determine a distribution of the datetime data for each column that was identified. The method further includes processing the set of expressions to generate a directed graph including more than one nodes and a plurality of edges, where each node represents one of the columns or a transformation applied by one of the expressions to one of the columns. The method additionally includes generating a weight for edges of the directed graph according to a distribution of the datetime data in the columns and a usage index of a corresponding expression.

Tables time zone adjuster

A method includes training, using first real data objects, a generative adversarial network having a generator model and a discriminator model to create a trained generator model that generates realistic data, and training, using adversarial data objects and second real data objects, the discriminator model to output an authenticity binary class for the adversarial data objects and the second real data objects. The method further includes deploying the discriminator model to a production system. In the production system, the discriminator model outputs the authenticity binary class to a system classifier model.

Adversarial detection using discriminator model of generative adversarial network architecture

A method implements efficient counterfactual search. The method includes receiving a request corresponding to an input vector, processing the input vector with a model to generate an output vector that does not correspond to a selected class, and processing the input vector using a component, of a plurality of components, to generate a counterfactual vector to the selected class. The plurality of components includes a number of dimensions that is less than a number of features of the input vector. The method further includes processing the counterfactual vector to generate a recommendation and presenting the recommendation.

Efficient counterfactual search

Described herein are example implementations for generating a perturbation seed for the perturbation of electronic data. A system obtains a plurality of datapoints (with one or more statistics calculated from the plurality of datapoints to be perturbed based on a perturbation seed). The system calculates one or more metrics from the plurality of datapoints. The system also generates, for each of the one or more metrics, a rounded metric by rounding the metric. The system further generates the perturbation seed. Generating the perturbation seed includes hashing the one or more rounded metrics. Rounding a metric may be to a defined place value (such as the second most significant place value), and a binary output of hashing the one or more rounded metrics may be converted to a number. The system may perturb one or more statistics based on the perturbation seed and output the one or more perturbed statistics.

Seed generation for electronic data perturbation

Systems and methods are described for automatically documenting queries and dynamically populating interactive graphical user interfaces with query recommendations. A computing system receives an initial query from an interactive graphical user interface and asynchronously parses the query for strings matching predetermined phrases. In response to determining that the initial query recites strings matching the predetermined phrases, the system extracts metadata and identifies a table name in the initial query, and modifies a table data in key-value data structure corresponding to the table. Subsequent queries related to the initial query cause the system to asynchronously populate the graphical user interface with query recommendations related to the initial query.

Database auto-documentation systems and methods

Described herein are example implementations for handling of phishing attempts. A system receives a request to perform an electronic transaction, with the request including information regarding a user account. The system generates one or more probabilities of the request being valid based on the request and processing of a plurality of electronic transactions associated with one or more user accounts, identifies whether the request is valid based on the one or more probabilities, and in response to identifying that the request is not valid, provides an indication that the request is not valid. Processing the plurality of electronic transactions includes, for a plurality of electronic transactions including information regarding user accounts, generating a numerical representation associated with each electronic transaction, clustering the plurality of electronic transactions into one or more clusters based on the numerical representations, and, for each cluster, generating one or more aggregates of user account information.

Phishing detection and mitigation

A method implements a table discovery service. The method includes receiving a query string, converting the query string to a query graph, and identifying a selected graph, of a set of graphs, that matches the query graph. The method further includes transmitting a notification identifying a previously generated table corresponding to the selected graph, receiving a notification response to accept the previously generated table, and transmitting data from the previously generated table in response to the query string.

Table discovery service

Techniques for detecting fraud may include obtaining a merchant's financial data; determining, via a machine learning model, a first prediction of the merchant's industry; generating a first probability matrix based on the first prediction and the declared information regarding the merchant's industry; determining, via the machine learning model, a second prediction of the merchant's industry; generating a second probability matrix based on the second prediction and the declared information regarding the merchant's industry; obtaining a declared industry of a subject merchant in a runtime environment; determining, via the machine learning model, a predicted industry for the subject merchant; obtaining, based on the declared industry and the predicted industry of the subject merchant, a first value from the first probability matrix and a second value from the second probability matrix; and labeling the subject merchant for further investigation.

Industry-profile service for fraud detection

A method including monitoring, using a machine learning model, network events of a network. The machine learning model generates fraud scores representing a corresponding probability that a corresponding network event is fraudulent. The method also includes detecting a failure of the machine learning model to generate, within a threshold time, a given fraud score for a given network event. The method also includes determining, by the machine learning model and after the threshold time, the given fraud score. The method also includes logging, responsive to detecting the failure, the given network event in a first table, including logging the given fraud score. The method also includes determining a metric based on comparing the first table to a second table which logs at least the given fraud score and the fraud scores. The method also includes generating an adjusted machine learning model based on the metric.

Evaluating machine learning model performance by leveraging system failures

Certain aspects of the present disclosure provide techniques for placing targeted messages in communications within a software application using machine learning models. An example method generally includes retrieving, from a repository, a data set of targeted messages. For each respective targeted message in the data set of targeted messages, an effectiveness score for a party associated with the respective targeted message, a distance score between the party associated with the respective targeted message and a host party, and a match score between the party associated with the respective targeted message and the host party. Based on the effectiveness score, the distance score, and the match score for each respective targeted message in the data set of targeted messages, a message is selected to be included in one or more communications by the host party. The one or more communications including the selected message are generated and output for transmission.

Growing community of inventors

Ramat Gan, Israel

Aviv Ben Arie