Maplebear Inc.

Facebook, Inc.

Google Inc.

Haixun Wang

Tejaswi Tenneti

Taesik Na

Xiao Xiao

Min Xie

Xinyu Li

Sharath Rao Karikurve

Xuan Zhang

Saurav Manchanda

Li Tan

Bo Zeng

Krishnakumar Subramanian

Ruoming Jin

Jian Li

Hao Yang

Zhen Wen

Asif Haque

Shishir Kumar Prasad

Vinesh Reddy Gudla

Shih-Ting Lin

Guanghua Shu

Jonathan Newman

Zhihong Xu

Yuqing Xie

Ze He

Allan Stewart

Negin Entezari

Charles Perng

Ruhan Zhang

Xiaochen Wang

An online system configures one or more system AI agent instances that interact with user AI agents and performs one or more tasks on behalf of the online system. Thus, responsive to detecting the presence of a user AI agent representing a particular user, the online system directs the session for the user to communicate and interact with a system AI agent.

AI agent-driven interaction model for applications

An online system generates a template shopping list for a user by accessing a machine learning model trained based on historical order information associated with the user, applying the model to predict likelihoods of conversion for item categories by the user, and populating the template shopping list with one or more item categories based on the predicted likelihoods. The system ranks one or more item types associated with each item category in the template shopping list and determines a set of collection rules associated with one or more item categories/types based on the historical order information. The system generates a suggested shopping list by populating each item category in the template shopping list with one or more item types and a quantity of each item type based on the ranking and rules and sends the suggested shopping list and rules for display to a client device associated with the user.

Generating a suggested shopping list by populating a template shopping list of item categories with item types and quantities based on a set of collection rules

A computer system finetunes a machine-learned language model to generate a personalized response to a user request. The system may generate a user representation for each of a plurality of users by applying a transformer model to a sequence of tokens representing a sequence of activities of the user. The system may train an evaluation model coupled to receive a user representation and a response to a user request and generate an estimated evaluation score indicating a level of personalization of the response to the user. The system may finetune a first machine-learned language model to generate a second machine-learned language model. The finetuned machine-learned language model is configured to provide personalized responses for customer services at an online concierge system.

Personalized machine-learned large language model (LLM)

An online system updates the labels on negative examples to account for the possibility that the example is a false negative. The system generates a set of initial training examples that each include a query input by the user and item data for an item presented as a result to the user's query. Each training example also includes an initial label, which represents whether the user interacted with the item presented as a search result. The online system updates the initial label for a negative training example by identifying a set of bridge queries and computing a similarity score between the query for the training example and the bridge queries. The online system computes an updated label for the negative example based on the similarity scores and updates the training example with the updated label.

False negative prediction for training a machine-learning model

An online system leverages stored interactions with items made by users after the online system received queries to determine display of items satisfying the query. For example, the online system trains a model to predict a likelihood of a user performing an interaction with an item displayed after a query was received. As different items receive different amounts of interaction from users, limited historical interaction with certain items may limit accuracy of the model. The online system generates embeddings for previously received queries and uses measures of similarity between embeddings for queries to generate clusters of queries. Previous interactions with queries in a cluster are combined, with the combined data being used for determining display of items in response to a query.

Clustering data describing interactions performed after receipt of a query based on similarity between embeddings for different queries

A system may generate a prompt based in part on a search query from a customer client device. The prompt instructs a machine learned model to provide item predictions. And the model was trained by: converting structured data describing items of an online catalog to annotated text data (unstructured data), generating training examples based in part on the annotated text data, and training the model using the training examples. The system may receive item predictions generated by the prompt being applied to the machine learned model, the item predictions may have corresponding item identifiers. The item predictions are processed to identify a recommended item from the item predictions. The processing includes determining item information for the recommended item using an item identifier associated with the recommended item. The item information is provided to the customer client device.

Machine learned models for search and recommendations

An online system maintains various items and maintains values for different attributes of the items, as well as an item embedding for each item. When the online system receives a query for retrieving one or more items, the online system generates an embedding for the query. Based on measures of similarity between the embedding for the query and item embeddings, the online system selects a set of items. The online system identifies a specific attribute of items and generates a whitelist of values for the specific attribute based on measures of similarity between item embeddings for items in the selected set and the embedding for the query. The online system removes items having values for the selected attribute outside of the whitelist of values from the selected set of items to identify items more likely to be relevant to the query.

Accounting for item attributes when selecting items satisfying a query based on item embeddings and an embedding for the query

An online concierge system maintains various items and an item embedding for each item. When the online concierge system receives a query for retrieving one or more items, the online concierge system generates an embedding for the query. The online concierge system trains a machine-learned model to determine a measure of relevance of an embedding for a query to item embeddings by generating training data of examples including queries and items with which users performed a specific interaction. The online concierge system generates a subset of the training data including examples satisfying one or more criteria and further trains the machine-learned model by application to the examples of the subset of the training data and stores parameters resulting from the further training as parameters of the machine-learned model.

Training a machine learned model to determine relevance of items to a query using different sets of training data from a common domain

An online concierge system receives a delivery order containing a list of items, generates a suggested picking sequence for picking the delivery order in a warehouse, and transmits the suggested picking sequence to a mobile device of the shopper. Generating the suggested sequence includes applying a trained item sequence model to the delivery order. Training the item sequence model includes accessing data about a set of historical orders, determining a pairwise distance between each pair of aisles in the warehouse based on the data about the set of historical orders, and generating a distance graph based on the pairwise distance between each pair of aisles in the warehouse. The plurality of nodes represent a plurality of aisles in the warehouse, and the plurality of edges represent pairwise distances between pairs of aisles.

Picking sequence optimization within a warehouse for an item list

An online concierge system receives unstructured data describing items offered for purchase by various warehouses. To generate attributes for products from the unstructured data, the online concierge system extracts candidate values for attributes from the unstructured data through natural language processing. One or more users associate a subset candidate values with corresponding attributes, and the online concierge system clusters the remaining candidate values with the candidate values of the subset associated with attributes. One or more users provide input on the accuracy of the generated clusters. The candidate values are applied as labels to items by the online concierge system, which uses the labeled items as training data for an attribute extraction model to predict values for one or more attributes from unstructured data about an item.

Weakly supervised extraction of attributes from unstructured data to generate training data for machine learning models

An online concierge system trains a classification model as a domain adversarial neural network from training data labeled with source classes from a source domain that do not overlap with target classes from a target domain output by the classification model. The online concierge system maps one or more source classes to a target class. The classification model extracts features from an image, classifies whether an image is from the source domain or the target domain, and predicts a target class for an image from the extracted features. The classification model includes a gradient reversal layer between feature extraction layers and the domain classifier that is used during training, so the feature extraction layers extract domain invariant features from an image.

Training a classification model using labeled training data that does not overlap with target classifications for the classification model

An online concierge system allows users to order items from a warehouse, which may have multiple warehouse locations. The online concierge system provides a user interface to users for ordering the items, with the user interface providing an indication of whether an item is predicted to be available at the warehouse at different times. To predict availability of an item model at different times, the online concierge system selects data from historical information about availability of items at one or more warehouses based on temporal, geospatial, and socioeconomic information about observations of historical availability of items at warehouses. The online concierge system accounts for distances between observations and a time and geographic location in a feature space to select observations for predicting item availability at the time and the geographic location.

Selecting a warehouse location for displaying an inventory of items to a user of an online concierge system based on predicted availabilities of items at the warehouse over time

An online concierge shopping system identifies candidate items to a user for inclusion in an order based on prior user inclusion of items in orders and items currently included in the order. From a multi-dimensional tensor generated from cooccurrences of items in orders from various users, the online concierge system generates item embeddings and user embeddings in a common latent space by decomposing the multi-dimensional tensor. From items included in an order, the online concierge system generates an order embedding from item embeddings of the items included in the order. Scores for candidate items are determined based on similarity of item embeddings for the candidate items to the order embedding. Candidate items are selected based on their scores, with the selected candidate items identified to the user.

Personalized recommendation of complementary items to a user for inclusion in an order for fulfillment by an online concierge system based on embeddings for a user and for items

Growing community of inventors

Palo Alto, CA, United States of America

Haixun Wang