Maplebear Inc.

Tejaswi Tenneti

Tyler Russell Tate

Haixun Wang

Taesik Na

Jonathan Lennart Bender

Xiao Xiao

Vinesh Reddy Gudla

Aditya Subramanian

Manmeet Singh

Qingyuan Chen

Jason E Scott

Sharath Rao Karikurve

Xuan Zhang

Guanghua Shu

Logan William Murdock

Zhihong Xu

David Vengerov

Shishir Kumar Prasad

Tilman Drerup

Reza Sadri

Levi Boxell

Shrikar Archak

Raochuan Fan

Yuqing Xie

Wideet Shende

Ruhan Zhang

Michael Kurish

Xiaochen Wang

An online system displays items to a user in search results based on appeasement scores for the items, adjusted according to how specific the search query is. The online system receives a search query from a user of an online system. The online system computes a query specificity score, a measure of the specificity of the search query. The online system accesses candidate items from a database that potentially match the search query. For each candidate item, the online system may compute or predict an appeasement score. The online system adjusts the appeasement score based on the query specificity score such that a more specific query weights the appeasement score lower than a less specific query. The online system may then compute a ranking score based on the adjusted appeasement score and display the candidate items to the user based on their ranking scores.

Ranking search results based on appeasement signals and query specificity

An online system generates a template shopping list for a user by accessing a machine learning model trained based on historical order information associated with the user, applying the model to predict likelihoods of conversion for item categories by the user, and populating the template shopping list with one or more item categories based on the predicted likelihoods. The system ranks one or more item types associated with each item category in the template shopping list and determines a set of collection rules associated with one or more item categories/types based on the historical order information. The system generates a suggested shopping list by populating each item category in the template shopping list with one or more item types and a quantity of each item type based on the ranking and rules and sends the suggested shopping list and rules for display to a client device associated with the user.

Generating a suggested shopping list by populating a template shopping list of item categories with item types and quantities based on a set of collection rules

An online system updates the labels on negative examples to account for the possibility that the example is a false negative. The system generates a set of initial training examples that each include a query input by the user and item data for an item presented as a result to the user's query. Each training example also includes an initial label, which represents whether the user interacted with the item presented as a search result. The online system updates the initial label for a negative training example by identifying a set of bridge queries and computing a similarity score between the query for the training example and the bridge queries. The online system computes an updated label for the negative example based on the similarity scores and updates the training example with the updated label.

False negative prediction for training a machine-learning model

An online system leverages stored interactions with items made by users after the online system received queries to determine display of items satisfying the query. For example, the online system trains a model to predict a likelihood of a user performing an interaction with an item displayed after a query was received. As different items receive different amounts of interaction from users, limited historical interaction with certain items may limit accuracy of the model. The online system generates embeddings for previously received queries and uses measures of similarity between embeddings for queries to generate clusters of queries. Previous interactions with queries in a cluster are combined, with the combined data being used for determining display of items in response to a query.

Clustering data describing interactions performed after receipt of a query based on similarity between embeddings for different queries

A system uses a contextual bandit model for query processing. The system receives, from a client device, a user query for identifying one or more items by the system. The user query is described by one or more query features. The system obtains one or more contextual features describing a context of the user query. The system applies a contextual bandit model to the query features and the contextual features to select a query processing model from a plurality of query processing models. The system applies the selected query processing model to the user query to obtain query results. The system transmits the query results for display on the client device.

Contextual bandit model for query processing model selection

An online concierge system generates an item graph connecting item nodes with attribute nodes of the items. When the online concierge system receives a search query to identify one or more items from a customer, the online concierge system parses the search query into combinations of terms and identifies item nodes and attribute nodes related to the search query. The online concierge system may determine that no item nodes meet presentation criteria. The online concierge system may determine that a reformulated search query has a higher conversion probability than the search query received from the customer. The online concierge system reformulates the search query. The online concierge system selects item nodes as search results. The online concierge system transmits the search results to the customer.

Query reformulations for an item graph

An online system maintains item embeddings for items. As a number of items maintained by the online system increases, maintaining a single index of the item embeddings is increasingly difficult. To increase scalability, the online system partitions item embeddings into multiple indices, with each index corresponding to a value of a specific attribute maintained by the online system for items. For example, an online system generates indices that each correspond to a different warehouse offering items. To expedite retrieval of item embeddings, the online system allocates each index to one of a number of shards. When the online system receives a query, the online system determines an embedding for the query and retrieves an index from a shard based on metadata received with the query. Based on distances between the query for the embedding and the item embeddings in the retrieved index, the online system selects one or more items.

Distributed approximate nearest neighbor service architecture for retrieving items in an embedding space

An online system maintains various items and maintains values for different attributes of the items, as well as an item embedding for each item. When the online system receives a query for retrieving one or more items, the online system generates an embedding for the query. Based on measures of similarity between the embedding for the query and item embeddings, the online system selects a set of items. The online system identifies a specific attribute of items and generates a whitelist of values for the specific attribute based on measures of similarity between item embeddings for items in the selected set and the embedding for the query. The online system removes items having values for the selected attribute outside of the whitelist of values from the selected set of items to identify items more likely to be relevant to the query.

Accounting for item attributes when selecting items satisfying a query based on item embeddings and an embedding for the query

An online concierge system maintains various items and an item embedding for each item. When the online concierge system receives a query for retrieving one or more items, the online concierge system generates an embedding for the query. The online concierge system trains a machine-learned model to determine a measure of relevance of an embedding for a query to item embeddings by generating training data of examples including queries and items with which users performed a specific interaction. The online concierge system generates a subset of the training data including examples satisfying one or more criteria and further trains the machine-learned model by application to the examples of the subset of the training data and stores parameters resulting from the further training as parameters of the machine-learned model.

Training a machine learned model to determine relevance of items to a query using different sets of training data from a common domain

An online concierge system suggests subsequent search queries based on previous search queries and whether the previous search queries resulted in conversions. The online concierge system trains a machine learning model using previous delivery orders and whether initial and subsequent search queries in the previous delivery orders resulted in conversions. When the online concierge system receives a search query to identify one or more items from a customer, the online concierge system parses the search query into combinations of terms and identifies items related to the search query. In response to the search query resulting in a conversion, the online concierge system retrieves a conversion graph and presents a suggested subsequent search query based on the conversion graph. In response to the search query not resulting in a conversion, the online concierge system retrieves a non-conversion graph and presents a suggested subsequent search query based on the non-conversion graph.

Providing search suggestions based on previous searches and conversions

An online concierge system generates a graph connecting items with attributes of the items and other items. Hence, the graph includes nodes corresponding to attributes and nodes corresponding to items, with an item connected to attributes of the item in the graph. Example attributes include a brand, a category, a department, or any other suitable information about the item. When the online concierge system receives a search query to identify one or more items from a customer, the online concierge system parses the search query into combinations of terms and compares different combinations of terms to the graph to determine connections between different combinations of terms in the graph. Based on measures of connectedness between combinations of terms and connections in the graph, items are identified from one or more combinations of terms. Information about the identified items is presented to the customer.

Identifying items offered by an online concierge system for a received query based on a graph identifying relationships between items and attributes of the items

An online concierge system generates an item graph connecting item nodes with attribute nodes of the items. Example attributes include a brand, a category, a department, or any other suitable information about the item. When the online concierge system receives a search query to identify one or more items from a customer, the online concierge system parses the search query into combinations of terms and identifies item nodes and attribute nodes related to the search query. The online concierge system identifies item nodes and attribute nodes that are likely to result in a conversion. Information about the identified nodes is presented to the customer. The customer may select an item node to purchase the item, or an attribute node to execute a new search query based on terms associated with the attribute node.

Attribute node widgets in search results from an item graph

An online system provides options for selection by a user. The online system receives a query entered on a client device. The online system queries an item database to retrieve a set of items related to the query and assigns each item to a product category in a predefined taxonomy that maps items to product categories. The online system inputs each item into a prediction model trained to predict a probability that an item is available at a warehouse location. The online system determines that a first product category has low availability based on predicted probabilities for items in the first product category. Responsive to determining that a first product category has low availability, the online system generates a generic item for the first product category and sends a list of items including the generic item to the client device for display responsive to the query.

Growing community of inventors

Fremont, CA, United States of America

Tejaswi Tenneti