Maplebear Inc.

Microsoft Technology Licensing, LLC

Taesik Na

Tejaswi Tenneti

Haixun Wang

Xiao Xiao

Daniel Lo

Eric S Chung

Vinesh Reddy Gudla

Haishan Zhu

Zhihong Xu

Guanghua Shu

Douglas Christopher Burger

Xuan Zhang

Bita Darvish Rouhani

Li Tan

Manmeet Singh

Reza Sadri

Raochuan Fan

Jian Li

Eric Sen Chung

Wideet Shende

Ruhan Zhang

Lee Cohn

Yuqing Xie

Xiaochen Wang

An online system updates the labels on negative examples to account for the possibility that the example is a false negative. The system generates a set of initial training examples that each include a query input by the user and item data for an item presented as a result to the user's query. Each training example also includes an initial label, which represents whether the user interacted with the item presented as a search result. The online system updates the initial label for a negative training example by identifying a set of bridge queries and computing a similarity score between the query for the training example and the bridge queries. The online system computes an updated label for the negative example based on the similarity scores and updates the training example with the updated label.

False negative prediction for training a machine-learning model

An online system provides a search interface for a user to identify items. The search interface may present suggested search queries to the user, allowing the user to select a suggested search query rather than manually entering search terms to form a search query. To identify search queries most likely to be selected by the user, the online system gets a set of candidate search queries and generates a relevance score for each candidate search query by applying a trained query relevance model to each candidate search query. The scored candidate search queries are selected and ranked using the relevance scores, and the selected candidate search queries are displayed using the ranking in the search interface. The query relevance model is a transformer-based small language model receiving a user sequence of prior search queries and items with which the user interacted and the candidate search terms as input.

Search engine for recommending search queries based on user interactions using a transformer-based language model

An online system leverages stored interactions with items made by users after the online system received queries to determine display of items satisfying the query. For example, the online system trains a model to predict a likelihood of a user performing an interaction with an item displayed after a query was received. As different items receive different amounts of interaction from users, limited historical interaction with certain items may limit accuracy of the model. The online system generates embeddings for previously received queries and uses measures of similarity between embeddings for queries to generate clusters of queries. Previous interactions with queries in a cluster are combined, with the combined data being used for determining display of items in response to a query.

Clustering data describing interactions performed after receipt of a query based on similarity between embeddings for different queries

An online system maintains item embeddings for items. As a number of items maintained by the online system increases, maintaining a single index of the item embeddings is increasingly difficult. To increase scalability, the online system partitions item embeddings into multiple indices, with each index corresponding to a value of a specific attribute maintained by the online system for items. For example, an online system generates indices that each correspond to a different warehouse offering items. To expedite retrieval of item embeddings, the online system allocates each index to one of a number of shards. When the online system receives a query, the online system determines an embedding for the query and retrieves an index from a shard based on metadata received with the query. Based on distances between the query for the embedding and the item embeddings in the retrieved index, the online system selects one or more items.

Distributed approximate nearest neighbor service architecture for retrieving items in an embedding space

A system may generate a prompt based in part on a search query from a customer client device. The prompt instructs a machine learned model to provide item predictions. And the model was trained by: converting structured data describing items of an online catalog to annotated text data (unstructured data), generating training examples based in part on the annotated text data, and training the model using the training examples. The system may receive item predictions generated by the prompt being applied to the machine learned model, the item predictions may have corresponding item identifiers. The item predictions are processed to identify a recommended item from the item predictions. The processing includes determining item information for the recommended item using an item identifier associated with the recommended item. The item information is provided to the customer client device.

Machine learned models for search and recommendations

An online system displays search results in response to a query by receiving a query from a customer. An online system accesses a set of candidate items and computes a relevance score and personalization score for each item. The online system computes the relevance score based on query data and item data and may normalize the relevance score. The online system computes the personalization score based on item data, such as an item embedding, and user data, such as a user embedding. The online system computes a query specificity score and adjusts the personalization score with the query specificity score such that generic queries have high personalization scores and specific queries have low personalization scores. The online system combines the relevance and personalization scores for each candidate item into a ranking score and displays the candidate items to the customer based on their ranking scores.

Providing and displaying search results in response to a query

An online system maintains various items and maintains values for different attributes of the items, as well as an item embedding for each item. When the online system receives a query for retrieving one or more items, the online system generates an embedding for the query. Based on measures of similarity between the embedding for the query and item embeddings, the online system selects a set of items. The online system identifies a specific attribute of items and generates a whitelist of values for the specific attribute based on measures of similarity between item embeddings for items in the selected set and the embedding for the query. The online system removes items having values for the selected attribute outside of the whitelist of values from the selected set of items to identify items more likely to be relevant to the query.

Accounting for item attributes when selecting items satisfying a query based on item embeddings and an embedding for the query

An online concierge system maintains various items and an item embedding for each item. When the online concierge system receives a query for retrieving one or more items, the online concierge system generates an embedding for the query. The online concierge system trains a machine-learned model to determine a measure of relevance of an embedding for a query to item embeddings by generating training data of examples including queries and items with which users performed a specific interaction. The online concierge system generates a subset of the training data including examples satisfying one or more criteria and further trains the machine-learned model by application to the examples of the subset of the training data and stores parameters resulting from the further training as parameters of the machine-learned model.

Training a machine learned model to determine relevance of items to a query using different sets of training data from a common domain

Bounding box quantization can reduce the quantity of bits utilized to express numerical values prior to the multiplication of matrices comprised of such numerical values, thereby reducing both memory consumption and processor utilization. Stochastic rounding can provide sufficient precision to enable the storage of weight values in reduced-precision formats without having to separately store weight values in a full-precision format. Alternatively, other rounding mechanisms, such as round to nearest, can be utilized to exchange weight values in reduced-precision formats, while also storing weight values in full-precision formats for subsequent updating. To facilitate conversion, reduced-precision formats such as brain floating-point format can be utilized.

Neural network training with decreased memory consumption and processor utilization

Technology related to training a neural network accelerator using mixed precision data formats is disclosed. In one example of the disclosed technology, a neural network accelerator is configured to accelerate a given layer of a multi-layer neural network. An input tensor for the given layer can be converted from a normal-precision floating-point format to a quantized-precision floating-point format. A tensor operation can be performed using the converted input tensor. A result of the tensor operation can be converted from the block floating-point format to the normal-precision floating-point format. The converted result can be used to generate an output tensor of the layer of the neural network, where the output tensor is in normal-precision floating-point format.

Growing community of inventors

Issaquah, WA, United States of America

Taesik Na