Pinterest, Inc.

Andrew Huan Zhai

Charles Joseph Rosenberg

Dmitry Olegovich Kislyuk

Kelei Xu

Naveen Gavini

Kevin Yushi Jing

Nikil Pancha

Kaifeng Chen

Adam Jay Barton

Marcelo Reis E Silva De Queiroz

Yushi Jing

Kunlong Gu

Eric Kim

Chenyi Li

Zhiyuan Zhang

Maesen Churchill

Yue Li Du

Zhefei Yu

Described are systems and methods for determining complementary and/or matching objects based on an input query object. The described systems and methods can generate an embedding representative of the provided object, which can be transformed to generate a style embedding by a trained system, such as a machine learning system. The style embedding can then be used to identify one or more complementary objects from a corpus of classified objects. Aspects of the present disclosure also relate to creation of the training dataset, as well as training the machine learning system.

Systems and methods for identifying complementary objects having similar styles

Systems and methods for identifying content for an input query are presented. A mapping model is trained to map elements of an input query embedding vector for a received query into one or more elements of a destination embedding vector. In response to receiving an input query, an input query embedding vector is generated that projects into an input query embedding space. The input query embedding vector is processed by the mapping model to map the input query embedding vector into one or more elements of a destination embedding vector in a destination embedding space, resulting in a partial destination embedding vector. Items of a corpus of content are projected into the destination embedding space and the partial destination embedding vector is also projected into the destination embedding space. A similarity measure determines the most-similar items to the partial destination embedding vector and at least some of the most-similar items are returned in response to the input query.

Identifying similar content in a multi-item embedding space

Described is a system and method for enabling dynamic selection of a search input. For example, rather than having a static search input box, the search input may be dynamically positioned such that it encompasses a portion of displayed information. For example, a user may touch a touch-based display using two fingers to invoke the dynamic search input and then determine a size and a position of the dynamic search input by moving their fingers on the display. An image segment that includes a representation of the encompassed portion of the displayed information is generated and processed to determine an object represented in the portion of the displayed information. Additional images with visually similar representations of objects are then determined and presented to the user.

Dynamic search control invocation and visual search

Systems and methods for responding to a subscriber's text-based request for content items are presented. In response to a request from a subscriber, word pieces are generated from the text-based terms of the request. A request embedding vector of the word pieces is obtained from a trained machine learning model. Using the request embedding vector, a set of content items, from a corpus of content items, is identified. At least some content items of the set of content items are returned to the subscriber in response to the text-based request for content items.

Identifying content items in response to a text-based request

Described is a system and method for enabling visual search for information. With each selection of an object included in an image, additional images that include visually similar objects are determined and presented to the user.

Object based image search

Described is a system and method for enabling dynamic selection of a search input. For example, rather than having a static search input box, the search input may be dynamically positioned such that it encompasses a portion of displayed information. An image segment that includes a representation of the encompassed portion of the displayed information is generated and processed to determine an object represented in the portion of the displayed information. Additional images with visually similar representations of objects are then determined and presented to the user.

Dynamic search input selection

Mixed type image based search results

Growing community of inventors

Berkeley, CA, United States of America

Andrew Huan Zhai