Google Inc.

Fujifilm Corporation

Pinterest, Inc.

Kunlong Gu

Charles Joseph Rosenberg

Thomas J Duerig

Samy Bengio

Jason Edward Weston

Erik Murphy-Chutorian

Congcong Li

Andrew Huan Zhai

Akira Hasegawa

Yun Zhou

Eric Kim

Huzefa Neemuchwala

Chenyi Li

Mingchen Gao

Sean Arietta

Described are systems and methods for determining complementary and/or matching objects based on an input query object. The described systems and methods can generate an embedding representative of the provided object, which can be transformed to generate a style embedding by a trained system, such as a machine learning system. The style embedding can then be used to identify one or more complementary objects from a corpus of classified objects. Aspects of the present disclosure also relate to creation of the training dataset, as well as training the machine learning system.

Systems and methods for identifying complementary objects having similar styles

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for analyzing image search result relevance. In one aspect, a method includes receiving result data specifying a search query and responsive image search results that reference images that are responsive to the search query. A determination is made that the search query matches an indexed query. An image relevance model is identified for the indexed query. The image relevance model can output a relevance score adjustment factor for an image search result based on image feature values of the image that is referenced by the search result. A relevance score adjustment factor is determined for each image search result using the identified image relevance model. A relevance score for each image search result is adjusted using the image's image relevance score adjustment factor. The images are ranked based on the adjusted relevance scores.

Online image analysis

Methods, systems and apparatus for refining image relevance models. In general, one aspect includes receiving a trained image relevance model that generates relevance measures of content feature values of images to a query, identifying a first threshold number of common content feature values for the set of training images, the common content feature values being identified as a set of content feature values that are each shared by at least a portion of the training images, identifying a subset of the set of training images having a quantity of the common content feature values greater than a second threshold number of content features values, and generating a re-trained image relevance model based on content feature values of the set of training images, wherein content feature values of the subset of training images are weighted higher than content feature values of the training images not in the subset.

Refining image relevance models

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating a query image model. In one aspect, a method includes receiving a set of images determined to be responsive to a query and ranked according to a first order; determining a positive image signature from a first subset of images selected from images ranked highest in the first order, determining a negative image signature from a second subset of images selected from images ranked lowest in the first order, determining a query image signature for the query based on a difference of the positive image signature and the negative image signature; and applying the query image signature to each image in the set of images to rank the images according to a second order that is different from the first order.

Query image model

A method includes receiving a search query comprising one or more query terms, receiving image relevance models that each generate relevance measures of content feature values of images to a query, each image relevance model being a predictive model that has been trained based on content feature values of a set of training images, and each of the queries being a unique set of one or more query terms received by a search system as a query input, identifying an image relevance model for a different query that has been identified as similar to the received search query, and calculating a fractional adjustment multiplier for search results responsive to the received search query, the fractional adjustment multiplier being based on a relevance measure generated by the identified image relevance model for the different query and based on a degree of similarity between the different query and the received search query.

Applying query based image relevance models

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for identifying images responsive to a search phrase are disclosed. In one aspect, a method includes identifying a set of responsive images for a search phrase that includes two or more terms. Interaction rankings are determined for images in the set of responsive images. Two or more sub-queries are created based on the search phrase. Sub-query model rankings are determined for images in the set of responsive images. A search phrase score is determined for the image relevance model. Based on the search phrase scores for the sub-queries, one of the sub-query models is selected as a model for the search phrase.

Sub-query evaluation for image search

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for classifying images. In one aspect, a method includes receiving training samples for a particular data dimension. Each training sample specifies a training value for the data dimension and a measure of relevance between the training sample and a phrase. A value range is determined for the data dimension. The value range is segmented into two or more segments. A predictive model is trained for each segment. The predictive model for each segment is trained to predict an output based on an input value that is within the segment. A classification sample specifying an input value is received. A classification output is computed based on the input value, the predictive model for the segment in which the input value is included, and the predictive model for an adjacent segment.

Image classification

Methods and apparatuses perform thickness compensation in anatomical images. The method according to one embodiment accesses digital image data representing an image including a breast; estimates thickness of the breast at multiple locations inside the breast using an image data characteristic and a reference tissue in the breast; compensates thickness of the breast using a thickness model; and refines compensation of breast thickness from the compensating step.

Method and apparatus for thickness compensation in mammographic images

A method and an apparatus automatically detect eyeglasses in an image. The method according to one embodiment accesses digital image data representing an image including a face; detects eyeglasses in the image by using nose ridge masking; and outputs a decision about presence or absence of eyeglasses in the image.

Method and apparatus for automatic eyeglasses detection using a nose ridge mask

A method and an apparatus automatically detect and remove eyeglasses from an image. The method according to one embodiment accesses digital image data representing an image including a face; detects eyeglasses in the image to produce a report about presence or absence of eyeglasses in the image; normalizes illumination of the image to obtain a normalized image; and removes eyeglasses from the normalized image to obtain a face image without eyeglasses.

Growing community of inventors

Belmont, CA, United States of America

Kunlong Gu