Xerox Corporation

Naver Corporation

Conduent Business Services, LLC

Naila Murray

Florent C Perronnin

Luca Marchesotti

Albert Gordo Soldevila

Diane Larlus

Sandra Skaff

Jon Almazan

Rafael Sampaio De Rezende

Cesar De Souza

Diane Larlus-Larrondo

Eleonora Vig

Bojana Gajic

Michal Kucer

Jon Almazán Almazán

Saumya Jetley

A generative adversarial network, a method of training a generator neural network, and a method of generating images using the generator network is provided. The generator neural network is configured to process an input comprising a noise vector and a pair of conditioning variables to generate an image according to the conditioning variables. The generator neural network includes a mixed-conditional batch normalization layer. The mixed-conditional batch normalization layer is configured to normalize a network layer output to generate a normalized network layer output, comprising transforming the network layer output in accordance with mixed-conditional batch normalization layer parameters to generate the normalized network layer output, wherein the mixed-conditional batch normalization layer parameters are computed by applying an affine transformation to the conditioning variables.

Generative adversarial network for generating images

A system includes: a training dataset including first objects of a first modality and second objects of a second modality different than the first modality, where the second objects include text that is descriptive of the first objects; a first matrix including first relevance values indicative of relevance between the first objects and the second objects, respectively; a second matrix including second relevance values indicative of relevance between the second objects and the first objects, respectively; and a training module configured to: assign ones of the second objects to bins based on distances between the ones of the objects and a query; determine a ranking measure based on a number of the ones of the second objects assigned to the bins; determine losses based on the ranking measure and the first and second matrices; determine a final loss based on the losses; train embedding functions based on the final loss.

Cross-modal search systems and methods

A training system includes: a training dataset including first objects of a first modality and second objects of a second modality that are associated with the first objects, respectively; a first matrix including first relevance values indicative of relevance between the first objects and the second objects, respectively; a second matrix including second relevance values indicative of relevance between the second objects and the first objects, respectively; and a training module configured to: based on similarities between ones of the second objects, generate a third matrix by selectively adding first additional relevance values to the first matrix; based on the similarities between the ones of the second objects, generate a fourth matrix by selectively adding second additional relevance values to the second matrix; and store the third and fourth matrices in memory of a search module for cross-modal retrieval in response to receipt of search queries.

A method and system generate an ensemble image representation for cross-domain retrieval of a fashion item image from a database by using a three-stream Siamese triplet loss trained convolutional neural network to generate a first retrieval descriptor corresponding to an inputted query image; using an average precision loss trained convolutional neural network to generate a second retrieval descriptor corresponding to the inputted query image; concatenating both the first retrieval descriptor and the second retrieval descriptor; and I-normalizing the concatenated result to generate the ensemble image representation. During a first stage of the method and system, database items are cropped using a trained fine-grained fashion item detector.

Training and using an ensemble of complimentary convolutional neural networks for cross-domain retrieval of fashion item images

A method for re-identification of a subject in an image by pre-training a convolutional neural network to recognize individuals within a closed set of possible identifications and further pre-training the convolutional neural network using classification loss; training the pre-trained convolutional neural network by sequentially processing a plurality of triplet of images, each triplet containing a query image degraded by adding random noise to a region of the query image, a positive image corresponding to an image of a same subject as in the query image, and a negative image corresponding to an image of a different subject as in the query image by (a) ranking the triplets by the triplet loss computed, (b) selecting a subset of triplets among the plurality of triplets, and (c) retraining the pre-trained convolutional neural network on each of the triplets of the subset of triplets.

Training and using a convolutional neural network for person re-identification

A system and method enable semantic comparisons to be made between word images and concepts. Training word images and their concept labels are used to learn parameters of a neural network for embedding word images and concepts in a semantic subspace in which comparisons can be made between word images and concepts without the need for transcribing the text content of the word image. The training of the neural network aims to minimize a ranking loss over the training set where non relevant concepts for an image which are ranked more highly than relevant ones penalize the ranking loss.

Latent embeddings for word images and their semantics

A method for generating a system for predicting saliency in an image and method of use of the prediction system are described. Attention maps for each of a set of training images are used to train the system. The training includes passing the training images though a neural network and optimizing an objective function over the training set which is based on a distance measure computed between a first probability distribution computed for a saliency map output by the neural network and a second probability distribution computed for the attention map for the respective training image. The trained neural network is suited to generation of saliency maps for new images.

End-to-end saliency mapping via probability distribution prediction

Authentication methods are disclosed for determining whether a person or object to be authenticated is a member of a set of authorized persons or objects. A query signature is acquired comprising a vector whose elements store values of an ordered set of features for the person or object to be authenticated. The query signature is compared with an aggregate signature comprising a vector whose elements store values of the ordered set of features for the set of authorized persons or objects. The individual signatures for the authorized persons or objects are not stored; only the aggregate signature. It is determined whether the person or object to be authenticated is a member of the set of authorized persons or objects based on the comparison. The comparing may comprise computing an inner product of the query signature and the aggregate signature, with the determining being based on the inner product.

One-to-many matching with application to efficient privacy-preserving re-identification

A system and method enable similarity measures to be computed between pairs of images and between a color name and an image in a common feature space. Reference image representations are generated by embedding color name descriptors for each reference image in the common feature space. Color name representations for different color names are generated by embedding synthesized color name descriptors in the common feature space. For a query including a color name, a similarity is computed between its color name representation and one or more of the reference image representations. For a query which includes a query image, a similarity is computed between a representation of the query image and one or more of reference image representations. The method also enables combined queries which include both a query image and a color name to be performed. One or more retrieved reference images, or information based thereon, is then output.

Discriminative embedding of local color names for object retrieval and classification

A method for generating an image representation includes generating a set of embedded descriptors, comprising, for each of a set of patches of an image, extracting a patch descriptor which is representative of the pixels in the patch and embedding the patch descriptor in a multidimensional space to form an embedded descriptor. An image representation is generated by aggregating the set of embedded descriptors. In the aggregation, each descriptor is weighted with a respective weight in a set of weights, the set of weights being computed based on the patch descriptors for the image. Information based on the image representation is output. At least one of the extracting of the patch descriptors, embedding the patch descriptors, and generating the image representation is performed with a computer processor.

Weighting scheme for pooling image descriptors

A method, a system, and a computer program product for extracting one or more images from a storage medium. A search model is selected based on the availability of a semantically related aesthetic model. A search model includes a generic aesthetic model if the semantically related aesthetic model for query is not available. A semantic score and an aesthetic score are computed based on the selected search model. The images are further ranked based on the semantic and aesthetic score.

Methods and systems for ranking images using semantic and aesthetic models

A system and method are disclosed for image selection based on photographic style in which photographic style annotations are learned using a data-driven approach. The method includes assigning a style value for each of a set of photographic style categories to each of a set of database images with a trained classifier of a computing device. A user's selection of a subset of the photographic style categories, such as three style categories, is received. A user interface is generated for assigning values to each of the selected photographic style categories. A set of database images is identified, based on the assigned values for each of the selected photographic style categories and the style values for each of the selected photographic style categories of the database images.

Image selection based on photographic style

A method for color transfer includes retrieving a concept color palette from computer memory corresponding to a concept selected by a user. The concept color palette includes a first set of colors, which may be statistically representative of colors of a set of predefined color palettes which have been associated with the concept. The method further includes computing an image color palette for an input image. The image color palette includes a second set of colors that are representative of pixels of the input image. Colors of the image color palette are mapped to colors of the concept color palette to identify, for colors of the image color palette, a corresponding color in the concept color palette. A transformation is computed based on the mapping. For pixels of the input image, modified color values are computed, based on the computed transformation, to generate a modified image.

System and method for image color transfer based on target concepts

A system and method are provided for modeling a chromatic object, such as an image. For a set of colors of a chromatic object that are expressed as color values in a perceptual color space, the method includes optimizing a convex objective function which is a log likelihood function of a combination of weighted kernels centered on each color in the set over each of the other colors in the set. A number Nof weighted kernels in the optimized function which each have a weight which is at least greater than 0 is identified. The chromatic object is modeled with a mixture model in which the complexity of the model is based on the identified number N.

Growing community of inventors

Grenoble, France

Naila Murray