Microsoft Technology Licensing, LLC

Amazon Technologies, Inc.

Ebay, Incorporated

Kevin Shude Lin

Robinson Piramuthu

Zicheng Liu

Qiaosong Wang

Fan Yang

Lijuan Wang

Kun Luo

Hilliard Bruce Siegel

Andrew Stuart Huntwork

Frédéric Johan Georges Deramat

Saiprasad Satya Kapila

Arvind Srinivasan

Wasiq Bokhari

Ana Sande Do Vale Pinto Da Silva

Johan Christer Liedgren

Disclosed are systems, methods, and non-transitory computer-readable media for using adversarial learning for fine-grained image search. An image search system receives a search query that includes an input image depicting an object. The search system generates, using a generator, a vector representation of the object in a normalized view. The generator was trained based on a set of reference images of known objects in multiple views, and feedback data received from an evaluator that indicates performance of the generator at generating vector representations of the known objects in the normalized view. The evaluator including a discriminator sub-module, a normalizer sub-module, and a semantic embedding sub-module that generate the feedback data. The image search system identifies, based on the vector representation of the object, a set of other images depicting the object, and returns at least one of the other images in response to the search query.

Adversarial learning for finegrained image search

Systems and methods to measure and affect focus, engagement, and presence of users may include measuring a variety of aspects of users engaged in particular activities. Individual user characteristics or preferences and attributes of activities may be taken into account to determine levels of focus for particular users and activities. A variety of sensors may detect aspects of users engaged in activities to measure levels of focus. In addition, a variety of output devices may initiate actions to affect levels of focus of users engaged in activities. Further, a variety of processing algorithms, including machine learning models, may be trained to identify desired levels of focus, to calculate current levels of focus, and to select actions to change or boost levels of focus. In this manner, activities undertaken by users, as well as interactions between multiple users, may be made more engaging, efficient, and productive.

Systems and methods to measure and affect focus and engagement

A computing system is provided. The computing system includes a processor configured to execute a convolutional neural network that has been trained, the convolutional neural network including a backbone network that is a concatenated pyramid network, a plurality of first head neural networks, and a plurality of second head neural networks. At the backbone network, the processor is configured to receive an input image as input and output feature maps extracted from the input image. The processor is configured to: process the feature maps using each of the first head neural networks to output corresponding keypoint heatmaps; process the feature maps using each of the second head neural networks to output corresponding part affinity field heatmaps; link the keypoints into one or more instances of virtual skeletons using the part affinity fields; and output the instances of the virtual skeletons.

Neural network for skeletons from input images

A speech processing system configured to convey linked content to a user in a voice user interface (VUI) environment is described. The system may create output audio data that includes an audible indicator (e.g., a beep) that indicates to a user linked content. An audible indicator may be located at the beginning and/or end of linked content in the output audio data. The system may also output non-linked content in a first voice while outputting linked content in a second voice. The system may further overlay audio atop the linked portion of the output audio data, with the overlaid audio indicating the linked content to the user. A user may invoke linked content in output audio by speaking the linked content, or a portion thereof, back to the system. The system may then output the linked, additional content to the user.

Linked content in voice user interface

Growing community of inventors

Seattle, WA, United States of America

Kevin Shude Lin