A9.com, Inc.

Amazon Technologies, Inc.

Google Inc.

Raghavan Manmatha

Mark Andrew Ruzon

Sunil Ramesh

Donald O Tanguay

Max Delgadillo

Arnab S Dhua

Gurumurthy D Ramkumar

Supratik Bhattacharyya

Gautam Bhargava

Xuefu Wang

Son Dinh Tran

Wei-Hong Chuang

Arnab Sanat Kumar Dhua

Christopher Richard Uhlik

Luc Vincent

Ilya Vladimirovich Brailovskiy

Ashwin Swaminathan

Bo Pang

Shai Mazor

Jonathan Wu

Srikar Appalaraju

Oron Anschel

Shahar Tsiper

Romain Thibaux

Siqi Deng

Hao Yang

Roee Litman

Ron Litman

Ali Caner Turkmen

Varad Gunjal

Christopher Michael Russell

Matthaeus Kleindessner

Shaolei Feng

Bhargava Urala Kota

Kailash Budhathoki

Donald Tanguay

The present disclosure generally relates to systems and methods for searching media content. In some implementation examples, a search system receives an input query, generates a query embedding of the input query, and generates a bias mitigation transformation associated with a sensitive attribute. Based on the query embedding and the bias mitigation transformation, the search system generates a transformed query embedding that suppresses at least a portion of the query embedding related to the sensitive attribute. Using the transformed query embedding, the search system executes a similarity search in a media embedding model to identify one or more media embeddings that are similar to the transformed query embedding and transmits the one or more media embeddings.

Mitigating bias in multimodal models via query transformation

Representations of sets of descriptors of reference objects are stored in a repository, with individual descriptors including information about entities identified in the reference objects. In response to a request to extract content from a particular data object, a reference object which satisfies a similarity criterion with respect to the particular data object is identified from the repository using the descriptors. A structural comparison of the particular data object and the reference object is performed to determine an entity related to another entity identified in the particular data object.

Content extraction using related entity group metadata from reference objects

Techniques for recognizing text in an image are described. An exemplary method may include receiving a request to recognize text in an image; extracting features from the image and generating a visual feature sequence from the extracted features; performing selective contextual refinement at least one selective contextual refinement block of a stack of selective contextual refinement blocks to generate a text prediction by: generating a contextual feature map and combining the contextual feature map with the visual feature sequence into a visual feature space, and applying a selective decoder that utilizes a two-step attention on the visual feature space to generate a text prediction, wherein the two-step attention includes performing a 1-D self-attention computation to generate attentional features and decoding the attentional features to generate the text prediction; and outputting the generated text prediction.

Residual context refinement network architecture for optical character recognition

Techniques are generally described for enhanced compression of video data. In various examples, the techniques may include receiving first video data representing a scene in an environment. The techniques may further include generating illumination map data representing illumination of the scene in the first video data. The techniques may further comprise generating reflectance map data representing a reflectance of at least one object in the first video data. In some examples, the techniques may include sending, to a second computing device, the illumination map data and the reflectance map data. The techniques may further include receiving second video data representing the scene. The techniques may include determining a first illumination difference between the second video data and the first video data. The techniques may comprise sending, to the second computing device, the first illumination difference.

Enhanced compression of video data

Tasks such as object classification from image data can take advantage of a deep learning process using convolutional neural networks. These networks can include a convolutional layer followed by an activation layer, or activation unit, among other potential layers. Improved accuracy can be obtained by using a generalized linear unit (GLU) as an activation unit in such a network, where a GLU is linear for both positive and negative inputs, and is defined by a positive slope, a negative slope, and a bias. These parameters can be learned for each channel or a block of channels, and stacking those types of activation units can further improve accuracy.

Activation layers for deep learning networks

Various approaches enable a user to capture image information (e.g., still images or video) about an object of interest such as the sole of a shoe or other piece of footwear (e.g., a sandal) and receive information about items that are determined to match footwear based at least in part on the image information. For example, an image analyze service or other similar service can analyze the images to determine a type of shoe included within the images based at least in part on patterns of other distinguishing features of the sole of the shoe. The image analysis service can aggregate the results and can provide information about the results as a set of matches or results to be displayed to a user in response to a visual search query. The information can include, for example, descriptions, contact information, availability, location data, pricing information, and other such information.

Object retrieval

Systems and approaches for searching a content collection corresponding to query content are provided. In particular, false positive match rates between the query content and the content collection may be reduced with a minimum content region test and/or a minimum features per scale test. For example, by correlating content descriptors of a content piece in the content collection with query descriptors of the query content, the content piece can be determined to match the query content when a particular region of the content piece and/or a particular region of a query descriptor have a proportionate size meeting or exceeding a specified minimum. Alternatively, or in addition, the false positive match rate between query content and a content piece can be reduced by comparing content descriptors and query descriptors of features at a plurality of scales. A content piece can be determined to match the query content according to descriptor proportion quotas for the plurality of scales.

Content collection search with robust content matching

A method, system and computer program product for encoding an image is provided. The image that needs to be represented is represented in the form of a Gaussian pyramid which is a scale-space representation of the image and includes several pyramid images. The feature points in the pyramid images are identified and a specified number of feature points are selected. The orientations of the selected feature points are obtained by using a set of orientation calculating algorithms. A patch is extracted around the feature point in the pyramid images based on the orientations of the feature point and the sampling factor of the pyramid image. The boundary patches in the pyramid images are extracted by padding the pyramid images with extra pixels. The feature vectors of the extracted patches are defined. These feature vectors are normalized so that the components in the feature vectors are less than a threshold.

Method and system for matching an image using normalized feature vectors

Various embodiments of the present invention relate to a method, system and computer program product for detecting and recognizing text in the images captured by cameras and scanners. First, a series of image-processing techniques is applied to detect text regions in the image. Subsequently, the detected text regions pass through different processing stages that reduce blurring and the negative effects of variable lighting. This results in the creation of multiple images that are versions of the same text region. Some of these multiple versions are sent to a character-recognition system. The resulting texts from each of the versions of the image sent to the character-recognition system are then combined to a single result, wherein the single result is detected text.

Method and system for detecting and recognizing text in images

Present invention relates to a method and system for automatic searching for information on a network in response to an image query sent by a user. The image query includes an image that is captured by using a mobile communications device with a camera. The image is processed to detect the text present in it. The detected text is then recognized using an OCR. Subsequently, the text is searched for matches in the corresponding domain database, selected from the various domain databases present in the network. Thereafter, selected matches and additional related information is sent to the user.

Method and system for searching for information on a network in response to an image query sent by a user from a mobile communications device

Method and system for matching an image using image patches

Growing community of inventors

Belchertown, MA, United States of America

Raghavan Manmatha