Amazon Technologies, Inc.

Jonathan Wu

Meng Wang

Joseph Patrick Tighe

Xiang Xu

Wei Xia

Hao Wu

Rahul Bhotika

Zheng Zhang

Raghavan Manmatha

Hao Chen

Vijay Mahadevan

Shai Mazor

Ranju Das

Yuanjun Xiong

Oron Anschel

Shuo Yang

Joaquin Zepeda Salvatierra

Kunwar Yashraj Singh

Shahar Tsiper

Song Cao

Ron Litman

Roee Litman

Hao Zhou

Erhan Bas

Mingze Xu

Tianchen Zhao

Systems and techniques are disclosed for authorizing users based on biometric data. Images are collected from an authorized user are used to generate three-dimensional representations of the user that are stored for use in authorization operations. Images accompanying a request for authorization are first processed using liveness detection operations and then, if the images are associated with a live person, a three-dimensional representation of the person is generated using the images. If a correspondence to a verified user's three-dimensional representation is identified for the three-dimensional representation of the person requesting authorization, the request is granted.

Mobile device user authorization system

Techniques for liveness detection based on gesture validation, facial expression analysis, and concurrency validation. The techniques include selecting a color light pattern challenge and sending the color light pattern challenge to a personal computing device for display on a display screen of the personal computing device. A set of target images (video) is received by a liveness detection service in a provider network from the personal computing device as a response to the challenge. The liveness detection service analyzes the set of target images for macro-facial expression and micro-facial expressions. A liveness determination is made by the liveness detection service as to whether the user of the personal computing device is a live genuine user or an impersonated user based on the analysis of the macro and micro-facial expressions detected in the set of target images.

Liveness detection based on gesture validation, facial expression analysis, and concurrency validation

Techniques for liveness detection using a motion, face, and context cues. The techniques can be implemented to prevent against successful presentation attacks, video injection attacks, and deepfake attacks. In some examples, the techniques encompass receiving a set of video frames from a personal computing device. A first liveness determination can be made using a motion-based model based on the received video frames. A second liveness determination can be made using a face-based model based on the received video frames. A third liveness determination can be made using a context-based model based on the received video frames. A final liveness determination can be made based on the first, second and third liveness determinations.

Liveness detection based on motion, face, and context cues

A system for evaluating a biometric authorization system is described. The biometric authorization system is configured to apply a facial recognition model to image data to make an authorization determination based on detection of synthesized image data and based on matching a reference image to the image data. The system is also configured to execute one or more synthetic image data attack protocols to evaluate the biometric authorization system. The system also generates, according to one or more synthetic image data generation techniques, an evaluation set of image data comprising synthesized representations of a target and sends one or more authorization requests using the evaluation set of image data to the biometric authorization system. The system generates an evaluation of the biometric authorization system for synthetic image data attack analysis based on respective responses to the one or more authorization requests received from the biometric authorization system.

Evaluating biometric authorization systems with synthesized images

Techniques for creating machine learning models for direct homography regression for image rectification are described. In certain embodiments, a training service trains an algorithm on a source view of a training image and a homography matrix of the training image into a machine learning model that generates a normalized homography matrix for an input of the source view. The normalized homography matrix may then be utilized to generate a target view of an image input into the machine learning model. The target view of the image may be used in a document processing pipeline for document images captured using cameras.

Machine learning models for direct homography regression for image rectification

Techniques for recognizing text in an image are described. An exemplary method may include receiving a request to recognize text in an image; extracting features from the image and generating a visual feature sequence from the extracted features; performing selective contextual refinement at least one selective contextual refinement block of a stack of selective contextual refinement blocks to generate a text prediction by: generating a contextual feature map and combining the contextual feature map with the visual feature sequence into a visual feature space, and applying a selective decoder that utilizes a two-step attention on the visual feature space to generate a text prediction, wherein the two-step attention includes performing a 1-D self-attention computation to generate attentional features and decoding the attentional features to generate the text prediction; and outputting the generated text prediction.

Residual context refinement network architecture for optical character recognition

Techniques for keypoint-based multi-label word segmentation and localization are described. A machine learning model identifies bounding regions of text within an image, and then generates multiple channel matrices representing predicted keypoints of the text within the bounding regions. The keypoints can be used to rectify the corresponding graphical content from the image including the text to improve the ability to perform optical character recognition and identify the text. Line and word segmentation and localization can be performed together.

Keypoint-based multi-label word segmentation and localization

Embodiments of the present disclosure provide systems and processes for automatically determining a layout of text within an image that makes sense from a semantic perspective. In certain embodiments, the systems disclosed herein receive bounding box information relating to one or more bounding boxes that surround text within the image. The systems compare the received bounding box information to determine a clustering of bounding boxes that have an above threshold probability of including words that when read in order make sense semantically. For example, systems herein can determine whether words in a cluster correspond to a line of text.

Semantic ordering of image text

A method and system for analyzing text in an image is disclosed. A text localization and classification system accesses an annotated image comprising a plurality of text location identifiers for a given item of text. A neural network predicts the location of the given item of text using at least a first location identifier and a second location identifier. Optionally, the first location identifier comprises a first shape and the second location identifier comprises a second shape. A first loss is generated using a first loss function, the first loss corresponding to the predicated location using the first location identifier. A second loss is generated using a second loss function, the second loss corresponding to the predicated location using the second location identifier. The neural network is enhanced with backpropagation using the first loss and the second loss.

Image text localization

A method and system for analyzing text in an image. Classification and localization information is identified for the image at a word and character level. A detailed profile is generated that includes attributes of the words and characters identified in the image. One or more objects representing a predicted source of the text are identified in the image. In one embodiment, neural networks are employed to determine localization information and classification information associated with the identified object of interest (e.g., a text string, a character, or a text source).

Growing community of inventors

Seattle, WA, United States of America

Jonathan Wu