Amazon Technologies, Inc.

Joseph Patrick Tighe

Vuong Van Le

Jonathan Wu

Xiang Xu

Vivek Bhadauria

Gerard Guy Medioni

Meng Wang

Hao Chen

Alessandro Bergamo

Shubham Chandra Gupta

Hao Wu

Davide Modolo

Stephen Gould

Zheng Zhang

Prathyusha Reddy Cheruku

Stefano Stefani

Andrea Olgiati

Gaurav D Ghare

Rahul Bhotika

Roman Goldenberg

Wei Xia

Ranju Das

Vasant Manohar

Jonathan Hedley

Yuanjun Xiong

Nataliya Shapovalova

Gowtham Jeyabalan

Nitin Singhal

Kaustav Kundu

Hao Zhou

Michael Quang Thai Lam

Praveen Kumar Udayakumar

Fanyi Xiao

Mingze Xu

Tianchen Zhao

Kuang Han

Renjun Zheng

Hao Li

Xinyu Li

Manchen Wang

Systems and techniques are disclosed for authorizing users based on biometric data. Images are collected from an authorized user are used to generate three-dimensional representations of the user that are stored for use in authorization operations. Images accompanying a request for authorization are first processed using liveness detection operations and then, if the images are associated with a live person, a three-dimensional representation of the person is generated using the images. If a correspondence to a verified user's three-dimensional representation is identified for the three-dimensional representation of the person requesting authorization, the request is granted.

Mobile device user authorization system

Techniques for liveness detection based on gesture validation, facial expression analysis, and concurrency validation. The techniques include selecting a color light pattern challenge and sending the color light pattern challenge to a personal computing device for display on a display screen of the personal computing device. A set of target images (video) is received by a liveness detection service in a provider network from the personal computing device as a response to the challenge. The liveness detection service analyzes the set of target images for macro-facial expression and micro-facial expressions. A liveness determination is made by the liveness detection service as to whether the user of the personal computing device is a live genuine user or an impersonated user based on the analysis of the macro and micro-facial expressions detected in the set of target images.

Liveness detection based on gesture validation, facial expression analysis, and concurrency validation

Techniques for liveness detection using a motion, face, and context cues. The techniques can be implemented to prevent against successful presentation attacks, video injection attacks, and deepfake attacks. In some examples, the techniques encompass receiving a set of video frames from a personal computing device. A first liveness determination can be made using a motion-based model based on the received video frames. A second liveness determination can be made using a face-based model based on the received video frames. A third liveness determination can be made using a context-based model based on the received video frames. A final liveness determination can be made based on the first, second and third liveness determinations.

Liveness detection based on motion, face, and context cues

Techniques for unique label recognition and notification for streaming video. A video stream from a camera-equipped electronic device is received via the API endpoint of a stream processing service in a provider network. A request for unique label recognition and notification is received via an API endpoint of a computer vision service in the provider network. The computer vision service analyzes a video fragment of the video stream. If a unique label is recognized in the video fragment, then a notification service in the provider network sends a notification that the unique label was recognized in video captured by the camera-equipped electronic device.

Unique label recognition and notification for streaming video

Techniques for label recognition and notification for streaming video from non-overlapping cameras. A stream processing service of a provider network receives a first video stream from a first camera-equipped electronic device via an API endpoint of the stream processing service. The stream processing service also receives a second video stream from a second camera-equipped electronic device an API endpoint of the stream processing service. Meanwhile, a request for label recognition and notification is received at a computer vision service of the provider network via an API endpoint of the computer vision service. In response, the computer vision service recognizes a label in a video fragment of the first camera video stream and recognizes a label in a video fragment of the second camera video stream, and then identifies whether the two labels are the same label. If so, a notification service of the provider network sends a notification indicating that the label was recognized across non-overlapping cameras.

Label recognition and notification for streaming video from non-overlapping cameras

A system for evaluating a biometric authorization system is described. The biometric authorization system is configured to apply a facial recognition model to image data to make an authorization determination based on detection of synthesized image data and based on matching a reference image to the image data. The system is also configured to execute one or more synthetic image data attack protocols to evaluate the biometric authorization system. The system also generates, according to one or more synthetic image data generation techniques, an evaluation set of image data comprising synthesized representations of a target and sends one or more authorization requests using the evaluation set of image data to the biometric authorization system. The system generates an evaluation of the biometric authorization system for synthetic image data attack analysis based on respective responses to the one or more authorization requests received from the biometric authorization system.

Evaluating biometric authorization systems with synthesized images

Techniques for training a visual neural network are described. In particular, the training of the visual neural network may be performed using one or more contrastive learning loss functions including one or more of a visual-to-visual contrastive learning function using the visual neural network on positive and negative video clips according to a first loss function, a secondary-to-secondary contrastive learning a secondary neural network on secondary positive and negative information generated from the positive and negative video clips, and a secondary-to-visual contrastive learning according to a third loss function using the visual neural network on positive and negative video clips and using the secondary neural network secondary positive and negative information generated from the positive and negative video clips.

Secondary distillation for self-supervised video representation learning

Personal equipment detection may utilize pose-based detection. Input image data may be evaluated to detect persons in the image data. For detected persons, regions of the persons may be determined. Personal equipment may be detected for the detected persons in the image data and compared with the regions of the persons to determine whether the detected personal equipment is properly placed on the person.

Pose-based personal equipment detection

Methods, systems, and computer-readable media for content moderation using object detection and image classification are disclosed. A content moderation system performs object detection on an input image using one or more object detectors. The object detection finds one or more elements in the input image. The content moderation system performs classification based at least in part on the input image using one or more image classifiers. The classification determines one or more values indicative of one or more content types in the input image. The content moderation system determines one or more scores for one or more content labels corresponding to the one or more content types. At least one of the scores represents a finding of one or more of the content types in the input image. The content moderation system generates output indicating the finding of the one or more of the content types.

Content moderation using object detection and image classification

A camera with a field of view of a shelf may acquire images of items on the shelf at subsequent times. An analysis of each image yields a set of estimated locations of the items on the shelf. Based on a working volume of the shelf, a subset of valid estimated locations of the items at the shelf may be determined. Thereafter, a count of the items on the shelf at a particular time may be determined using the subset of valid estimated item locations. By comparing the count of items at the shelf at two subsequent times, a change in quantity of the items on the shelf may be determined. Interaction data may be generated in response to detecting the change in quantity of items at the shelf.

Detecting inventory changes by comparing image data

A data processing pipeline controller receives a request, from a data iterator associated with a machine learning model, for a data output of a module in the data processing pipeline, wherein each module in the data processing pipeline has an associated cache. The controller determines whether a data output of the module is stored in the associated cache and responsive to the data output being stored in the associated cache, provides the data output from the associated cache to the data iterator without processing data through the module.

Data iterator with automatic caching

Image data representative of an inventory location may be acquired by cameras. Described are techniques to process such image data to remove one or more perspective effects. For example, transformation data that corresponds to the inventory location may first be determined. Such transformation data is determined based on associations between points in a common plane and alternate points in a virtual camera plane. Transformed image data may then be generated by applying the transformation data to the image data that is representative of the inventory location.

Image rectification using transformation data

Images of a fixture may be acquired by cameras positioned with a field-of-view of the fixture. Such images are processed to identify estimated tops of items at the fixture. Using the estimated tops and item data for items designated for stowage at the fixture, one or more estimated locations of items (such as bounding boxes representative of the items) may be determined. Each estimated location for an item is tested for validity. For example, each estimated location is checked to see if the estimated location is within a working volume of the fixture. If the estimated location of the item is within the working volume, the item is determined to be valid. Otherwise, the item is deemed invalid.

Testing validity of items using image data analysis

Techniques for analyzing stored video upon a request are described. For example, a method of receiving a first application programming interface (API) request to analyze a stored video, the API request to include a location of the stored video and at least one analysis action to perform on the stored video; accessing the location of the stored video to retrieve the stored video; segmenting the accessed video into chunks; processing each chunk with a chunk processor to perform the at least one analysis action, each chunk processor to utilize at least one machine learning model in performing the at least one analysis action; joining the results of the processing of each chunk to generate a final result; storing the final result; and providing the final result to a requestor in response to a second API request is described.

Analysis of video content

Images of a fixture such as a shelf holding items may be acquired by cameras and processed to count the quantity of items at the fixture. A top of an item is determined in the image. Given information about the items designated for stowage at the fixture and the location of the top, a three-dimensional (3D) bounding box indicative of a volume is determined relative to the fixture. Bounding boxes which extend outside the boundaries of the fixture are disregarded. Remaining bounding boxes may then be analyzed to determine a measured height of the item(s) in a stack. The measured height may be divided by a per-item height to determine a quantity of items in the stack. The quantities in multiple stacks may be summed to determine a quantity at the fixture.

Growing community of inventors

Seattle, WA, United States of America

Joseph Patrick Tighe