Amazon Technologies, Inc.

Dilip Kumar

Gerard Guy Medioni

Manoj Aggarwal

Nishitkumar Ashokkumar Desai

Sridhar Boyapati

Gianna Lise Puerini

Sudarshan Narasimha Raghavan

Jayakrishnan Kumar Eledath

Jin Dong Kim

Ehud Benyamin Rivlin

Jason Michael Famularo

Kushagra Srivastava

Ammar Chinoy

Amber Autrey Taylor

Liefeng Bo

Thomas Meilandt Mathiesen

Xiaofeng Ren

Roman Goldenberg

Ofer Meidan

Korwin Jon Smith

Gopi Prashanth Gopal

Nikhil Chacko

Varadarajan Gopalakrishnan

Daniel Bibireata

Subram Narasimhan

Alexander Clark Prater

Jared Joseph Frank

Shrivatsan Vasudhevan

Omar FawazHashim Zakaria

Kartik Muktinutalapati

William Spencer Worley, Iii

Ohil Krishnamurthy Manyam

Ramanathan Palaniappan

Michel Leonard Goldstein

Paul Eugene Munger

Igor Kviatkovsky

Steven Kessel

George Leifman

Jaechul Kim

Alon Shoshan

Rajeev Ranjan

Robert Crandall

Dmitri Veikherman

Chang Yuan

Michael Lee Brundage

Joseph Xavier

Prithviraj Banerjee

Lior Zamir

Nadav Israel Bhonker

Hoi Cheung Pang

Wyatt David Camp

Elie Micah Kornfield

David L Selinger

Boris Cherevatsky

Michelle K Anderson

Shaonan Zhang

Minmin Chen

Yasser Baseer Asmi

David William Bettis

Alexander Michael McNamara

Jon Robert Ducrou

Aaron Craig Thompson

Waqas Syed Ahmed

Danny Guan

Félix Joseph Étienne Pageau

Christopher Andrew Stephens

David Echevarria Ignacio

Kevin Alexander Lee

Gary Paolo Raden

Muralidhar Koka

Jiajun Zhao

Yacov Hel-Or

Natan Peterfreund

Joseph M Alyea

Eli Osherovich

Sora Kim

Sanjay Dash

Hoi-Cheung Pang

Paat Rusmevichientong

Longlong Zhu

Anil Reddy Musku

Lekha A Bhargavi

Daniel M Grossman

Ivan Stankovic

Rui Zhao

Gang Hua

Li Guan

Somasundaram Niranjayan

Emilio Ian Maldonado

Navid Shiee

Christopher Robert Towers

Nikolai Orlov

Ronald Eugene Huebner

Jordan Tyler Williams

Abhinav Kashyap

Steven Augustine Wilson

Baomin Wang

Kun Qian

Keunhong Park

Zheng Tang

During operation a biometric identification system acquires an image of at least a portion of a user that is processed by a representation model to determine representation data. The representation data may be securely stored for later use. The image is not stored. The stored representation data is smaller than equivalent image data. The representation model may comprise a subset of a first recognition model that has been trained using image training data. The image training data may be processed by the trained representation model to determine training representation data. The representation data contains sufficient information to be used as training data for recognition models, instead of acquired images. Training of recognition models using representation data is more computationally efficient compared to training with image data. Recognition comparisons using query representation data are more computationally efficient compared to query image data.

Storable representation of biometric data

An image of at least a portion of a user during enrollment to a biometric identification system is acquired and processed with a plurality of models to determine embeddings that are representative of features in that image in respective embedding spaces. Transformers are trained to accept as input an embedding from a respective model and produce as output a transformed embedding consistent with a common embedding space. These transformed embeddings are aggregated to produce transformed aggregated embedding data and stored for later comparison to identify the user. Subsequently, one or more embedding models may be used to process a query image to determine a query embedding. The query embedding may be transformed into the common embedding space for comparison.

System to reduce data retention using aggregated embedding data

This disclosure describes architectures and techniques to provide information to a user about items with which the user interacts. In some instances, a user may utilize a wearable device that is configured to interact with one or more components of an information discovery system to obtain information about items in the user's environment.

Item information presentation system

This disclosure describes a system for automatically transitioning items from a materials handling facility without delaying a user as they exit the materials handling facility. For example, while a user is located in a materials handling facility, the user may pick one or more items. The items are identified and automatically associated with the user at or near the time of the item pick. When the users enters and/or passes through a transition area, the picked items are automatically transitioned to the user without affirmative input from or delay to the user.

Item transitions

During enrollment to a biometric identification system an image of at least a portion of a user is acquired and processed to determine a compact representation (CR). The CR may be stored for later comparison to identify the user, to train embedding models, and so forth. The image is not stored. A machine learning system that comprises a variational autoencoder is trained to produce the CR with a loss function that includes a distortion loss, an embedding distance loss, and in some implementations a bitrate loss. The trained encoder is used to determine the CR from an input image, while the decoder is not stored. The CR contains sufficient information to be used as training data for embedding models, instead of acquired images. Training may be more computationally efficient using the CR. Recognition comparisons may be more efficiently performed using the CR, compared to query images.

System to determine compact representation of data

Multiple video files that are captured by calibrated imaging devices may be annotated based on a single annotation of an image frame of one of the video files. An operator may enter an annotation to an image frame via a user interface, and the annotation may be replicated from the image frame to other image frames that were captured at the same time and are included in other video files. Annotations may be updated by the operator and/or tracked in subsequent image frames. Predicted locations of the annotations in subsequent image frames within each of the video files may be determined, e.g., by a tracker, and a confidence level associated with any of the annotations may be calculated. Where the confidence level falls below a predetermined threshold, the operator may be prompted to delete or update the annotation, or the annotation may be deleted.

Multi-video annotation

A biometric identification system may store biometric data for later assessment. Data storage parameters, such as cryptographic keys used to encrypt and decrypt the biometric data, may be determined based on the biometric data. In one implementation, the biometric data comprises embedding data in an embedding space. During enrollment and storage, the embedding data is assessed to determine nearest anchor data in the embedding space. Cryptographic parameters, such as an encryption key, are determined based on 'k' anchor data that are within a threshold distance of the embedding data in the embedding space. During query, query embedding data is similarly processed to determine cryptographic parameters, such as a decryption key. The decryption key may then be used to attempt decryption of the encrypted at-rest biometric data. If successful, the decrypted biometric data may then be compared to the query embedding to assert an identity.

System for determining storage parameters for biometric data

This disclosure describes a system for managing inventory as it transitions into a materials handling facility, as it transitions between locations within a materials handling facility and/or as it transitions out of a materials handling facility. In some instances, a user (e.g., picker or picking agent) may retrieve an item from an inventory location and place the item into a tote. The systems described herein detect the item when it is added to or removed from the tote.

Image-based item detection and allocation

Maintaining the security of biometric data is an utmost priority. Biometric data is secured using one or more techniques. With one technique, biometric input such as images of a user's palm is used to generate first primary data (PD). The original biometric input is deleted from temporary secure storage while the first PD is securely stored. The first PD may then be processed later to determine a second PD. The first PD may then be deleted, and the second PD subsequently used. With another technique, biometric input or a PD may be processed by a first model to determine first secondary data (SD) that is representative of features of a particular user within a first embedding space. Later the PD may be processed by a second model to determine a second SD in a second embedding space. The first SD is deleted, and the second SD subsequently used.

System to manage biometric data

This disclosure describes a system for disambiguating between multiple potential users that may have performed an item action (e.g., item removal or item placement) at an inventory location. For example, if there are three picking agents (users) standing near an inventory location and one of the agents removes an item (item action) from the inventory location, the example systems and processes described herein may utilize various inputs to disambiguate between the users and determine which of the potential users performed the item action.

Disambiguating between users

Described is a system and method for presenting event information to a user and, if necessary, obtaining confirmation of different aspects (user, item, action) of the event. In some implementations, an event includes a user, an action, and an item. For example, an event may include a user picking an item from an inventory location, a user placing an item into a tote associated with the user, etc. If the aspects of the event cannot be determined with a high enough degree of confidence, a user interface may be generated and sent to the user requesting confirmation of one or more of the aspects of the event.

Event determination and presentation

This disclosure describes a system for tracking removal or placement of items at inventory locations with a materials handling facility. In some instances, a user may remove an item from an inventory location and the inventory management system may detect that removal and update a user item list associated with the user to include an item identifier representative of the removed item. Likewise, if the user places an item at an inventory location, the inventory management system may detect that placement and update the user item list to remove an item identifier representative of the placed item.

Detecting inventory changes

Enhanced training data representative of possible inputs is used to train a machine learning system. For example, a machine learning system to determine identity based on an image of a human palm may be trained using enhanced training data comprising images. The enhanced training data may comprise source images that have been modified to appear to depict synthetic artifacts that attempt to simulate human palms, augmented images of dirty hands, and so forth. A synthetic artifact image may be produced by selectively removing some data from a source image. An augmented image may be produced by selectively blending the source image with features extracted from sample images. These images may then be used as training data to train the machine learning system.

System for generating enhanced training data

This disclosure is directed to techniques in which a first user in an environment scans visual indicia associated with an item, such as a barcode, before handing the item to a second user. One or more computing devices may receive an indication of the scan, retrieve image data of the interaction from a camera within the environment, identify the user that received the item, and update a virtual cart associated with the second user to indicate addition of the item.

Analyzing sensor data to identify events

A biometric identification system acquires a multimodal image. The system is trained to determine three embeddings: a first embedding associated with features present in a first modality, an intersection embedding associated with features present in both the first and a second modality, and an XOR embedding associated with features that are not shared between the first modality and the second modality. Once trained, the system may process a multimodal query image to determine query embedding data. Additional information, such as minutiae depicted in the multimodal query image may also be determined. The query embedding data, and in some implementations the additional information, may be compared with enrolled embedding data associated with a previously enrolled user. If the comparison exceeds a threshold value, an identity may be asserted.

System for single image multimodal biometric recognition

Growing community of inventors

Seattle, WA, United States of America

Dilip Kumar